The joy of 360° images and 360° virtual tours

Once I discovered the joy of wide angle images I barely photographed with any other focal length. I like being able to capture more than the eye can take in in a single glance. Occasionally I played with stitched panoramic images and liked the results. In fact my first ever photography competition win was a stitched panoramic photograph. I also enjoyed playing around with imagery constructed from multiple images, not Photoshop montages, but taking multiple images and arranging them, often physically with separate prints rather than stitching and blending. I like this, there is an element of craft, of creation, of building an image rather than just 'pointing a camera and pressing a button'. Yes, I know, there is more to photography than that, but this is how I feel, it's why I enjoy the constructed image.

As those of you who follow the highs and lows of my photography know, I came to a point last year where I ran out of steam. I was really struggling to engage with the medium. It's not that photography is lacking, it's my desire to capture more of the experience of being in a place. I'm not an artist, I'm a high degree experientialist. Right, that's my colours nailed firmly to the mast!

And virtual tours come a lot closer to portraying the experience in a place to the viewer. They have enhanced my experience of the moment and the location as I build up detailed soundscapes and make supporting photography and video to be embedded in the tour that give a sense of a place, that tell it's story.

My current year long comission to produce virtual tours for The Royal Society of Wildlife Trusts Living Landscape Scheme has not only invigorated my desire to make landscape photographs, but is giving me a chance to really explore the virtual tour medium in lots of exciting and innovative ways. Sadly I can't share the results until 2012, but watch this space.

The perception of people to virtual tours is interesting, but especially by other photographers. My mum, herself a good photographer and someone who used to assist an uncle who was a pro photographer, looked at a few compared to my usual work, and then quickly commented that "well, its just like standing in a spot and making a photograph all round" implying that it is not art, but simply a record of a location. In some ways she is right, but she is also very wrong. I think we all need to draw our own conclusions about this. I'd love to hear yours, please leave comments below.

And in this age of touch computing (and digital cameras), where people pass photographs around again, photographs that they can interact with, it feels like this form of photography has enhanced meaning to people. The first time I took an iPad to a client demo and passed round a virtual tour was an awesome moment as each person interacted with the imagery, and obviously enjoyed, the experience. And with recent advances in virtual tour tech allowing video and even 360° video the world of virtual tours is even more immersive.

The process of stitching the panoramas, of toning the HDR files, of constructing the virtual tour and it's story take me back to the days of film when there was more process, more anticipation of whether the image would work or not. Watching the image stitching on the screen is very akin to watching the magic of a print in a tray of developer. It feels like magic. It looks magic. I love it!

I love the result so much I often turn on full screen mode of the virtual tour with the sound up and just let it spin away, like my own relaxing screensaver.

Equipment and software

Hardware

• Nikon D700 camera
• Nikon 16mm fisheye lens
• Nodal Ninja NN3 panoramic head
• Laser pointer in hotshoe mount
• Manfrotto 055MF4 with 460Mg head
• Panasonic GF1 with 20mm f/1.7 lens
• Marantz PMD661 solid state sound recorder
• BeyerDynamic DT100 headphones
• Sennheisser K6 microphone connected via XLR cable with Rycote windgag
• iPhone with Panavision Panascout app to record a geotagged photograph. (Not the best geotagging app, but it's a bit tarty!)

Software setup for virtual tours

• MacBook Pros (various configs, maxed out RAM, SSD drives)
• Mac OS X Finder
• Adobe Lightroom
• Photomatix Pro
• PT Gui Pro
• Adobe Photoshop CS5 with Noise Ninja and Photokit Sharpener
• Kolor Panotour Pro
• krpano with google maps and e-card plugins, amongst others

I am very seriously considering selling my Nikon gear right now and building a micro 4/3 system based around the Panasonic GF1 or the Olympus E-P3. Great image quality, compact (which really matters on a bike), lightweight, I can carry two bodies for less weight and cost than a good DSLR, and for harsh environments I could use an underwater housing (which would also allow me to do underwater virtual tours.) Food for thought.

Taking the photographs

Setting up a panoramic head can initially be quite daunting, but I've done a few now and I followed the instructions carefully and it was no problems. The Nodal Ninja has little quick set plates that can be locked in place so the setup is only required once if you use a single lens, which I do. Easy.

The hardest part of making 360x180° panoramic photographs is finding the right spot. I pre-visualise the sort of photograph I want to make. The location can be chosen based on the story of the location, it may be necessary to chose a central point that illustrates key characteristics of the location rather than the most visually attractive point. Finding the right centre point, in terms of X, Y and Z coordinates is important to the overall balance of the composition. Will the composition work in both the virtual tour and a projected equirectangular image print? Considering how moving objects will affect the stitch. How will light and the suns position effect the final image? How will the image look once it is flattened out into a equirectangular projection? Just because the image will look great as a virtual tour, doesn't mean that the projected equirectangular image will look great. How will the extreme perspective distortion of the fisheye push even fairly close objects further away? Are there any nearby edges that will cause parallax issues when I make the 'tripodless' nadir (downward) image? How will I get out of the way of the all seeing fisheye lens, especially given my love of virtual tours on the edges of cliffs and up trees? What sounds do I need to record? What supporting images and video to further the story?

Actually taking the photographs is pretty easy after that. I can actually be setup and have made a full 360x180° in under a minute. It's even possible to do grab shots, of slow moving objects of course, and I've made some very effective virtual tours on rivers with boats and of animals such as fields of cows or geese.

I like a good amount of overlap between shots to maximise control point matching and also to allow extra margins for moving objects so they are easier to include or exclude in the final stitch. To achieve this I make six images going round, or one every 60°. (That works for me with full frame and a 16mm fisheye.) I then make a zenith shot pointing straight up. Then the first nadir shot pointing straight down which will include the tripod. I then use the laser pointer, which is mounted on a fixed position on the camera, to identify a reference location on the ground, move the tripod to the side a bit, then realign the camera to the same position using the laser pointer and the reference location. With most nadir shots it doesn't matter if the camera position is a little different, but if there are any near/far edges such as steps, railings, furniture or branches it is critical to achieve very similar camera positions or the stitching process will become very complicated with much Photoshop post-processing.

I also shoot exposure bracketed shots so I can produce HDR files to stitch. I will normally shoot five exposures per position at 0, +1, +2, -1 and -2EV. The +1EV and -1EV files will normally be deleted as they aren't needed to make an HDR file. I wish camera makers would take note of the amount of people shooting HDR and make the very simple firmware changes that would give us photographers more control over exposure bracketing e.g. being able to shoot 0, +2 and -2EV.

Creating soundscapes for virtual tours

Location sound recording is an art in itself and I have a lot to learn. I try to listen closely to the soundscape of an area and identify characteristic sounds in the same way that I identify characteristic features to build into the visual composition. One of the nice features of krpano is that with a little coding it is possible to add directional sounds into virtual tours so that the soundscape changes as you move around inside the virtual tour. It needs to be done subtly, and with cheap PC speakers the effect may be lost, but with even half decent speakers it is possible to compose very interesting soundscapes.

That said, due to time constraints on location and load time bandwidth on viewers computers, I will normally make a maximum of 2-3 sound recordings per location. Once I have identified the best sound source directions I will make a few recordings, usually with verbal notes to describe what, why and the direction of recording. 

Location sound is often windy making for sometimes overpowering wind noise in recordings. The mic windgag helps. Notionally I could invest in a 'zeppelin' wind isolator, but cost and the large size of them keeps me away. I usually find it easier to slightly change position and/or shield the mic with my body or a nearby object such as a tree or building.

Pre-processing

First I delete the +1 and -1EV bracketed shots. I open the memory card folder in Finder, make the window larger so that the files are five across (the same number of shots per bracketed exposure for each camera position) and drag select the un-needed files then delete them. I'll then resize the Finder window so there are three shots wide (the remaining bracketed shots) and check I have all the files needed.

I then ingest the files into Adobe Lightroom as DNG files with the usual keywording process.

Then colour code each panorama, rank them and make any group wide corrections, such as setting white balance to be the same across all images, then save all changes into the metadata. (Yep, I don't follow the rules of setting a fixed exposure and white balance at the point of capture. This would be a mistake with single frame jpegs, but with HDR and RAW files the results are better for me with auto exposure and set the WB after.)

The final step in Lightroom is to export 16 bit TIFF files with a ProPhoto colour space.

HDR 1

Photomatix Pro is a really useful tool for processing lots of images as there is a well thought out batch processing tool. I'll set up a batch process to produce HDR files to the following setup:

• reduce ghosting
• noise reduction
• reduce chromatic aberration
• align features using horizontal/vertical shifts if the location was precarious or windy
• output Radiance HDR files

And then leave it to do it's thing. Easy. BTW I output Radiance files rather than OpenEXR as I have found that PTGui recognises control points much better, something to do with contrast, and I think they also stitch quicker, but I've never done an accurate timing test.

Stitching panoramas

PTGui is my tool of choice for panorama stitching. I used to use Realviz Stitcher as it was very easy to use with great results every time. I tried PTGui a few times for interest, but the interface and learning curve were overwhelming, so I stuck with Stitcher. However Autodesk have failed to innovate with Stitcher and needing more advanced output options I was recently forced to review available software. PTGui still seemed very complex, but with supportive words on the forums and from Joost, the writer of the software, I persevered, and it clicked. Suddenly I got the awesome power of the software, and appreciated the great output quality. It probably still requires a bit more effort, and more frequent intervention, than Stitcher, but I have more control, especially when things go wrong and I need to fix them.

I've setup a template for my Nikon 16mm fisheye lens that retains all the most commonly used settings, so after adding nine HDR files I apply the template and run the align images routine. The zenith (upward) shot will often need manually aligning or control points adding as it will often have a lack of features other than soft clouds that have probably moved and changed shape.

The panorama editor can then be used to change the centre point for the equirectangular projection and to straighten the horizon if it needs it. Mine usually do as I think the D700 weight slightly twists the panoramic head causing the horizon in the individual shots to be a little wonky.

The output options are then set to 10000 x 5000px and again for Radiance HDR format. I could output larger, but this is plenty pixels for my prints and I will downsize for the virtual tours anyway.

If I am stitching a single panorama I will do it then, usually takes about 2-3 minutes (on a i7 MacBook Pro). If there are a few or a lot then I will send them to the batch stitcher and leave that running while I do other things.

HDR 2

Photomatix Pro is my HDR toning tool of choice. I like a fairly normal looking HDR image so the settings are turned right down, but season to suit your own taste. Don't forget to tick the 360° image checkbox so that the toning will be uniform round the edges for seamless transitions in the virtual tour. If I have a lot of images to tone for a single job/client where I need a consistent look then I will use the batch process in Photomatix Pro to tone all the HDR files at once and output a folder of TIFF files.

Post processing

The Photoshop part is probably the worst part of the process. I'll examine the resulting 200MB TIFF file at 100% to check for stitching artefacts. It may be possible to correct some in Photoshop using the clone/patch/heal/warp/content aware fill tools, or it may be necessary to reopen the PTS file in PTGui and change control points then re-output the file. Some images just won't stitch perfectly, normally when I have screwed up the initial image capture. Sometimes the wind vibrates the camera on the relatively wobbly head, sometimes the location is difficult such as up trees, sometimes things like clouds or large vehicles have changed/moved, sometimes I haven't aligned the nadir shot accurately enough and a parallax error results.

In cases like this there are a few options, but only try these after you have read the excellent faqs on the PTGui site, in particular the faq on aligning nadir shots.

In PTGui:

• Select the viewpoint checkbox in the optimiser.
• Try only using a few control points on the awkward image and mask out the rest that isn't needed.
• Import the awkward image multiple times and align/mask different parts of it to recreate the desired image. This is a last ditch way and will require much Photoshop work to tidy up, but it can work, sometimes.

In Photoshop:

• Be very careful with corrections or localised changes near the edges if you want smooth seams in the virtual tour.
• The patch or content aware fill tool is awesome for filling in black areas if you had too much mask or there are blending artefacts.
• The clone tool needs to be used with care. I often set it to an opacity of 50% and gently soften errors rather than try and create totally new pixels.
• Sharpening should also be done with care as when the image is downsized and tiled in the virtual tour software any over sharpening will quickly show. Better a soft image or sharpen print and vr tour images seperately. I use Photokit Sharpener on middle settings eg Edge Sharpen 2.
• Really awkward blends can be corrected by selecting the relevant area, creating a new layer with it, then using the warp transform tool to pull it into the right shape. The eraser tool (with a soft brush) and a curve layer (maybe with a brushed on mask) can then be used to blend the new layer in.

Once I've got the panorama how I want it, then I will save the 10000 x 5000px as a 16bit TIFF for print purposes, downsize it to 6000 x 3000px (which I find to be a good compromise between detail in the virtual tour and download speed), change the colour profile to sRGB and save an 8bit high quality JPG. (On the subject of colour profiles, when I open the TIFF for the first time in Photoshop I will manually assign it the AdobeRGB profile. I usually work in ProPhoto and the exported TIFFs are tagged with ProPhoto, the software is all set to work with ProPhoto, but for some reason when I open the toned TIFF in Photoshop it looks better if I assign the AdobeRGB profile. Maybe I'm doing something wrong somewhere, maybe one of the programs doesn't handle colour profiles correctly, maybe it is a side effect of HDR toning but it seems to work best like this for my eye.)

Other content for virtual tours

Supporting still photography will be prepared as necessary in Photoshop.

I use Garageband for audio post production and export high quality MP3 files, usually 2-3 per panorama in the tour, with a length of about 30seconds each. I try to do very little to the audio files as it is easy to mess them up if you don't know what you are doing. I'll usually give them a boost around 2k as this enhances voice and natural sounds like birdsong to the human ear and I'll slightly drop low frequencies. If there was a lot of wind noise I'll use a high pass filter to cut this out. In theory I could tune compression or set loudness for a better sounding sound, but I prefer to keep the soundspace more natural sounding.

Video will be edited in Apple Final Cut.

Constructing the virtual tour

I love this next part using Kolor Panotour Pro to pull all the content together. There's no real difficulties here, everything is intuitive and straightforward. If I have created a custom interface, for example to match a clients brand, then this will be setup. The hardest part is trying to create an intuitive and logical navigation from one panorama to the next if the locations are not visible by line of sight. The Google Maps plugin is a big help here.

I'll output the virtual tour to an appropriate directory, then the fun begins in a text editor. Visual effects like rain or snow, setting up the directional sound, adding and customising the Google Maps, custom pointers, mobile device specific functionality or restrictions (due to lack of Adobe Flash support) and many other things need to be coded into the XML files. It was a bit intimidating at first, but the documentation and the forums (both Kolor and krpano) were and are really helpful.

It's also worth noting at this point that some functionality, like hyperlinks, may not work locally due to the security settings for the Flash player. This can be fixed by opening the Flash player settings and adding the local folder where the tour(s) are as trusted location.

Hosting

Nearly there, I use the Transmit ftp program to upload to the server as it's really fast and reliable. 

Now sit back and enjoy your tour. To see some of my tours visit www.tours.mikemcfarlane.co.uk. (This is just a temporary site for my tours, a redesign is in the brain for discussion with my fab web designer Brian Coult of etheya.)

1 Comments

Leave a comment

* denotes required fields

Name: *

Email: *

URL:

Your comment: *

All comments are moderated before posting live. Basic XHTML is allowed (a href, strong, em, code). Off-topic or inappropriate comments will be edited or deleted. Email addresses will never be published.

< back to previous page