In the first part of my timelapse blog series I wrote about different types and techniques for longer and really long (seasonal) timelapse movies. In this article I want to describe a few specific techniques and tools that are useful (and often mandatory) to finalize those timelapses.
But first I'll lead into this timelapse article with my new video "Mountains and Clouds 2015" that I've shot over the course of about one year in the South Island of New Zealand on various trips.
One thing I wish I had done from the start is shoot all the frames for timelapses in full resolution and as RAW files. Always keep future use of your files in mind! Unless I got lucky and the scene was really well exposed, I couldn't correct as much as when grading from RAWs and because of the better resolution the final video will also look sharper. I take bets on Twitter (@tobiaswulff) on which scene was the one shot in RAW ;) .
The first step to processing timelapse RAW frames is to "develop" them from being a digital negative to a bitmap photo file. When exporting the image it is important to choose a format that will retain the full bit depth of the original RAW file: options are TIFF or 16-bit PNG. JPEGs should not be used since they only store 8-bit of color depth which will not fare well in color correction and grading later.
There is another way, though: keeping the files in their RAW format and using a video editing software that can deal with RAW footage, such as Davinci Resolve 12. For Resolve to ingest the RAW files they have to be converted to DNG. My photo management software of choice digikam has an in-built DNG converter and I believe so does Apple Aperture, Adobe Bridge and/or Lightroom. While working with a RAW video is pretty neat, because this is not a recognized format from a video camera (such as RED) the possibilities to adjust the image are limited and a proper RAW photo program such as Darktable or Lightroom is much better suited for the job. Nevertheless, you are importing 10+ bit image data into the editing/color grading software which gives much, much more room for colour and exposure adjustments.
As described earlier, Resolve can ingest DNG RAW files so it is possible to do editing and color grading with the source material. For the timelapse video posted at the beginning of this article, however, I compiled each sequence into a video first, which can be done using one of the following two Linux programs: Blender or ffmpeg (CLI tool). This use of "baked" videos will make the edit smoother because the program doesn't have to deal with as much data.
ffmpeg is a command-line tool, so it is easy to batch-process or automate converting image file sequences into video files. ffmpeg supports all the usual video file formats and containers, including ProRes which is a 10 to 12-bit 4:2:2 or 4:4:4 codec and the preferred format for Resolve. However, I found that the picture didn't quite come out the way I wanted, in particular darker areas got too dark so for example stars in night time sequences (like the one at the end of my video) almost disappeared. I'm sure this can be adjusted using the codec settings but for now I have turned to Blender.
Blender is first and foremost a 3D modelling and animation program. However, in recent years it also became a more and more powerful video editor and VFX pipeline, and it can be used to turn any bitmap sequence into various video formats. Once an image sequence has been imported, it can be modified (scaled, rotated, color corrected, composited with a 3D scene, etc) using nodes as shown in this screenshot:
For exporting I chose AVI RAW since it gave the best quality and could be converted to ProRes for Resolve, again without any loss in quality. It might be possible to export directly to ProRes or to use ffmpeg under the hood but I haven't explored the export and encoder settings too deeply yet.
Long Timelapse Processing Techniques
The biggest problems with long-term timelapses in the outdoors (i.e. outside a controlled environment) are changing lighting conditions and that it's basically impossible to get the camera set up 100% exactly the same way every time: the tripod will be positioned slightly differently, a zoom lens will make the focal length setting inaccurate, pointing the camera at the same spot will still be a millimeter or two off ... Luckily, both issues can be dealt with fairly successfully in software as described below.
Stabilizing and Aligning Photos
In order to make the transition from one frame to the next as smoothly looking as possible, non-moving objects in each frame really shouldn't move or jump around. Therefore, it is necessary to either align all the photos before they are compiled into a sequence (or video), or to stabilize the final video. There are at least three very different ways to achieving this. These different approaches can differ greatly in terms of time and effort, and quality of the end result.
1) Align photos automatically using Hugin. Hugin is an HDR and panorama toolkit but it can also be used to align a sequence of photos without exposure bracketing or stitching them into a panorama. There are several algorithms to choose from when aligning photos (I usually use "Position(y, p, r)" but its results are not perfect). The algorithm will look at all the photos that are next to each other in the sequence and find common control points in the picture that it uses to align (translate, rotate and scale) them. Control points can also be manually added, removed and shifted to improve the alignment. In terms of speed this is the easiest and fastest approach. I usually roughly follow this tutorial - something you will need because it's a complex piece of software!
2) Align photos by hand in GIMP (or Photoshop): there are various plugins for those image editing programs that allow a user to specify two common control points in two pictures and the plugin will then do the alignment. The results are near-perfect but it will take a long time because you have to do it for each frame in your sequence (what's that - 60, 90, 200?). A professional suite like Adobe's probably contains automatic tools similar to Hugin as well.
3) Stabilize the final video: all professional editing and/or VFX programs such as Resolve, Hitfilm, After Effects and also Blender have built-in stabilizers. I haven't tested this method yet but because they work on a frame-by-frame basis they should be able to stabilize the footage very well. However, as described in the next step, I like to blend (or blur) my frames which will definitely make stabilizing the video more inaccurate so doing it before the video is compiled seems more robust to me.
As shown in some of the examples from other photographers in the first timelapse article, we often simply blend images or videos together to make for a smoother (and longer) final product. This can either be done between sequences (say you show a locked down timelapse of autumn, then blend it into another timelapse of the same spot in winter) or between every single frame. Using the free toolkit ImageMagick, this can be done with one command:
convert frame_a.jpg frame_b.jpg -evaluate-sequence mean frame_ab.jpg
Ideally, your original source files would be organized to have odd sequence numbers (frame001, frame003, ...) and the generated blended images will fill the gaps (frame002, frame004, ...). This way you'll end up with all the frames for the timelapse video, now much smoother because even though lighting conditions and your camera setup change dramatically between each frame, there is now a frame in between that combines both conditions and makes it look much more pleasant on the eyes.
As you can see, there is quite a difference between the blended and the original image sequence - apart from the speed that it loops at of course since one has twice the frame-count of the other. Note that this is not a perfect alignment and also that I haven't done any RAW processing yet (these are simply out-of-camera JPEGs) so for a final product I would first process all the frames so that they are similar in exposure and saturation. Left is original (rough and unpleasant), right is blended (smoother):
Please head over to Google+ or Twitter @tobiaswulff (see links on top of the page) to discuss this article or any of my photography and videography work. My Flickr, 500px and Vimeo pages also provide some space to leave comments and keep up to date with my portfolio. Lastly, if you want to get updates on future blog posts, please subscribe to my RSS feed. I plan to publish a new article every Wednesday.