Outdoor Photography and Videography

Tobi Wulff Photography

Home / Tags / photography / Essays

Challenges Adjusting Time in JPGs and RAWs

I recently found myself in a situation where I had to adjust the date and time on all my photos from overseas, JPEGs and RAWs. Lesson learned: it is much easier to remember (if you do) to change the setting on the camera when you are switching time zones.

JPEGs

It is fairly easy to change EXIF and IPTC metadata in JPEGs because pretty much all the tools support it. Apart from just writing data directly, most of the tools (and luckily there are many options) I looked at also allow for automatic and intelligent data/time adjustments, so you only have to specify the offset in minutes or hours or whatever unit you require, and it will set the date and time accordingly. In the end this means the choice of the specific program comes down to personal preference. In Linux, there are several options, both for the CLI and as a GUI.

In digikam, the time adjustment can be found in the batch processing editor. To get there, select the photos you want to adjust, then hit B. You can select the individual destinations for the adjust times and I usually go with all the EXIF tags and the digikam timestamp (IPTC wasn't set when the files came out of camera). After the adjustments have been made to the files, it is important to re-read the photos back into digikam. To do this, select the photos again, then go to the menu Item and click "Reread Metadata".

On the CLI, the job is much easier in my opinion (as is often the case). To get a console for the album you want to edit, right-click on the album in the album view (sorry, this technique can't work when you want to edit photos based on tags or other filters; in that case you have to use the GUI method described above) and select "Open in Terminal". Now we can use (if installed) several programs to fix the date/time:

- exiftool: Does not have a date/time adjust option so for JPEGs I would not use it
- exiv2: Can read and write all the tags in JPEGs (and other formats, but not all RAW formats, see below) and has a handy date/time adjust function: "exiv2 ad -a -10 *.JPG" will subtract 10 hours from the EXIF timestamps. It can also be used to rename the files according to the timestamp ("exiv2 mv") but I like to use digikam for that (it can make filenames unique automatically if necessary).
- jhead: Functionality around timestamps and renaming is similar to exiv2 so it comes down to personal taste and specific use cases: "jhead -ta-10:00 *.JPG" will subtract 10 hours.

RAWs

This is were things get a bit trickier and depending on your camera's RAW format some of the programs will not work, e.g. exiv2 supports ORF but not RW2, and the GUI alternatives (digikam or UFRaw) didn't contain any options to write arbitrary metadata. exiv2 can work on some formats as described above (which is nice because it is the shortest and simplest command) but failed to write RW2 (Panasonic). What did work was exiftool. One slight quirk is that while exiftool displays pretty field names when you print all the metadata within a file (no arugments, just "exiftool file.RW2"), it requires the arguments for time adjustment to be the technical, compressed names of all the individual fields that you want to write, so: exiftool -"ModifyDate"-=10 -"DateTimeOriginal"-=10 -"CreateDate"-=10 *.RW2

I hope someone else who is on the search for the right tool to adjust their photos' metadata will find this information useful. I'll keep it as a reference for the future because I'm sure I will forget to set my camera to the right time zone again.

Please use the comment section below or head over to Google+ or Twitter @tobiaswulff to discuss this article or any of my photography and videography work. My Flickr, 500px and Vimeo pages also provide some space to leave comments and keep up to date with my portfolio. Lastly, if you want to get updates on future blog posts, please subscribe to my RSS feed. I plan to publish a new article every Wednesday.

Best of 2015 Photography Portfolio

2016 has been my second year of picking up photography. I've done a Best of Portfolio for last year as well, just never published it, but this year I decided to write a quick blurb about each picture and make my progress over time public.

The Process

I got the idea from the Martin Bailey Photography Podcast. Martin talks about how important it is to develop the skills of narrowing all your photos and great moments of the year down to just the very best 10 pictures. In the end I couldn't get below 12 and figured it's a good number because there are 12 months (my photos are not strictly by month though). There is also a bonus photo which has been my wallpaper for over a year and I still love it - both on my desktop and on my phone. I've divided the 12 best photos up into three panels and will talk very briefly about each photo, starting at the top left, going around clockwise.

12 Best Photos of 2015

Rolling hills: shot in central Otago, NZ, on the Pisa range near sunset. Probably my strongest light-and-shadow photo so far which is why I like it so much. The composition could probably be improved but the light was changing fast and this one was the best of the lot. The moon adds a really nice central accent.
Butterfly: Shot at Singapore airport, again at sunset. Since the butterfly garden is in an airport with buildings on 3 sides of it, this was a very lucky moment and the light was gone just 5 minutes later. There are many butterflies in the garden and they constantly feed on fruit nectar and flowers so one doesn't have to wait long to get a shot with a beautiful insect like that. However, it was the only spot in the gardens at that time that received the golden sunlight onto the nice contrasting white flowers.
Pine tree: This is a shot from the Black Forest in Germany. I was walking through swampy marshlands (on wooden boardwalks) and noticed that the trees had those pollen containers (Biologists please send me the correct terminology) primed and ready to go at the slightest touch. So I shook the twig and took a rapid burst of photographs as the branch swayed back and worth and shook out the cloud of pollen. This photo was the best of the lot where framing, sharpness, and the swirl in the pollen cloud worked out best.
Ominous bridge: This is taken crossing the Top Butler river on the West Coast of NZ. When I took the picture I thought it just looked nice because a swingbridge high over a wild torrent always looks good. But at home I noticed how ominous and dark everything looked and after fiddling with the sliders in Darktable for a while I came up with this photo that (in my opinion) captures the harsh and mysterious conditions when hiking on the rough West Coast really well. Unfortunately, it comes out quite dark on some monitors and so far I haven't decided if I should just up the exposure or work on the darker bits individually which would ruin some of the high contrast look.

Beach: another sunset (must be a theme among photographers) taken in Sumner, Christchurch, NZ. I like there is so much activity on the beach: people walking, surfing, paragliding, birds flying - and you can see the spray from the ocean really well.
Power lines: This one is taken among some wheat fields in Germany while out on a bike ride. That part of Germany consists of continuous rolling hills yet in this photo it's got a neat "Great Plains" look. I also like the colours and the very limited colour palette.
Snowy ridge: Back in NZ in the middle of winter. As the photographer I know exactly what the scene looked like in real life but I think the photograph successfully plays some tricks on the eye in terms of perspective. Because we happened to come down the steep part of this mountain at just the right time, less than an hour before sunset I think, we got this really string shadow line on the ridge. The hiker sort of (but not completely) gives the scene a sense of scale.
Tops hut: Not a particularly hard or lucky photo but it's got all the elements of a picture-perfect NZ backcountry scene: some snow, clouds, a cozy hut, and just a very pleasant to look at mix of colours.

Waterfall: I'm not a huge practitioner of blurred water and waterfall photography but when you are out in the mountains you do spot a lot of pretty looking waterfalls. This one seemed to work best in black & white because it's got a lot of texture but not really any colour that would be missed (the water wasn't glacier blue or anything like that). My only complaint about it is that the rocks on the right are a bit too shiny. I find it quite tricky to balance my circular polarizer effect when doing long-exposure of water because you want to take the glare of rocks but the agitated water actually looks better with the reflections left in.
Sand columns: Another one where it's nearly impossible to get a good sense of scale. These are actually very small pebbles and other debris that prevent the river sand on a bank from getting eroded by rainfall. It's quite a unique photo and perspective which is why I chose it for my Best of collection.
Swirly clouds sunset: Taken from one of the Canterbury (NZ) foothills at sunset while low clouds got blown around our mountain from the West. It was actually a really strong wind but we camped behind a ridge line where we had this great view just above the tree line.
Dragonfly: This was the first trip after buying the Olympus 45 mm prime lens for MFT and I was blown away but its detail and sharpness. This scene might be a little bit busy but it's a neat scene with a clear subject in its environment with some really nice colours and detail. A fairly lucky shot for sure.

Bonus

As promised, here is the link to the wallpaper in high resolution. This is one of my all time favourite photos because I love the perspective, the colours, and even the (yes, gimmicky) diorama effect. I hope someone else will also enjoy it as their wallpaper. If you want to redistribute the wallpaper, please link back to this blog post or to the photo in my gallery or on Flickr.

Please use the comment section below or head over to Google+ or Twitter @tobiaswulff to discuss this article or any of my photography and videography work. My Flickr, 500px and Vimeo pages also provide some space to leave comments and keep up to date with my portfolio. Lastly, if you want to get updates on future blog posts, please subscribe to my RSS feed. I plan to publish a new article every Wednesday.

RAW Timelapse Workflow with Darktable and Davinci Resolve

I shot a new timelapse in the mountains, this time exclusively recording all the frames in full resolution and RAW (unlike my previous outdoor timelapse). It was recorded with the Olympus OM-D E-M1 (de) and Olympus 12-40mm F2.8 PRO (de) lens.

Out in the field

Basic outdoor timelapse 101: all manual settings, that is ISO, white balance, aperture, shutter speed. White balance was obviously daylight and I kept the ISO at its minimum (200). Shutter speed should be set to something "video-like". Video and film cameras usually use something called a 180 degree shutter which essentially means that the shutter speed is 1/(2 x frame rate). So for a 24 fps video that means 1/48 or (because photo cameras usually don't offer this setting) 1/50. Anything faster than that runs the risk of making the timelapse feel jittery and too sharp. For fast movements, like people or clouds, I like to go even slower and aim for something like 1/20 - 1/40. This gives the video a more dreamy and pleasing look.

I record every frame in RAW. I like to store JPEGs as well so I can generate a quick timelapse when I get home without having to go through the RAW workflow (described below) first.

To do the actual timelapse recording, there are several option depending on your circumstances and your equipment:

  • Using the camera's in-built timelapse function: most compact solution and works well on the E-M1 except when you want faster than 1 second intervals;
  • using a remote shutter release or remote timer: works great but you have to dial in the intervals using the anti-shock functionality and it's an extra cable flapping in the wind;
  • a slider or panning head triggering the camera: whenever the E-M1 sits on the panning head (see next section), it will receive it's shutter releases from the Genie. The result: accurate intervals perfectly timed with the stops between motions of the moving parts of the timelapse setup.

For filters I often use a graduated ND filter to make the bright sky and the darker ground a bit more even. This is particularly important at sunrise and sunset because the ground will be really dark. I also have a circular polarizer that lives on my lens 95% of the time: vegetation looks more lush, colours more vibrant, and annoying reflections of leaves or glaring surfaces disappear. It can also cut through a lot of haze and mist on a more cloudy day. Time in Pixels just released an excellent article about filters for video with many visual examples.

Getting moving

I've written about my DIY slider before and it is actually undergoing some major upgrades right now to make it more usable and flexible. However, I don't usually take it very far because it is heavy and big. In order to have something that always fits in even the smallest bag, is compact and rugged (not weather-proof, though) and "just works", I got myself a Genie Mini which is actually being developed here in NZ. It's controlled from a smartphone via Bluetooth so setting it up takes a few minutes since my phone is usually off when I'm in the outdoors (no reception anyway) but it's very intuitive and flexible (watch the videos on their website). All the shots in the video at the top of the page that have some side-to-side movement are done with the Genie Mini.

RAW workflow

The out of camera JPEGs are alright but (especially for landscapes) don't look nearly as good as they could when I develop my own final images from RAW: better colours, more dynamic range, more wiggle room in the highlights (and some in the shadows). This is particularly important when photographing sunsets, sunrises, or rapidly changing lighting conditions because the exposure can be adjusted so much in post. I load all my RAWs from one scene into Darktable, then do all my adjustments on one of them (shadows, highlights, general exposure, Velvia/saturation filter, contrast, noise reduction, but no cropping - I can always do that later when editing the video). Then, I copy the settings to all the other RAWs and export everything to bitmap files with a high bit rate, such as 16-bit PNG or TIFF. In theory, one could also make fine adjustments to individual frames at this stage.

Editing

The last step is to edit the photos into a timelapse video and maybe add some music and sound effects. I mainly use Davinci Resolve for editing because it also has colour grading built in but the colours should already be fairly correct and good looking from the last step. Davinci can directly import image sequences (i.e. individual files) and display them as video clips.

Please use the comment section below or head over to Google+ or Twitter @tobiaswulff to discuss this article or any of my photography and videography work. My Flickr, 500px and Vimeo pages also provide some space to leave comments and keep up to date with my portfolio. Lastly, if you want to get updates on future blog posts, please subscribe to my RSS feed. I plan to publish a new article every Wednesday.

My (rather late) Guide to some Olympus OM-D E-M1 Features and Quirks

During my first big outdoor trip with the Olympus OM-D E-M1 (de) I found a few new ("hidden") features and quirks that I want to share.

HDR modes

In the dense Fiordland forests I experimented with the E-M1's HDR modes and found that, unfortunately, the camera itself doesn't really produce any useful, good looking HDR images (JPEGs). However, whether HDR1/2 modes or one of the multi-frame +/- EV modes from the HDR menu are selected, the camera will also automatically engage the high-framerate interval shooting mode so that the shutter only has to be pressed down once to capture all the exposures at a (theoretical) speed of 10 fps. If you want to make use of this feature you shouldn't set the "H" framerate to anything lower than the maximum. Note that in the modes HDR1 and HDR2 you won't get the individual exposures as RAWs - you only end up with one HDR image produced by the camera.

As described in the next section, the last used HDR mode cannot be toggled on and off with a press of the HDR button on the left-hand side of the camera the way you can toggle the other bracketing modes. Therefore, I had to sacrifice another Fn button and chose the one on the Olympus PRO lens since this is the only lens I regularly use for this kind of photography. My smaller lenses that lack the L button are typically used for portraits, wildlife or timelapses.

Quickly toggle bracketing

As opposed to the HDR functionality (which annoyingly has to be switched on and off with a separate Fn button), bracketing can be toggled with a quick press on the HDR button when the button lever is in position 2 and "lever2+left buttons" (the last item in the gear B menu) is enabled. The really cool thing is that a long press (more than 1 second) of the same button will take you to the bracketing menu where you can select between different modes (ISO bracketing which uses different ISO levels, AE bracketing which is similar but in addition varies the shutter speed as well, ART bracketing among others, and the new focus bracketing). Why can't toggling HDR work exactly the same way, it's basically the same thing??

ISO bracketing

I think this is my new favourite HDR mode for handheld shooting because it takes one exposure but delivers 3 different RAWs at +/- 1 EV (e.g. at ISOs LOW, 200 and 400). For many situations, that is enough coverage unless you're dealing with extremely high-contrast scenes. However, this also means that there is only one exposure and the camera delivers 3 different files with different light amplification so it is not the same as actually producing different exposures with longer or slower shutter speeds. Anything other than the base ISO of the camera (200 on the E-M1) will introduce noise and/or limit colours and dynamic range. Here are some interesting, yet slightly confusing discussions around this topic on Photography StackExchange.

One more note on AE bracketing which is similar to ISO bracketing because it does vary the ISO but also takes multiple true exposures at different shutter speeds: contrary to the E-M1's HDR modes, high-framerate interval shooting does not engage automatically so you'll have to set this separately if you don't want to press the shutter multiple times.

Interval Quirks

This, to me, is clearly a bug (although I can see why Olympus did it) and destroys the expectations of precision one might expect from the built-in interval timers used for timelapses and multiple self-timer exposures. Which features exactly do I mean? First, there is a self-timer mode where the user can enter a custom value (in seconds) for the camera to wait, and how many pictures to take at which intervals (also in seconds). The other feature is the timelapse mode where the user can set the interval between shots. Unfortunately, both modes suffer from one common problem: the interval timer only starts once the image (RAW, JPEG, or both) has been written to the SD card. Not only does this introduce an additional delay, it's also inconsistent as writing to the card does not always take the same amount of time.

I can see why Olympus did this, at least for continuous shooting like it is done in the timelapse mode: if it takes a while to write all data to the card, shooting more photos during that time will result in the internal buffer slowly filling up until the camera won't be able to take any more pictures. However, I didn't actually find this to be a problem, and here is both how I tested it and how I work around it when I want to shoot timelapses with interval times faster than the normally achievable 2-3 seconds.

You'll have to use a remote trigger than plugs into the camera. These can be bought for very little money on sites like Amazon. Then, enable the H framerate anti-shock mode (the one with the rhombus) and set your desired interval time (which can be fractions of a second) as the anti-shock delay time. Unfortunately, you'll have to go into the main menu to do this. An option to access this mode quicker is to save it as a MySet and make it accessible through a Fn button or a position on the mode dial. However, to change the interval time you'll still have to do a lot of navigating in the gear menus. Once the shutter remote is pushed down and locked, your camera will happily shoot timelapses at interval times below 1 second. This has worked fine with a reasonably fast and large SD card like the Transcend 64 GB UHS-3 Flash Memory Card (de) down to 1/2 and 1/4 s for many hundreds of shots.

Quickly access focus peaking settings

While the viewfinder or LCD is displaying focus peaking (either because assist is enabled and you've turned the focus ring, or because you've pushed a Fn button that has been assigned to peaking), a press of the INFO button will pop up a small menu that lets you adjust peaking colour and strength without having to trawl through the main menu. I wouldn't be surprised if this quick-access feature works for other things as well but I haven't noticed any yet.

Please use the comment section below or head over to Google+ or Twitter @tobiaswulff to discuss this article or any of my photography and videography work. My Flickr, 500px and Vimeo pages also provide some space to leave comments and keep up to date with my portfolio. Lastly, if you want to get updates on future blog posts, please subscribe to my RSS feed. I plan to publish a new article every Wednesday.

New Olympus OM-D Firmware (4.0)

Olympus has just released firmware version 4.0 for the OM-D E-M1 (de) and version 2.0 for the OM-D E-M5 Mark II (de). I haven't seen an announcement on their website yet but the updater software can already download and install both camera and lens updates. One tip that took me a few tries to figure out: when connecting the camera to the computer, select "Storage" from the camera screen, otherwise the Olympus updater software won't be able to see the device.

It's a free upgrade and brings a lot of exciting new features to both of those cameras. Unfortunately for me as an E-M1-only owner, some of those are for the E-M5II only. Before upgrading keep in mind that the upgrade process will wipe all your settings from the camera!

Update: After having played with the new firmware for a day now, I've updated the sections below with some observations and new discoveries (in bold). I will also try to put a sort of E-M1 guide page together with useful settings and little quirks.

Electronic Shutter

It looks like finally the E-M1 is getting an electronic shutter mode which will be great for situations where the loud mechanical shutter is not appropriate. I don't know who made that decision but the heart symbol for the silent shutter (next to the familiar rhombus for the anti-shock mode) is kinda cute. However, be aware of the limitations: rolling shutter (if panning while taking a shot) is worse and flickering lighting such as florescent lights or projector lamps can make photos shot with electronic shutter nearly unusable.

Update: The electronic shutter setting is in the second camera menu (menu button, then on the second page down). Select Anti-Shock/Silent, then pick a silent delay (0 seconds for no delay but no mechanical shutter), then half-press the shutter button to go back to photo shooting mode and select drive mode Single Silent (heart). This is a lot of setup but once it's done you can quickly switch between mechanical and silent shutter using the drive/HDR button on the top left of the camera body. I'm looking forward to using the electronic shutter in my timelapses to go a bit easier on the mechanical shutter mechanism.

Focus Stacking and Bracketing

The biggest feature additions are the new modes for focus stacking and bracketing. Both do essentially the same thing, that is taking a whole bunch of pictures with the same exposure settings but slightly different focus points. This is particularly useful in macro photography where the depth of field usually is very small. It works with compatible auto-focus lenses such as Olympus's M.ZUIKO PRO lens series (de) by automatically shifting the focus point after each photo.

In focus bracketing you will end up with all of those photos and you can post-process them however you like (similar to Panasonic's new Lytro-like focus-later technology). However, when focus stacking is selected, the camera will do all the magic inside and produce one photo out of 8 individual ones, all with slightly different focus points. This should result in a macro shot where the whole subject is in focus.

Update: It works - as long as nothing in the frame moves. Focus stacking only works with the electronic shutter and it's so quick that it can easily be done hand-held. I don't have a dedicated macro lens so I couldn't really shoot any meaningful examples but it turns long focal length f2.8 into "everything is in focus" which is pretty cool. When focus stacking is selected, it also keeps all 5 individual files on the card so you can post-process them later. Some of the little but great improvements that I haven't mentioned in the original article are:

  • the menu system remembers where you left off last time so you can quickly play with settings without having to go through pages and pages to find it again,
  • not only are there more colours for focus peaking (red or yellow is so much better than black or white!) but the intensity can also be changed,
  • histogram, level gauge and over/under exposure indicators can now be displayed at the same time: this is huge because previously I had to jump through all the different options with the Info button to get my camera level, then get the exposure right; you can selected two different custom modes to cycle through using the Info button and selected which parts you want on each screen - the settings are under Menu - Gear D - Info Settings - LV-Info.

Simulated Optical Viewfinder

The S-OVF mode disables some of the "live view" features in the viewfinder, such as boosting the light levels. This means that it won't assist the photographer in bad lighting conditions but on the other hand, you'll see exactly what a true optical viewfinder would see, that is it depends entirely on the currently selected aperture on your lens. White-balance compensation is also turned off for a "truer" image. I think most of the time a appreciate the assisting features of the EVF and I use the histogram to accurately determine whether my exposure is good, so I can't see myself using this mode too much but it's still a free new feature that could come in very handy in certain situations (e.g. when not using the histogram for some reason).

Update: I probably didn't get it fully right in the paragraph above because I didn't know how optical viewfinders used to work. When S-OVF is selected, exposure compensation is completely disabled so you see pretty much exactly what your eye would see outside the camera. If you want to judge exposure you have to go by the metering number - the histogram doesn't help at all because it only turns what's currently in the EVF into a graph which means it won't change as you alter ISO, aperture or shutter speed (because the OVF doesn't change). To see the photo as it will turn out when you press the shutter you have to do two things: 1) go back into normal EVF mode, and 2) turn off Live View Boost under Menu - Gear D - second page. This is my preferred setting because it gives the least unwanted surprises, and I've mapped the S-OVF to Fn2 so I can change to it if I want a more realistic view.

Video

There are a few upgrades that apply to video only, such as a new picture profile (E-M5II only) and synchronised recording with an Olympus audio recorder. I don't own either so sadly, video won't receive any useful improvements (I was really hoping for focus peaking during recording but at least they are adding more colours to choose from for the outlines). Another minor addition is the slate tone generator which I assume can be assigned to a button. Using this probably looks more professional than snipping your fingers in front of the camera when recording audio with an external recorder.

For the E-M1 there is another good and bad update for video: a new framerate. It's great that Olympus has added 24p but it is also still missing 60p to become a useful sports and documentary video camera (which otherwise the rugged and weatherproof body and the in-body stabilisation makes it perfectly suited for). I don't quite understand why Olympus is adding features like timecode (and those awful movie effects) first before improving on the essentials.

Lenses

The PRO lenses will also receive a new firmware which will add support for disabling the MF clutch. I only usually use the clutch to switch to a true manual focus while shooting video. When shooting photos I have previously pressed my back-button-focus button just to find it didn't do anything because the clutch was still on manual focus. This update might help in those situations.

So overall it's a great update and we should keep in mind that not all manufacturers release such improvements for free. However, there are still features missing that I'm sure the camera would be capable of handling. They might arrive in the future with another free upgrade despite the E-M1 Mark II probably not being too far away anymore. I'm optimistic because in this upgrade Olympus has added features to the E-M1 that at first looked like they were for the E-M5II only.

Please head over to Google+ or Twitter @tobiaswulff (see links on top of the page) to discuss this article or any of my photography and videography work. My Flickr, 500px and Vimeo pages also provide some space to leave comments and keep up to date with my portfolio. Lastly, if you want to get updates on future blog posts, please subscribe to my RSS feed. I plan to publish a new article every Wednesday.

Backup strategy for photos and videos on Linux

This is a continuation of my previous Workflow article. In the workflow article I've linked to Chase Jarvis' excellent videos and blog posts about working with digital media and keeping it safe. I highly recommend to watch them if you are interested in a more in-depth look at workflow and backups.

When I started getting into photography beyond simple point-and-shoot or cellphone snapshots, I realised that leaving all my valuable photos in one location without a decent backup plan would be too risky. I've never had a hard drive suddenly die on me but I'm sure that one day it will happen - suddenly or, maybe even worse, gradually. Not that my old snapshots weren't valuable but RAW photography suddenly involved much more data and also more high-quality prints, competitions, and a growing portfolio. Of course, I had done backups to an external hard drive and this is often as far as most people take it. This prevents data loss due to a sudden one-disk failure. However, even my older photos which were taken during some of the most memorable and important periods of my life were always just backed up to that other drive in the same room. A violent power surge while it is plugged in, a fire, water damage, or theft could easily render all those important files inaccessible.

Requirements

The requirements were easy to jot down:

  • Full system protection against a one-disk failure so I don't get stopped dead in the tracks if something happens to my hardware,
  • off-site backups for the most important data so that even my house being swallowed off the surface of the earth doesn't lead to significant data loss,
  • ability to take snapshots of folders or partitions so that I can experiment with files without the risk of corrupting or losing any data (this also helps with taking backups),
  • protection against data corruption and bit flipping which does happen.

Redundancy

Multi-terrabyte hard disks are become pretty affordable and if your priced digital possessions are shot with expensive cameras, there's no good reason not to invest another $100-200 to make your data and ideally your whole system fully redundant. A RAID1 mirrors all data on the disk so if one disk suddenly fails the other one can keep running. It is recommended to use disks from different batches - or even models - so that a manufacturing issue does not propagate across both drives. RAID1 can easily be done in software by the Linux kernel and there are no complicated algorithms that could lead to issues recovering data further down the road. It is as simple as: both disk have exactly the same information stored on them.

There are also a few other benefits of RAID1. First, while write speeds are slower, read speeds are twice as fast. Once the photo or video data has been offloaded from the memory cards, editing software can take advantage of that for a quicker and more fluid workflow. Additionally, when using a file system like btrfs, any data corruption on one disk can be repaired with data from the other disk. I'll talk about this further down.

For hardware I use two Western Digital Red 4TB NAS Hard Drives (de). WD has an excellent reputation and the Red drives are designed for workstations and Network Attached Storage (NAS) systems where disks can run many hours at a time (think heavy editing or transcoding) or even 24/7. For a great big-scale reliability study of current hard drives check out this Backblaze article about the drives in their data centre - personally, I'd stay away from Seagate.

Off-site backups

All your important data must be stored securely off-site. While it is luckily quite unlikely to fall victim to a house-destroying catastrophe or serious theft it is still possible, and it is the one point were complete data loss could happen when it is expected the least. I use two identical external drives and one always lives off-site in a safe location. I swap them out every one to two weeks depending on the flow of new photos, videos and editing files.

For my big backup disks I use the more affordable Western Digital Green 3TB Hard Drives (de) which are not designed to run for long periods of time so they shouldn't be used in workstations or servers. For backups they are ideal because they are only spun up every other week and it means cheaper storage: all of my data with room to spare for at least the next half year for under $100. To attach the backup disk to my computer I use a cheap and fast Sabrent USB 3.0 to SATA External Hard Drive Docking Station, then rsync to copy new and changed data from my workstation to the disk.

Since I don't really need any of my older, smaller drives for my workstation (if I run out of space it is easier to buy new big multi-TB disks rather than reuse old sub-TB ones and assemble them into RAID0s and RAID1s), I'm also planning to copy some of my finished projects and previous years of photography to those smaller drives and to keep them in yet another off-site location as an archive. My only concern is that after a year of not spinning them up and running a btrfs scrub, data corruption might start to get noticeable.

Data integrity

The Linux file system btrfs provides multiple features that come in very handy when keeping data safe and running backups. For starters, btrfs scrub start /media/data will start a check of all my multimedia files and fix any issues found. Btrfs keeps metadata (which includes checksums) and actual data separate so if there is any damage to the real data there is a chance it can be recovered from the metadata. Furthermore, in a RAID1 system the data can be restored from the other disk which shouldn't show the same (random) data corruption.

The next great feature is snapshots: btrfs subvolume snapshot photography photography-backup creates a snapshot of all my photos and keeps changes that happen from now on separate (through a mechanism called copy-on-write). So if I accidentally delete a file in my photography/ folder I can get it back from the backup folder, yet it doesn't use up any additional space on the drive if files stay the same. This can easily be automated using cronjobs to create and rotate snapshots on a daily or weekly basis. It is also handy to create a snapshot before copying files off to an external hard disk so that I can keep working on my photos and videos while the backup is running in the background.

Other, smaller backups

I also have rsnapshot running every hour to take backups of my XMP sidecar files that get created by Darktable, my RAW photo processing application. Since Darktable automatically saves any changes made to a photograph as they happen, it is quite possible to accidentally delete the whole editing history (the RAW file itself is never modified, though, by the way). However, I have to say that this hasn't happened to me at all in almost two years of using the program. Anyway, keeping the small XMP files available for up to a month is an easy, inexpensive and great way to keep my mind at peace.

Conclusion

Just today, Caleb Pike released a video on his workflow and backup strategy as well. While fairly different due to different tools and computer expertise, the main principles and requirements stay the same and I recommend watching his take on the topic, especially if you found mine too technical or Unix centric.

Please head over to Google+ or Twitter @tobiaswulff (see links on top of the page) to discuss this article or any of my photography and videography work. My Flickr and Vimeo pages also provide some space to leave comments and keep up to date with my portfolio. Lastly, if you want to get updates on future blog posts, please subscribe to my RSS feed. I plan to publish a new article every Wednesday.

Long-run timelapses across multiple seasons - Part 1

Some links to products in this blog post are Amazon Affiliate links that earn me a few cents or dollars if a reader buys any product on Amazon through this link. The price of the product does not increase so it is a free way to support this site by using the links provided. The main product link goes to Amazon.com and the "(de)" leads to Amazon.de.

This is the first part of a longer series of blog posts about timelapses. I have started planning for and taking long-run timelapses that span many weeks and months, and I want to talk about how these ideas and visions can be accomplished in a reasonably efficient workflow by an amateur photographer. I say "reasonably" because processing timelapses from RAW files and working on such long running sequences will always involve a lot of work.

Who, and why

Apart from an article on Photo Sentinel, there aren't many interesting articles or howtos available. I highly recommend reading the article if you're interested in timelapses because it showcases different techniques and links to some great videos in each category. However, it belongs to a company that sells specialised long-term timelapse equipment which does not really fit the kind of subjects I'm shooting. On Youtube and Vimeo there are only a couple of videos that portrait certain subjects in nature over the course of many months but there are some amazing and award-winning short videos and films that I will link to further down.

The most impressive executions of this sort of timelapse - and the aforementioned howto talks about it as well - are several features by the BBC such as The British Year and of course Planet Earth. The team which shot the timelapses for The British Year talk at length about planning, shooting, editing and various tips in a blog post. I highly recommend Chad's blog and all the content on his website as it is a wealth of timelapse stories, workflow tips, and kit reviews.

How-to

The obvious but most time extensive way to shoot a timelapse across multiple seasons is to take individual photos of the same subject under similar lighting conditions and from the same spot over a long, long time. Another technique is to take multiple "normal" timelapses, that is sequence of an hour or a few hours, and then blend them together such as in the Youtube video "4 Seasons 1 Tree". Unfortunately, the blending will be very obvious, and there also isn't much movement or change within the individual sequences themselves. On the other hand, there is no flickering due to abrupt changes in lighting or weather. This could be enhanced by doing some masking and selective blending to change some areas of the image before others which can also be seen in the video as the ground changes before the tree does.

The easiest way to accomplish a long running timelapse is to have a camera that can be left in a fixed spot and orientation. The photo above is actually a blend of two individual frames, one with different lighting and more leaves on the tree. It shows that blending and aligning photos on the computer can produce a very smooth result even if the original photos are totally unaligned and taken in completely different conditions. In amateur nature photography, it usually isn't an option to have an absolutely fixed camera spot because the locations are too exposed to the elements. Even in urban environments you wouldn't leave your camera or tripod anywhere except inside your own house or apartment - and then you wouldn't be able to take it somewhere else.

Therefore - unfortunately - we have to re-set up the camera and point it at the same spot every single time. This gets very complicated if movement of the camera is involved but even with a static shooting position there will be slight variations due to uneven ground, zoom lens variations (zooms are not "clicked" after all) and inaccuracies when pointing the camera at the subject. A very sturdy tripod is important but because I usually travel on foot or bike and also take my equipment on hikes into the mountains, I couldn't just go for the most sturdy one out there. So my tripod is the Manfrotto BeFree Compact Aluminum Travel Tripod which I love because it fits even into a normal day pack, yet it can extend to eye level and is reasonable rigid. However, pushing down on it will bend the legs in their joints so it is tricky to get it set up 100% exactly the same way every time.

So there will be variations in tripod position, tripod height, camera attitude and focal length. Luckily, those issues can be resolved almost completely in post-production and I will talk about methods and tools to align photos and blend them in the next blog post in this series. Apart from dedicated software and plugins for Lightroom there are also a bunch or free tools available that do a very good or even perfect job at the expense of a maybe not so polished user interface or some efficiency.

Ongoing work and ongoing articles

Something like the video "Fall" from NYC Central Park is probably the closest inspiration to what I am planning to achieve. I didn't know the video when I started my project. There is also a year-long timelapse from the Canadian Rocky Mountains which employs some really nice blending and obviously beautiful outdoor scenery.

As I shoot individual frames and sequences for my own long-run timelapse video, I will add more parts to this series talking about specific shooting tips and releasing some more snippets of the ongoing work. Towards the end I'm sure it will all become fairly editing and video post-production heavy.

Please head over to Google+ or Twitter @tobiaswulff (see links on top of the page) to discuss this article or any of my photography and videography work. My Flickr and Vimeo pages also provide some space to leave comments and keep up to date with my portfolio. Lastly, if you want to get updates on future blog posts, please subscribe to my RSS feed. I plan to publish a new article every Wednesday.

Peak Design Camera Clips and Straps for the Outdoors

Ever since I started using my first piece of Peak Design camera accessory, I have become a big fan, and it's getting better and more useful every time I take it with me to an event or on a trip outdoors. It keeps your hands free and your gear secure.

Obvious disclaimer that I'm not affiliated with Peak Design (I wish my blog was that successful) - I just love those products ever since I got them, and a few people have asked me about them so I decided to write it all up. Some links to products in this blog post are Amazon Affiliate links that earn me a few cents or dollars if a reader buys any product on Amazon through this link. The price of the product does not increase so it is a free way to support this site by using the links provided. The main product link goes to Amazon.com and the "(de)" leads to Amazon.de.


Peak Design started making the Capture Camera Clip (de) pictured above in 2010 and has been expanding their product range with new, exciting products ever since. This is a great article on SmugMug that goes over the idea, the history, the people and the Kickstarter campaign behind the clip. I used to have my old camera, an Olympus OM-D E-M5 with kit zoom lens, strapped onto the shoulder strap of my backpack using a Maxpedition Janus. This worked reasonably well as long as I didn't have to jump off boulders or duck under trees that had fallen over onto the track. Given the wrong angle or too much force, the camera and lens could fall out of the bungee cord strap that held it in place. For a small compact camera or a light camera with a fairly long lens I still think that this is a very good and cheap system.

I saw a friend use the Capture Camera clip and decided that it would fit my use case perfectly. It became a necessity when I upgraded to an Olympus OM-D E-M1 since it is bulkier and the zoom lens would no longer fit into the strap I had been using previously. One great feature is that the plate that attaches to the camera's tripod mount can slide into the clip in any 90 degree orientation so you can have the camera facing down, sideways or even up, for instance to change lenses. One thing I don't like so much is that you need a hex key to fasten and loosen the plate. It can be done by rotating the whole plate which is what I do most of the time but after a few times it gets to the skin on your fingers ...

I went for the standard version which is constructed from aluminium as well as "glass-reinforced nylon". So far it seems to be built to last a long time, although I can see how the Pro version that is all metal could have the edge after many years of abuse. The plate that screws onto the camera as well as the parts that hold the plate in place are mostly metal so I'm confident it will always hold the camera securely. The main weak points made out of nylon are the screw holes on either side which maybe could snap one day if I screw it on too tightly to a thick strap or belt. Furthermore, the Pro version comes with a plate that can act as a quick release plate for Arca-style and RC2 tripods.

The Capture clip works great on a backpack shoulder strap (sliding the camera in from above), on a belt at events (sliding the camera in sideways), or the shoulder strap of a messenger bag (works either from the top or the side).

My next purchases from Peak Design were the Cuff (de) wrist strap to secure my camera to my arm or my backpack via a carabiner, and the Anchor Links (de) to convert my original Olympus neck strap into a more useful detachable strap. Right away I have to say what's great about a lot (but not necessarily all) of their accessories is that you get quite a few spare parts: I've only attached one anchor link to the PD plate and one to the right-hand side of my camera, and now I've got 4 (!) anchor links left that can be used once the first two wear out. I'm not sure how fast this will happen but I'm sure it will last me quite a long time.

Another bonus of the clip/unclip system as opposed to a fixed neck strap or something like Blackrapid shoulder strap is that I can quickly change from wrist cuff to safety leash to shoulder strap. When I use my original E-M1 as a shoulder strap with anchor links on the bottom and right-hand side of the camera, I find it sits much more securely and comfortably than if I use a Blackrapid (clone) where the strap only goes through the bottom of the camera. The only disadvantage is that you loose the sliding action up and down the strap but I haven't found that to be a problem yet. I've also heard great things about the slide shoulder strap, however I'm quite happy with the original Olympus strap. From what I can see the Peak Design Slide offers much quicker length adjustments but I can't say that I needed to change the length of my neck or shoulder strap on the fly very much.

Now, they are diving into the realms of camera bags with the Everday Messenger Bag which has been designed together with New Zealands Mr. HDR Landscapes: Trey Ratcliff. While I'm in no need to purchase any more shoulder bags or backpacks at the moment, I'll keep an eye on their developments because I'm sure it will be exciting, functional and very well made.

Photo Management Workflow in Linux

Ever since I started taking RAW photographs and developing them on my computer, I put a lot of thought ("obsessing over") into how I want to organize my workflow and in particular how to structure my files.

Many photographers might start by simply dumping all their pictures into a folder for each trip or shoot, hopefully ordered by date. If your folder names do not start with YYYY-MM-DD you're going to have a hard time quickly finding and grabbing your photos with a file manager - on the other hand, photo management software can simply use the date and time in the EXIF data to sort and find digital assets. However, what can happen if you take that approach is that you lock yourself into one specific workflow with one specific application.

Apart from that, one folder sounds fine, after all, different files for different purposes have different file extensions: RAW as the original ("digital negative") that never gets modified, XMP sidecar files that store additional metadata and RAW processing steps, XCF/PSD work-in-progress editing files, and finally JPEGs. Nevertheless, I keep my JPEGs and my RAWs in different folders so that I can quickly copy all the JPEGs to an external device to show them to other people. Having to go through the export functionality of a photo management tool would complicate this process unnecessarily.

A great video to watch is Chase Jarvis' TECH blog post and video on his company's workflow, and if you've got some extra time to kill: Chase Jarvis LIVE Q&A on workflow which goes into much more useful detail (but also rambles on about less important stuff from time to time). There is a lot of solid advice from highly successful professionals in there but it can easily be applied to your personal needs by scaling it down a notch. After all, most of what they do still applies to one-man-bands and enthusiasts:

  • Use redundant hardware to prevent data loss due to technical failures
  • Backup regularly and off-site to prevent data loss due to theft or human error
  • Have a standardized workflow so that all your folders are organized consistently
  • Tag and rate to find photos once you've accumulated thousands

Requirements

Availability

I want to be able to quickly get to my photos and show them to someone or copy them to a USB flash drive without having to go through my photo management software. By having separate folders for each photo shoot sorted by date and within those separate folders for each file type ("digital negative", 'developed photos", "project files"), I can always use the CLI or a file manager to get exactly the files I want without reading their EXIF etc data.

Interoperability

Interoperability might not be very important if you know that you will always use one specific program and that you can rely on this program being available, up-to-date and meeting your needs for many years to come (we are talking potentially decades here). Personally, I wouldn't put that much trust into it, and while proprietary photography applications have a slightly better track record than something like MS Office, keep in mind that things might still change very suddenly and your favorite program might not meet your requirements anymore or work in a very different, non-backwards compatible way (like when FCPX came out).

My photo management program of choice, digikam, relies on my own file system structure of my albums, and displays them basically exactly like they are stored on disk. However, it can also browse all photos by date, tag, rating, etc. This way, I can quickly search and filter for specific criteria, or just browse my albums as they are stored in folders. For much, much more information on digikam I highly recommend the eBook digikam recipes which is easily worth the little money it costs if you're looking into using Linux for your photography workflow.

Backups

I don't want to rely solely on my photo management program to do the backups. This is another reason why I started this blog post and my workflow considerations with a sane and well organized file system structure: any backup program will be able to grab those files (all of them or a subset) and copy them somewhere else doing full, incremental and differential backups. Restoring them is also easier if the photo or photo shoot in question can be found quickly.

I will talk more about my backup strategy and how I've implemented it (including tools around rsync and btrfs) in a future blog post.

Personal Workflow

My personal directory structure which reflects most of the workflow:

  • It starts with a folder for the YEAR (e.g. 2014),
  • within each year I have a sub folder DATE_PROJECT/TRIP (e.g. 2015-09-30_Trip_Location),
  • if there are a lot of photos: DAY (full date) or other categorization (e.g. by camera, sub-project, etc)
  • Further sub folders based on workflow (see below)
  • File names are usually DATE_TIMESTAMP (with suffix _1, _2 etc if multiple photos have the same timestamp). I've also seen many people keep the camera's original file name but add date and time at the front. Personally, I don't see any point in keeping the camera's numbering scheme - it doesn't convey any useful information apart from avoiding duplicates if date/time are the same.
These are the sub folders that I use to organize the files within each shoot:
  • jpg: final JPEGs, often straight from camera and the ones developed by myself; I want all the final pictures in one folder so I can browse them easily
  • orf: my current camera's (Olympus) RAW files
  • liveworks: contains RAW files and their XMP sidecars

This covers 95% of my usual projects which are trips or events with lots of out-of-camera JPEGs and a few jewels I want to work on from RAW. These are the ones you see show up in my portfolio. I usually don't have HDR, panorama or other composite shots but if I need a place for them I would put them in a new folder in liveworks. JPEGs that get a final touch-up in GIMP also go into a new "edit" folder.

There is also a discussion to be had about whether to keep developed files or not. While it should always be possible and easy to get to any photo - if developed: from RAW plus sidecar, if edited: from GIMP's XCF- it also makes it quite difficult to quickly access the final image, or the one I uploaded to my website because I also want to publish it somewhere else, or the one I adjusted for printing. Therefore, I keep all the final images around in the "jpg" folder and name them using a flexible system of suffixes. Basically, there is no strict system as long as it is clear what the image and its intention are. It is nice to have the final image appear first in alphanumerical sorting order. Here are some examples:

  • DATE_TIMESTAMP_final_no_wm.jpg: final image (this is usually the final photo, the way I like it most, no watermark therefore not directly for publishing,
  • DATE_TIMESTAMP_final_wm.jpg: same as above but with a watermark/signature and therefore suitable for publishing,
  • DATE_TIMESTAMP_bw.jpg: a black and white version of the photo if I feel like both versions, color and monochrome, work well,
  • DATE_TIMESTAMP_Ax.jpg: image in A format for printing on A4, A3, etc,
  • DATE_TIMESTAMP_dark.jpg or _bright.jpg: different exposures from the final image if they are worth keeping, for instance if it gives the image a different atmosphere,
  • no suffix at all usually means the jpg comes straight out of the camera.

Digikam and most other photo management software can group images. When I keep all developed and processed JPEGs in one folder, I can group them under the "_final_no_wm" version so that all the different varieties won't clutter the album but I can still quickly access all versions by expanding the group.

Conclusion

It's been 1.5 years since I started doing photography with RAW files and this workflow has worked really well for me. I have hardly modified it apart from renaming a few folders. I could see myself dividing my files even more in the future, say, between outdoor trips and events, but for now the quantity of photos is perfectly manageable as described in this blog post. I might look again into digikam's import functionality and what it can do for me but it didn't convince me the first time I tried so I stuck with using the CLI to create sub directories and copying the files from the camera's SD card to their respective folders.

Please head over to Google+ or Twitter @tobiaswulff to discuss this article or any of my photography and videography work. My Flickr and Vimeo pages also provide some space to leave comments and keep up to date with my portfolio. Lastly, if you want to get updates on future blog posts, please subscribe to my RSS feed. I plan to publish a new article every Wednesday.