PetaPixel

A Look at Reducing Noise in Photographs Using Median Blending

noisegalaxy

Between a recent post here on PetaPixel about the Beauty of Space Photography, and my own experiments on blending series of images using averaging techniques, I noticed some rather interesting alignments in technique.

In image averaging, I had previously blended images together by averaging the value of each pixel in an image to produce something entirely new.  The way that this process relates to astrophotography is that the general method is commonly used as a means of noise reduction.  It becomes interesting when you realize that you don’t have to be blending images or taking photos of the cosmos to benefit from the same methods of noise reduction.  It works incredibly well for any images that are relatively static.

Have a look at this image (click it for a larger version):

ISO25600-Desk-median

Although a relatively boring image, the interesting thing about it is that it was shot at ISO 25,600. To be fair, there is a little bit of cheating going on. The result above is derived from combining 10 images of the same scene. This is the power of using median image stacking to increase the signal-to-noise ratio in images.

Here are a few 100% crops to demonstrate what is happening (single ISO 25,600 left, median stack right):

100-crop-1

100-crop-3

100-crop-2

100-crop-4

The last image really shows the strength in this technique.  The words in the banner “Fast, Focused, Fearless” are not even distinguishable in the single ISO 25,600 shot.  Stacking 10 images cleans up the noise enough to clearly read the text.

What Median Stacking Does

Median stacking will consider all the values of a pixel at a single location across each image in the stack, then choose the final value for that pixel based on the median result.

Assuming you took 5 images for a stack, then for a given pixel location, there will be at most 5 different pixel values:

Red  ( 115, 120, 130, 190, 191 )
Green( 150, 166, 169, 200, 210 )
Blue ( 200, 205, 209, 211, 220 )

A Median stack will give a final pixel value as RGB( 130, 169, 209 ). Basically, median will pick the value as the number that falls in the middle of the ordered list. If there are an even number of values, it will average the two values that fall on either side of the middle. This means that values that are spurious or  outside of the range will be disregarded.

This works wonderfully, as noise is random, while the signal should remain constant at a location. It is this property of median stacking that is so useful for astrophotographers who often need to shoot at higher gain to capture their subjects. By combining multiple images they can significantly increase their Signal-to-Noise ratio.

Pushing the Results

An interesting result of this method is that it can be useful even when shooting in ideal conditions.  Granted, you may not shoot a still subject at ISO 25,600 – but even shooting at a cameras base ISO, you can decrease that noise even further than what a camera is capable of in a single shot.

Below is a 200% crop from an image shot at my cameras base ISO 200:

200-crop-base

Here is the same portion from a median stack of 6 images:

200-crop-median

You can see that even at my cameras base ISO 200, there is still some noise when pixel-peeping (200%).  The extra effort to create the median stack result was trivial overall.  In fact, most modern enthusiast cameras can be setup for high-speed capture – so firing off 6 frames is effortless once everything is set up already.

This technique can be exploited by others as well – not just for astrophotography.  It should be immediately apparent that this can be quite useful for any still image being captured.  Some great uses for this technique can be applied in other areas as well:

  • When shooting stock images you will sometimes shoot static scenes, and if it requires a very noise-free image, then this is a perfect technique to consider.
  • I’ve seen some tutorials recently about digitizing negatives/slides using a dslr – this would again be a perfect technique to minimize the noise coming from the digital sensor, and to maximize the quality of the final scan.
  • If you need to capture an image of a highly trafficked area, and want to easily remove people, cars, etc. from the scene quickly and easily (see below).

Creating a Median Stack

I’m primarily a Free/Open Source Software user – so the tools to create these stacks were already in my basic toolkit (Hugin/Imagemagick/GIMP).  Median stacks can also be created using Photoshop (from CS3 Extended on).

Photoshop

Creating the median stack in Ps is relatively straightforward (disclaimer: I’m not a Ps user personally – so I apologize if there is some other method of achieving this that I am overlooking).

  1. Open each of the images in your stack as layers in Ps.
  2. Align them (Edit → Auto-Align Layers…).  Auto should work fine here.
  3. Select all of the layers and turn them into a Smart Object (Convert to Smart Object).
  4. Now apply the Median Stack mode (Layer → Smart Objects → Stack Mode  Median).

Free/Open Source Tools (Hugin/Imagemagick)

Hugin is an open source panorama software that gets fairly heavy use on my system for a myriad of things, but in this case you only need the command align_image_stack to automatically align your images.  It’s fairly straightforward:

/path/to/align_image_stack -m -a OUT_PREFIX file1 file2 ...

Where file1, file2, etc… are the images to be stacked.

Once this command is finished, there will be a series of images that are now aligned and named with the out_prefix from the command.  At this point it only takes one more command using Imagemagick to create the median blended output:

convert OUT_PREFIX* -evaluate-sequence median OUTPUT.jpg

This will stack all of the OUT_PREFIX images, evaluate them using a median stack, and save the results to OUTPUT.jpg.

Interesting Side Effects

An interesting side effect of doing median stacks is that objects that move between the different images in the stack might be removed.  In fact – this is another common use of median stacking.  A good example is if you want to capture an image of a location that has people/cars/etc. moving about, but you don’t want those objects in the final image.  A great example of this is trying to shoot touristy locations, but the desire is to remove all of the other tourists from the final result.  Manually masking out each object could be time consuming, whereas a median stack might remove them quickly and easily.

Here is a simple example to illustrate:

ultramarine-combined

The top 5 images have an object that is moving across the frame, and the bottom image shows the result of doing a median stack.  As far as the median stack is concerned, the pixel data that makes up the moving object is spurious data, and is not considered for the final output.

Overall, median stacking can be an incredibly powerful method of producing very clean and noise-free images.  The only caveat is that the subject be static between each of the frames.


 
  • Jonathan Maniago

    If your camera is steady enough for taking multiple images for later stacking, wouldn’t it be more time-saving if you simply took a longer exposure at a lower ISO? Outside of astrophotography, what other applications would benefit from this method of noise reduction?

  • http://www.facebook.com/people/James-Conner/100000343007259 James Conner

    The technique is an effective way of reducing noise in the shadows of scanned images. VueScan has built-in functions for multiple pass scans.

  • Banan Tarr

    Did you read the article?

  • Zachary Larsen

    Isn’t this basically what the Nokia 808 does, except that it stacks from neighboring pixels to create a smaller image from its 41mp sensor? Also, I think my S100 does something similar in one of it’s pretty cartoon picture night mode thingies.

  • Ilker Sen

    Not necessarily. Instrument stability over time is always a consideration in signal processing. If you will do this regularly, you would do a series of noise comparisons, say a shot with say 200 s exposure vs. 4×50 s exposures averaged, and find your sweet spot. Every instrument is different…

  • Ilker Sen

    I would like to see a comparison of median averaged vs. mean averaged noise in photos. Technically mean average should reduce noise by a factor of square root of # of averages, whereas median average is prone to artifacts (which can be a positive thing in the case of the moving object).

    Also, camera noise is usually not random. That’s why astrophotographers remove dark frames.

  • http://blog.patdavid.net/ Pat David

    I mentioned it in the article, but this method allows you to push the noise down further even moreso than optimum settings on your camera (ie: less noise than you can capture in a single photograph).

  • Jonathan Maniago

    Skimmed, with preconceived notions based on experiences with DeepSkyStacker. Hooray, illiteracy.

    Anyway, my typical solution for moving crowds is using ND filters for a significantly longer exposure, but that technique isn’t applicable for fisheye lenses (bulging front element, ridiculous field of view, and no threads). Median stacking might just do the trick.

  • rwodaski

    This can actually get way more sophisticated than what’s discussed here. The issue are outliers: pixels that are way off what they should be. Noise is one thing (random deviations from what’s ‘true’). Outliers occur from actual errors (e.g. hot pixels), and unusual outliers (e.g., cosmic-ray strikes in astro photos). A combination of stacking images and smoothing to remove outliers is typically the best overall strategy. Products like PixInsight (for astrophotography) offer amazing statistical tools for stacking/combining images that optimize these two factors (better optimizations, of course, require more individual images to work). A tremendous amount is already known about this stuff; it’s just not widely known in photography circles because the very idea of multiple images never comes up in most photography circles. Having been proselytizing about these issues for almost two decades now (I write books about astrophotography), I can tell you that the #1 obstacle to understanding this stuff is that the whole idea of ‘noise’ is very counterintuitive – bringing everyday expectations to learning about noise just does not fly. :-) A good starting point: noise is really uncertainty, what you don’t know. The only way to reduce it is to increase the information; that’s what all forms of stacking images do.

  • Willi Kampmann

    Good morning: This isn’t only old news, but some of Sony’s cameras are even able to do exactly that *in-camera* – and have been for a couple of years now. The article is definitely interesting nonetheless, but the “oh hey look what i found out” sentiment seems a bit strange.

  • iowapipe

    to be a bit more accurate about dark frame subtraction: it removes the element of fixed pattern noise. (dead/hot pixels, unique response of each pixel…) So your statement isn’t correct about camera noise not usually being random. Only ‘one source’ of camera noise tends not to be random. Having said that; even that noise isn’t particularly set either. Often, you use an average of dark frames to calculate your final subtraction. There is variance in the dark current pattern of each frame.

  • http://blog.patdavid.net/ Pat David

    Sony cameras do *in-camera* median blending of multiple exposures? I must have missed that somewhere, could you point me to some information about it?

  • Willi Kampmann

    In the NEX cameras it’s called “Twilight Handheld Mode”. They capture a series of photos in rapid succession (I believe 5 photos) and then merge them to create a low-noise JPEG. This can create images at ISO5000 that look like ISO1000, it’s pretty amazing. I’m pretty sure that’s median merging (it’s certainly averaging each pixel). If there’s motion in the scene then there’s a similar mode that blends only static parts of the frame but leaves out the parts that are changing (so they’ll stay artifact free but also noisier).

  • Willi Kampmann

    You still get better results doing this manually with RAW images of course (though with much more work), but it’s pretty amazing what these cameras can do these days.

  • http://csabaszilvasi.com/ Csaba

    Underwater photography could be a perfect application of this method of noise reduction. Sure it would only work with static subjects. Or would natural light beams moving around because of the surface of the water make it impossible to use? Hmmm…

  • http://csabaszilvasi.com/ Csaba

    Underwater photography? (See my comment above)

  • http://alphacorner.eu/ Sky

    It was introduced in cybershot, than Sony added it to the DSLRs… A500 was the first one if I remember well. Now it’s a standard in A-mount cameras and most of E-mount (NEXes) got it too. It’s really nice thing, as it tries to be smart and, for example, skips any blurred frames if you got shaky hands (something Photoshop won’t do).

  • http://blog.patdavid.net/ Pat David

    Thanks for that! TIL about “twilight mode”. Makes sense to implement something like this in-camera if the processing power is there. :)

  • MarceloMetayer#Aline

    Man, this is amazing for people like me, still fighting with the small sensor of compact cameras! Thank you! I’ve tested the method and works really fine.

  • Andre Gasket

    Using mean is a little like a single long exposure and includes a bit of info from each of the photos in the stack. Using median removes the effect that any outliers play.

    for example with the following value 10, 11, 12, 13, 14 100
    if using median, the result would be 12.5 (rounded to 13) – the 100 simply ‘falls off’ the end.
    if using mean, the result would be 26.6 (rounded to 27). The 100 is included in the ‘averaged’ and still creates a hotspot.

    if the 100 were replaced with 256, again the median is 12.5 (13), but the mean is 52.7 (53) – an even more significant hotspot.

    As the number of images in the stack increases, this outlier becomes less of an issue, but other things like (eg movement, or the sun moving) can creep in.

    There is no correct one to use, mean gives nice motion trails/blurs, whilst median tends to remove a moving object altogether.

  • Ilker Sen

    Yes, this is all correct. What I was trying to get at was noise reduction rather than removing outliers. If you mean average 4 photos, you’ll have 1/2 the noise (technically S/N, signal to noise ratio). 100 photos will be 1/10 the noise. I wanted to see how median average progresses.

  • laurence

    try it and show the results?

  • Bobby

    Why criticize someone that is sharing info? Not being rude, but congrats that you already knew that. Not everyone does.

  • Willi Kampmann

    I didn’t criticize the sharing of the information; I criticized the writing style which suggested the author has had an epiphany and came up with something new. It just sounds strange and kind of inappropriate. I think my post was pretty clear on that.

  • http://www.shinyphoto.co.uk/ Tim

    No. You’d still suffer from long-exposure noise (which takes the form of a fixed pattern of hot & dead pixels, rather than the Poisson distribution of ISO noise, and therefore is not amenable to reduction by averaging).

  • http://www.shinyphoto.co.uk/ Tim

    Yes. Bingo. `Information’ is just the right word for it: image-quality might reasonably be defined as the amount of information put in front of your eyes that originates from photons (as distinct from, say, some “noise-reduction” algorithmic approximation).

    Random experiment, because I like doing these things: comparing two images processed identically from RAW, the addition of (delicate, to-taste) pyramid noise-reduction and Wiener sharpening is sufficient to change an *average* of 2 bits per pixel in the L channel. That’s a quarter of the image’s bit-space changed, just like that.

    And you can use enfuse to blend biassed towards image entropy. That’s my favourite.

  • http://www.shinyphoto.co.uk/ Tim

    Kudos to the article for sharing align_image_stack; that’s probably a key to a whole thing.

    Upscale all the images at the point of RAW conversion (say using Lanczos3 in RawTherapee or Photivo), maybe to 1.33x larger. Now you’ve got a grid of higher resolution for align_image_stack to work with – effectively, what would have been sub-pixel offsets might now become adjacent-pixel statements in a new, larger, grid. And that is known as super-resolution. (In a random spray of images, some pixels will interleave and some will coincide, so that’s a hybrid of super-resolution and noise-reduction.)

    Notes:
    1) if shooting in daylight, the tripod is now your enemy – hand-held in burst mode gives you a brace of images with random offsets required, quickly and easily
    2) people pay for prints of images simply upscaled with lanczos3 – heck, stupid photographers pay for the ability to *use* Lanczos3, and fret about whether it’s better than bicubic or whatever.
    3) conventional noise-reduction and sharpening algorithms change maybe 1/4 of the bits in an image based on some algorithm’s computed approximation. This enhances the bits with photons from the scene instead.
    4) Using multiple images with random offsets does help counter long-exposure noise as well as ISO.

    5) you can use it for extending the exposure time; and more, you can use it to choose a subset of frames for shorter long-exposures up to the total time. E.g. taking a photo of water flowing around rocks: one 16s exposure is final, nothing to change; but by taking 8*2s exposures, you can choose a subset of frames for their sharpness, or to avoid certain patterns of errant spray, finishing with maybe a 10s total exposure time instead.

  • CartierBesson

    Just like other many typical high ISO examples, this I done in places with plenty of light. Can you do this or the images that are shot in Dark places? I don’t know why would anybody shoot high iso when there is good light.

  • chris77ve

    Have you tried this with 16-bit TIFF images? I’m trying but the result is a empty image using -evaluate-sequence median, whereas it is all fine with JPEGs…

  • http://blog.patdavid.net/ Pat David

    If you’re unfamiliar with imagemagick, you may find that you have to remove any alpha from the tiff first (and, of course, make sure you are using a 16-bit imagemagick as well).

  • Wah!

    I think median stacking is just a particular case of sigma clipping average stacking. Do you agree?