X-Trans vs Bayer Sensors: Fantastic Claims and How to Test Them
Since the introduction of the Fujifilm X-Series line of cameras, reviewers and consumers have struggled to compare them directly to the competition. Fujifilm’s is a tightly integrated system, wherein everything is a little bit different.
The effect of all these confounding factors, intentional or not, along with Fujifilm’s hyperbolic and cryptic marketing copy, has been to lead consumers to draw incorrect conclusions when comparing Fujifilm against other brands of camera, specifically regarding noise, moiré, and detail resolution. If you compare a Fujifilm camera to another brand of camera without accounting for these various factors, you may think the Fujifilm performs better in every regard. You may even think that there’s something magical about it.
Given this confused situation, I wanted to perform a comparison which eliminates all of these factors and compares the Bayer CFA to the X-Trans CFA as directly as possible, without involving lighting, optical aberrations, lens light transmission, ISO ratings, noise reduction, optical lowpass filters, etc., and see if the X-Trans CFA really does offer any of the advantages Fujifilm claims it does when compared against Bayer — with no marketing funny-business.
As the saying goes, the devil’s in the details, and there are a lot of details involved, so grab your spectacles and wand: we’re going in search of that Fuji X Magic!
Methodology
In order to remove the complicating factors of optics, base sensor technology, etc., these comparisons are performed with synthesized raw images. This is what raw data from a sensor with a Bayer and an X-Trans CFA looks like, respectively, before demosaicking:
This methodology allows for a direct comparison of the output with the input images (ground truth), and is the same technique employed by the researchers who develop demosaicking algorithms. The goal is to simulate an AA-filterless sensor, differing only in CFA (Bayer vs. X-Trans). The synthetic raw images are generated by filtering the target images through the respective CFA patterns. The resulting data is then fed into a demosaicking algorithm.
DCRaw is used for all demosaicking because it conveniently allows us to provide our own raw pixel data without having to wrap it in a container. The target images themselves have been downscaled significantly from their original size in order to eliminate any noise and false colors from the input.
Because Fujifilm’s own X-Trans demosaicking algorithm is proprietary, it could not be used for this comparison. Instead, I use Frank Markesteijn’s algorithm (in highest quality 3-pass mode). However, I’ve done tests and found that this algorithm is at least as good as (and perhaps better than) Fujifilm’s.
For the Bayer images, we use AHD, a similar high quality algorithm for demosaicking Bayer, which shares some properties with the Markesteijn algorithm. There are better algorithms available for Bayer, but this is the best DCraw supports. (I prefer AMaZE, myself.)
Performance
Fujifilm representatives have tossed around various figures of their own for X-Trans performance (“30% slower”) and have also hinted that one of the reasons they choose Bayer for their new medium format GFX 50S camera was because demosaicking 50 megapixel X-Trans images would be too slow.
In these tests, X-Trans demosaicking took approximately 3.27 times as long as Bayer. So perhaps what Fujifilm really meant to say is that X-Trans demosaicking is 30% as efficient as Bayer. In any case, it’s significantly slower, a fact that is uncontested.
Bayer vs. X-Trans
The difference between these two patterns of red, green, and blue sensitivity is what this article is all about.
The Bayer CFA is common and very well established. Invented by Bryce Bayer of Kodak in the early 1970s, the Bayer CFA has been part of digital photography since its inception. Fujifilm introduced X-Trans promising that it offered many improvements over Bayer, most of them incredible.
Here are Fujifilm claims regarding X-Trans:
The unique random color filter array reduces moiré and false colors without an optical low-pass filter. These color filters also have the effect of increasing the resolution so, when shooting with a high-resolution Fujinon lens, the camera delivers images with a perceived resolution far greater than the actual number of pixels used.
As you can plainly see for yourself in the above figure, there’s absolutely nothing random about the X-Trans CFA. It’s just a larger pattern, 6×6 vs Bayer’s 2×2. To call it random is extremely misleading, but that seems to be the theme for the entire brochure.
Here Fujifilm elaborates on the claim that APS-C X-Trans can match the performance of full-frame (presumably higher resolution as stated above) Bayer:
The FUJIFILM X-M1 is equipped with a large APS-C X-Trans CMOS Sensor, which offers picture quality comparable to that of full-frame sensors. The sensor’s unique colour filter array minimises moiré and chromatic aberration without the need for an optical low pass filter, while dramatically boosting resolving power even at identical pixel counts to deliver sharp and texture-rich pictures.
Fujifilm seems to be conflating false color (a type of aliasing) and chromatic aberration (a property of lenses) here, but it’s the dramatically boosted resolving power that’s fantastic. All of these are rather bold claims, to say the least, which have never, to my knowledge, been backed up by any evidence. But that’s OK. We can test these claims ourselves — cast a spell of knowledge, if you will…
Moiré and False Color
Let’s begin with a standard test-chart like image designed to show the limits of resolution.
Well, the moiré/false color certainly looks different. Whether it’s reduced or not appears to depend on the hue of the subject.
I think we can do better than charts though, let’s try a subject where one typically encounters moiré in the real world: fabric. (Note that the input image is completely monochrome.)
Well, so much for that… This example looks far worse with X-Trans! The fact is, some patterns and orientations will look better with Bayer and some will look better with X-Trans. This is simply because the CFA patterns are different and will therefore interfere with different subject patterns. Just different, still patterns — neither are random. I see no evidence that is one inherently more resistant to moiré than the other.
So how does Fujifilm deal with this? How have their claims of moiré reduction gone unchallenged? Well, as we discovered in the first article of this series, the in-camera processing does more than just demosaic the image, it also applies heavy chroma noise reduction and color profiles which reduce overall saturation (these are called Film Simulations by Fujifilm). Let’s simulate this effect by applying strong bilateral filtering to the chroma and reducing the saturation a bit (by about as much as STD/Provia does).
Well, that’s more like it. Now the moiré is significantly attenuated. However, there’s a problem: we could have done just the same kind of filtering with Bayer (or any other CFA)! Using X-Trans didn’t buy us anything. But that’s not the only problem with this approach. Let’s see what happens when we apply the same processing to an image with color:
Ah, there’s the rub. This spell has a catch! The butterfly wings didn’t fare too poorly (the high contrast edges make it easy for the bilateral filter), but look at the unnatural color of the fingernail in the NR’d image! This happens because the chroma NR strength required to eliminate moiré, which is, in-camera, naively applied to all images, regardless of the actual presence of moiré, is much greater than the chroma NR strength required to eliminate chroma noise.
By using a technique designed to treat color noise to treat both color noise and false color (which have similar appearance but different causes), fine and especially subtle color variations are lost, even in low noise, low ISO images. Fujifilm doesn’t mention it anywhere in their marketing copy, but this is how their X-Trans cameras suppress moiré. Not optically, not in the particular arrangement of the CFA, but purely in the digital domain, algorithmically, and in a general way that is equally applicable to AA-filterless Bayer images.
If you think the effect on a fingernail is bad, take a look at was this kind of processing does to a face:
Observe the color of the teeth and eyes, and how the skin has taken on a waxen, lifeless appearance. You can pump the saturation back up all you want, but you can never recover the fine color detail after this kind of processing has been performed.
Resolution and Fidelity
For each example we present the ground truth, followed by the Bayer (left) and the X-Trans (right) results, below these we show the respective difference images (that is, the difference from the ground truth).
Example 1
Example 2
Example 3
Example 4
False colors are most apparent with high contrast monochrome subjects, which are instructional if not realistic. (Image courtesy WikiMedia Commons.)
PSNR Stats
PSNR is a standard measurement for quantifying image degradation. In this case it measures the difference between the ground truth and the demosaicked output. The bigger the number, the higher the fidelity.
Bayer wins overall. From looking at the difference image, it seems likely that if AHD were doing a better job interpolating the near-diagonal lines in Example 2, it would have won across the board. Notably, X-Trans performed poorly on Example 1, which contained a lot of red. This is because there are fewer red and blue sites in the X-Trans pattern compared to Bayer.
The poor performance of X-Trans on Example 4 is more interesting. With a monochrome subject, the extra green in the X-Trans pattern is supposed to produce a perceived higher fidelity result than Bayer. The reality is that the false colors wash out this supposed advantage and it actually performs worse. There is no evidence of “dramatically boosted resolving power.”
What About Noise?
The preceding tests were conducted with images containing essentially no noise. It has been stated by reviewers that the X-Trans CFA offers a noise advantage over Bayer, producing lower color noise and having a more “film like” grain. Let’s see if that’s true. In order to test this, we generate a noise image and apply the very same noise image to the raw Bayer and X-Trans data before demosaicking, which simulates how noise occurs in a sensor. This direct comparison is also completely free of the complication of Fujifilm’s different standard of ISO rating, something which often thwarts camera to camera comparisons.
There can be no more direct a comparison than this. No false color suppression or noise reduction is employed. Bring on the magic!
Example 1
Example 2
Example 3
Example 4
Huh. I wonder what happened to the magic. Can you see the difference? I can’t. They all look crusty as a Leprechaun’s corduroys on St. Patrick’s Day. (Leprechauns are magical right?) Do any of these examples look “film like?” Let’s see what happens when we apply a little noise reduction to one of these. Let’s use a Bayer image, with its supposedly un-film-like characteristics…
Ah, there we go. Well, I wouldn’t call this film-like either, but at least the color noise is gone. This is the same thing that happens in Fujifilm’s in-camera processing and in 3rd party RAW processors which don’t allow the user to completely disable noise reduction for X-Trans files.
PSNR stats with noise
Surprisingly, X-Trans does have the win here — albeit a rather marginal one. You can see for yourself in the images how much of a visual difference these few decimal points of PSNR correspond to. Even if we don’t call this a tie, differences of this order would be completely swamped by 8-bit quantization, JPEG compression, optics and other real-world factors. Poof.
Versus an AA Filter
AA-filterless sensors (or sensors with negated AA-filters) have become popular in part because their output appears sharper straight off the sensor, without digital sharpening. As we’ve seen, this comes with the cost introducing false color and moiré artifacts.
A sensor with an AA filter requires digital sharpening, but with it can appear almost as sharp (more on this below) as an image from an AA-filterless sensor while displaying fewer artifacts. AA-filterless sensors require little or no digital sharpening, but are subject to false color and moiré effects (which as we’ve seen require digital noise reduction filtering to suppress).
An AA filter, however, does nothing to help with noise, which happens in the sensor, so false colors introduced by high ISOs (as in the noise examples above) are unaffected.
Where AA-filterless sensors may have an advantage is when the final image is to be converted to monochrome, and especially when the subject itself is monochrome (i.e. documents). And to a lesser extent when photographing subjects which contain no patterning and no high contrast fine detail (i.e. some types of nature/landscape images). Also, there exists computational diffraction reduction technology (which may be what Fujifilm is using in their so called Lens Modulation Optimizer) which actually relies on aliasing to function.
In order to simulate an AA filter, we apply a blur filter to the high resolution target image, downscale it to the testing resolution, perform the RAW conversion and demosaicking, and then apply an unsharp mask filter to the output. No noise reduction or desaturation is required.
Here’s the image from Example 4, with a simulated AA filter Bayer sensor:
The AA-filterless example looks like it has more detail, but does it really? And at what cost? Is it real detail or just aliasing and false color? That superfine “detail” (which is really just aliasing) isn’t going to be visible when you zoom out, but the false color may, especially with examples of moiré.
And let’s take a look at that moiré target with a simulated AA filter:
AA-Bayer vs X-Trans
Now let’s compare our simulated AA-filter Bayer output to X-Trans. This was the original case for X-Trans, touted as a superior alternative to AA Bayer…
Conclusions
The Markesteijn algorithm does a better job of interpolating near-diagonal lines than does the AHD alogrithm, but this isn’t directly attributable to anything about X-Trans or Bayer; other Bayer demosaickers perform better in this regard, and other X-Trans demosaickers perform worse. (Add to this that AHD hasn’t been tuned for unantialiased input, while Markesteijn is doing extensive 3-pass luminance interpolation.)
However, even with the algorithmic lead, X-Trans appears to offer no advantage over AA-filterless Bayer, and in fact produced a lower fidelity result than Bayer in all but one test case. Predictably, AA-filterless Bayer and X-Trans suffer from similar levels of false color, X-Trans being slightly worse/chunkier due to the courser pattern. X-Trans tends to produce line-like artifacts, appearing smeared in aggregate, whereas Bayer produces more speckle-like artifacts. The X-Trans pattern changes the character of moiré, but does not appreciably reduce it, and certainly doesn’t eliminate it.
The real moiré and false color reduction of Fujifilm’s cameras comes not from the choice of sensor CFA, but from noise reduction occurring in the image processing pipeline. As shown here and in my previous articles, this level of post-processing, which is applied globally and indiscriminately, has the side-effect of significantly reducing fine color resolution. X-Trans is more sensitive to subject color, performing its worst on subjects with predominant red or blue hues. X-Trans provided a marginally higher PSNR than Bayer in the presence of noise (the results are so close that things like a different choice of Bayer algorithm, JPEG compression, and certainly any application of NR would wipe out the differences). Any apparent larger noise advantage found in other comparisons must be due the confounding factors of underlying sensor technology (Sony makes the sensors, FYI), ISO rating, electronic/thermal noise, and noise reduction baked in to the X-Trans demosaicking algorithm in use.
Even though X-Trans lost the battle, the results were very close. An AA-filterless Bayer sensor and an X-Trans sensor of the same resolution are fairly evenly matched. A sensor with an AA filter, however, will beat them both hands down when it comes to eliminating moiré and false color — and without reducing color resolution in the process (but one must apply the appropriate amount of sharpening for the best results). This leaves us in a bizarre situation because Fujifilm insists on continuing to use X-Trans in their midrange/high-end cameras (except for the medium format GFX 50S), while using Bayer (with AA-filter of course) in their low-end X-A range. That means you can get better IQ by buying a low-end camera (X-A3) that costs a third what the high-end model (X-T2) does — from the same manufacturer (although you won’t necessarily get better JPEGs if the level of chroma NR being applied in-camera is the same in the X-A line).
True AA-filterless sensors (where the AA filter isn’t simply negated) might have a slight physical sensitivity advantage due to receiving light which would have otherwise been absorbed or scattered by the AA filter. However, given the amount of noise reduction required to treat the false colors introduced by the omission of the AA filter, it seems unlikely that there is much, if any, net benefit.
Finally, and probably most practically, X-Trans requires significantly more processing time/power and, at the time of writing, all but one of the commercial RAW processing programs on the market produce lower quality output than the free-software Markesteijn algorithm used in preparing the examples for this article.
So there you have it: We disenchanted the marketing copy, saw through the legerdemain, traced the rainbow back to its very source, and found the truth about X-Trans. And that truth is: all the “magic” is nothing but smoke (sans mirrors). But, I have to admit, it’s a clever trick.
About the author: Jonathan Moore Liles is a photographer, writer, musician, and software architect living in Portland, Oregon. The opinions expressed in this article are solely those of the author. You can find his work on his website, Instagram, and Bandcamp. This article was also published here.
