PetaPixel

Inspecting an ‘In Spec’ Lens

lensrentals8

I’m going to open a can of worms today. I’ve been getting more and more emails from people telling me the same story that goes like this:

I’ve got this lens. It’s awful. I’ve sent it in for adjustment and the service center tells me it’s ‘in spec’ and nothing is wrong with it. Am I crazy?

Second only to the dreaded “impact damage — warranty void” statement, the “lens is in spec” statement seems to be some factory service center’s answer to far too many complaints. Don’t get me wrong, sometimes there is impact damage and the warranty should be void.

But I can’t think of any reason why this seems to happen only to certain brands and never to others. Similarly, lenses a customer thinks are bad can be ‘in spec’. The problem is, since the factory service center doesn’t have to tell us what “in spec” means, it’s open to a lot of abuse.

Now before I go further, let me say that I feel the service center’s pain, too. Most of those emails I get are accompanied by some useless JPEGs. Maybe 1 person in 10 who emails me actually has taken images that would indicate if there was a problem or not. So I’m certain the techs at the service center become a bit cynical over time.

So if you think a hand-held autofocus image, an ISO 6400 image, or a picture of some isolated object right in the center of the image is helpful in deciding if a lens is OK, then you aren’t allowed to read this post. Instead read about how to test a lens, how to check for decentering, or what a lens with a tilted element looks like. Then you can come back and read this.

An Example

First of all, I’m not going to mention what lens or company this example is taken from. The reason is simple — I’ve got plenty of similar examples (over 100 actually) with other lenses and other factory service centers.

What we have here is a high-quality, fairly expensive zoom lens. It was sent to factory service because it had much lower resolution on the left side than the right. It came back looking the same. The second time it came back better at the wide end, but worse at the long end along with a note saying it had been tested and was in spec. The third time it came back no better with a note that it had been tested and was in spec. We didn’t see much sense in trying a 4th time.

I won’t bore you with optical bench, lens test projector, or Imatest results for the lens (we did them, but I want to keep this simple and they aren’t necessary). I’ll just show you images taken of an ISO12233 test chart. I like this chart because it lets us look at resolution in the center and all 4 corners in one shot, if done correctly.

Here’s a picture of the chart, in case you aren’t familiar with it:

isochart1

The lens that came back from factory service for the 3rd time was OK in the center, so I’ll just show you close-ups of the 4 cross-shaped resolution charts in the corners. Here are the corners at the wide & zoom ends of the zoom range.

Four corners at widest zoom,

Four corners at widest zoom, “in spec” according to factory service.

Four corners at long end of zoom,

Four corners at long end of zoom, “in spec” according to factory service.

To make sure you have an idea of how bad this “in spec” lens is, in the image below I’ve put the top two corners at the long end from the “factory in spec” lens with the top two corners of another copy of the lens taken off our shelf. Again, because someone who doesn’t read the whole article is bound to say it — these shots are all focus bracketed and the best image used. Autofocus has nothing to do with it — this is purely optics.

Top two corners from the factory

Top two corners from the factory “in spec” lens above; top two corners from an off the shelf lens below.

Like we all learned from Sesame Street, one of these things is not like the other. According to the factory service center, they are both fine. But if I sent that “in spec” lens out to one of my customers, I would have one less customer.

I talked Aaron into opening this lens up and adjusting it in-house. Adjusting it involved removing the mount and making some 17 trial-and-error adjustments to the rear group, testing after each adjustment. Then we removed the front barrel and made a half dozen adjustments at the front group, followed by taking the mount off again and making some more tweaking adjustments to the rear element.

It was a several-hour long process. But at the end of the process, the lens was now within my “spec” as you can see below:

Four corners at wide end after adjustment.

Four corners at wide end after adjustment.

Four corners at long end after adjustment.

Four corners at long end after adjustment.

It’s not absolutely perfect. If you noticed, the left lower corner at the wide end still has some astigmatism with the vertical lines not resolving nearly as well as the horizontal lines. But even that corner is much better than when it came back from the factory in spec (and is better at infinity than in this fairly close-up shot). The other 7 corners are all excellent. Maybe we could have gotten that last corner better, but my experience is that most of these lenses have a slightly weak corner at one end of the zoom range.

It’s the same lens, no new parts, just a very lengthy optical adjustment.

My point is not that we’re so great. We’re not. We’re just a couple of self-taught guys trying to figure stuff out mostly by trial and error. My point is that if some half-trained guys with a few screwdrivers and some optical testing equipment could do this, then the trained technicians at the factory service center could have done it, too. They certainly could have done it faster. In theory they could have done it better.

But in three tries they didn’t do it at all. They just told me, in writing, that the lens was “in spec.” They won’t share exactly what those “specs” are, but in this case the picture says it all. Their specs are pretty damn low.

Let me repeat for you fanboys who are certain your brand doesn’t do this — chances are that it does. I just chose this lens because it happened to come back yesterday. I have this kind of result several times a week (out of dozens of repairs a week, mind you).

So Why Doesn’t the Service Center Do It?

I’m doing some educated guessing here, and not all reasons would apply all of the time. But here are some of them.

  1. Some lenses can’t be optically adjusted. The factory has the choice of giving you a new one, telling you the one you have is OK, or randomly replacing some of the lens elements and groups and hoping that makes it better.
  2. Service centers don’t have nearly as much optical testing equipment as you think they do. None of them talk openly about what equipment they have, but I’ve had a lot of conversations with techs that let slip that they don’t have certain capabilities. They have lots and lots of computerized adjustment software. But most don’t have optical benches, collimators, or even high-quality test targets. Some use lens-test projectors, which aren’t particularly accurate, but are better than nothing. Some don’t even have those. My favorite anecdote is a lens we fought with the service center over for 3 months. The last time it came back we were told their chief of optical testing had adjusted it himself. Unfortunately the chief left his memory card in the camera that we’d sent along with the lens: 62 images of a bookshelf were used to make the optical adjustments. Even when looking at a bookshelf, though, you could tell it still wasn’t right.
  3. Some service centers apparently can only adjust optics for the center of the lens. They apparently don’t have equipment for off-axis adjustments or assessment. (That’s may be the case with this brand; they always seem to get the center perfectly adjusted but often send lenses back with horrid corners.)
  4. These adjustments can be really, really difficult and time-consuming. The techs are on a clock, expected to do a repair in a given amount of time, so they must dread optical adjustments. Or maybe “in spec” just means time was up.
  5. When a customer says their lens is soft, the most common problem is that they can’t tell backfocus from their back end. So the service center tends to “electronically adjust” the lenses. That is fast, computerized, and works great when the problem really is backfocus or frontfocus — and that’s usually the problem. But eventually, if all you have is a hammer, everything starts looking like a nail.
  6. So many people claim their lens is bad when it really isn’t, that the techs get rather cynical. I’ve gone up the food chain often enough, and have heard the surprise in someone’s voice when they say, “Oh, you really do know what you’re talking about,” to know this is true.

So What to Do?

I don’t expect my writing a blog post is going to make any factory service center rush out and make changes. But it may help some of you who are fighting these battles to know it isn’t just you. Just because you’re paranoid doesn’t mean they aren’t out to get you.

I also think that maybe, just maybe, if people ask politely with their wallets, some manufacturer will decide it will be to their competitive advantage to offer better optical services. Sigma recently got a ton of positive press when they showed the equipment they had placed in the assembly factory to test every lens before it was boxed for shipment. People want that kind of assurance.

If you run into this problem, you might ask to see the optical test results. They aren’t going to show them to you. Looking at the example above (and lots of similar examples like the ones above), I doubt they even have any. But maybe if enough people ask, in these days of lowered sales expectations, one or two companies might decide enough customers are interested in that service to provide it. Actually, we only need one company to do that. The others will have little choice but to follow along.

I think that a service center that actually sent you a printout of their optical tests, showing that your lens really did (or didn’t) meet the expected test results, would win a lot of loyal customers. Heck, the oil change place always shows me the old filter they replaced just to make sure I know they really replaced it. The service center could at least show me that they’d actually tested the lens.

Actually, I suspect there are one or two companies that already do have optical testing equipment available in their service centers, and who use it to make accurate optical adjustments. If they’d like to show it off, I’d be happy to write about it for them.


About the Author: Roger Cicala is the founder of LensRentals. This article was originally published here.


A Note About Who Shot John:

I knew it would happen — that people would be angry that I don’t name names. Let me be clear as to why.

First and foremost, I haven’t gone back, gathered the data on 10,000 repairs, hand sorted which ones were repeat optical repairs (because that parts not a sortable field in our database), compared that to all optical repairs for that brand, number of copies and rental weeks for each lens, etc. That’s a 60 hour job and I’m busy at the moment. But without that it would be totally unfair for me to put up a note saying “this manufacturer has more problems,” etc. Do I have some ideas in my head? Sure. But I’m the analytical type. I want numbers and facts before I spew off negativity.

Second, I’d really like to see changes made. Backing someone into a corner and thrashing them is certainly fun, but it rarely results in change. It usually just results in damage control and denial. If I did that today I already know what the response would be — “we have far superior methods of testing our own lenses, but they are secret and proprietary and we can’t show them to the public.” Given the way most camera manufacturers seem to work these days, they might change things quietly and secretly. A few years from now things might be better (or they might not). But they would never, ever let us know there had been changes, because there was never a problem.

I’d much rather that one company, having not been backed into a corner, decides it will be great marketing to show the purchasing public how they test, to provide test results when you send a lens in for optical adjustment, to get lavish praise in all the forums for that, and watch the serious photographers, who buy lots of lenses, flock to their brand. Is it going to happen? Nah, probably not. But I’m an optimist.

But for those that want to investigate what companies might be better at optical adjustments, here’s a good place to start. Look up what companies make both lenses and optical testing equipment. You’d kind of figure they’d have their own machines in their own service center, wouldn’t you?


 
Get the hottest photo stories delivered to your inbox.
Get a daily digest of the latest headlines:
  • Todd Klassy

    I personally am sick of everyone claiming their lenses suck because of lens tests like above. There are far too many variables in lens tests. Human input, tripod set-up, level of the wall, etc. The only constant is the lens. Real world results are the only thing that should matter. I have owned several maligned lenses that others had claimed were susceptible to being out whack. They would send them in for repair and get a “lens is within spec” and then complain about the results. Sorry, but I trust Canon’s or Nikon’s lab tests more than I do the average photographer’s.

  • opticsguy

    Lens performance in a volume product is statistical, with hundreds of manufacturing parameters (thickness, radii, index, tilts, decenters) varying randomly to produce a distribution of performance across an ensemble of lens units. Manufacturers optimize the lens design’s sensitivity to these variations to balance performance and cost, and choose an image quality specification to set what fraction of the performance distribution is acceptable and therefore “in spec” (again trading performance and cost). That threshold is by definition below nominal (ideal) performance, and thus performance may vary noticeably from lens to lens. The manufacturing process is not designed to optimize every parameter for best lens performance (rather to produce a specific level of yield at a given cost), and so it is also not surprising that a given lens’s performance can be improved by someone with good feedback and time on their hands. Of if it’s a good on you might just screw it up. :-)

  • Tom Dragonetti

    I will say, I’ve NEVER had a bad experience with Canon. I’ve probably sent at least 6 lenses in as a consumer before I became a CPS member. Now that I’ve become a CPS member, the turn around is AMAZING on lens cleaning and the one lens with a loose first lens element. I’ve never had a lens come back to me that I was unhappy with..

  • Andy Umbo

    Had a new Hasselblad 50mm CF back in the day, it was so “soft” looking compared to my previous 50mm C T*, and the rest of my CF lenses, I sent it in under warranty; came back as “in spec”. Shortly after, Hasselblad redesigned the lens with a floating element, then redesigned it again with a floating element and a bigger piece of galss needing a 70mm filter instead of their usual 60mm. Here I sit 25 years later with a lens I still rarely use because it is so soft, I should have traded it years ago. When is “in spec” just bad lens design?

    BTW, Tom Dragonetti, if I had to send six different lenses back to a camera company for specing or repair, I’d start looking for a new camera company. As a pro, I don’t think I’ve sent six lenses in for repair in my entire 40 year career!

  • gratayua506

    мʏ ƈʟαѕѕмαтɛ’ѕ нαʟғ-ѕιѕтɛʀ мαĸɛѕ $84/нʀ օɴ тнɛ ιɴтɛʀɴɛт. ѕнɛ нαѕ вɛɛɴ աιтнօυт աօʀĸ ғօʀ ѕɛʋɛɴ мօɴтнѕ вυт ʟαѕт мօɴтн нɛʀ քαʏƈнɛƈĸ աαѕ $15984 ʝυѕт աօʀĸιɴɢ օɴ тнɛ ιɴтɛʀɴɛт ғօʀ α ғɛա нօυʀѕ. ɢօ тօ աɛвѕιтɛ fox200&#46­com

  • darylcheshire

    I have been fortunate that I have four Canon L lenses without apparent problems. I read the fredmiranda forums and note the whingers there who send lenses back sometimes more than once.
    The Canon 5D mk III can micro adjust a lens. This would be a move by Canon to reduce the number of service requests to adjust lenses.
    Also regarding the 50mm f/1.2L lens, lots of people complained about it. I wonder if the took into account the very narrow depth of field? Also the focus shift? The 50mm f/1.2 is almost tricky to use and you need some experience to determine if a problem is a fault or lack of experience.
    Interestingly, the complaints dropped off and people were more happier with the 50mm f/1.2 lens and some later reviewers wondered if Canon did a ‘silent’ upgrade to the lens.
    Mine was a later model and I have not noticed any focus shift.
    Some time later Canon dropped the practice of date coding their lenses.

  • oscar

    They should use DXO before and after, it is automated and with computerized, fast, reliable and repeatable results. I wonder if somebody does that service.

  • oscar

    You could use DXO before and after, it is automated and with
    computerized, fast, reliable and repeatable results. I wonder if
    somebody does that service. And there is a database with results from most the modern lenses in the market, so you could compare.

  • WKYA_Radio

    Ive had very few issues with lenses, save for the craptastic 50 1.4.

    My issues are with the bodies. Ive had AF failures on every canon body ive owned, save for my pretty new M. I view EVERY image at 100%. Pixel peeper, and proud to be one.

    Every body had to be sent back to canon at least twice, because what came back after electrical adjustments did not work. The 7d i had whacked out after a year, and never regained proper AF. Canon tried and failed 4 times, had to sell it.

    I bought a refurb 6d a few weeks ago. Its on its way to repair. Multiple FoCAL sessions helped, but “never a sharp image the camera would make.” (yoda voice) I went down to bh to try out a demo model- and bam – everything perfect. The Af accuracy is so laughably poor in the one they sold me that its amazing that canon didnt catch it.

    Unlike some other people, i dont like testing out lenses for a week to make they work properly. All of my main lenses, when first purchased, were duds. 24-105, 70-200 2.8 II, and the dreaded 50 1.4.

    At this point every single piece of canon gear i purchase has to go through the rounds.