Results 1 to 14 of 14

Thread: Evaluating Lens or Camera Performance with JPEGs

  1. #1
    Senior Member
    Join Date
    Jan 2002
    Location
    UK
    Posts
    3,911

    Default Evaluating Lens or Camera Performance with JPEGs

    This is just a warning to everyone out there that you cannot make judgements on a camera or lens' performance from low resolution JPEGs. Sure look at them and appreciate them for their worth, but to gauge resolution and sharpness, you need high resolution images on which to make your judgement.

    I've posted three pictures as follows:

    The original



    These are two full resolution crops of the relevant area. The one on the left has had 5 pixel motion blur added to it, resulting in awful full resolution quality. The one on the right is untouched. I've also included the lower resolution version of each crop, the equivalent of a 550 pixel image of the original, as little insets in each image. Try and tell them apart.



    Apologies for the lack of technical/aesthetic quality of the shots but they were just set up quickly to illustrate that at 550 pixels across, you really cannot tell anything about whether a lens is a stellar performer or not. Or even if it is a complete dud or not.

  2. #2
    Deregistered
    Join Date
    Mar 2003
    Location
    ClubSNAP community
    Posts
    2,775

    Default


  3. #3

    Default

    Quote Originally Posted by Azure
    JPEG cannot really be used for evaluation because JPEG is a lossy format, even when high resolution shots are used for comparsion.

    Just bringing up a point.
    Last edited by mervlam; 1st January 2004 at 01:54 AM.

  4. #4
    Senior Member
    Join Date
    Feb 2003
    Location
    The heart of the Abyss
    Posts
    2,307

    Default

    Duh!

    This should be bloody obvious for anyone who has gone past the basics of digital photography . Thanks to Jed to show it with a real example

    Lossy, blurring, 8-bit formats good for generic sharing of photos but hardly the suitable one for any top end work. Furthermore, not every JPG engine is equal.

    Pet Peeve: Graphics web designers use JPG to do icons or text label with plain background <-stupid all the way.

  5. #5
    Senior Member
    Join Date
    Jan 2002
    Location
    UK
    Posts
    3,911

    Default

    Quote Originally Posted by mervlam
    JPEG cannot really be used for evaluation because JPEG is a lossy format, even when high resolution shots are used for comparsion.

    Just bringing up a point.
    Actually, it can.

    For starters, look at the above comparison and tell me that you cannot make a meaningful evaluation from it. And yet that's exactly how bad alleged pictures from a lens currently in the spotlight are.

    Secondly, there shouldn't be major problems at all using JPG to evaluate images as long as compression is not excessive and the compression is equal. This is because JPG works primarily on the basis that the human eye responds less to colour change than it does to visible detail change. So it compresses colour over losing detail/sharpness, so the original detail/sharpness of the image should still remain until compression gets ridiculous. What does get introduced are JPG artifacts, but this is easily distinguishable from original detail/sharpness, so it shouldn't prove an impediment.

    Thirdly, you shouldn't be trying to evaluate colour from JPGs at all, not without a strictly colour controlled environment from capture through to output.

    Fourth, if you have 13Mb to play around with an your web browser displays it or you have a copy of Photo Desk, I would be glad to let you have a copy of the original of the above so you can draw your own conclusions. Or anyone else for that matter.

  6. #6
    Senior Member
    Join Date
    Jan 2002
    Location
    UK
    Posts
    3,911

    Default

    Quote Originally Posted by Watcher
    This should be bloody obvious for anyone who has gone past the basics of digital photography . Thanks to Jed to show it with a real example

    Lossy, blurring, 8-bit formats good for generic sharing of photos but hardly the suitable one for any top end work. Furthermore, not every JPG engine is equal.
    You're welcome. Yeah, you said it not me, obvious, yet there are lots of people on the Internet, and even some on this forum, who show people low res JPGs as evidence of lens quality, and lead many newbies and inexperienced photographers into false conclusions. Not necessarily that the conclusion is false, but that it wasn't reached via proper steps.

    No, you're right that not every JPG engine is created equal, that's why for my workflow with some cameras I don't bother shooting anything other than JPG. And with some I only ever shoot RAW. Some cameras are good enough to handle the JPG transformation to a very good level, others less so.

  7. #7

    Default

    I see. So if I am thinking of buying a lens and want to evaluate its image quality, I should do so using at least a full-sized image (say 6 megapixels) instead of a web-sized picture, which essentially tells me nothing about the lens quality...

    Interesting...

  8. #8

    Default

    What's required is just a 100% crop, not the full sized image.

  9. #9
    Senior Member
    Join Date
    Jan 2002
    Location
    UK
    Posts
    3,911

    Default

    Essentially, yes, but I'm sure you knew that already

    Of course, temper all this with the fact that you need to be aware of your own use. If you're only purchasing the lens to only ever use it to generate web-sized pictures, then you can use those to evaluate it for your needs without any problems. But you should steer clear from general comments about a lens' optical quality unless you've at least looked at high resolution pictures taken with it. To do so otherwise and call the lens excellent or a dream would be potentially misleading to other people.

  10. #10
    Senior Member
    Join Date
    Jan 2002
    Location
    UK
    Posts
    3,911

    Default

    Quote Originally Posted by Zerstorer
    What's required is just a 100% crop, not the full sized image.
    What's required is a GOOD 100% crop, and you need a few of them, one will not suffice. Ideally you need a full res image, but yes, selective 100% crops do a good job. But one thing that a web-sized image will tell you that a 100% crop is unlikely to is anything about lens distortion, although selective 100% crops should cover coma, chromatic aberration, sharpness, performance in corners, edges and centre, light fall off (you need comparison swaths for this), low contrast and high contrast performance.

    If viewing a digital file it's probably also a good idea to keep in mind what camera was used and have a yardstick to judge by. But in general, all my cameras fall short of the quality capable by all of my lenses.

  11. #11
    Senior Member
    Join Date
    Feb 2003
    Location
    The heart of the Abyss
    Posts
    2,307

    Default

    I fully agree with Jed ^^^

    Of course, the best is if pictures taken and selected without bias show case the strength and weakness of a lens. The point is to isolate the camera from the lens itself. The camera should provide as little alteration or at least, unbias information as possible. The only exception is of couse, when the said lens can only be used on a body but even then, that means that in future, it has to be reevaluated again when another body can make use of it.

    Eg the only EF-S lens currently on sale.

    Many think that JPEG is the be all and end all of a lossy compression of a bitmap. This is not even technically true: JPEG is just a file format; the encoding is actually JFIF. In JPEG200 format, it uses either one of the following encoding method: JFIF (FFT-based), the wavelet-based and even a lossless codec!

    Coming back to JPEG/JFIF, the encoding engine follows a standard. However, the quality differs in terms of speed, quality, etc. There are parameters to use to balance (or compromise) between quality and speed.

    Coming back to the original topic of lens evaluation and JPEG/JFIF, the main issue is this: a JFIF file will regardless of implementation, make a image more soft. That is why it is a lossy compression in the first place. There are other issues as well.

    The best way, IMO, is to use good selective 100% crop to showcase the strength and weakness and publish it in PNG format. This lossless, 24 (but can go up to 32)-bit format can be viewed by IE6 and the NS 6 and above IIRC. Most people don't even know that it exist!
    Last edited by Watcher; 2nd January 2004 at 01:00 AM.

  12. #12
    Senior Member
    Join Date
    Sep 2003
    Location
    In this small world
    Posts
    3,142

    Default

    Good points made, learned something new from people who have first hand experiences. My personal experience is JPEG compression (Super) from my G5 is near or almost as near to Raw->Tiff conversion in terms of quality on Photoshop. But I shoot Raw exclusively to learn more about editing of photos in case I need to salvage one some day.

    PNG has no compression (or lossless format?), same or near the same as Tiff? Not sure about that, needs to read up more since I am not a photographer/designer in practise, rather a web programmer.


    Anyway, my practice on the web is not to believe in anything other people say. Just go out and find out the truth, which may be 180 degrees in difference. A lot of properganda is found, from young kids who make statements without thinking, and refusing to accept the true sometimes.

  13. #13
    Senior Member
    Join Date
    Feb 2003
    Location
    The heart of the Abyss
    Posts
    2,307

    Default

    Quote Originally Posted by theITguy
    PNG has no compression (or lossless format?), same or near the same as Tiff? Not sure about that, needs to read up more since I am not a photographer/designer in practise, rather a web programmer.
    You should then know of PNG . It was created as an alternative to GIF when part of the file format was claimed not to be royalty free. So the whole group (W3C ?) decided that for the web, a new format that can do a lot of non-lossy stuff is needed. Why non-lossy? There is already JPEG/JFIF. So they instead made it a replacement for GIF which can only have 1) 256 (8-bit) colors only 2) 1-bit transparency, etc. PNG can do a lot more...

  14. #14
    Senior Member
    Join Date
    Jan 2002
    Location
    UK
    Posts
    3,911

    Default

    Quote Originally Posted by theITguy
    A lot of properganda is found, from young kids who make statements without thinking, and refusing to accept the true sometimes.
    Wise man speaking.

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •