Best Quality for RAW to JPEG conversion

[kineticworks]

What is the most efficient method for RAW to JPEG conversion with highest quality output (16 -> 8bit)?
How do you guys overcome quality loss and reduction in exposure after conversion?

 

[David Kwok]

What is the most efficient method for RAW to JPEG conversion with highest quality output (16 -> 8bit)?
How do you guys overcome quality loss and reduction in exposure after conversion?

JPEG encoding is pretty efficient nowadays. JPEG can only cope with 8bits RGB not 16bits.
You should perform your post processing in your software after loading from the RAW. Quality Loss due to compression is unavoidable, it is a balance you seek between file size and quality. If you have concern about quality lost, you can either choose to save in 16bits TIFF with lossless compression to preserve both the colour bits and image quality, or you can choose to save as lossless jpeg but limited to 8bits colour.

Generally for good jpeg compression, you will first need to remove as much noise as possible. That will give a smaller file size for the same quality setting.
It's a trial and error more than science to it.

 

[kineticworks]

TIFF is not a good choice as the batch of images are for my client (regular desktop viewing or small maybe prints).
Try my best not to reprocess after conversion.

 

[Rashkae]

TIFF is not a good choice as the batch of images are for my client (regular desktop viewing or small maybe prints).
Try my best not to reprocess after conversion.

If that's all they do, you don't need to worry about the "best conversion". Seems like they have low standards. Just do the normal sharpen after resizing.

 

[kineticworks]

Low requirements does not mean low standards. If you are responsible photographer and genuinely proud of your works, you will not settle for acceptable standards for your deliverables.
I'm hoping for experienced photog who can suggest a good workflow that I am not aware off.

@David
Thanks for your filesize saving pointer.

Noise was not a huge problem for that low light situation. However, the JPEG has a darker shadow and midtone compared to the RAW. I understand that quality loss is inevitable but I really hope to minimize the effect.

 

[Rashkae]

Low requirements does not mean low standards. If you are responsible photographer and genuinely proud of your works, you will not settle for acceptable standards for your deliverables.

All depends on how much they are willing to pay. You don't work for hours for peanuts. Responsible photographers who do not want to destroy the industry will maintain payment standards as well.

I'm hoping for experienced photog who can suggest a good workflow that I am not aware off.

I use spatial frequency data separation for the commercial shoots.

 

[David Kwok]

Noise was not a huge problem for that low light situation. However, the JPEG has a darker shadow and midtone compared to the RAW. I understand that quality loss is inevitable but I really hope to minimize the effect.

Characteristic of JPEG is to exploit on human eyes deficiencies. Hence sacrificing on the accuracy of shadow and extreme highlights as well as subsampling on chroma channels is what JPEG depends on to give you good compression with minimal adverse effects on the final product.


There are specialize JPEG encoders that can help to squeeze out slightly more quality out of less capable encoders, but seriously that would complicate your workflow with really negligible benefits from it. I understand that since JPEG is the one of the most common way of sharing your final images with your client, then you will really need to seek a balance between size and quality.


When you clip down from a source 12~14bits RAW to a 8bits images, you are definitely going to loss on tone granularity. Highlights are very much less discernible from the human eyes. Humans seems to be able to see more from the shadow areas or at least we tend to emphasize more on it. There is really nothing much you can do in this aspect. But I'm curious, can you actually make a lot of differences out of it. To be frank, if you client is really so particular about such details, then images that maintain the raw information is the best you can do. Even so, there isn't a lot of display out there that can reveal much in the shadow areas.


If JPEG is your way to go, then I say you just make do with it. Rather than keeping the shadow area which can look distracting if it isn't smooth, a better approach will be to de-noise it so that it is smoother and less distracting to cause discomfort when viewing the overall image. I'm not a professional photographer, but from what I have to study when it comes to imaging, there isn't much you can do about such as it's the technology limitation that you have chosen.


I did a look up in this area since you mentioned and this is what I found. Is this similar to what you are referring to ?
Adobe Forums: Exported JPEGs Distorted in Shadow Areas


That example is where the gamma is brought up a lot to see such an effect, is inherent to JPEG quantization process where bits are discarded from the images. It is especially pronounced in shadow areas, that I'm aware of.


For simplicity, quantization process is where if the original pixel value is 17 for example, it will first divide by a divisor for example, least say it is 3, which yield an integer of 5. Then this value is processed further down the JPEG encoding. During decoding, the reverted value 5 is multiply back to give you 15. The inaccuracy is therefore the side effect from the quantization process. So as you suspect, original values between 15 and 17 all gives you 3 after quantization, hence the originally smooth tone yields back a stepped effect.


Good JPEG decodes attempt to rescue such artifacts too, so at the end of the day, JPEG enc/dec is a combination to achieve good reproduction of the original image.


For your case where you are more concern about the quality from a simplistic perspective, I say you either convince yourself, if not your clients on the expectation on what to expect from the images based on the image file format. Just quote them that based on your professional opinion, if they are concern about image quality, then stick with file format that are designed to preserve good image quality, if not use another that gives good compression but sacrifice on image quality.

 

[David Kwok]

I use spatial frequency data separation for the commercial shoots.

Interesting, didn't know about this technique. After reviewing it and try it, it's seems exactly what Portraiture is performing Nice to know that's the technique they are using.

 

[Rashkae]

Interesting, didn't know about this technique. After reviewing it and try it, it's seems exactly what Portraiture is performing Nice to know that's the technique they are using.

Works well! and if you rally dig into it, you'll get far more control than what Portraiture offers, and the tools are open source (free).

 

[David Kwok]

Works well! and if you rally dig into it, you'll get far more control than what Portraiture offers, and the tools are open source (free).

Tools ? I was experimenting it in PS. Pretty tedious versus sliding levers in the plugin ? keke. But yes I agree, you get a lot more controls. Need quite a fair bit of practice to know where to tweak to get the results one wanted.
None the less, the part where manually tweaking the mask is indeed more control over what Portraiture offers, though I somewhat feel the latest version of it has a bug, adding up the mask using the "+" colour dropper don't seems to work the same compared to a few versions back. Previously it will really add up the mask part by part, now it don't seems to perform the same.

Anyway, that's great info you have reveal! Thanks

 

[kineticworks]

Guess I have to bump up exposure, in order to compensate for loss of details after conversion.

 

[catchlights]

Guess I have to bump up exposure, in order to compensate for loss of details after conversion.

are you referring to downsize low resolution jpg? if yes, please search internet for how to optimize web friendly jpg, such type of images are mostly highly sharpen, highly saturated, high contrast, only good for viewing but will be printed horribly.

anyway, whatever it is, the exposure/wb should nailed on before export to jpg, for whatever useage.

 

[kineticworks]

@Catchlights.
We are referring to full resolution for RAW to JPEG conversion. IQ loss due to downsize will be of another issue
It happened to be a theme party where most guests wored black for an indoor event. Like what David have mentioned, JPEG conversion will sacrifice accuracy of shadow.

This is what everyone is aiming for but at times it may not turn out as expected. My shoot was a fast pace event, not studio 1 to 1 with cam tethered to computer ;p.

exposure/wb should nailed on before export to jpg, for whatever useage

Regretted not shooting RAW+JPEG.

 

[catchlights]

using lightroom to do correction, correct a few and sync to the rest in same/similar scenes, and export to jpg,

 

[David Kwok]

@kineticworks:
Since you have started on this question, hence I'm also interested to know what is the kind of sacrificial in quality when converted from RAW to JPEG. Here I have carried out a couple of experiments and reviewed the changes, they are not scientific review but rather on the basis on observation.

I started with a 16bits grey scale black on the left and white on the right as you can see below:

Now I converted it into 8bits in PS save it as PSD in lossless compression, reload it and then placed it as a separate layer into the same grayscale in 16bits. I did a difference between the 2 layers. On screen, you will not see anything much except everything is in BLACK, means visually they are the SAME. But when you put them under the histogram, and then use levels to adjust to extreme, you will see something like this

As you can see across the full grayscale spectrum, extreme level shows differences of cyan and red. Based on this we can observe changes but that's when the gamma is at extreme scale. Also you will notice the frequency of changes increase along from the black to white region. Having a lower frequency and larger band will means your eyes will more likely to notice the changes in the shadow area versus the highlight areas.

Below is a 100% crop in the shadow area.

It's the extreme shadow and changes are pretty uniform, hence while 16bits to 8bits conversion did indeed lose information, it's the least of your concern since the loss is pretty uniform.

Now lets compare between a 8bits RAW and 8bits JPEG at the highest (12) quality that PS offers and what kind of loss are we experiencing. The loss due to the JPEG comparison is in similar fashion but even at this point you are observe more erratic behaviour, lets go into the 100% crop of the shadow area

Now at the extreme shadow, you see the quantization process gives very bad results after decoding. Also the banding basically stretch across each interval. But still I must say this is pretty good even it's not observable under normal circumstances. This is again an extreme gamma tweaking to reveal the inaccuracy of colour representation due to JPEG encoding.

How about if we choose the medium encoding for JPEG ? I will go straight into the 100% crop as shown below

Now this is drastic right ? Not only is the banding bad looking, but compared to the highest quality, it's not uniform and also the quantization artifacts are very pronounced. But still I must say it doesn't look too obvious in normal circumstances.

Hence back to your issue, now that you mentioned your source is noisy and started with dark scenes, hence you might just wanna de-noise as much as possible for the dark regions so that it doesn't look so bad when encoded using JPEG. But if you keep your JPEG encoding to the highest level, I say it should be good enough for your client. Your client eyes are not histogram and also they should refrain from extreme post processing on the JPEG media since it is lossy in nature and every re-save will loss more information.

I hope this helps for you.

 

[hairyice]

@kineticworks:
Since you have started on this question, hence I'm also interested to know what is the kind of sacrificial in quality when converted from RAW to JPEG. Here I have carried out a couple of experiments and reviewed the changes, they are not scientific review but rather on the basis on observation.

I started with a 16bits grey scale black on the left and white on the right as you can see below:

Now I converted it into 8bits in PS save it as PSD in lossless compression, reload it and then placed it as a separate layer into the same grayscale in 16bits. I did a difference between the 2 layers. On screen, you will not see anything much except everything is in BLACK, means visually they are the SAME. But when you put them under the histogram, and then use levels to adjust to extreme, you will see something like this

As you can see across the full grayscale spectrum, extreme level shows differences of cyan and red. Based on this we can observe changes but that's when the gamma is at extreme scale. Also you will notice the frequency of changes increase along from the black to white region. Having a lower frequency and larger band will means your eyes will more likely to notice the changes in the shadow area versus the highlight areas.

Below is a 100% crop in the shadow area.

It's the extreme shadow and changes are pretty uniform, hence while 16bits to 8bits conversion did indeed lose information, it's the least of your concern since the loss is pretty uniform.

Now lets compare between a 8bits RAW and 8bits JPEG at the highest (12) quality that PS offers and what kind of loss are we experiencing. The loss due to the JPEG comparison is in similar fashion but even at this point you are observe more erratic behaviour, lets go into the 100% crop of the shadow area

Now at the extreme shadow, you see the quantization process gives very bad results after decoding. Also the banding basically stretch across each interval. But still I must say this is pretty good even it's not observable under normal circumstances. This is again an extreme gamma tweaking to reveal the inaccuracy of colour representation due to JPEG encoding.

How about if we choose the medium encoding for JPEG ? I will go straight into the 100% crop as shown below

Now this is drastic right ? Not only is the banding bad looking, but compared to the highest quality, it's not uniform and also the quantization artifacts are very pronounced. But still I must say it doesn't look too obvious in normal circumstances.

Hence back to your issue, now that you mentioned your source is noisy and started with dark scenes, hence you might just wanna de-noise as much as possible for the dark regions so that it doesn't look so bad when encoded using JPEG. But if you keep your JPEG encoding to the highest level, I say it should be good enough for your client. Your client eyes are not histogram and also they should refrain from extreme post processing on the JPEG media since it is lossy in nature and every re-save will loss more information.

I hope this helps for you.

Darn, this is one neat experiment. Thank you very much!