The Problem with Proprietary RAW files

jUzMoO · 26th April 2005

Hi,

Dunno if this a right thread to post this. Check out this OpenRaw site. What do you think? Are shooters/consumers like us gonna suffer in the future?

Or is there advantages about proprietary file format from each company?

Let's heard from your views then...

loupgarou · 26th April 2005

erm.

a) photoshop supports all raw formats in existence I believe.

b) adobe has the dng format and convertors to the dng format. whether the raw -> dng format convertor program exists/is free, I havent' checked since I don't use it. but I guess any third party application which can process dng wouldn't be left out.

c) the only problem proprietary raw formats cause are to application developers, not end users. because all 3rd party programs (like capture 1) need to update to support them anyway.

d) is the dng format free? (ie: no licensing charges to the camera makers)

e) are camera manufacturers restricted to making changes if they stick to a common standard raw format?

--
I think point e is the most pertinent, can you imagine what would happen if a camera sensor was developed that can record in 64 bit mode / or in CMYK , would any "common raw format" currently even accomodate that?

jopel · 26th April 2005

adobe has problem now with nikon over the raw issue.

as comsumer we can't do much accept to buy that which we think is good in term of it performance and support.

loupgarou · 26th April 2005

so don't buy nikon cameras. 8)

theITguy · 26th April 2005

There is a reason why I still shoot slide film instead of going digital (yeah I am poor in fact to go digital).

Watcher · 26th April 2005

Then don't buy Canon since Canon decided to stop support for D30 cameras in its new DPP even though Nikon's D1 series was launched 15 months earlier...

erwinx · 28th April 2005

Frankly, its no big loss if Canon stopped support of D30 in DPP, it has freely released the SDK to any third party who wants to support it, eg: Breezebrowser, Captureone, photoshop etc.

Breezebrowser is a great example of Canon's enlightened approach. It is basically a simple third party application enhancing Canon's SDK programmed by an enthusiast and competes directly with DPP. Canon could easily make life hard for such small software houses through 'encryption' or 'restrictive licensing' but it didn't.

Watcher · 28th April 2005

Originally Posted by erwinx

Frankly, its no big loss if Canon stopped support of D30 in DPP, it has freely released the SDK to any third party who wants to support it, eg: Breezebrowser, Captureone, photoshop etc.

Breezebrowser is a great example of Canon's enlightened approach. It is basically a simple third party application enhancing Canon's SDK programmed by an enthusiast and competes directly with DPP. Canon could easily make life hard for such small software houses through 'encryption' or 'restrictive licensing' but it didn't.

Hmmm, 2 Points:

1) Nikon freely releases the SDK to third parties too. See IMatch as one example. So others too can support even troublesome parts of the RAW files. The SDK was offered to Adobe btw, and Adobe rejected it. If Canon officially stops the support of D30, I wonder when they will stop supporting it on the SDK too. I mean, if supporting it on the SDK is trivial, then why not continue to support it in DPP as well?

2) From the maker of dcraw, the opensource RAW converter, the interview on DPReview, he basically said most camera makers obfruscate the metadata. If you see point 5, where he was asked "Which RAW format was the first you worked on which showed signs of having its metadata deliberately encrypted / obsfuscated? Can you give us examples of other formats which have been made 'hard to decode' by the manufacturers?", he said:

Quote:

The Canon PowerShot G6, S60, S70, and Pro1 apply a trivial XOR to the metadata related to color balance. Phase One encrypts the entire image in a slightly more complicated way.

So it seems that Canon is leading the way on this issue, encrypting metadata. We can all thank Canon for being in the forefront.

loupgarou · 28th April 2005

apparently trivial xor (you do know what xor is right) and a possibly dmca protected whitebalance encoding/encryption is the same to you.

8)

it is entirely logical to based a purchasing decision with the phasing out dpp support of their "first" generation digital slr, , despite the fact that adobe and most raw convertors still work and many people don't even use dpp at all, or the fact the original free raw convertor software that came with your camera still works.

this profundity of logic also means that one shouldn't buy microsoft windows XP because windows 98 is phased out and unsupported, or one shouldn't buy OSX tiger because mac os 8.6 is phased out.

8)
---
in anycase, shouldn't purchasing decisions be determined by what you consider important? so if you consider encryption of meta data and not being supported natively by adobe photoshop to be a big issue, then shouldn't that be valid to not buy?

BUT, in this case, read http://www.dpreview.com/news/0504/05...ninterview.asp

Quote:

. Are you ever concerned about the legal implications of reverse-engineering proprietary file formats?

If anyone sued me, I'd be the biggest free software hero since Jon Johanson. It's better for the camera makers to ignore me and hope I lose interest.

so its crackable (as is anything), so unless nikon doesn't care to sue indefinitely, then adobe can do the same thing, but given the litiginous business environment in the states, I doubt it.

since nikon says its SDK is free for use by bona fide software companies, why isn't adobe biting?

Watcher · 28th April 2005

Originally Posted by loupgarou

apparently trivial xor (you do know what xor is right) and a possibly dmca protected whitebalance encoding/encryption is the same to you.

A lock is a lock. DMCA makes no difference in the difficulty of the encryption. Let's put it this way: if you break a lock to enter a house illegally, it is a felony on breaking and entering, regardless of the lock.

And if you read the code, the WB "encryption" is:

Originally Posted by dcraw.c

ci = xlat[0][serial & 0xff];
cj = xlat[1][fgetc(ifp)^fgetc(ifp)^fgetc(ifp)^fgetc(ifp)];

You know what "^" is in C, right? For those who are not conversant in C, it is the operator, xor, the one mentioned above as mentioned above. So the "strength" are both the same "trivial", but again, who came first and set the precedence? Canon.

Originally Posted by loupgarou

it is entirely logical to based a purchasing decision with the phasing out dpp support of their "first" generation digital slr, , despite the fact that adobe and most raw convertors still work and many people don't even use dpp at all, or the fact the original free raw convertor software that came with your camera still works.

this profundity of logic also means that one shouldn't buy microsoft windows XP because windows 98 is phased out and unsupported, or one shouldn't buy OSX tiger because mac os 8.6 is phased out.

Well, Win16 or Win32 programs written in even in 1992 can mostly run now on Win XP. Image processing software == OS, camera == Programs bought. So 13 years of support is definately longer than oh 5 years. D1 was released 15 months before and is still supported by both official software (NC), SDK-based (IMatch) and unofficial (ACR) ones.
<chopped>

Originally Posted by loupgarou

so its crackable (as is anything), so unless nikon doesn't care to sue indefinitely, then adobe can do the same thing, but given the litiginous business environment in the states, I doubt it.

s/nikon/canon/g and both are in the same situation. Read this and this

Originally Posted by loupgarou

since nikon says its SDK is free for use by bona fide software companies, why isn't adobe biting?

Because Adobe felt that they want to have the full power of the RAW instead of going through an abstraction.

timlim · 29th April 2005

Originally Posted by Watcher

A lock is a lock. DMCA makes no difference in the difficulty of the encryption. Let's put it this way: if you break a lock to enter a house illegally, it is a felony on breaking and entering, regardless of the lock.

so you did get it after all!

http://forums.clubsnap.org/showthrea...1&page=2&pp=20

Watcher · 29th April 2005

Originally Posted by timlim

so you did get it after all!

http://forums.clubsnap.org/showthrea...1&page=2&pp=20

I got it all along

In this discussion, I was giving the full benefit of the doubt and took the assumption or the stand (for the sake of this debate and background to the quote) that XOR (aka "^") is encryption, thus my comments and the line of reasoning. Many already made up their minds on DPReview and here that XORing is encryption so base on the popular standard, I took that level for the sake of discussion

.

However, I personally don't think XOR is encryption, since it is a many-to-one mapping in a domain of 16-bit values that at worse, can be made into a LUT. There are heaps of encryption that are more secure than just XORing it without changing the size nor the complexity or speed of the camera. IMPO, it is more an obfruscation, rather than encryption.

nightpiper · 29th April 2005

for what i understand, companies make their RAW file proprietary is becos they want to safe keep their techology secrets (both strength & flaws). i do know Oly & Nikon has the best quality & colour if u use their s/w. thats why, i m still sticking to Oly's s/w to do conversion.

did a pseudo comparison with different converters & the native support from camera makers still win in the quality dept.
http://forums.clubsnap.org/showthread.php?t=127113

i dun see any 'open source' for RAW format in the coming 10yrs, my guess is that Adobe DNG format is not going to be widely used by manufacturers unless they have ways to keep their secrets in that format & at no cost in adapting it. maybe manufacturers can use DNG for free but 3rd party who writes converters will have to pay Adobe. either way, i dun see a common format coming soon. so i guess its up to u to gauge if its worth the price.

erwinx · 13th May 2005

Latest release from Phase One/Capture One:

"Capture One does not support the Nikon D2X white balances, due to the encrypted white balance format recently introduced by Nikon Corporation. We have emulated all the white balance presets so that your images will look very similar to the original Nikon D2X white balance presets. If you have shot your images using auto white balance, Capture One will automatically perform an auto white balance one your images when they are opened for the first time. If you have shot your images with a custom white balance, you will have to adjust the white balance manually in Capture One."

26th April 2005	#1
jUzMoO Member Join Date: Aug 2004 Location: Singapore Posts: 396	The Problem with Proprietary RAW files Hi, Dunno if this a right thread to post this. Check out this OpenRaw site. What do you think? Are shooters/consumers like us gonna suffer in the future? Or is there advantages about proprietary file format from each company? Let's heard from your views then...

26th April 2005	#2
loupgarou Member Join Date: Sep 2003 Posts: 1,794	erm. a) photoshop supports all raw formats in existence I believe. b) adobe has the dng format and convertors to the dng format. whether the raw -> dng format convertor program exists/is free, I havent' checked since I don't use it. but I guess any third party application which can process dng wouldn't be left out. c) the only problem proprietary raw formats cause are to application developers, not end users. because all 3rd party programs (like capture 1) need to update to support them anyway. d) is the dng format free? (ie: no licensing charges to the camera makers) e) are camera manufacturers restricted to making changes if they stick to a common standard raw format? -- I think point e is the most pertinent, can you imagine what would happen if a camera sensor was developed that can record in 64 bit mode / or in CMYK , would any "common raw format" currently even accomodate that? Last edited by loupgarou; 26th April 2005 at 10:48 AM.

26th April 2005	#3
jopel Senior Member Join Date: Dec 2004 Location: Punggol Central Posts: 1,015	adobe has problem now with nikon over the raw issue. as comsumer we can't do much accept to buy that which we think is good in term of it performance and support. __________________ Photography is all about you.

26th April 2005	#5
theITguy Senior Member Join Date: Sep 2003 Location: In this small world Posts: 2,042	There is a reason why I still shoot slide film instead of going digital (yeah I am poor in fact to go digital).

26th April 2005	#6
Watcher Senior Member Join Date: Feb 2003 Location: The heart of the Abyss Posts: 2,319	Then don't buy Canon since Canon decided to stop support for D30 cameras in its new DPP even though Nikon's D1 series was launched 15 months earlier...

28th April 2005	#7
erwinx Senior Member Join Date: Jan 2002 Posts: 2,492	Frankly, its no big loss if Canon stopped support of D30 in DPP, it has freely released the SDK to any third party who wants to support it, eg: Breezebrowser, Captureone, photoshop etc. Breezebrowser is a great example of Canon's enlightened approach. It is basically a simple third party application enhancing Canon's SDK programmed by an enthusiast and competes directly with DPP. Canon could easily make life hard for such small software houses through 'encryption' or 'restrictive licensing' but it didn't.

29th April 2005	#13
nightpiper Member Join Date: Oct 2003 Posts: 1,801	for what i understand, companies make their RAW file proprietary is becos they want to safe keep their techology secrets (both strength & flaws). i do know Oly & Nikon has the best quality & colour if u use their s/w. thats why, i m still sticking to Oly's s/w to do conversion. did a pseudo comparison with different converters & the native support from camera makers still win in the quality dept. http://forums.clubsnap.org/showthread.php?t=127113 i dun see any 'open source' for RAW format in the coming 10yrs, my guess is that Adobe DNG format is not going to be widely used by manufacturers unless they have ways to keep their secrets in that format & at no cost in adapting it. maybe manufacturers can use DNG for free but 3rd party who writes converters will have to pay Adobe. either way, i dun see a common format coming soon. so i guess its up to u to gauge if its worth the price.

13th May 2005	#14
erwinx Senior Member Join Date: Jan 2002 Posts: 2,492	Latest release from Phase One/Capture One: "Capture One does not support the Nikon D2X white balances, due to the encrypted white balance format recently introduced by Nikon Corporation. We have emulated all the white balance presets so that your images will look very similar to the original Nikon D2X white balance presets. If you have shot your images using auto white balance, Capture One will automatically perform an auto white balance one your images when they are opened for the first time. If you have shot your images with a custom white balance, you will have to adjust the white balance manually in Capture One."