#1  
Old 05-12-2019, 08:45 PM
The_bluester's Avatar
The_bluester (Paul)
Registered User

The_bluester is offline
 
Join Date: Feb 2011
Location: Kilmore, Australia
Posts: 3,342
Camera gain and ADC bit count

Just looking to check my thinking, I was actually considering posing this question to ZWO.

My ASI294MC Pro has 65K (Approx) pixel wells. It also has a 14 bit ADC. That means approx 16K output levels.

I have been working to improve my star colours for a while. They have a tendency for the brighter stars to look blown out and white even at relatively mild stretches.

I was looking at the ADC specs and I wonder if anyone else would share my thinking. At unity gain (Which approximates 1 electron = 1 ADC count) any pixel well over approx 16K electrons will be effectively saturated as the ADC has no more to give? The output files still show 65K levels, so I am assuming some multiplication factor is at play in the camera firmware to expand the output range to match the pixel well depth. It is irrelevant though it 65K output actually equals 16K off the ADC.

The way I am seeing it, to get much better results I need to layer in shorter subs for stars and bright areas to keep them from hitting more than 16K and effectively saturating as while I can spread the dynamic range by reducing gain, the data becomes non linear as more than one electron is required to change the output of the ADC, up to 4 in fact.

The gain setting looks to me to be something that is masking an ADC that really needed to be 16 bit to begin with, the ASI1600 would be even worse with 12 bit ADC, it sort of explains why they seem to produce great results out of short subs, anything longer and the output effectively saturates.

Should I just grit my teeth and save up for a 16 bit camera like the coming ASI2600? I actually have my eye on those ones but I will be letting others take the plunge first to see if they have cooling or flat fielding issues like my 294. An ADC that can cover the entire well depth of the pixels would be a pretty big advantage as I see it.

Anyone reckon I am just flat wrong in thinking that they have to be effectively wasting a lot of the sensor dynamic range on cameras like the ASI294MC with a 14 bit ADC?
Reply With Quote
  #2  
Old 05-12-2019, 09:33 PM
Atmos's Avatar
Atmos (Colin)
Ultimate Noob

Atmos is offline
 
Join Date: Aug 2011
Location: Melbourne
Posts: 6,980
The ADC actually doesn’t matter that much as what’s more important is the dynamic range of the camera. Take the KAF-8300 sensor, it has about 11.3 stops of dynamic range and a 16-bit output. Given that it has less than 12 stops it doesn’t matter if it’s 12-bit or 16-bit.

Converting from 14-bit to 16-bit involves a 4x multiplication. Let’s say you have a well depth of 65,000 and a pixel fills to 48,618. You’re working with a gain of 0.252e/ADU. This correlated to a value of 12,251.736 or 12,252 ADU. It’s then multiplied 4x to get to the 16-bit values used in a FITS file and 48,607 ADU. So, there is a little absolute inaccuracy introduced from all of the converting but that all gets averaged out over multiple exposures.
Reply With Quote
  #3  
Old 05-12-2019, 10:05 PM
The_bluester's Avatar
The_bluester (Paul)
Registered User

The_bluester is offline
 
Join Date: Feb 2011
Location: Kilmore, Australia
Posts: 3,342
that more or less covers what I reckon the limitation is. The IMX294 sensor is advertised as having approx 64ke pixel Wells, but the ADC on the camera is 14 bit, so anything above approx 16k count on the sensor will max the conversion at unity gain. I have found results to generally be better at unity gain than at lowest gainwith better faint detail, but brighter stars blow out.

It more or less comes down to good discrimination of levels with reduced effective dynamic versus a high apparent dynamic range but an effective loss of QE. Does it matter in the end if it takes say one versus four photons to produce an electron for 100 versus 25% QE, or 100% QE (Perfect world example) and one versus four electrons per ADC count? It still equates to a significant loss of discrimination regards to captured light versus data.

Your KAF 8300 example is a good one. The ADC can faithfully represent the entire dynamic range of the sensor as the ADC range is greater. Many (most?) of the current CMOS cams are the other way around with the ADC having a smaller range than the sensor.
Reply With Quote
  #4  
Old 05-12-2019, 10:25 PM
Atmos's Avatar
Atmos (Colin)
Ultimate Noob

Atmos is offline
 
Join Date: Aug 2011
Location: Melbourne
Posts: 6,980
Take the ASI094 I have, it has a 56k full well which means it’s at a gain of 0.293 which means that it takes 3.4 electrons for every ADU. This has nothing to do with QE and it also isn’t a bad thing. At gain 0 my camera has bang on 14 stops of dynamic range which means its using all 14-bits of its output. This is what’s important. At unity gain I have about 13.4 stops of dynamic range so it’s actually more difficult to control bright stars compared to the background.
Reply With Quote
  #5  
Old 05-12-2019, 11:13 PM
The_bluester's Avatar
The_bluester (Paul)
Registered User

The_bluester is offline
 
Join Date: Feb 2011
Location: Kilmore, Australia
Posts: 3,342
My impression from using the ASI294MC for about a year is that in practice it isn’t working like that. Unity gain has proven to bring out faint detail better using the same sub lengths as before, where zero gain previously showed better star colours on brighter stars but the subtle stuff visibly lost out.

The way I see it is this. 14 bits to 16 bits for the fits file, fine. Multiply it by four to fill the data set, you can rebuild the original data by simple division. The issue is that reducing gain to fit the dynamic range of the sensor into a smaller ADC range is reducing resolution to fit the data in. On the ASI294 at zero gain effectively only every fourth electron is recorded, similar to dividing the output excepted that you can’t multiply by four to get back to what the sensor read was. Every ADU count carries uncertainty of up to four electrons and that information has been discarded.


IMO, only recording every fourth electron to fit the range of the ADC is very much the same effect as having an ADC which can represent in discrete steps every electron each pixel can carry but having a less efficient sensor that records less of the photons striking it.
Reply With Quote
  #6  
Old 05-12-2019, 11:41 PM
The_bluester's Avatar
The_bluester (Paul)
Registered User

The_bluester is offline
 
Join Date: Feb 2011
Location: Kilmore, Australia
Posts: 3,342
Just to add to what I wrote, I found a post on CN which stated what I am thinking in perhaps clearer terms.

If you have an ADC with significantly fewer discrete levels than the pixel electro count then by reducing gain reduces precision in the data as it is effectively division with rounding and the net effect is a noisier output. I could certainly see that in the faint stuff when I used to run at zero gain and even ZWO’s specs seem to support that with approx 7.5e read noise at lowest gain versus around 1.5e at higher gain. The trade off for me has been flat looking stars versus noisy looking background and more noisy faint signal.
Reply With Quote
  #7  
Old 06-12-2019, 10:54 AM
Camelopardalis's Avatar
Camelopardalis (Dunk)
Drifting from the pole

Camelopardalis is offline
 
Join Date: Feb 2013
Location: Brisbane
Posts: 5,425
OK, I’ll bite ... It’s not like they’re cheating you or anything

If the sensor only has 14-bit dynamic range, it only needs a 14-bit ADC. The 6200 is IIRC the first CMOS sensor to come near astro consumers that has the full well depth large enough and with sufficiently low read noise to effectively utilise a 16-bit ADC, even then, it doesn’t absolutely need it. On other sensors, this subtle mystical resolution from a 16-bit ADC of which you speak would just be lost in the noise.

Any rounding “errors” in the data are taken care of during stacking == averaging.

My take on it is not to drop the gain beyond the peak DR sweet spot, as that’s where posterisation/quantisation errors can creep in.
Reply With Quote
  #8  
Old 06-12-2019, 11:30 AM
The_bluester's Avatar
The_bluester (Paul)
Registered User

The_bluester is offline
 
Join Date: Feb 2011
Location: Kilmore, Australia
Posts: 3,342
There is the rub, the sensor in the 294 has four times the dynamic range as the ADC it is supplied with. The effective change to the noise profile is pretty visible, at the gain that brings the highest dynamic range the read noise is about four times what it is at unity gain. At unity gain, bright stars will be effectively saturated quite quickly when the ADC runs out of range. It sort of comes down to a choice between nice looking stars or a nice background and faint detail. Or possibly a workflow that looks like it was made by a glass blower with hiccups to lay in nice, unsaturated stars on a nice, smooth, low noise background.

I a not feeling cheated, they are a entry level camera after all and I think that it has produced some really nice results for me, sort of the story of my whole setup. Lots of work to produce reasonably solid results out of “modest” gear.

My real take out I think is that if you can afford it you will get the best out of a camera where the ADC range matches the sensor range. 5 minutes around M42 probably has a lot greater dynamic range than you would get in the IMX294’s native use as a security camera sensor.
Reply With Quote
  #9  
Old 06-12-2019, 02:48 PM
Camelopardalis's Avatar
Camelopardalis (Dunk)
Drifting from the pole

Camelopardalis is offline
 
Join Date: Feb 2013
Location: Brisbane
Posts: 5,425
I think you misunderstand how dynamic range is measured...

Dynamic range is the ratio of the highest signal to the lowest signal level, not the number of absolute steps in between, since it is equally applicable to analogue waveforms as digital signals. The well depth is the highest signal level, whereas the read noise (and any other noise contributions) will determine the lowest detectable signal - but the latter must be distinguishable above the noise, hence why lower noise cameras are more desirable than noisier ones, regardless of well depth.

In the case of our sensor situation, the 294 has deep wells but the read noise is such that it limits the dynamic range to 13 stops (bits)...therefore, a higher resolution ADC is not necessary, by definition. The 294 is complicated somewhat by the HCG mode - it's actually pretty noisy before HCG kicks in at unity.

As in Colin's example above, the popular KAF-8300 may have a 16-bit ADC but is limited to relatively meagre (by today's standards) dynamic range, on paper. That hasn't prevented owners from putting out millions of great pictures with them over the years though

The ASI1600, for example, is capable of a solid 12 stops of dynamic range (at around gain 75)...this is a case where the sensor is indeed limited by the 12-bit ADC...not an ideal scenario, but doesn't prevent effective use. Likewise the 183...even with its tiny pixels, it can muster 12 stops of dynamic range (again the limit of the ADC).

It's probably not worthwhile getting hung up on the ADC issue, as it's less significant that it might first appear.
Reply With Quote
  #10  
Old 06-12-2019, 03:59 PM
The_bluester's Avatar
The_bluester (Paul)
Registered User

The_bluester is offline
 
Join Date: Feb 2011
Location: Kilmore, Australia
Posts: 3,342
I am not hung up on it as such, just that the 14 bit ADC is ultimately a limitation on the camera. From what I have found you have to make a compromise, either you can make use of up to the full dynamic range of the sensor by reducing gain, with a penalty in read noise and discrimination as information is effectively discarded by the division and rounding to a whole number (Using the term discrimination in place of resolution as it will just be a confusion) or, make use of the best noise profile and most faithful input/output conversion for faint stuff at the effective expense of dynamic range, as the ADC output will be maxed at approx a quarter of the sensors well depth. It is a question that will never be answered as what Sony built it with is what they built it with, but I would bet you a nice bottle of scotch that it would be able to produce measurably better results with a 16 bit ADC.


In my case it will be more a matter of informing my next buying decision rather than being grumpy about the one I have. I still enjoy the camera and will be using it for many months yet, but if I dropped it in the dark and it was not economic to repair, I would buy a different cam as a replacement.
Reply With Quote
  #11  
Old 06-12-2019, 04:49 PM
The_bluester's Avatar
The_bluester (Paul)
Registered User

The_bluester is offline
 
Join Date: Feb 2011
Location: Kilmore, Australia
Posts: 3,342
Out of interest I had a look at some old data today, the noise difference between 30 second subs at zero gain and 120 gain (Just above unity where the high gain conversion mode starts, however it does what it does) is quite stark
Reply With Quote
  #12  
Old 06-12-2019, 05:07 PM
Camelopardalis's Avatar
Camelopardalis (Dunk)
Drifting from the pole

Camelopardalis is offline
 
Join Date: Feb 2013
Location: Brisbane
Posts: 5,425
Sony knew what they were doing.

What you're discussing is well depth, not dynamic range.

CCDs with with well depths beyond 65535 BUT with 16-bit ADCs are not hard to find.
Reply With Quote
  #13  
Old 06-12-2019, 06:19 PM
The_bluester's Avatar
The_bluester (Paul)
Registered User

The_bluester is offline
 
Join Date: Feb 2011
Location: Kilmore, Australia
Posts: 3,342
I think that this is going around in circles.


From my point of view, understanding what input saturates the output file is the important part here. At zero gain on the ASI294 and using a fictional "Perfect" sensor where every arriving photon is converted to an electron, approx 65K photons are needed to saturate the system but it is effectively only sensitive to every fourth one and the other three won't be reflected in the output file, you might overcome that with integration of heaps and heaps of subs. At unity gain, only about 16K photons are needed. Faint stuff will be better recorded but brighter stars end up as white blobs as over 16K arrive in each colour channel, anything beyond that is discarded. "Range" wise, one gain is able to record 65K and the other, 16K photons and that is what I am most interested in. I am not concerned as much with "Mystical" discrimination that comes with actually reporting each electron as a seperate ADU step, but the fact that you can faithfully record and capture in the output file on the same image both a feature which produces just enough electrons to overcome read noise by one electron AND a feature which produces as many electrons as the pixel well can hold, and everything in between.


I am never going to tell anyone else what cam they should buy, but any future ones where the ADC count does not at least match the sensor pixel wells won't have me reaching for my wallet.


And yes, Sony knew what they were doing when they designed this sensor, for a quite different application as a security camera sensor, I am just happy we have clever engineers scouring the lists of vendors for sensors than can be re purposed as the innards of astro cameras..

Last edited by The_bluester; 06-12-2019 at 06:52 PM.
Reply With Quote
  #14  
Old 06-12-2019, 07:27 PM
Camelopardalis's Avatar
Camelopardalis (Dunk)
Drifting from the pole

Camelopardalis is offline
 
Join Date: Feb 2013
Location: Brisbane
Posts: 5,425
Going in circles...because your belief of what dynamic range is, is wrong

To answer your original question: yes, you are flat wrong in thinking Sony wasted DYNAMIC RANGE by equipping the IMX294 with a 14-but ADC.

Better now?
Reply With Quote
  #15  
Old 06-12-2019, 08:12 PM
The_bluester's Avatar
The_bluester (Paul)
Registered User

The_bluester is offline
 
Join Date: Feb 2011
Location: Kilmore, Australia
Posts: 3,342
OK, I will pose a question, what term would you use to define the difference between the smallest and largest recordable signals for the system? I will add that I don't really care what you want to call what I am measuring here. In the context of my original post it is about as relevant as an ex workmate of mine who went to work for NBN and started telling people that they should not voice their opinion if they did not know that the NBN box on the wall was a "Network Terminating Device" not a "Network Terminating Unit"



From my own data and Colins post here regards the 14 bit data value being multiplied by 4 for use in the 16 bit fit files I can show that the useful range of the camera as a system at just above unity gain, where the noise profile improves markedly (And I can tell you, visibly) is approximately one quarter of the full well depth of the sensor, ignoring noise and dark current for simplicity. An area of a 30 second sub I have yields an ADU off the sensor of approx 3500 (Divide the 16 bit by four) Again ignoring dark current and noise, a 300 second sub (Which was shot right after the 30 second one) would yield about half of the full well depth, call it 35,000. except from around 150 seconds exposure time the feature in question will hit the conversion limit at unity gain and that is that, though the sensor will go on accumulating electrons.



Given that the noise profile improves so much around unity gain and that the output effectively saturates at about a quarter of the sensors range I would argue that it is demonstrable that the 14 bit conversion is a limiting factor for astro use, and a symptom of that appears to be saturated and relatively colourless stars that are the reason for my OP. And note that I have never said that Sony "Wasted" dynamic range by designing it with 14 bit conversion or that they don't know what they are doing, they did not design the chip for this application and for that we can be thankful as it means there are cameras out there that people can actually afford that produce some pretty good results.


Again, I don't care what anyone else wants to buy, if they make a camera with 100,000e wells and 12 bit ADC and people want to buy it that is up to them if it has some compelling selling point. I won't be buying a new camera for this purpose in future unless the ADU count meets or exceeds the pixel depth as it is one less compromise to consider, in the mean time I will understand where the ASI294's limitations are and work within them, the known flat fielding and uneven cooling issues are one, this is another.

Last edited by The_bluester; 06-12-2019 at 09:05 PM.
Reply With Quote
  #16  
Old 06-12-2019, 10:24 PM
Camelopardalis's Avatar
Camelopardalis (Dunk)
Drifting from the pole

Camelopardalis is offline
 
Join Date: Feb 2013
Location: Brisbane
Posts: 5,425
So what you're saying is that you don't want technical answers to technical questions expressed using the correct technical terminology, or your terminology corrected to correlate with the technical terminology, even when they are overlapping terms

In which case, I won't waste either of our time by caring to answer any further....
Reply With Quote
  #17  
Old 07-12-2019, 08:21 AM
The_bluester's Avatar
The_bluester (Paul)
Registered User

The_bluester is offline
 
Join Date: Feb 2011
Location: Kilmore, Australia
Posts: 3,342
I will go back first to my question in the post above.

What term would you use to define a measurement of the difference between the minimum input signal and maximum input signal which the system as a whole can capture and then output without modification in any given configuration?


And to state it flat, none of this going around in circles about terminology goes anywhere near a point of contact with my original post, which is a question about early effective saturation of the output in a given condition related to a 14 bit ADC.
Reply With Quote
  #18  
Old 07-12-2019, 11:19 AM
Atmos's Avatar
Atmos (Colin)
Ultimate Noob

Atmos is offline
 
Join Date: Aug 2011
Location: Melbourne
Posts: 6,980
When I get some time later on today I’ll go into some detail but quickly; dynamic range is a description of the number of discreet usable steps in the data.

Take my ASI094 at Gain 0. It has a well depth of about 55,000e- and a read noise of 3.4e- RMS. it’s dynamic range is 55000/3.4= 16,176 steps

It has a 14-BIT ADC which has 2^14 discreet steps available or 16,364 steps.

Or the KAF-16803 has 100,000 well depth and maybe 8.5e- RMS read noise.
100,000/8.5= 11,765 BUT it has a 16-Bit ADC with 65,536 steps.

Neither of these cameras need more than a 14-Bit ADC due to the number of steps in the data produced by the sensor.

As I said earlier there is a little imprecision introduced by having the 14-bit ADC in a 16-bit FITS file BUT this is LESS than the photon shot noise and given a handful of exposures averages itself out anyway.
Reply With Quote
  #19  
Old 07-12-2019, 11:50 AM
The_bluester's Avatar
The_bluester (Paul)
Registered User

The_bluester is offline
 
Join Date: Feb 2011
Location: Kilmore, Australia
Posts: 3,342
And having spent some hours mowing for the fire season then some more researching, I am still at the same point.

In my view the 14 bit ADC imposes inherent limitations on the camera for this application, just as the 12 bit in the ASI1600 and similar cameras. Regardless of saying that dynamic range is expressed as a ratio, in this context of these you can relate it back to absolute numbers because they are what you use to determine the ratio. It eventually has to go back to real numbers or it is a measurement without a unit and a completely useless abstraction.

Back to my original question/discussion point. The 14 bit ADC is used to quantise the input signal. At just above unity gain the read noise is markedly reduced, that wins back dynamic range on paper as the ratio between the minimum signal that can be discriminated from the noise floor (the "grass" in RADAR terms, I was a RADAR tech) and the maximum that can be measured is back to approximately what it is at lowest gain, but the absolute maximum (Which matters) has been reduced by two stops, it is a quarter of what it was before, as is the maximum.

Assuming the same noise profile existed, would it or would it not be an advantage (And potentially a big step in dynamic range) if it had a 16 bit ADC and the measurement range did not go from approx 7 to 65K electrons at minimum gain to 1.7 to 16K at unity but 1.7 to 65K at unity instead? The dynamic range would potentially improve by another two stops around unity gain and THAT is what I am interested in the discussion of.

And again, I am not critiquing Sony, who designed the sensor for a very different application, nor ZWO (And others) for adapting it to use in an astro cam and have to take what Sony dish up and make the best of it, just what limitations apply to particular hardware and what improvement changes might make make.
Reply With Quote
  #20  
Old 07-12-2019, 12:32 PM
Atmos's Avatar
Atmos (Colin)
Ultimate Noob

Atmos is offline
 
Join Date: Aug 2011
Location: Melbourne
Posts: 6,980
The answer to that in short is no, it would not make any difference at all. Going from a 14-bit ADC to a 16-bit ADC does nothing but change the gain setting.

Just as I described a few posts back that there is some error introduced (about 0.05%) going from 60k electrons to 16k ADU bits to 65k ADU. That same amount of error is caused with a 16-bit ADC with a camera with 12.5 bits of dynamic range. Because there is a lot more values in the file than there is coming off the sensor it just gets spreads out more but the error is still there.

In short, having more bits doesn’t mean it has more dynamic range when the bits are only being spread out in a less precise manor.

EDIT:
How about I put it this way. With a 14-bit ADC you end up with values in blocks of 4; 220,224,228,232... ect.
This creates a blocky kind of error.

With a 16-bit ADC you end up with a smooth profile with more random error. This means you’ll end up more discreet values (not blocks of 4) BUT each value has more error. Both images would have the same absolute dynamic range but the 2-bits of error is displayed in two different ways.
The 14-bit ADC blocky where as the 16-bit ADC would be randomised. Both of which are average stacked out which is the point of stacking.

Last edited by Atmos; 07-12-2019 at 01:12 PM.
Reply With Quote
Reply

Bookmarks

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +10. The time is now 01:46 AM.

Powered by vBulletin Version 3.8.7 | Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Advertisement
Bintel
Advertisement
Testar
Advertisement