Bias files in DSS cause noise and granulation in stacking RAW files from Canons ? ! ?

[jsmoraes]

:shrug: I don't know explain, but this is the result of my tests.

Target: NGC 6496

Top photo - with Bias
Bottom photo - without Bias.

First photo: Canon T3
Second photo: Canon 350D - without internal IR filter.

Both photos have HDR convertion, adjustment in curves and level, in Phothop CS3.

Without Bias the image has less noise and the stars have better shape.

With Canon 350D I needed to reduce the intensity of red information. I haven't, yet, a good custom white balance to avoid the general redish color. So I have loose of stars color information, mainly blue. It was necessary to adjust it with reduction of Red in Selective Color-Red.

[cometcatcher]

I don't use bias frames. Just darks and sometimes flats.

[jsmoraes]

I was reading again the documentation of DSS. And they tell that dark, flat and bias add noise. They improve some details of the photo, but they really add noise. The more files, more noises.
The documentation has an example of improvement and shows the noise, too: without no files, with two files and with 20 files, over a 32 lights.

[PeterEde]

Canon RAW has a tool for reducing long exp noise before export as TIFF.
2nd tab. sorry can't remember the name of the tool (Luminance I think). But does a pretty good job reducing noise before processing

[Camelopardalis]

:confused2: a n00b knows not where to turn :sadeyes:

Interesting experiment though Jorges :thumbsup:

[Amaranthus]

I don't understand why, Kevin? (unless you use Flat Darks). Bias should be subtracted from the darks, flats and lights to remove read noise. Bias frames are easy to take, and easy to apply. Why not do it?

My straightforward procedure:
1. Darks subs - Bias Master (BM), then combine to create DM (or Bad Pixel Map)
2. Flat subs - BM, combine to FM
3. (Light subs - BM - DM)/FM

[raymo]

There seems to be as many opinions as there are imagers. I have several
reputable books on astrophotography, and they all, as I recall, state that
most of the time bias frames are not needed. It seems from what I have
read that just about the only time these authors use bias frames is when scaling darks. I bow to their experience, and don't use them.
raymo

[Amaranthus]

The calibration maths is pretty clear Raymo. You need to remove read noise in some way. If you leave this in your darks, and then take flat darks to remove the dark current + bias from your flats, then I agree you don't need them. If you don't take flat darks, then you need bias. (and if your flats are short duration, then bias == flat dark).

[cometcatcher]

I didn't think they would make much difference. I can see obvious benefits in darks and flats, but if in Jorge's example they actually increase noise.... I'll have to experiment and see what happens.

[nebulosity.]

I have never used bias frames, I thought that bias is in the darks and would be removed along with the dark subtraction.

If you subtract a dark and a bias, wouldn't that subtract the bias twice? (assuming that bias is in the darks) Which would then add in noise?

:confuse3:

Jo

[vlazg]

From a newbie, what i have read the darks have the bias component but if flats are taken bias should be removed from them before a master flat

[codemonkey]

That's my understanding. I wouldn't use both unless I'd already subtracted the bias from the dark (like you would if you were scaling your dark frames).

[Amaranthus]

Have another look at my workflow :) there is no double subtraction

[jsmoraes]

We must analyse the issue in both side of the coin.

1) the main noise in lights is reduced by amount of these lights. The noise isn't created by the camera.

2) there are others electronic noises:
a) in light, dark, flat and bias
b) hot pixels

3) DSS can subtract these electronic noises and clean the image, enabling enhancement of details in photo. But these files will carry electronic noise, too.
And it will be the same for lights. The amount of files can reduce these noises.

I will experiment with much more bias files and stack then again to see the difference in the result. I will post it here. I used only 10 bias files. (How many bias files I will need with my set of equipments and my Canons T3 - 350D ? :question: )

Some very important texts from DSS documentation about the matter:

[cometcatcher]

It's all too complicated. I think I'll go back to film. ;)

[jsmoraes]

:lol:
You don't need to be so radical, Kevin. Only professional astrophotographers need to deal with this issue so serious. We, amateurs, can get GOOD images with less care.

As soon they are ready I will post new photos with 100 bias files.

[raymo]

Judging by your last post we are kindred spirits Kevin. You could write
my posts for me, and save me the trouble.
raymo:)

[jsmoraes]

Canon T3 presents more noise. Therefore, here is what happens with 100 bias, no bias and 10 bias.

It seems that 100 bias has less noise than 10 bias and a little more noise than no bias.

All photos are from the same CR2 RAW files, and was used dark and flat files.
No change was done in Autosave.tiff - 32 bits - in DSS enviroment.
All photos were converted from 32 bits to 16 bits with HDR curve in Photoshop and was applied adjustment in curve and level, only.

Conclusion:

1) the improvement of bias files depends from camera noise creation. My Canon T3 can don't like of bias; others Canon T3 can like it.

2) as bias files are used for calibration of the image, some times theirs use can be useful, despite of noise, since they can enhance the details in the photo.

3) the amount of files used to create masters are important to reduce the inherent noise they carry in. A large amount will not enhance its performance.

4) I need to use more than 10 dark, flats and bias. Perhaps 20 or 30 files. But it is impossible work with 100 dark and flats with long time of exposition. Only if it is to sell the photo or to win any competition.


5) the main solution for noise issues is a good camera, with refrigeration. Those with price around dozen thousands dolares.

6) as I haven't that money... :prey: God bless the noise ! I love them ! They are beatiful as the spikes distortion are!

[cometcatcher]

That sounds like a good enough excuse for me not to bother with them then. :)

[jsmoraes]

It is not only excuse, Kevin. It is the reality. Some cameras produce more noise than others. And the files for calibration of photo don't clean all them.

A hundred bias file is better than ten files, but the noise will be there. Bias are with short time of exposition. But think about 100 dark files with 4 minutes each one. 400 minutes in the same session of lights to have the same temperature. Unreal, right ?

Therefore... the noise will exist and will be cleanned with graphic processing. We can use those calibration files to reduce noise, but only to reduce, and make more easy the task of cleanning.

And another issue, not so evident: the documentation of DSS says that the dark, flat and bias are to calibrate the photo. Subtracting the dark signal, flat signal (vignetting ?) and bias signal. Some electronic noise will be cleanned by colateral action, not as a main goal of these files. Am I right ?

Do you have any description of what is dark signal, flat signal and bias signal ? They aren't the noise, at least only the noise. They carry noise to lights.

[jsmoraes]

A result of process without use of bias can be seen at http://www.iceinspace.com.au/forum/attachment_browse.php?a=168933

I liked it, mainly because of the stars' shapes. It was done with Canon 350D mod and with only 5 lights of 4 minutes. :)

I did Halfa files, but they didn't improve the colors of stars. I didn't use them.

[cometcatcher]

100 files take up some serious disk space. If I was to take 100 darks, 100 flats and 100 bias, that would be (for my camera) 7.5 GB of calibration files even before I take any lights! Okay so the bias doesn't have to be taken every time. Or the darks, but I do anyway because it's hard to match temperature from a different set. Not to mention how long DSS would take to process 400 frames....

[Amaranthus]

You don't need to take bias frames each time you image -- thee are measuring the electronic read current of your camera, which is instantaneous and so unaffected by temperature. They are also taken at the shortest possible shutter speed. So if you are concerned about noise, do a one-off session where you take 500-1000 of them, and create a Master Bias. Then re-use this again and again, and stop worrying.

[Amaranthus]

Kevin, once you've made the Master Dark, Flat and Bias, you can delete the 100 files and just keep the single-file Masters. (The flats can definitely only be used for a given session, but you still delete the subs once done).

[cometcatcher]

That reminds me, I have to clean up my hard drive...

[LightningNZ]

FWIW. I use 100 bias images and 50 flats. It took me about 10 minutes to take them all, cause the bias shots are 1/4000th of a second and the flats are 1/40th of a second, imaging my LCD monitor.

I haven't bothered with any darks so far because the D5100 just doesn't generate any heat noise to speak of with even 2 minute subs here in winter. I'll likely have to take some in summer, but I'll take them during the day, inside at about 20'C. Again, I'll be going for around 50 darks. Any less and the variation will just cause more noise in my final images than without.

[nebulosity.]

Cam, I'm looking into buying a D5100 and would be really interested to see some images you've taken with it. Have heard that the noise is pretty good and want to see that it is like compared to say, an 1100D. If it's not to much trouble.

Cheers
Jo

[Grimmeister]

An interesting read I found on another forum, I was under the impression Canon cameras removed noise at the time the image was taken so I do not bother with BIAS frames on my Canon camera. Seems this is the reason why you don't need to.

I just thought I would chime in here. I found this topic while searching on a similar topic.

When it comes to Canon DSLRs, they do not really subtract a bias frame from the image. There isn't a separate exposure or anything like that. Canon CMOS image sensors simply use CDS, or Correlated Double Sampling. This is an active process that occurs during readout. CDS units on each column sample the dark current noise flowing through the sensor at "reset" time. When the sensor is read for the full image, each pixel is sampled, and the dark current sample from the reset is subtracted. That's it. CDS greatly reduces the noise contribution from dark current (it doesn't eliminate it, but it does reduce it considerably.) Canon sensors do use a bias offset (calibrated via a border of masked pixels on the sensor that are also saved to the RAW file), so negative signal is preserved. Canon does have some noisy downstream electronics, and a lot of the banding that shows up in Canon RAW images is thanks to those downstream electronics, not the sensor itself (their sensors are actually quite good, and their CDS is top notch, so if Canon can ever figure out how to reduce downstream noise contribution, they would make even better astro cams than they do now.)

I've always wondered about the value of a Bias Frame myself. I've only more recently gotten deeper into astrophotography, but I've experimented it for some time. In my experience, every time I have involved bias frames, my noise seems to get worse (at least with Canon sensors). Not exactly more, however it does take on a less desirable characteristic...usually noise ends up becoming patterned, forming barely visible angled bands across the background of the image. When I exclude bias frames from my calibrations, and only using dark frames, noise maintains the nicer, entirely random aesthetic.

If you use a Canon DSLR for your astrophotography, you might want to forgo bias frames. Or at the very least, try processing a few of your images with and without them, and compare the quality of the noise. Specifically the noise quality, not the amount of noise, as overall the amount doesn't seem to change...it just seems to get more unsightly with bias frames. I would also experiment with CCDs. To my knowledge, CCD sensors were using CDS long before CMOS sensors ever came along. I don't know as much about the actual construction of astro imaging CCDs on the market today (they mostly seem to be Kodak, but beyond that I don't know the specifics of their construction). I would expect CCDs to use CDS...its about the most fundamental and ubiquitous form of hardware level noise reduction there is for imaging sensors.

Where you should really be concerned is if you use a Nikon camera. Nikon has long been known to clip negative pixel values, rather than use a bias offset. They have also been known to use a variety of sensors from a variety of sources. That makes it difficult to fully know the characteristics of Nikon cameras in general. You have to investigate the specifications and behavior of each Nikon camera to know how it will affect your images. Nikon also has indeed employed additional forms of hardware noise reduction beyond just CDS in their RAW images. Again, this differs from model to model, but it can impact your results.

If I were to decide whether to use bias frames or not, it would be as follows:

Canon: Probably not
Astro CCD: Maybe
Nikon: Who knows!

Well, hope this helps.

cheers

Anthony

[cometcatcher]

Very useful post Anthony. Thanks.

[LightningNZ]

If I can make some comments in response to Anthony's post:

CDS is a technique that's been used in CMOS and CCD detectors for at least a decade now - I dare you find a mainstream sensor that doesn't use it. It's not a magic bullet as pixels can still retain charge (or charge deficit) and the bias images will clean this up. You need lots of them though or you will introduce more Read Noise into the image than the Bias Noise you'll reduce.

I used to get terrible streaks in my images when I tried using only 10-20 bias images with my Canon 300D. Now I've moved on to the Nikon D5100 I'm not stuffing around and have gone for a solid 100 bias images. The result is a visibly cleaner background - a lot of colour noise that was there gets totally nuked by the bias images.

The information about Nikon cameras is still true though they don't hack the images up as much as they used to. You can also install custom firmware from https://nikonhacker.com/ to fix these problems completely.

Cheers,
Cam

[John K]

Very interesting discussion guys and something that I too would like to get to the bottom of regarding bias frames for my modified Canon 400D.

Whilst I do a run on some of the data I have taken recently - ie. use DSS to stack with, and without, bias frames - can I suggest that those on the thread and forumn that may have done the same post the results here.

Good discussion - and as always not a straight forward answer as is the case with this astrophotography hobby!

Clear skies,

John K.

[LightningNZ]

I thought I should respond to this one - only darks need to be matched to a specific temperature. Bias shots are 1/2000th or 1/4000th of a second - far too short for heat to play any part, provided that you're not doing this close to the breakdown temperature of the sensor, like 80*C. Also you really don't need that fine a temperature control - temperature changes during an evening anyway, so within 5*C would be plenty fine enough.

So really you can have as many correcting frames as you like and only need to keep the master frames - the averages of these.

Only flats need to be taken reasonably frequently as dust may come and go on the optics, or field rotation may change.

At the end of all of this, you drop them all into DSS, CCD stacker, PI or Nebulosity and the go have breakfast/lunch/dinner. When you come back - Hey Presto!

Cheers,
Cam