#1  
Old 25-05-2016, 06:44 PM
Slawomir's Avatar
Slawomir (Suavi)
Registered User

Slawomir is offline
 
Join Date: Sep 2014
Location: North Queensland
Posts: 3,240
MasterBias vs SuperBias

Ahoy Comrades!

I have been pondering about the real gain in using a SuperBias as opposed to a MasterBias in calibrating Lights. It only takes a click of a button to create a SuperBias in PI, but does it really make any difference to the final image? One would need to calibrate data with MasterBias and SuperBias and then process both sets of data in exactly the same way, and then measure final result for noise...

I may do that one day, but I thought that using PixelMath and Statistics tools might give me some indication.

Here are the results:

Single Bias: St.dev 14.7 ADU
Master Bias (150 frames): St.dev 2.143 ADU
Super Bias: St.dev 1.756 ADU

All frames were captured at -15C.

But here is my problem - SuperBias is meant to be noise free, so is there a better way than using the Statistics tool on the entire frame? I wanted to avoid using a small area in a bias frame, because of fixed patterns across the frame - it could also give me a skewed indication of noise.

Then I thought - why not subtract my MasterBias from the SuperBias? I added a fixed value to SuperBias to avoid clipping data. (the first screenshot shows a MasterBias, SuperBias and a difference between the two plus a fixed value).

Now the Statistics tool gave St.dev of 1.267 ADU for the difference between SuperBias and MasterBias.

Would it be correct then to assume that when calibrating with this MasterBias I would be adding 1.267 ADU of noise to all Light frames, while if using the SuperBias I would add no noise to Light frames?



On a slightly different topic, since I have just recollected a full set of Bias frames for the first time since I sent my camera to QSI for a WSG-8 upgrade and for removal of the infamous amp glow, I thought of re-measuring my camera too. Until now I have been using a mix of old and new bias frames...I know, I am lazy...

Interestingly, it looks like gain after the fix has changed from 0.16e/ADU to 0.26e/ADU, resulting in a change in well depth from about 10,000 to 16,500 electrons. Dynamic range remains about the same at 4265 steps (from memory), and read noise measured with three different methods gave values from 3.775e to 3.887e (see second screenshot for reference).

Thank you for reading, I know I can waffle on and on...
Suavi
Attached Thumbnails
Click for full-size image (QSI_May_2016_2.jpg)
96.5 KB30 views
Click for full-size image (QSI_May_2016.jpg)
130.5 KB27 views

Last edited by Slawomir; 25-05-2016 at 08:12 PM.
Reply With Quote
  #2  
Old 25-05-2016, 07:16 PM
Atmos's Avatar
Atmos (Colin)
Ultimate Noob

Atmos is offline
 
Join Date: Aug 2011
Location: Melbourne
Posts: 6,982
I too have wondering as to how much of an influence using the Superbias routine has over just a simple MasterBias. Like you, I have been too lazy to do any quantitative analysis

Just did similar to what you did, just got:
Master Bias: ~2.25
SuperBias: ~1.25

When I subtracted the Master Bias from SuperBias I ended up with a SD of ~4. If I did it without adding a pedestal and ended with 4.645 ADU average.
Reply With Quote
  #3  
Old 25-05-2016, 07:48 PM
Slawomir's Avatar
Slawomir (Suavi)
Registered User

Slawomir is offline
 
Join Date: Sep 2014
Location: North Queensland
Posts: 3,240
Hi Colin,

St.dev values for our MasterBias and SuperBias are very similar, but I wonder why you got such a large value for the difference between the two

It would indicate that your SuperBias is significantly different from MasterBias, unless there was an error in subtracting one from another
Reply With Quote
  #4  
Old 25-05-2016, 07:54 PM
Atmos's Avatar
Atmos (Colin)
Ultimate Noob

Atmos is offline
 
Join Date: Aug 2011
Location: Melbourne
Posts: 6,982
I think it is more likely the first one, they do look a bit different. Sometimes think it may be because the Master Bias with a ICX694 sensor is so clean!
Reply With Quote
  #5  
Old 25-05-2016, 08:03 PM
Slawomir's Avatar
Slawomir (Suavi)
Registered User

Slawomir is offline
 
Join Date: Sep 2014
Location: North Queensland
Posts: 3,240
But if your SuperBias is significantly different from MasterBias, then calibrating Lights with such SuperBias might yield inferior results.

To my understanding, after subtracting a SuperBias from MasterBias (or vice versa), what is left should be just noise without fixed patterns introduced by the camera, and StDev for such difference should be less than in the MasterBias.
Reply With Quote
  #6  
Old 25-05-2016, 08:10 PM
Atmos's Avatar
Atmos (Colin)
Ultimate Noob

Atmos is offline
 
Join Date: Aug 2011
Location: Melbourne
Posts: 6,982
Totally agree, when I subtract them however it looks just like noise without any variations, pure noise, which as I understand it should.
Reply With Quote
  #7  
Old 25-05-2016, 08:16 PM
Slawomir's Avatar
Slawomir (Suavi)
Registered User

Slawomir is offline
 
Join Date: Sep 2014
Location: North Queensland
Posts: 3,240
That's good

Maybe then noise approximated by the Statistics tool can be at times unreliable? I guess we will never know for sure...that bugs me!

EDIT: Just a thought, are you using 24bit STF for inspecting Bias frames?
Reply With Quote
  #8  
Old 25-05-2016, 08:26 PM
strongmanmike's Avatar
strongmanmike (Michael)
Highest Observatory in Oz

strongmanmike is offline
 
Join Date: May 2006
Location: Canberra
Posts: 17,170
Quote:
Originally Posted by Atmos View Post
I have been too lazy to do any quantitative analysis
I'm so lazy I don't do bias, super bias, nuttin, not even darks...laaaazeeee

....sorry

but I do flats!

Mike
Reply With Quote
  #9  
Old 25-05-2016, 08:40 PM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
Suavi,

Using your data above and assuming the worst case gain of 0.26 you're only talking about a difference of 0.1e- noise between the master bias and superbias. In practice that's going to be swamped by read noise and shot noise.

I've done some comparisons between using a master bias and superbias with SRO data and it doesn't make a measurable difference to my final results (I went through the whole process of calibrating and integrating several sets of data.) If we did a poor job of collecting bias frames then the results might be different, but with an adequate supply of bias frames (50+ in this case) it didn't really help.

Cheers,
Rick.
Reply With Quote
  #10  
Old 25-05-2016, 08:46 PM
Slawomir's Avatar
Slawomir (Suavi)
Registered User

Slawomir is offline
 
Join Date: Sep 2014
Location: North Queensland
Posts: 3,240
Quote:
Originally Posted by strongmanmike View Post
I'm so lazy I don't do bias, super bias, nuttin, not even darks...laaaazeeee

....sorry

but I do flats!

Mike
And that is perfectly okay when you have big...huh...pixels and you end up with stacking 10-20 long subs, but most of us with small...huh...pixels , doing many short subs, can in some circumstances benefit from subtracting bias.

And now some folks are taking about stacking hundreds of very short subs, so I would imagine that subtracting a bias would be essential, otherwise fixed camera patterns may show up with aggressive stretching due to a low SNR in a single short sub.
Reply With Quote
  #11  
Old 25-05-2016, 08:49 PM
Slawomir's Avatar
Slawomir (Suavi)
Registered User

Slawomir is offline
 
Join Date: Sep 2014
Location: North Queensland
Posts: 3,240
Quote:
Originally Posted by RickS View Post
Suavi,

Using your data above and assuming the worst case gain of 0.26 you're only talking about a difference of 0.1e- noise between the master bias and superbias. In practice that's going to be swamped by read noise and shot noise.

I've done some comparisons between using a master bias and superbias with SRO data and it doesn't make a measurable difference to my final results (I went through the whole process of calibrating and integrating several sets of data.) If we did a poor job of collecting bias frames then the results might be different, but with an adequate supply of bias frames (50+ in this case) it didn't really help.

Cheers,
Rick.
Thank you Rick for your explanation.

EDIT: However, the image resulting from subtracting SuperBias from Master Bias has a st.dev of about 1.2ADU, so wouldn't that indicate of injecting 1.2x0.26~0.3e of noise into light frames during calibration?

I guess I was somehow hoping for finding a quick n easy way of improving SNR in my images...in the end it looks that I might have to move from Paddington to some darker place...

Last edited by Slawomir; 25-05-2016 at 09:05 PM.
Reply With Quote
  #12  
Old 25-05-2016, 09:08 PM
strongmanmike's Avatar
strongmanmike (Michael)
Highest Observatory in Oz

strongmanmike is offline
 
Join Date: May 2006
Location: Canberra
Posts: 17,170
Quote:
Originally Posted by Slawomir View Post
And that is perfectly okay when you have big...huh...pixels and you end up with stacking 10-20 long subs, but most of us with small...huh...pixels , doing many short subs, can in some circumstances benefit from subtracting bias.

And now some folks are taking about stacking hundreds of very short subs, so I would imagine that subtracting a bias would be essential, otherwise fixed camera patterns may show up with aggressive stretching due to a low SNR in a single short sub.
Hey you may be right, I was just being slightly flippant ...but my pixels are only 4.5 micron btw and I generally only use 5 or 10min subs sometimes less

Mike
Reply With Quote
  #13  
Old 25-05-2016, 09:33 PM
Atmos's Avatar
Atmos (Colin)
Ultimate Noob

Atmos is offline
 
Join Date: Aug 2011
Location: Melbourne
Posts: 6,982
Quote:
Originally Posted by strongmanmike View Post
Hey you may be right, I was just being slightly flippant ...but my pixels are only 4.5 micron btw and I generally only use 5 or 10min subs sometimes less

Mike
Suavi probably thinks your using that 16803 you have stashed away in your cupboard
Reply With Quote
  #14  
Old 25-05-2016, 09:47 PM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
Quote:
Originally Posted by Slawomir View Post
EDIT: However, the image resulting from subtracting SuperBias from Master Bias has a st.dev of about 1.2ADU, so wouldn't that indicate of injecting 1.2x0.26~0.3e of noise into light frames during calibration?
Suavi,

I was using the difference of the stdevs (2.143-1.756) and converting to e-. I don't know that either answer is incredibly accurate. I think the approach of using a master and a super on real data and measuring the end result is more reliable.

Cheers,
Rick.
Reply With Quote
  #15  
Old 26-05-2016, 06:21 AM
Slawomir's Avatar
Slawomir (Suavi)
Registered User

Slawomir is offline
 
Join Date: Sep 2014
Location: North Queensland
Posts: 3,240
Quote:
Originally Posted by Atmos View Post
Suavi probably thinks your using that 16803 you have stashed away in your cupboard
That was my thinking exactly!

Quote:
Originally Posted by RickS View Post
Suavi,

I was using the difference of the stdevs (2.143-1.756) and converting to e-. I don't know that either answer is incredibly accurate. I think the approach of using a master and a super on real data and measuring the end result is more reliable.

Cheers,
Rick.
I understand and agree. Ultimately, it is the end result that matters, not noise indicators used on individual bias, MasterBias or light frames. Thank you Rick, Colin and Mike for your replies; it is precisely discussions such as this one that help me to get a better grasp of what is going on with my data when I click buttons and tick boxes in PI
Reply With Quote
  #16  
Old 28-05-2016, 08:51 AM
gregbradley's Avatar
gregbradley
Registered User

gregbradley is offline
 
Join Date: Feb 2006
Location: Sydney
Posts: 17,885
I might be talking about a different thing as I am not sure how this super bias is calculated (is this the same thing as overscan bias?) but if it is overscan bias then that is only going to give a gain on a camera that has significant bias drift.

Some cameras where the electronics are not up to scratch and the bias values can drift over time.

Greg.
Reply With Quote
  #17  
Old 28-05-2016, 09:01 AM
codemonkey's Avatar
codemonkey (Lee)
Lee "Wormsy" Borsboom

codemonkey is offline
 
Join Date: Jul 2013
Location: Kilcoy, QLD
Posts: 2,058
Superbias is basically an attempt to remove the noise from a normal bias. I'm not sure what overscan is though :-)

I briefly tried calibrating with superbias vs normal bias after seeing this thread. I didn't record the numbers, so I know I'm not of much use here, but I did see a small decrease in noise in the superbias-calibrated image as measured by the PI noise evaluation script.
Reply With Quote
  #18  
Old 28-05-2016, 09:03 AM
Atmos's Avatar
Atmos (Colin)
Ultimate Noob

Atmos is offline
 
Join Date: Aug 2011
Location: Melbourne
Posts: 6,982
The Superbias works by doing a modified 3x3 (default) Median Transform or something to that effect.... And now I am explaining things in PI talk... What have I come too
Reply With Quote
  #19  
Old 28-05-2016, 04:38 PM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
Quote:
Originally Posted by gregbradley View Post
I might be talking about a different thing as I am not sure how this super bias is calculated (is this the same thing as overscan bias?) but if it is overscan bias then that is only going to give a gain on a camera that has significant bias drift.

Some cameras where the electronics are not up to scratch and the bias values can drift over time.
As the guys have said, Superbias is a PI process that reduces the noise in a master bias. It uses multiscale NR and the knowledge that there are typically column (or row) based structures in bias data.

I've never quite managed to think through how a superbias master would interact with overscan calibration. I guess I should try it. I normally take a lot of bias frames (why not, it's quick & easy?) so my master bias is very low noise anyway.
Reply With Quote
  #20  
Old 28-05-2016, 05:25 PM
gregbradley's Avatar
gregbradley
Registered User

gregbradley is offline
 
Join Date: Feb 2006
Location: Sydney
Posts: 17,885
Lee,

Overscan is an area of a CCD not used in the image but records some pixel information. I think its used for internal settings for the CCD. When you see a sensor that has say 15mp of pixels and 14.8 effective the difference is in the overscan area. Its blacked out and the pixels don't receive light.

Greg.
Reply With Quote
Reply

Bookmarks

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +10. The time is now 02:21 PM.

Powered by vBulletin Version 3.8.7 | Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Advertisement
Bintel
Advertisement
Testar
Advertisement