#21  
Old 29-05-2016, 11:12 AM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
Quote:
Originally Posted by gregbradley View Post
Overscan is an area of a CCD not used in the image but records some pixel information. I think its used for internal settings for the CCD. When you see a sensor that has say 15mp of pixels and 14.8 effective the difference is in the overscan area. Its blacked out and the pixels don't receive light.
A sensor will have active pixels which are exposed to light, some rows and/or columns which are masked so that light doesn't reach them and also some rows and/or columns which are clocked out and read but don't correspond to actual physical pixels. This group of virtual pixels is the overscan region (some of the more sophisticated CCDs in professional use may have multiple overscan regions.)

The data read from the pixels of the overscan region is an overscan value (a component of bias) plus read noise. There's no per pixel bias, no dark current/dark noise and no light signal. The overscan data can be used in the calibration process and this has benefits if you have a camera which shows variation in overall bias over time - it corrects for this variation.

I use overscan calibration with my Apogee U16M which shows a fair amount of bias drift. I think this drift is fairly common but most people don't notice or choose to ignore it. I've seen it on SBIG and SX cameras as well. The difference is typically only a few e-, but if you're going for very dim targets that might be as much as your signal

The only processing packages that I've come across that support overscan calibration are PixInsight and Mira.

Cheers,
Rick.
Reply With Quote
  #22  
Old 29-05-2016, 11:43 AM
Slawomir's Avatar
Slawomir (Suavi)
Registered User

Slawomir is offline
 
Join Date: Sep 2014
Location: North Queensland
Posts: 3,240
I have decided to compare non-calibrated lights with flat and bias calibrated ones, to pick the most optimal pre-processing routine for my data.

Variables:
10-minute Ha 3nm lights (33 of them)
151 bias frames
50 flats
CCD temperature: -15C
Camera: QSI 690

Noise indicators: AVG (Average Absolute Deviation) and MAD (Median Absolute Deviation)

All subs were star-registered against the same sub and also the same sub has been used as a reference for integration (Linear Fit with the same parameters).

Results:

Single entire subs:

Not calibrated: AVG = 27.1, MAD = 23.7
Calibrated with Master Bias and flats: AVG = 27.077, MAD = 24.054
Calibrated with SB and flats: AVG = 27.056, MAD = 24.028

Single subs - small area with background only:

Not calibrated: AVG = 24.4, MAD = 23.7
MB and flats: AVG = 24.644, MAD = 24.579
SB and flats: AVG = 24.667, MAD = 24.748

These results suggest that there is a bit of noise being injected into single subs while calibrating them with bias and flats, while at the same time there is little or no difference between calibrating with either Master Bias and Super Bias, at least in terms of noise.

Stacks of 33 subs:

Entire subs:

Not calibrated: AVG = 7.801, MAD = 4.685
MB and flats: AVG = 7.798, MAD = 4.609
SB and flats: AVG = 7.937, MAD = 4.703

Stacks - the same small area with background only:

Not calibrated: AVG = 4.187, MAD = 4.206
MB and flats: AVG = 4.287, MAD = 4.295
SB and flats: AVG = 4.340, MAD = 4.357

Surprise surprise. For this data set, the stack of subs calibrated with Master Bias apparently injected less noise (and/or more effectively removed fixed patterns) than the same set of data calibrated with a Super Bias!

Out of curiosity, I have also subtracted New Master Bias from Old Master Bias (from over a year ago), and then aggressively stretched the difference. Then I repeated the same for Super Bias frames. The difference between two Super Biases resulted in somehow funny patterns, that are not present in the difference between Master Bias frames.

Mind you, mean ADU values for both differences were less than 0.5.

I think that in the end I managed to produce a few meaningless numbers

Thank you for reading.
Suavi

EDIT: Just read Rick's post - wondering if this difference between old and new bias frames indicates a drift of bias over time, or is it a result of the "amp glow" fix that QSI did a few months ago.
Attached Thumbnails
Click for full-size image (OldSB-NewSB.jpg)
141.7 KB14 views
Click for full-size image (OldMB-NewMB.jpg)
24.9 KB12 views

Last edited by Slawomir; 29-05-2016 at 11:54 AM.
Reply With Quote
  #23  
Old 29-05-2016, 12:24 PM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
Quote:
Originally Posted by Slawomir View Post
EDIT: Just read Rick's post - wondering if this difference between old and new bias frames indicates a drift of bias over time, or is it a result of the "amp glow" fix that QSI did a few months ago.
Suavi,

The bias drift I see is apparent over short time frames. If I do my typical run of 400 or 500 bias frames to produce a new master you can see the values wandering around. So, it's probably the amp glow fix

You have to be very careful comparing noise estimates. The method I use is borrowed from here: http://pixinsight.com/forum/index.ph...54909#msg54909

Cheers,
Rick.
Reply With Quote
  #24  
Old 29-05-2016, 01:27 PM
Slawomir's Avatar
Slawomir (Suavi)
Registered User

Slawomir is offline
 
Join Date: Sep 2014
Location: North Queensland
Posts: 3,240
Thank you Rick,

I am always keen to learn new tricks and will certainly need to investigate whether I can measure bias drift in my little camera

I have checked single frames with the method from the link you provided, and these are noise estimates (biweight midvariance) I got:

Single un-calibrated sub: 0.9828
MB and Flat calibrated: 0.9842
SB and Flat calibrated: 0.9851

I am assuming the smaller the number the better, so these results also indicate that I would be better off with Master Bias (and possibly could add more bias frames to it, as well as it probably would not hurt to take a few more dozens of flats).


EDIT: Possibly a silly question...in PI, would overscan only apply to calibrating Bias frames (BatchPreProcessing Script)? And can I collect new bias frames with overscan enabled and use them to calibrate Lights and Flats that were captured without overscan enabled?

Last edited by Slawomir; 29-05-2016 at 01:48 PM.
Reply With Quote
  #25  
Old 29-05-2016, 03:01 PM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
Suavi,

Quote:
Originally Posted by Slawomir View Post
I have checked single frames with the method from the link you provided, and these are noise estimates (biweight midvariance) I got:

Single un-calibrated sub: 0.9828
MB and Flat calibrated: 0.9842
SB and Flat calibrated: 0.9851

I am assuming the smaller the number the better, so these results also indicate that I would be better off with Master Bias (and possibly could add more bias frames to it, as well as it probably would not hurt to take a few more dozens of flats).
Yes, the numbers are scaled noise estimates so smaller is better. Because they are estimates you should take them with a grain of salt. I'd run several examples and check for consistent behaviour.

Quote:
Originally Posted by Slawomir View Post
EDIT: Possibly a silly question...in PI, would overscan only apply to calibrating Bias frames (BatchPreProcessing Script)? And can I collect new bias frames with overscan enabled and use them to calibrate Lights and Flats that were captured without overscan enabled?
Sorry, overscan calibration is applied to the calibration frames and the lights as well, so they all need to include the overscan area. You can't apply it to old data which doesn't include it.

Probably best to check your camera for drift first. If the bias is stable then you won't gain anything except additional hassle by overscan calibration

Cheers,
Rick.
Reply With Quote
  #26  
Old 29-05-2016, 03:39 PM
Slawomir's Avatar
Slawomir (Suavi)
Registered User

Slawomir is offline
 
Join Date: Sep 2014
Location: North Queensland
Posts: 3,240
Thank you Ray - your method is certainly easy to apply and understand

Rick- I have just measured mean value for a central area in several bias frames collected over 45 minutes. The greatest difference was 1.2 ADU, which I estimate to be around 0.3e. Would it be worthwhile using overscan in my case? I managed to find a way to turn overscan on and off, and it gives access to about 150 additional columns of pixels on the right-hand side of the image.
Reply With Quote
  #27  
Old 29-05-2016, 04:00 PM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
Quote:
Originally Posted by Slawomir View Post
Rick- I have just measured mean value for a central area in several bias frames collected over 45 minutes. The greatest difference was 1.2 ADU, which I estimate to be around 0.3e. Would it be worthwhile using overscan in my case? I managed to find a way to turn overscan on and off, and it gives access to about 150 additional columns of pixels on the right-hand side of the image.
Suavi, it's probably not worth the trouble for a third of an electron. In my case the difference was a lot more than that.

Cheers,
Rick.
Reply With Quote
  #28  
Old 29-05-2016, 04:19 PM
Slawomir's Avatar
Slawomir (Suavi)
Registered User

Slawomir is offline
 
Join Date: Sep 2014
Location: North Queensland
Posts: 3,240
Quote:
Originally Posted by RickS View Post
Suavi, it's probably not worth the trouble for a third of an electron. In my case the difference was a lot more than that.

Cheers,
Rick.
Thank you Rick - low bias drift might be one of those few counted benefits of having a small sensor.

Anyway, it was really exciting learning more about read noise, overscan and functioning of my camera. Does it qualify me as a geek?
Reply With Quote
  #29  
Old 29-05-2016, 05:07 PM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
Quote:
Originally Posted by Slawomir View Post
Thank you Rick - low bias drift might be one of those few counted benefits of having a small sensor.

Anyway, it was really exciting learning more about read noise, overscan and functioning of my camera. Does it qualify me as a geek?
I think the KAF sensors may be more prone to drift. The Sony sensors generally have much nicer characteristics apart from size!

IMO you definitely qualify for a geek badge
Reply With Quote
Reply

Bookmarks

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +10. The time is now 09:22 PM.

Powered by vBulletin Version 3.8.7 | Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Advertisement
Bintel
Advertisement
Testar
Advertisement