#1  
Old 30-05-2014, 08:20 PM
pvelez's Avatar
pvelez (Pete)
Registered User

pvelez is offline
 
Join Date: Aug 2006
Location: Sydney
Posts: 1,250
Readout artifacts?

A call to all PI experts.

I have banding in my images. I suspect its introduced by the bias frames - but it might be the darks.

Have a look at the attached screenshot. This is an integration of 60 x 5 minute subs. Note the horizontal bands.

I can eliminate the bands by ditching the bias frames - but then this plays havoc with my flat division. As can be seen from the second attached screenshot.

Any thoughts? The Superbias module in PI gives me a similar result to the second attachment.

Pete
Attached Thumbnails
Click for full-size image (Screen Shot 2014-05-30 at 9.16.01 pm.jpg)
87.8 KB47 views
Click for full-size image (Screen Shot 2014-05-30 at 9.19.58 pm.jpg)
42.7 KB39 views
Reply With Quote
  #2  
Old 30-05-2014, 08:25 PM
pvelez's Avatar
pvelez (Pete)
Registered User

pvelez is offline
 
Join Date: Aug 2006
Location: Sydney
Posts: 1,250
FWIW - here is the Master Bias

I can't see the bands matching up

Also the Master Dark



Pete
Attached Thumbnails
Click for full-size image (Screen Shot 2014-05-30 at 9.22.42 pm.jpg)
45.8 KB29 views
Click for full-size image (Screen Shot 2014-05-30 at 9.24.02 pm.jpg)
185.0 KB28 views
Reply With Quote
  #3  
Old 31-05-2014, 01:57 PM
Shiraz's Avatar
Shiraz (Ray)
Registered User

Shiraz is offline
 
Join Date: Apr 2010
Location: ardrossan south australia
Posts: 4,787
hi Pete.

Crikey that's nasty.

I'm certainly not in the PI expert camp, but that looks like (rotated) column defects in the sensor to me - my guess is that will probably come from the flats. Maybe you could check the process for making the masterflat - could there be bias, dark or pedestal in there where it should not be - or missing where it should be included?

FWIW, I found that there are quite few settings that have to be right or the master cal images can get screwed up - I ended up using the following to the letter and all went well: http://www.pixinsight.com/tutorials/...mes/index.html good luck.

Last edited by Shiraz; 01-06-2014 at 05:14 AM.
Reply With Quote
  #4  
Old 01-06-2014, 10:42 AM
pvelez's Avatar
pvelez (Pete)
Registered User

pvelez is offline
 
Join Date: Aug 2006
Location: Sydney
Posts: 1,250
Quote:
Originally Posted by Shiraz View Post
hi Pete.

Crikey that's nasty.

I'm certainly not in the PI expert camp, but that looks like (rotated) column defects in the sensor to me - my guess is that will probably come from the flats. Maybe you could check the process for making the masterflat - could there be bias, dark or pedestal in there where it should not be - or missing where it should be included?

FWIW, I found that there are quite few settings that have to be right or the master cal images can get screwed up - I ended up using the following to the letter and all went well: http://www.pixinsight.com/tutorials/...mes/index.html good luck.
Thanks Ray

my diagnosis was the same.

I spent several hours playing with the Cosmetic Correction module in PI yesterday - with some success but its not as good as I'd like.

I am now taking new flats and will start from scratch with new bias and dark frames too.

That link is great - I have followed it to the letter to assemble the masters but I may have a setting wrong.

Will see how that goes

Its a pain - I have about 4.5 hours of each of R, G and B and a bit more Lum - I find that its my processing skills that are letting me down rather than my equipment - which is probably better than the other way around

Pete
Reply With Quote
  #5  
Old 01-06-2014, 12:57 PM
Geoff45's Avatar
Geoff45 (Geoff)
PI rules

Geoff45 is offline
 
Join Date: Jul 2006
Location: Sydney
Posts: 2,414
Quote:
Originally Posted by pvelez View Post
Its a pain - I have about 4.5 hours of each of R, G and B and a bit more Lum - I find that its my processing skills that are letting me down rather than my equipment - which is probably better than the other way around

Pete
Definitely better this way! Bad data is always going to be bad data, but processing, no matter how bad, can always be turned into great processing.
Reply With Quote
  #6  
Old 01-06-2014, 01:24 PM
pvelez's Avatar
pvelez (Pete)
Registered User

pvelez is offline
 
Join Date: Aug 2006
Location: Sydney
Posts: 1,250
Quote:
Originally Posted by ghsmith45 View Post
Definitely better this way! Bad data is always going to be bad data, but processing, no matter how bad, can always be turned into great processing.
You are not wrong there Geoff

The odd feature of this is I am now processing some data for Centaurus A - less RGB but more Lum - probably 3 hours each RGB and 5 Lum. I used the cosmetic correction module in PI - absolutely no lines to be seen.

I might go back and check my data to see if it is a feature of some images but not others

Pete
Reply With Quote
  #7  
Old 01-06-2014, 05:28 PM
ericwbenson (Eric)
Registered User

ericwbenson is offline
 
Join Date: Sep 2009
Location: Adelaide, Australia
Posts: 200
Hi Pete,
Take a short series of bias frames every night. See if the artifacts come and go (electronics flakyness or interference?) or are related to ambient temp. If the pattern never changes it's in the chip and you can calibrate it out.

Does the STX have the readout register on the side? I know the ST4000 did. Those single pixel dark streaks look like they are from hot pixel bleed (quite common in KAF chips), they are only found in columns. So either the image has been rotated 90 deg by processing, or the readout register is on the side.

I've attached a sample master bias frame from my camera (Apogee U16 with KAF16803, readout register at bottom), scaled down 25% and screen stretched to 1 ADU range (i.e. black = 1406.4 ADU and white = 1407.4 ADU) this is from 48 bias frames at -30C. The black lines are from non-uniformity in the readout register itself.

Good luck,
EB
Attached Thumbnails
Click for full-size image (Master_Bias004_4096x4096_Bin1x1_Temp-30C_ExpTime0ms.jpg)
171.5 KB13 views
Reply With Quote
  #8  
Old 01-06-2014, 05:56 PM
Peter Ward's Avatar
Peter Ward
Galaxy hitchhiking guide

Peter Ward is offline
 
Join Date: Dec 2007
Location: The Shire
Posts: 6,666
Peter,

As a zero second exposure is not possible with SBIG cameras, I am puzzled as to why anyone would want to use them, as they are already contained in the dark frame data.

A bias frame is simply a zero second dark, hence if you take any dark frame it consists of: a zero-second dark + x-second dark.

Also the dark map in SBIG cameras, from what I understand, while superbly consistent, is not truly linear.

Hence best practice to to use a library of darks taken at temperature x, and exposure time T, that obviously need to match your lights.

I've found much more value in simply getting excellent quality flats (I prefer sky flats....in fact they were the only flats that consistently worked with a back-illuminated chip I was running some years back )

Playing with Bias frames can often induce a bunch of artefacts that simply aren't in the data. I personally never use them.

Cheers
Peter
Reply With Quote
  #9  
Old 01-06-2014, 09:29 PM
Shiraz's Avatar
Shiraz (Ray)
Registered User

Shiraz is offline
 
Join Date: Apr 2010
Location: ardrossan south australia
Posts: 4,787
Pete

Peter may have identified the problem. I understand that Pixinsight automatically compensates for variations in chip temperature and exposure time by scaling the master dark, so bias subtraction is essential. If the SBIG cameras cannot provide bias data or if they have non-linear temperature/time profiles, the Pixinsight calibration scaling algorithm will not work properly. Not sure what you can do about it though, short of using some other software to calibrate your data, using a set of darks that exactly match the times and temperature of your light subs, as Peter suggests.

Last edited by Shiraz; 01-06-2014 at 09:43 PM.
Reply With Quote
  #10  
Old 01-06-2014, 10:24 PM
Joshua Bunn's Avatar
Joshua Bunn (Joshua)
Registered User

Joshua Bunn is online now
 
Join Date: Oct 2011
Location: Albany, Western Australia
Posts: 1,097
Maybe you could try CCDStack to dark scale your images and see if that gives a different result to PixInsight. I Use CCDStack for this process with great success. I use a master dark from 16 x 30min dark exposures with an SBIG STL11000.
Reply With Quote
  #11  
Old 02-06-2014, 07:20 AM
pvelez's Avatar
pvelez (Pete)
Registered User

pvelez is offline
 
Join Date: Aug 2006
Location: Sydney
Posts: 1,250
Quote:
Originally Posted by Peter Ward View Post
Peter,

As a zero second exposure is not possible with SBIG cameras, I am puzzled as to why anyone would want to use them, as they are already contained in the dark frame data.

A bias frame is simply a zero second dark, hence if you take any dark frame it consists of: a zero-second dark + x-second dark.

Also the dark map in SBIG cameras, from what I understand, while superbly consistent, is not truly linear.

Hence best practice to to use a library of darks taken at temperature x, and exposure time T, that obviously need to match your lights.

I've found much more value in simply getting excellent quality flats (I prefer sky flats....in fact they were the only flats that consistently worked with a back-illuminated chip I was running some years back )

Playing with Bias frames can often induce a bunch of artefacts that simply aren't in the data. I personally never use them.

Cheers
Peter
Understood Peter. I took the opportunity of lousy weather up at Coona yesterday to take a huge number of flats with an EL panel - 80 - 100 of each. The download pipe has been running hot. I've also taken plenty more darks.

Unfortunately, I have yet to find a way to calibrate files in PI without using a bias frame. Without it, the flats overcorrect. Time for some more research.

Quote:
Originally Posted by Shiraz View Post
Pete

Peter may have identified the problem. I understand that Pixinsight automatically compensates for variations in chip temperature and exposure time by scaling the master dark, so bias subtraction is essential. If the SBIG cameras cannot provide bias data or if they have non-linear temperature/time profiles, the Pixinsight calibration scaling algorithm will not work properly. Not sure what you can do about it though, short of using some other software to calibrate your data, using a set of darks that exactly match the times and temperature of your light subs, as Peter suggests.
Ray, you are right. There may be a way to do this in PI but I have yet to find it. I am trying to use PI for all processing and have overcome the UI hurdle. Now its down to getting it to play nicely.
Reply With Quote
  #12  
Old 03-06-2014, 02:58 PM
pvelez's Avatar
pvelez (Pete)
Registered User

pvelez is offline
 
Join Date: Aug 2006
Location: Sydney
Posts: 1,250
OK, I did some work on this last night.

At the weekend I compiled a new library of dark, bias and flat frames. A heap of them.

I then compiled master dark and master bias frames in PI last night. They were integrated separately ie the bias frames didn't come near the darks.

When severely stretched, I see readout artefacts in the same columns for both the master dark and master bias frames. They also line up with the lights.

So the bias frames aren't adding to the problem - its inherent in the camera readout.

So I'm back to 2 alternatives - a hot pixel that gets shuffled along a column till its read - assuming that the KAF16803 chip is read along the bottom rather than at the side - or I have a few column defects.

Could it be anything else?

Playing with Cosmetic Correction in PI last night, I also found that the most troublesome defect - which only extends about 1/3 of the way up a frame - wasn't being corrected as I had set the column number incorrectly. Moving it by 1 pixel seemed to deal with it. I need to play with it a bit more this evening but I'm hopeful that I can simply remove these in calibration.

I've posted on the other thread dealing with overscan. I'd like to play with that feature in PI but I don't think that it will solve this readout issue.

Pete
Reply With Quote
  #13  
Old 03-06-2014, 03:02 PM
pvelez's Avatar
pvelez (Pete)
Registered User

pvelez is offline
 
Join Date: Aug 2006
Location: Sydney
Posts: 1,250
Quote:
Originally Posted by ericwbenson View Post
Hi Pete,
Take a short series of bias frames every night. See if the artifacts come and go (electronics flakyness or interference?) or are related to ambient temp. If the pattern never changes it's in the chip and you can calibrate it out.

Does the STX have the readout register on the side? I know the ST4000 did. Those single pixel dark streaks look like they are from hot pixel bleed (quite common in KAF chips), they are only found in columns. So either the image has been rotated 90 deg by processing, or the readout register is on the side.

I've attached a sample master bias frame from my camera (Apogee U16 with KAF16803, readout register at bottom), scaled down 25% and screen stretched to 1 ADU range (i.e. black = 1406.4 ADU and white = 1407.4 ADU) this is from 48 bias frames at -30C. The black lines are from non-uniformity in the readout register itself.

Good luck,
EB
Eric

I must have missed your post yesterday.

What you present is precisely what I see in my bias frames. They don't vary from night to night.

If its the readout register itself, then presumably you calibrate them out. Do flats and darks alone deal with this or do you have some form of defect map applied as well?

Pete
Reply With Quote
  #14  
Old 04-06-2014, 12:28 PM
pvelez's Avatar
pvelez (Pete)
Registered User

pvelez is offline
 
Join Date: Aug 2006
Location: Sydney
Posts: 1,250
An update to my update

I calibrated about 48 frames in each of L, R, G and B using new master dark, flat and bias frames. I also applied the Cosmetic Correction module.

Result? No readout artifacts.

Will post some samples tonight

Pete
Reply With Quote
  #15  
Old 04-06-2014, 08:29 PM
ericwbenson (Eric)
Registered User

ericwbenson is offline
 
Join Date: Sep 2009
Location: Adelaide, Australia
Posts: 200
Hi Pete,

The flats need to have their bias subtracted, by either a master bias or a master short dark (sometimes called flat-darks), I never bother with flat-darks since the sensor is at -30C and there isn't much dark accumulation in 10 seconds! Ideally flats are calibrated before being combined into a master when using Sigma or SD combine.

All your images and your cal files have that built-in bias. This needs to be removed in order to scale a dark frame or divide/multiply two frames (flatten an image) otherwise the bias offset will contaminate the division. Darks only need to have the bias removed if they are to be scale to a different exposure length.

Bias frames are the easiest to acquire, there is no reason not to use them, unless of course the camera electronics drift!

EB
Reply With Quote
Reply

Bookmarks

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +10. The time is now 12:40 AM.

Powered by vBulletin Version 3.8.7 | Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
Advertisement
Bintel
Advertisement
OzScopes Authorised Dealer
Advertisement
Celestron Australia
Advertisement
SkyWatcher Australia
Advertisement
NexDome Observatories
Advertisement
Meade Australia
Advertisement
Lunatico Astronomical
Advertisement
Astronomy and Electronics Centre
Advertisement