Go Back   IceInSpace > Equipment > Astrophotography and Imaging Equipment and Discussions

Reply
 
Thread Tools Rate Thread
  #1  
Old 14-09-2011, 07:09 PM
peter_4059's Avatar
peter_4059 (Peter)
Big Scopes are Cool

peter_4059 is offline
 
Join Date: Jun 2007
Location: SE Tasmania
Posts: 4,532
Flats overcorrecting - anyone else had this?

I've recently been playing with some of my images taken at astrofest to try to understand why the flats are apparently overcorrecting causing light corners. I've tried a lot of different combinations of settings in Deep Sky Stacker as well as other calibration software including CCD Stack, Nebulosity and Pixinsight but get the same result. I've finally come across a work around to correct the issue but would like to understand what is causing it. I suspect it is something to do with the QHY8 being a one shot colour camera and the colour of the light in my light box but would be interested to know if others have seen the same thing and if so whether they found a way to resolve it?

I've found a good way to see if this is happening is to use the threshold function in PS after opening the stacked image. I've included a screen shot to show how this looks.
Attached Thumbnails
Click for full-size image (vignetting.JPG)
181.6 KB118 views
Reply With Quote
  #2  
Old 14-09-2011, 08:42 PM
pvelez's Avatar
pvelez (Pete)
Registered User

pvelez is offline
 
Join Date: Aug 2006
Location: Sydney
Posts: 1,250
Peter

I had something similar. Turned out my exposures were too short ie the light box was too bright. This resulted in shutter artifacts. This was apparent when I stepped down the light with some plain white material.

Looking at yours however, it seems only the corners so it may be different. How long are your exposures?

Pete
Reply With Quote
  #3  
Old 14-09-2011, 08:48 PM
RobF's Avatar
RobF (Rob)
Mostly harmless...

RobF is offline
 
Join Date: Jul 2008
Location: Brisbane, Australia
Posts: 5,716
I don't have all the answers (any answers ) Peter, but will be watching for people's thoughts with interest. From a dark sky site I find I can usually wing it better just using dynamic background correction in Pixinsight, but from a city location with greater skyglow or through my Ha filter the vignetting is too pronounced. I think I've seen Marc suggest a way of "backing down" flats to subdue the effect.

Sorry for lack of hard facts though
Reply With Quote
  #4  
Old 14-09-2011, 09:36 PM
Ross G
Registered User

Ross G is offline
 
Join Date: Apr 2007
Location: Cherrybrook, NSW
Posts: 5,006
Hi Peter,

I had a similar problem with my new QHY8L when I started using it early this year. My flats would give me light corners.
I found doing bias frames as well solved the problem.

Hope this helps.

Ross.
Reply With Quote
  #5  
Old 14-09-2011, 10:07 PM
Alchemy (Clive)
Quietly watching

Alchemy is offline
 
Join Date: Jul 2007
Location: Yarra Junction
Posts: 3,044
I've used a qhy8 for years no problems....

2 things come to mind

Firstly use a full calibration set, bias flat darks etc, ..... I always do

Secondly color is no issue, mine are always light blue as I do sky flats, it's only the luminosity thats applied, however make sure your flats are a mid range not too bright or too dark.
Reply With Quote
  #6  
Old 14-09-2011, 10:46 PM
gregbradley's Avatar
gregbradley
Registered User

gregbradley is offline
 
Join Date: Feb 2006
Location: Sydney
Posts: 17,877
I get this sometimes. My Planewave CDK17 setup is very sensitive to correct flat fielding.

Firstly, I take flats with a white cloth cover over the end of my scope and take about 6 flats in my observatory during the morning usually. The slight filtered light that makes its way into my observatory is perfect for flats as it turns out.

I aim for about 20,000 ADU. If the flat is too bright then I find it overcorrects.

Similarly I take a dark for the same exposure time for each filter's flat.These are subtracted when the flat is done. This is all at an exact temperature - the same as the light exposures you took. Also at the same focus and with the camera in the same orientation as the light exposures.

If you do adaptive darks - that is using a longer exposure dark for a shorter light but at the same temperature you also need a bias. That stops the weird corrections as the dark is scaled properly.

I take 5 or 6 flats darks and flats and then use median combine for the flats and subtract the sigma rejected combined darks from them. I shoot for 20,000 adu and make sure the exposure is long enough so there are no shutter artifacts from too slow an exposure that captures the shadows of the shutter opening.

I sometimes can get away with older flats if I haven't changed anything but I would update my flats if anything changed - camera orientation, filters, camera, changed telescope etc.

At a guess I would say your darks are mismatched to your lights and then your flat is not flat dark subtracted. Or your flats are overexposed.

Greg.
Reply With Quote
  #7  
Old 15-09-2011, 07:09 AM
multiweb's Avatar
multiweb (Marc)
ze frogginator

multiweb is offline
 
Join Date: Oct 2007
Location: Sydney
Posts: 22,060
Your flat needs to have the bias subtracted from it or it won't scale correctly when you divide it in your subs. That's quite an important point.

With the QHY8 I aim for 9500 ADU or where about in the brightest part of the flat.
Reply With Quote
  #8  
Old 15-09-2011, 08:51 AM
Terry B's Avatar
Terry B
Country living & viewing

Terry B is offline
 
Join Date: Mar 2006
Location: Armidale
Posts: 2,789
The other important thing is for the flat to be only exposed within the linear part of your CCD. If you have an antiblooming CCD then it will start to have a non linear response somewhere over 1/2 the full wells. Keeping the max exposure to 1/2 the full wells is a safe way of preventing this. There is an advantage to having the brightest exposure you can as it reduces noise but you must keep in the linear part of the CCD- a fine balance needs to be made.
Reply With Quote
  #9  
Old 15-09-2011, 12:48 PM
peter_4059's Avatar
peter_4059 (Peter)
Big Scopes are Cool

peter_4059 is offline
 
Join Date: Jun 2007
Location: SE Tasmania
Posts: 4,532
Quote:
Originally Posted by pvelez View Post
Peter

I had something similar. Turned out my exposures were too short ie the light box was too bright. This resulted in shutter artifacts. This was apparent when I stepped down the light with some plain white material.

Looking at yours however, it seems only the corners so it may be different. How long are your exposures?

Pete
Pete,
The flats I've been using are 4 second exposures however the qhy8 doesn't have a mechanical shutter so it's not a shutter issue.

Quote:
Originally Posted by RobF View Post
I don't have all the answers (any answers ) Peter, but will be watching for people's thoughts with interest. From a dark sky site I find I can usually wing it better just using dynamic background correction in Pixinsight, but from a city location with greater skyglow or through my Ha filter the vignetting is too pronounced. I think I've seen Marc suggest a way of "backing down" flats to subdue the effect.

Sorry for lack of hard facts though
Rob,
Pixinsight does do a nice job of removing the gradient with the dynamic background extraction however I'm trying to get to the bottom of why the flat is not correcting properly in the first place. The problem with dynamic background extraction is where to put the sample points when the object you are imaging fills the whole field of view (often the case with my setup). Marc did suggest using pixel math to "adjust" the flat and this is the workaround I referred to - if I add 28000 adu to the flat it corrects perfectly. I'm just trying to understand why I can't get the calibration to work with the un-adjusted flat.

Quote:
Originally Posted by Ross G View Post
Hi Peter,

I had a similar problem with my new QHY8L when I started using it early this year. My flats would give me light corners.
I found doing bias frames as well solved the problem.

Hope this helps.

Ross.
Ross,
I'm using bias and have also tried full calibration - ie lights, flats, bias, dark flat, darks but get the same result.
A few questions:
Are you using a light box?
What software are you using for calibration?
What ADU level are you shooting the flats at?
How long is your flat exposure?
How long is your bias exposure?

Quote:
Originally Posted by Alchemy View Post
I've used a qhy8 for years no problems....

2 things come to mind

Firstly use a full calibration set, bias flat darks etc, ..... I always do

Secondly color is no issue, mine are always light blue as I do sky flats, it's only the luminosity thats applied, however make sure your flats are a mid range not too bright or too dark.
Clive,
I've tried full calibration but get the same result. What software are you using for calibration? I think I've read DSS calibrates each channel seperately. My flats ahve a range of about 16000 to 36000 ADU. I've also tried iincreasing and reducing the exposure time for the flat however this doesn't appear to make any difference.

Quote:
Originally Posted by gregbradley View Post
I get this sometimes. My Planewave CDK17 setup is very sensitive to correct flat fielding.

Firstly, I take flats with a white cloth cover over the end of my scope and take about 6 flats in my observatory during the morning usually. The slight filtered light that makes its way into my observatory is perfect for flats as it turns out.

I aim for about 20,000 ADU. If the flat is too bright then I find it overcorrects.

Similarly I take a dark for the same exposure time for each filter's flat.These are subtracted when the flat is done. This is all at an exact temperature - the same as the light exposures you took. Also at the same focus and with the camera in the same orientation as the light exposures.

If you do adaptive darks - that is using a longer exposure dark for a shorter light but at the same temperature you also need a bias. That stops the weird corrections as the dark is scaled properly.

I take 5 or 6 flats darks and flats and then use median combine for the flats and subtract the sigma rejected combined darks from them. I shoot for 20,000 adu and make sure the exposure is long enough so there are no shutter artifacts from too slow an exposure that captures the shadows of the shutter opening.

I sometimes can get away with older flats if I haven't changed anything but I would update my flats if anything changed - camera orientation, filters, camera, changed telescope etc.

At a guess I would say your darks are mismatched to your lights and then your flat is not flat dark subtracted. Or your flats are overexposed.

Greg.
Greg,

Is the 20000 ADU the mean or the maximum value of the flat? My 4 sec flats range from 16000 to 36000 ADU with the mean around 26000. I take 20 flats for each imaging session to ensure same camera orientation and focus position. All the lights, flats, darks, bias etc are done at the same temperature.

I've tried shorter and longer flat exposures and get the same result.
I've always used darks, bias and dark flats and have also tried without dark flats and get the same result.

Based on all the testing I've done it appears the issue is the difference in contrast over the flat image - ie the difference in ADU count between the centre of the image and the corners. My understanding of the calibration process (ignoring darks and bias to simplify) is:
Calibrated light = Light / (Flat / mean of Flat)
(some software uses median or mode instead of mean but I believe these values will be similar in my case as there is a reasonably even distribution of ADU values across the flat according to CCD Inspector flat analysis).

So this means at the corners the light is being divided by 16000/26000 = 0.615 which is equivalent to multiplying the light by 1.625 - ie the dark corners get lightened by 63%
In the centre of the image the light is being divided by 36000/26000 = 1.385 which is equivalent to the centre being darkened by 28%
Somewhere between the dark corner and the light centre where the mean value is the light gets divided by 26000/26000 = 1 - ie. no change.
 
By adding 28000 to the flat using pixel math I've changed the calibration maths to:
at the corners the light is being divided by (16000+28000)/(26000+28000) = 44000/54000 = 0.815 which is equivalent to multiplying the light by 1.227 - ie the dark corners get lightened by 23%
In the centre of the image the light is being divided by (36000+28000)/(26000+28000) = 64000/54000 = 1.185 which is equivalent to the centre being darkened by 16%
Somewhere between the dark corner and the light centre where the mean value is the light gets divided by (26000+28000)/(26000+28000) = 1 - ie. no change.

What this means is the resulting calibrated image has the dark corners lightened by a similar amount to the amount the centre is being darkened whereas with the raw flat the corners are being lightened by twice as much as the centre is being darkened.
 
In any case when I adjust the flats this way and then review the resulting calibrated image using threshold I get a much more even illumination across the image. All the trials I've done so far are on an image of M20 as this was the only recent image I had with some plain "sky" in the corners. My next step is to take some images of a reasonably dark piece of sky with no structure and see if I get even illumination right to the centre.

Quote:
Originally Posted by multiweb View Post
Your flat needs to have the bias subtracted from it or it won't scale correctly when you divide it in your subs. That's quite an important point.

With the QHY8 I aim for 9500 ADU or where about in the brightest part of the flat.
Marc,

I'm definately subtracting the bias and have tried shorter flat exposures but still get the same result - adding using pixel math is the only thing that has provided a satisfactory result so far. Changing exposure is equivalent to multiplying by a constant (in the linear part of the sensor response which doesn't change the ratio I describe above as all values get scaled by an equal amount.

Quote:
Originally Posted by Terry B View Post
The other important thing is for the flat to be only exposed within the linear part of your CCD. If you have an antiblooming CCD then it will start to have a non linear response somewhere over 1/2 the full wells. Keeping the max exposure to 1/2 the full wells is a safe way of preventing this. There is an advantage to having the brightest exposure you can as it reduces noise but you must keep in the linear part of the CCD- a fine balance needs to be made.
Terry,

I ran some tests of the response vs exposure duration and I'm pretty confident I'm in the linear range. The results were in this thread...

http://www.iceinspace.com.au/forum/s...highlight=flat
Reply With Quote
  #10  
Old 15-09-2011, 12:53 PM
multiweb's Avatar
multiweb (Marc)
ze frogginator

multiweb is offline
 
Join Date: Oct 2007
Location: Sydney
Posts: 22,060
Quote:
Originally Posted by peter_4059 View Post
I'm definately subtracting the bias and have tried shorter flat exposures but still get the same result - adding using pixel math is the only thing that has provided a satisfactory result so far. Changing exposure is equivalent to multiplying by a constant (in the linear part of the sensor response which doesn't change the ratio I describe above as all values get scaled by an equal amount.
Can you post/PM a link to three FITS files? One sub, the master flat and the master bias. I'll play with it and see what's going on.
Reply With Quote
  #11  
Old 15-09-2011, 05:26 PM
peter_4059's Avatar
peter_4059 (Peter)
Big Scopes are Cool

peter_4059 is offline
 
Join Date: Jun 2007
Location: SE Tasmania
Posts: 4,532
Marc,

I've uploaded a light, flat and bias file (individual subs rather than a master) to this location:

http://www.sendspace.com/file/16ceas

Peter

Last edited by peter_4059; 15-09-2011 at 06:23 PM.
Reply With Quote
  #12  
Old 15-09-2011, 06:34 PM
multiweb's Avatar
multiweb (Marc)
ze frogginator

multiweb is offline
 
Join Date: Oct 2007
Location: Sydney
Posts: 22,060
Hi Peter, had a look and your flat is way way too bright. The center brightest pixels are well over 30000ADU. For this type of camera 9000 - 10000 ADU is the ball park. Try shooting flats about a third of the illumination you've got now and it's going to work no problem. The bias is fine. The over correction is not too bad. I've done way worse.
Reply With Quote
  #13  
Old 18-09-2011, 06:46 PM
gregbradley's Avatar
gregbradley
Registered User

gregbradley is offline
 
Join Date: Feb 2006
Location: Sydney
Posts: 17,877
Hi Peter,

Sorry I did not see your reply.

Are my flats 20,000 average or maximum you ask? Good question - I don't know. Its just the value CCDsoft puts out. Now that I think about it I think its likely to be the maximum.

As Marc has pointed out your flats are most likely too bright.

I have just seen with my CDK17 which really needs good flats as it vignettes quite a bit with my large chipped camera - that I have many times gotten crazy flat results where the flat makes the outer vignetted areas too bright. What worked for me is to make sure a bias was also subtracted (even if you subtracted flat darks from your flat). Usually though that was when I was using adaptive darks which are not ideal anyway. Adaptive darks is when you use say a 10minute dark on a 5 minute light and software adjusts it to suit your image.

I have also had it where a flat was too bright and it had that effect. I read a paper suggesting flats should be a lot brighter and are ok up to 80% of saturation so I am a bit unsure of the correctness of that statement. But in my experience that is what happened.
I'd have to read the paper again to see what was different about that approach. The concept though, was that modern CCDs sensitivity to light is very linear (unlike film) with length of exposure. In other words the CCD is the same sensitivity at 5 minutes as at 1 minute unlike film where sensitivity dropped the longer the exposure.

Greg.
Reply With Quote
  #14  
Old 18-09-2011, 09:46 PM
cventer's Avatar
cventer
Registered User

cventer is offline
 
Join Date: Jul 2005
Location: Melbourne Australia
Posts: 957
Interesting thread

I thought for flats you were supposed to expose histogram in half to 2/3 of full well ? So with full well of 65k I was aiming for median of around 40k. Sounds like this is too bright as well ?

What adu shoudl i aim for with 65k well depth ?
Reply With Quote
  #15  
Old 19-09-2011, 09:00 AM
Terry B's Avatar
Terry B
Country living & viewing

Terry B is offline
 
Join Date: Mar 2006
Location: Armidale
Posts: 2,789
Quote:
Originally Posted by cventer View Post
Interesting thread

I thought for flats you were supposed to expose histogram in half to 2/3 of full well ? So with full well of 65k I was aiming for median of around 40k. Sounds like this is too bright as well ?

What adu shoudl i aim for with 65k well depth ?
I think it depends on the CCD. for a non ABG CCD 2/3 max is usually still in the linear part of the ccd so is ok. I'm not sure about the modern ABG CCDs.
I think that the over correction that the original poster notes is a combination of problems. It is very hard to make perfect flats. T shirt flats are good but can still be inaccurate if angled ambient light falls on the shirt.
Dome flats probably mimic the sky the best providing you can very evenly illuminate the target.
Twighlight flats are also very good but a pest to make.
I use dome flats and find they are good most of the time. They do seem to fail a bit if I image at full moon resulting in uneven correction.
Reply With Quote
  #16  
Old 19-09-2011, 09:27 AM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
Wouldn't a flat with ADU counts up in the non-linear area of the sensor tend to under correct rather than over correct vignetting? As the wells fill they'd become less sensitive to further photons, so the brighter areas in the flat would tend to read lower than they would if the response was perfectly linear. That would result in a reduced correction to vignetted areas.

Cheers,
Rick.
Reply With Quote
  #17  
Old 19-09-2011, 07:52 PM
peter_4059's Avatar
peter_4059 (Peter)
Big Scopes are Cool

peter_4059 is offline
 
Join Date: Jun 2007
Location: SE Tasmania
Posts: 4,532
OK - done a lot more testing and here are the results:

1. For the images taken at astrofest adding 28000 to the flats using pixel math gave the best correction. A shorter flat (2s vs 4s made no noticable difference).

2. For an image taken from home under light polluted skies the adding 28000 resulted in very poor calibration. 2s flats were no better than 4s flats. There is a gradient from city light pollution.

I've included the response curves for my qhy8 pro to show the linear region vs exposure for my light box.

The flatstack attachment shows my light box is definetely illuminating the scope evenly - this is a stack of two flats taken with the light box rotated 180 degrees - one as a light and the second as the flat - the pixel values are the same from centre to edge and all edges are the same.
Attached Files
File Type: pdf FlatExposureTests_dark.pdf (213.4 KB, 32 views)
File Type: pdf FlatExposureTests_home.pdf (290.1 KB, 31 views)
File Type: pdf flat_tests.pdf (128.7 KB, 35 views)
File Type: pdf flat_stack.pdf (232.6 KB, 28 views)

Last edited by peter_4059; 20-09-2011 at 08:50 PM.
Reply With Quote
  #18  
Old 19-09-2011, 07:58 PM
peter_4059's Avatar
peter_4059 (Peter)
Big Scopes are Cool

peter_4059 is offline
 
Join Date: Jun 2007
Location: SE Tasmania
Posts: 4,532
and here are the histograms as reported by ps3
Attached Files
File Type: pdf flatshisto.pdf (232.2 KB, 32 views)
Reply With Quote
  #19  
Old 19-09-2011, 08:37 PM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
I'm glad you found a solution, Peter. It's still very mysterious behaviour...
Reply With Quote
  #20  
Old 19-09-2011, 08:58 PM
RobF's Avatar
RobF (Rob)
Mostly harmless...

RobF is offline
 
Join Date: Jul 2008
Location: Brisbane, Australia
Posts: 5,716
Peter, when you say there is a city light pollution gradient, do you mean the vignetting is magnified, or a top/bottom or left/right gradient?
Reply With Quote
Reply

Bookmarks

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +10. The time is now 01:36 AM.

Powered by vBulletin Version 3.8.7 | Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Advertisement
Testar
Advertisement
Bintel
Advertisement