I need some advice on addressing imaging artefacts in some recent M16 data that I grabbed whilst waiting for Neptune to climb out of the murky lights of Brisbane, using the C9.25 F10 with x0.63 Reducer/Corrector and Gerald’s ST2000/CFW10 combo.
When I came to process the LRGB frames in CCDStack I noticed what appeared to be “holes” in the data and these appeared at the same position in each frame so a Median combine couldn’t get rid of them. CCDStack reports them at around 6000-7000 ADU with the surrounding background starting at around 7,000 ADU.
I tried applying the “Hot/Cold Pixel” function in CCDStack and this failed to remove them. Other data rejections techniques in CCDStack look as if they remove good data in the frame when I try to “kill” these holes.
I’ve attached an x2 enlargement from the combined 12x5min Lum frames to show these “holes”.
I’d appreciate any advice on how to target these holes as I have 9x20 min sub frames revealing Nereid’s motion (moon of Neptune) and they are also peppered with these holes. Thanks!
Hi Dennis. I am sorry I cannot shed light on the cause but I can give you a method to remove these holes. It does appear to be some spots on some surface of the imaging train or in a calibration file.
Using CCDStack:
1. Select "Process", "Data Reget"
2. Select "Free Hand Draw" set pixel width to 1 pixel.
3. Mark all holes with a mouse click over them. Click apply.
4. Select "Grow" from reject menu. Set width to 5 pixels and apply.
5. From reject menu, select set pixel value to missing pixel and apply.
6. From reject menu select "Interpolate missing pixels" set value to 3 and iterations to 5. and apply.
This proceedure should remove these holes and fill them with an interpolated value based on the surrounding values.
I added an image repaired with a couple on the right side not repaired.
Sorry about the quality but it is a bit hard with a JPG.
I have seen similar with a Baader 7nm Ha filter. In that case I believe it ended up being poor coatings on my particular Baader Ha which generally have a good reputation. Some said it was likely a mismatch of darks and lights but even reshooting darks made no difference.
So its either:
1. defective coatings with holes in them
2. mismatched darks and lights. Can you post your dark?
Also if you use median combine you are less likely to see these sorts of artifacts that thrive if you use sum combine where all defects get added together with no outliers being removed.
Did you take any flats dennis? Are they in the flats as well?
Hi Robin
Thanks for the reply. The Lum frames have been reduced with Darks (5 min) and Flats (5 secs) and the Flats didn’t show any of these artefacts. They were nice and smooth, apart from the vignetting and dust bunnies!
Here’s the stretched Lum Flat.
Cheers
Dennis
Last edited by Dennis; 19-07-2010 at 06:08 PM.
Reason: Replace 20 min Dark with 5 min Dark
Hi Dennis. I am sorry I cannot shed light on the cause but I can give you a method to remove these holes. It does appear to be some spots on some surface of the imaging train or in a calibration file.
Using CCDStack:
1. Select "Process", "Data Reget"
2. Select "Free Hand Draw" set pixel width to 1 pixel.
3. Mark all holes with a mouse click over them. Click apply.
4. Select "Grow" from reject menu. Set width to 5 pixels and apply.
5. From reject menu, select set pixel value to missing pixel and apply.
6. From reject menu select "Interpolate missing pixels" set value to 3 and iterations to 5. and apply.
This proceedure should remove these holes and fill them with an interpolated value based on the surrounding values.
I added an image repaired with a couple on the right side not repaired.
Sorry about the quality but it is a bit hard with a JPG.
I hope this helps.
Hi Doug
Thanks for the quick and comprehensive reply, the results do look very promising. Your re-worked example is definitely superior to the “Patch” tool that I tried to use in Photoshop.
The anomalous pixels are sometimes single pixels and sometimes small clusters and their ADU value is mostly within 10’s to 100’s of the fainter background stuff although they appear quite dark in my example as the data is Scaled with DDP applied in CCDStack.
They appear to be in the same location on all each of the L,R,G and B frames so it looks as if those particular pixels seem to be less sensitive that their neighbours, albeit by small amounts, which then get reinforced through the stacking process?
I have seen similar with a Baader 7nm Ha filter. In that case I believe it ended up being poor coatings on my particular Baader Ha which generally have a good reputation. Some said it was likely a mismatch of darks and lights but even reshooting darks made no difference.
So its either:
1. defective coatings with holes in them
2. mismatched darks and lights. Can you post your dark?
Also if you use median combine you are less likely to see these sorts of artifacts that thrive if you use sum combine where all defects get added together with no outliers being removed.
Greg.
Hi Greg
Thanks for your reply. I have the benefit of having the LRGB frames in front of me to compare and it looks as if the holes are in the same place on each frame, so I think what is causing the holes is probably on the sensor rather than the filters?
The CFW10 belongs to Gerald and his LRGB filters are 1 ¼ inch Astronomik and they all appear to be parfocal too!
Dennis,
while no expert on this, I have had similar with some of my Ha shots.
I "think" or suspect it has something to do with the way I calibrate, or with the calibration frames.
Are these blemishes on the L, R, G, and B subs as well? Have you tried stacking with calibration? Just a though.
Gary
Dennis,
while no expert on this, I have had similar with some of my Ha shots.
I "think" or suspect it has something to do with the way I calibrate, or with the calibration frames.
Are these blemishes on the L, R, G, and B subs as well? Have you tried stacking with calibration? Just a though.
Gary
Hi Gary
Thanks for your reply. Yes – the “holes” appear to be in the same location in each LRGB sub frame as when I blink them, they don’t move at all.
Interestingly enough, I must have had a pretty accurate polar alignment as when each frame downloaded (via CCDSoft) on the PC, it registered perfectly with the previous frame as initially, I though that the camera wasn’t downloading, until I saw the filename change in the header.
This is looking more like a fixed pattern issue on the sensor I think?
Here is a quick assembly of the LRGB frames, a 1024x768 crop from the centre of the 1600x1200 fame.
I'm no CCD expert either but these look like hot pixels. They appear on your light and dark frames and most are rendered more or less invisible by calibration. The really bad (= bright = close to full well capacity) hot pixels though leave a relatively darker pixel after calibration that is made worse by stretching and become obvious in the higher signal areas of your image. The only way to remove these after the fact is to clone them out. The best thing to do is dither your light frames so they don't appear in exactly the same location in your lights. Then you can use a sigma reject algorithm and/or do a median combine to eliminate them altogether. Dithering is extra work but it is very effective at removing artefacts.
CCDsoft 2 also allows you to create bad pixel maps for your sensor and, I think, it can be applied to average the bad pixel using the surrounding pixels thereby eliminating them.
Darn Den, the one time you nail polar alignment it bites you in the proverbial, LOL.
Bump the tripod and then use median to get shot of them next time.
Gary
I agree with Marcus. Also CCDs do deteroriate over time. They get damaged by cosmic rays and a sensor that had virtually no defects a year ago may suddenly have new hot pixels. So you need to replace your dark library every now and then.
How old are the darks or are they fresh?
How did you combine the darks to form a master dark?
If the darks are old then try taking new darks. If the darks are fresh then perhaps the way they were taken didn't match for some reason.
Sigma reject is the usual dark combine method to get rid of non repeating elements like a random cosmic ray.
As mentioned dithering gets rid of these fixed noise patterns.
Thanks for your reply Marcus. Hmm, some new territory and concepts for me to explore with all that information – it seems the learning and tweaking never stops! Oh well, at least the climb is less steep now!
Much appreciated!
Cheers
Dennis
Quote:
Originally Posted by marc4darkskies
I'm no CCD expert either but these look like hot pixels. They appear on your light and dark frames and most are rendered more or less invisible by calibration. The really bad (= bright = close to full well capacity) hot pixels though leave a relatively darker pixel after calibration that is made worse by stretching and become obvious in the higher signal areas of your image. The only way to remove these after the fact is to clone them out. The best thing to do is dither your light frames so they don't appear in exactly the same location in your lights. Then you can use a sigma reject algorithm and/or do a median combine to eliminate them altogether. Dithering is extra work but it is very effective at removing artefacts.
CCDsoft 2 also allows you to create bad pixel maps for your sensor and, I think, it can be applied to average the bad pixel using the surrounding pixels thereby eliminating them.
Now I find that I have to delve into the depths of data normalisation, Sigma, Poisson & Gaussian stuff, statistical analysis, photon flux, dithering, etc.
Lucky that I’m a patient person and that we have some good brains here to help me out.
Cheers
Dennis
Quote:
Originally Posted by gbeal
Darn Den, the one time you nail polar alignment it bites you in the proverbial, LOL.
Bump the tripod and then use median to get shot of them next time.
Gary
Thanks for your reply. The darks were taken the same night as the M16 data and I used CCDStack to combine the 5 x 5 min Darks as follows:
Sigma reject Mean
Sigma multiplier 3
Iterations 1
Which are the default values (I think) as I haven’t changed these pending my better understanding of what is going on under the hood!
Cheers
Dennis
Quote:
Originally Posted by gregbradley
I agree with Marcus. Also CCDs do deteroriate over time. They get damaged by cosmic rays and a sensor that had virtually no defects a year ago may suddenly have new hot pixels. So you need to replace your dark library every now and then.
How old are the darks or are they fresh?
How did you combine the darks to form a master dark?
If the darks are old then try taking new darks. If the darks are fresh then perhaps the way they were taken didn't match for some reason.
Sigma reject is the usual dark combine method to get rid of non repeating elements like a random cosmic ray.
As mentioned dithering gets rid of these fixed noise patterns.
Were the darks taken at the same temperature and binning?
Is it possible you had autodark selected on the camera control software when you did the lights and subtracted the darks twice as a result when you callibrated the lights later?
The Darks were captured at the same temperature and binning as the light frames and with exposures of 5 mins I made sure that Auto Dark was set to none!
I took x5 off 5 min Darks at the start of the run.
In the meantime, I am going to run through my processing workflow once more, slowly, just to check that I was pointing to the correct files and making the right choices during the processing cycle.
I’ll report back when I have done this, probably in a day or two.
Thanks to everyone for their help and suggestions, all of it has been useful and much appreciated.
Cheers
Dennis
Quote:
Originally Posted by gregbradley
Were the darks taken at the same temperature and binning?
Is it possible you had autodark selected on the camera control software when you did the lights and subtracted the darks twice as a result when you callibrated the lights later?
Take some then use the adaptive dark settings in CCDStack. I use the RMS value. You need bias frames to calibrate the master dark frame. This should clear most of these overcompensated dark pixels (which are really bright pixels). Also, takes lots of really long darkframes. I use 30 minute darks for everything up to 30 minute subs. A nice cold cloudy night is perfect for making lots of long darks.
