“Holes” in M16 LRGB Frames?

[Dennis]

Hello,

I need some advice on addressing imaging artefacts in some recent M16 data that I grabbed whilst waiting for Neptune to climb out of the murky lights of Brisbane, using the C9.25 F10 with x0.63 Reducer/Corrector and Gerald’s ST2000/CFW10 combo.

When I came to process the LRGB frames in CCDStack I noticed what appeared to be “holes” in the data and these appeared at the same position in each frame so a Median combine couldn’t get rid of them. CCDStack reports them at around 6000-7000 ADU with the surrounding background starting at around 7,000 ADU.

I tried applying the “Hot/Cold Pixel” function in CCDStack and this failed to remove them. Other data rejections techniques in CCDStack look as if they remove good data in the frame when I try to “kill” these holes.

I’ve attached an x2 enlargement from the combined 12x5min Lum frames to show these “holes”.

I’d appreciate any advice on how to target these holes as I have 9x20 min sub frames revealing Nereid’s motion (moon of Neptune) and they are also peppered with these holes. Thanks!:)

Cheers

Dennis

[Tandum]

Did you take any flats dennis? Are they in the flats as well?

[Hagar]

Hi Dennis. I am sorry I cannot shed light on the cause but I can give you a method to remove these holes. It does appear to be some spots on some surface of the imaging train or in a calibration file.

Using CCDStack:
1. Select "Process", "Data Reget"
2. Select "Free Hand Draw" set pixel width to 1 pixel.
3. Mark all holes with a mouse click over them. Click apply.
4. Select "Grow" from reject menu. Set width to 5 pixels and apply.
5. From reject menu, select set pixel value to missing pixel and apply.
6. From reject menu select "Interpolate missing pixels" set value to 3 and iterations to 5. and apply.

This proceedure should remove these holes and fill them with an interpolated value based on the surrounding values.
I added an image repaired with a couple on the right side not repaired.
Sorry about the quality but it is a bit hard with a JPG.

I hope this helps.

[gregbradley]

I have seen similar with a Baader 7nm Ha filter. In that case I believe it ended up being poor coatings on my particular Baader Ha which generally have a good reputation. Some said it was likely a mismatch of darks and lights but even reshooting darks made no difference.

So its either:

1. defective coatings with holes in them
2. mismatched darks and lights. Can you post your dark?

Also if you use median combine you are less likely to see these sorts of artifacts that thrive if you use sum combine where all defects get added together with no outliers being removed.

Greg.

[Dennis]

Hi Robin

Thanks for the reply. The Lum frames have been reduced with Darks (5 min) and Flats (5 secs) and the Flats didn’t show any of these artefacts. They were nice and smooth, apart from the vignetting and dust bunnies!

Here’s the stretched Lum Flat.

Cheers

Dennis

[Dennis]

Hi Doug

Thanks for the quick and comprehensive reply, the results do look very promising. Your re-worked example is definitely superior to the “Patch” tool that I tried to use in Photoshop.

The anomalous pixels are sometimes single pixels and sometimes small clusters and their ADU value is mostly within 10’s to 100’s of the fainter background stuff although they appear quite dark in my example as the data is Scaled with DDP applied in CCDStack.

They appear to be in the same location on all each of the L,R,G and B frames so it looks as if those particular pixels seem to be less sensitive that their neighbours, albeit by small amounts, which then get reinforced through the stacking process?

Cheers

Dennis

[Dennis]

Hi Greg

Thanks for your reply. I have the benefit of having the LRGB frames in front of me to compare and it looks as if the holes are in the same place on each frame, so I think what is causing the holes is probably on the sensor rather than the filters?

The CFW10 belongs to Gerald and his LRGB filters are 1 ¼ inch Astronomik and they all appear to be parfocal too!

Here’s the dark frame (5x5 mins).

Cheers

Dennis

[gbeal]

Dennis,
while no expert on this, I have had similar with some of my Ha shots.
I "think" or suspect it has something to do with the way I calibrate, or with the calibration frames.
Are these blemishes on the L, R, G, and B subs as well? Have you tried stacking with calibration? Just a though.
Gary

[Dennis]

Hi Gary

Thanks for your reply. Yes – the “holes” appear to be in the same location in each LRGB sub frame as when I blink them, they don’t move at all.

Interestingly enough, I must have had a pretty accurate polar alignment as when each frame downloaded (via CCDSoft) on the PC, it registered perfectly with the previous frame as initially, I though that the camera wasn’t downloading, until I saw the filename change in the header.

This is looking more like a fixed pattern issue on the sensor I think?

Here is a quick assembly of the LRGB frames, a 1024x768 crop from the centre of the 1600x1200 fame.

Cheers

Dennis

[marc4darkskies]

I'm no CCD expert either but these look like hot pixels. They appear on your light and dark frames and most are rendered more or less invisible by calibration. The really bad (= bright = close to full well capacity) hot pixels though leave a relatively darker pixel after calibration that is made worse by stretching and become obvious in the higher signal areas of your image. The only way to remove these after the fact is to clone them out. The best thing to do is dither your light frames so they don't appear in exactly the same location in your lights. Then you can use a sigma reject algorithm and/or do a median combine to eliminate them altogether. Dithering is extra work but it is very effective at removing artefacts.

CCDsoft 2 also allows you to create bad pixel maps for your sensor and, I think, it can be applied to average the bad pixel using the surrounding pixels thereby eliminating them.

Cheers, Marcus

[gbeal]

Darn Den, the one time you nail polar alignment it bites you in the proverbial, LOL.
Bump the tripod and then use median to get shot of them next time.
Gary

[gregbradley]

I agree with Marcus. Also CCDs do deteroriate over time. They get damaged by cosmic rays and a sensor that had virtually no defects a year ago may suddenly have new hot pixels. So you need to replace your dark library every now and then.

How old are the darks or are they fresh?

How did you combine the darks to form a master dark?

If the darks are old then try taking new darks. If the darks are fresh then perhaps the way they were taken didn't match for some reason.
Sigma reject is the usual dark combine method to get rid of non repeating elements like a random cosmic ray.

As mentioned dithering gets rid of these fixed noise patterns.

Greg.

[Dennis]

Thanks for your reply Marcus. Hmm, some new territory and concepts for me to explore with all that information – it seems the learning and tweaking never stops! Oh well, at least the climb is less steep now!:)

Much appreciated!

Cheers

Dennis

[Dennis]

Hi Gary

Yeah – I figured that I had reached the summit;

Polar alignment – fixed
Mount balancing – fixed
Cables strain relieved and routed – fixed
Focuser braced with elastic bands – fixed
Motorised focuser – fixed
Carbon Fibre tube – fixed
Gerald’s ST2000 – fixed
Attitude – fixed

Now I find that I have to delve into the depths of data normalisation, Sigma, Poisson & Gaussian stuff, statistical analysis, photon flux, dithering, etc.:)

Lucky that I’m a patient person and that we have some good brains here to help me out.;)

Cheers

Dennis

[Dennis]

Hi Greg

Thanks for your reply. The darks were taken the same night as the M16 data and I used CCDStack to combine the 5 x 5 min Darks as follows:

Sigma reject Mean
Sigma multiplier 3
Iterations 1

Which are the default values (I think) as I haven’t changed these pending my better understanding of what is going on under the hood!

Cheers

Dennis

[gregbradley]

Were the darks taken at the same temperature and binning?

Is it possible you had autodark selected on the camera control software when you did the lights and subtracted the darks twice as a result when you callibrated the lights later?

Greg.

[Dennis]

Hi Greg

The Darks were captured at the same temperature and binning as the light frames and with exposures of 5 mins I made sure that Auto Dark was set to none!

I took x5 off 5 min Darks at the start of the run.

In the meantime, I am going to run through my processing workflow once more, slowly, just to check that I was pointing to the correct files and making the right choices during the processing cycle.

I’ll report back when I have done this, probably in a day or two.

Thanks to everyone for their help and suggestions, all of it has been useful and much appreciated.:)

Cheers

Dennis

[rat156]

Have you taken Bias frames?

Take some then use the adaptive dark settings in CCDStack. I use the RMS value. You need bias frames to calibrate the master dark frame. This should clear most of these overcompensated dark pixels (which are really bright pixels). Also, takes lots of really long darkframes. I use 30 minute darks for everything up to 30 minute subs. A nice cold cloudy night is perfect for making lots of long darks.

Cheers
Stuart

[mithrandir]

Can you dither when you are shooting? Trade off a few pixels at the edge of the frames for giving the stacking a chance to reject the bad pixels.

[multiweb]

Have you tried dithering? Maybe an easier way to get rid of those stuborns pixels? Really nice shot btw. :thumbsup:

[Dennis]

Thanks for all your help guys, I think that I may have found a candidate for the problem; me, operator error!:rolleyes:

Using CCDStack, after calibrating the raw files with the master Dark and Flat Field Frames, I would normally use the Data Reject procedure of “reject hot/cold pixels”. Although I’m not absolutely sure, when I ran the “hot/cold pixel” procedure to identify them based on the selection criteria, I think (?) that I may have forgotten the next step, to then impute those pixels. As a result, they remained in the frames and so appeared in the final image as the rogue pixels.:(

As part of my investigation I opened and I blinked the Darks and Flats to make sure there were no rogue frames in the sets and the image data appeared consistent from frame to frame, so I thought the problem would not likely lie here.

I then ran through my workflow and ticked off the procedural steps to make sure that I had followed them accurately. Doing this, the “holes” vanished!:)

So, once again, thank you for all your advice and suggestions in helping me resolve this problem – I really appreciated your efforts and please accept my apologies for my slack processing.:sadeyes:

But, we do have a happy ending. Here is M16 in LRGB using the C9.25 at F6.3 with the ST2000 and CFW10 with a 1st Qtr Moon some 90 degrees away. As Gerald had a set of Ha, OIII and SII filters in the CFW10, I also managed to grab 20mins through each filter although auto guiding wasn’t as accurate in these latter frames as I had gone past the meridian and the mount was “floating” in the gears.

L=12x5min, RGB=1x10 min each.
Ha, OIII and SII 1x20 min each.

Cheers

Dennis

[rat156]

So does this make in an unHoly M16?

Fear not, we've all done silly things whilst processing.

Cheers
Stuart

[gregbradley]

Well done for tracking down what the cause was. That gets rid of a mystery.

Nice image in the end.

Greg.