Weird Entropy weighted results

[LewisM]

I was experimenting with DSS 3.3.3 to see which stacking routine would give the best results on various DSO types - nebulae and galaxies.

I tried Entropy weighted average (HDR), and got a nice picture - good contras and depth etc, but then a problem when put into PS.

I have streaks of RGB all over the image - see below. I tried a median filter to remove them, which did work, but where the streaks run through the GX, of course removing them is obvious.

Anyone know WHY this does this, and it's only using Entropy Weighted average (doesn't matter which sub setting like 3x drizzle etc I use). I'd like to use it as the images are rather good otherwise, but...

Photo 1 is a close up crop straight out of DSS with no processing. Image 2 has had contrast adjusted to show the phenomena easier. The RGB streaks are present on the ENTIRE image.

[Shiraz (Ray)]

hot pixels?

[multiweb (Marc)]

Definitely hotpixels yes. There are two ways you can get rid of them.

1_ shoot darks and dark subtract.
2_ make what's called a bad pixel map, paint the pixels with it then interpolate them with the neighbouring pixels.

This has to be done to each individual sub prior to debayering or registering.

[LewisM]

Well, I ususally don't eed or take darks with the SXVR-M25C (and the manual suggests not important either, and can DEGRADE the result), so I don't even have a library of them.

I know all about hot pixels, but these same pixels do NOT show on ANY other image EXCEPT if I use Entropy Weighted Average as the stacking parameter. In my usual kappa-sigma median, none. In ALL the others, except Entropy (and Maximum) I get these RGB streaks.

I'll continue to probably NOT use darks and use kappa-sigma median, but just wondering what.why this occurs in Entropy/maximum - obviously because of the high tonal contrast with these settings?

[multiweb (Marc)]

Nah... these are hot pixels alright. The only way to get rid of them is as explained above.

[Shiraz (Ray)]

the manual warns against using maximum - suspect the same applies to entropy, which is probably going to try to maximise local contrast in some intelligent way.

"Maximum
This is and ultra simple method which should be use with a lot of care. The maximum value of all the pixels in the stack is computed for each pixel.
It may be useful to find what is wrong in a stack by exhibiting all the defects of all the calibrated image."

ie if you want defects, use this.

maybe the simple solution is don't use it

[LewisM]

Oh, definitely not using either, though the Entropy type shows good promise for MUCH easier post-processing. I guess I will make a series of dark runs next time out, just in case I do want to try Entropy Weighted again.

In the meantime, I will continue to use Kappa-Sigma, and live with the extra dance-step of contrast tweaking


Thanks for all responses.

[multiweb (Marc)]

Originally Posted by LewisM

Oh, definitely not using either, though the Entropy type shows good promise for MUCH easier post-processing. I guess I will make a series of dark runs next time out, just in case I do want to try Entropy Weighted again.

In the meantime, I will continue to use Kappa-Sigma, and live with the extra dance-step of contrast tweaking


Thanks for all responses.

Look the problem is very simple but you need to understand what's going on to remedy to it one step at a time. Forget about entropy and weights and all other type of combining algos for now, it's only confusing you. They might have their own issues but that's irrelevant to the orignal problem. The bottom line is that in each subs you have bad, hot, cold pixels, defect lines or whatever you want to call them. That is a signature of your sensor and it will change with temperature and age.

These will appear in a dark, in a flat or even in a bias if it's a long enough exposure and in any sub you take with your camera regardless of the exposure time. Nebulosity is very cheap and will allow you to make a bad pixel map. If you are worried that applying darks will introduce noise to your subs (and it might well do) then don't use darks. You don't have to. I know I never used any for my QHY8. What you are interested in is where those bad pixels and lines are. So shoot 1 dark for let's say 20min at a given temperature then make a bad pixel map in a program that supports it. Nebulosity, CCD Stack also does. Then apply that bad pixel map to your subs. That will fix all of your problems and it is easy and simple to implement.

[LewisM]

Merci beaucoup monsieur!

[WingnutR32 (Sam)]

Interesting results, Lewis. I am yet to have a play with the different stacking options in DSS.

Marc, what stacking options do you generally use? Perhaps not even DSS?

[multiweb (Marc)]

Originally Posted by WingnutR32

Marc, what stacking options do you generally use? Perhaps not even DSS?

I haven't used DSS in over 5 years so I'm not the best person to give you any info on it but every program has its own brew of stacking processes. I mostly use CCD Stack. There is a plugin called CCDIS which does a good job. Bicubic B-spline for registration. I always mean combine my subs. Regardless of how you stack or combine your subs a solid understanding of calibration is necessary to achieve good results.

[WingnutR32 (Sam)]

That is all good, Marc, was just curious. I have started to use MaximDL registering and stacking.

It was more just curiousity as I had not ever changed from defaults on DSS.

[multiweb (Marc)]

Originally Posted by WingnutR32

That is all good, Marc, was just curious. I have started to use MaximDL registering and stacking.

It was more just curiousity as I had not ever changed from defaults on DSS.

MAXIM DL is good too.

[naskies (Dave)]

Originally Posted by LewisM

Well, I ususally don't eed or take darks with the SXVR-M25C (and the manual suggests not important either, and can DEGRADE the result), so I don't even have a library of them.

I'm curious what they mean by "degrade" the result? If you're shooting a bright diffuse object with only say 1 or 2 subs, then sure - I can understand that dark subtracting hot pixels might leave you with a few unsightly black spots.

However, if you're stacking multiple shots and dithering between them, then dark subtraction is very useful for cancelling out the read noise and thermal noise.

If you're applying flats, then dark and dark flat subtraction is essential - otherwise the scaling will artificially increase your read/thermal noise.

I'll continue to probably NOT use darks and use kappa-sigma median, but just wondering what.why this occurs in Entropy/maximum - obviously because of the high tonal contrast with these settings?

I had a quick read of the academic papers that DSS refers to for entropy-weighted stacking. Here's a quick go at an intuitive explanation:

One way to think of entropy is "information content". A completely black image (RGB = 0 0 0) contains very little information (low entropy) because you can reproduce it with the instructions "every pixel is black". On the other hand, a well-exposed photo contains a lot of information (high entropy) because to reproduce it you basically need to know the RGB of each individual pixel - it takes a lot more instructions to reproduce it.

Think of the M42 core taken with long exposures: it will probably be blown out to white (low entropy) but the rest of the nebula will be nicely detailed (high entropy). However, with short exposures the core will be highly detailed (high entropy) but the rest of the nebula will be faint or black (low entropy).

If you were to stack together an M42 image from both long and short exposures, you'd want the core to take the detail from the short exposures, and the surrounding nebulosity to take the detail from the long exposures. That is, you maximise overall image detail by selecting each part of the image from the sub with the most entropy. This is basically what entropy-weighted stacking does.

In your case - think of the entropy in a relatively dark, uniform area of sky compared to a similar patch of sky with a few hot pixels thrown in. The hot pixels basically make the image more detailed (hence higher entropy) and so entropy-weighted stacking will preferentially use the hot pixels... does that make sense?

For each set of equal length exposures, kappa-sigma (or similar) stacking of calibrated subs should give you the "best" results. If you had multiple sets of varying length subs (e.g. short/long for M42), then you could stack those calibrated/stacked subs with entropy-weighting to maximise the amount of detail in your final output.