Crux w/ DSLR: lots of detail but almost no colour (?)

[janoskiss]

I've just started learning about software tools and processing raw image data from the weekend.

Here is the Cross from 9x30s exposures (+ one dark frame) at f/5.3 (IIRC), ISO 1600, 135mm Tamron lens, Nikon D3300. Pre-processed using dcraw, frames combined in DSS. DSS output further processed with Imagemagick and GIMP.

I am having a bit of trouble getting decent colour out of raw files. They don't look so bad (though still lacking a bit of lustre) after raw-to-tiff conversion but after DSS they either have very little colour or the colours are off, or there is a lot of hue noise.

I'm sure I'll figure it out in time but tips from experienced folks would be most welcome. Processing is definitely a steep learning curve.

Larger less compressed versions:
* Quarter size (1MB) (https://s6.postimg.org/6pxvtlc5b/crux_dss_02_imgk_01_g03.jpg)
* Half size (4MB) (https://s6.postimg.org/ou5hryz7l/crux_dss_02_imgk_01_g02.jpg)

[Camelopardalis]

Try increasing the colour saturation...

30 seconds just isn't a long exposure ;)

[janoskiss]

There is not much saturation there to increase. If I crank it up I just get hue noise. There is colour in the individual frames but in the DSS output the colour is either off mostly absent. I've tried various settings but I'll experiment more. ;)

Edit: Actually the colour is not great on my raw-to-tif conversions either. Still getting my head around white/colour balance stuff...

[bojan]

Steve,
DSS should be configured "per channel background calibration".
Also, save the result as tiff, but do not apply any adjustments..

Not sure how DSS handles Nikon raw format... there is no trouble with Canon though.

[Camelopardalis]

DSS has a habit of muting colours, but what signal there is, is there.

If you're finding colour noise when you stretch saturation, then your signal-to-noise ratio likely isn't high enough, so you need more subs. At f/5.6, I'd definitely suggest longer subs too.

Expose long enough such that the big peak in the histogram is exposed to about 25% from the left (going towards the right).

[janoskiss]

Thanks for the tips. I did make adjustments before saving from DSS. I figured at 16-bit it'd preserve more dynamic range if I stretched out the values some. Otherwise the images look very dark. But I'll heed your advice, Bojan. Thanks! ;)

I did not use raw with DSS but tiff converted from raw with dcraw. The reason for that is to cut down on DSS' workload. (That way I can also reduce the size of the images before feeding them to DSS. I only have a Linux PC, and DSS won't run in Wine, so I've been using my spouse's Win7 laptop when she's not using it. It's not a particularly powerful machine. I already got one out-of-memory error with the 4GB RAM it has.)

In the long run I hope to get away from DSS and do all the processing with lower level tools, but as it's early days I'd prefer to see some results and want to understand all the steps before I go full-on nerdtard with the processing.

[raymo]

If your ambient temp was much above about 11-12* at the time of image
capture, you needed a lot more subs to subdue the noise[say 40-50].
Why are you hoping to get away from DSS later? its probably the most
widely used stacking software out there.
raymo

[Camelopardalis]

Yeah you definitely don't want to do anything to the images before DSS doing its stuff. Use the raw files instead. Debayering a raw image is not particularly computationally intensive.

What consumes most of the time when stacking is the star registration, and the default settings in DSS exacerbate this if you have a widefield with lots of stars (intentionally!). When you go to Register Checked Pictures and the Register Settings dialogue pops up, click the Advanced tab and increase the Star Detection Threshold. For it to work well, you only need about 200 stars, so increase the threshold accordingly. This will reduce (sometimes dramatically) the time taken to process the images.

Once you get the result, it's a 32-bit TIFF file...you will need to bring the black and white ends of the histogram in quite a bit to see your data. That is normal :) Once you've got the "luminance" about right, then you can think about stretching the saturation. From an unmodded DSLR, you shouldn't have too much work to do to get the colour balance right.

[janoskiss]

Might as well expose my ignorance (only way I'll learn):
What's a sub?


That's just how I roll. I have a maths, physics and scientific computing background. Building my own numerical tools lets me optimise things to my needs, have more control, and add new features as I see fit. I've written a fair amount of special purpose image processing software in the past - stuff that is just not available commercially because it's too niche. DSS is great though, don't get me wrong.

[janoskiss]

Great tips Dunk, thanks! I'll have a go at that asap.

[Camelopardalis]

sub = sub-exposure, one of the many that will make up your total exposure.

Don't be afraid to ask - there are no stupid questions, only stupid answers ;)

I don't want to sound like a stuck record, but as I said in your DSLR thread, exposing the image correctly is absolutely key to getting good results. The best thing to do is to expose for long enough that the background sky level (the big peak in the histogram in most images) has higher values than the read noise (+thermal noise).

[Camelopardalis]

On the subject of software, and this is just IMO with my own data/experiences...DSS is a good start, but PixInsight does a better job of stacking.

Star registration and image stacking is well-trodden ground from an algorithmic perspective though.

[janoskiss]

Thanks, Dunk. This was my very first dark sky weekend astro-imaging (or any sort of AP), so exposures were as long as I could make them given mount + camera mechanics restrictions. I'm definitely getting an intervalometer at the very least next payday.

I do appreciate the need to get above the noise floor. Multiple short-exposure frames (or "subs") cannot effectively replace a single equal-total-time continuous shot unless the signal is well above the noise in each frame.

PixInsight looks great. Never heard of it before. I guess it's because it's not free like DSS that it's not so popular. (But I don't mind paying for software if it's a good product and value for money.) I'll give the trial version a go for sure! :thumbsup:

[raymo]

For people such as yourself its not a problem, but PixInsight has a steeper
learning curve than DSS, which is a problem for some old f---s like me.
raymo

[janoskiss]

It runs on Linux, which is a big bonus for me.

... and as far as writing my own software when I can and have the time to do so, it's also a matter of hobby and getting to the nuts and bolts of the process. It's the same reasons others might cite in response to questions along the lines of: Why do you spend all that time, effort and money taking astrophotos when there are heaps better images freely available online from Hubble and other amazing telescopes?

[RickS]

A saturation boost will work best if you limit the effect to the brighter parts of the image. As you've apparently noticed, applying a boost to the dim areas will just emphasise chrominance noise.

In PixInsight this is easily done with a clipped luminance mask. Dunno how to do it in other environments.

Cheers,
Rick.

[janoskiss]

You guys rock and IIS rocks. So much good advice. Trying to put it all into practice finishing processing Omega Cent + Hamburglar 5128 --- meanwhile 45 mins of Virgo cluster are slow roasting in the DSS furnace . :thanks:

One final bottleneck is that I don't have Photoshop or any other similarly user friendly tool capable of handling 32- or even 16-bit images. I do use Imagemagick but it's practically command line only so a fair bit of trial and error and patience is required (there is a GUI but it's very rudimentary and crippled compared with command line).

[Camelopardalis]

If you work through the maths on signal to noise ratio from stacking multiple (suitably exposed) subs, you might find the SNR is better than a single long sub ;)

PixInsight is THE processing software :lol: as raymo says it has a bit of a learning curve, but so did Photoshop first time I used it...



Download one of the development snapshots of GIMP (2.9+), as they have implemented higher bit depths now.

[raymo]

You can subscribe to the latest Photoshop [you get Lightroom as well]
for $11.99 per month, and you keep getting all the latest updates. You can
pull out whenever you wish. If you stop paying, it stops working after 3 months.
raymo

[janoskiss]

Yeah sure once you're above a certain threshold. But only then; as you were implying with your "broken record" comment. Otherwise we would not need tracking. We'd just take short snaps, or a video even of deep sky and stack like mad, like panetary imagers do.

Thanks Raymo $12/month no strings could be worthwhile.

@Camelo yeah I know about Gimp 2.9 and I have the sources sitting on my computer, but the dependencies for compiling it on my current system are a pain I'm yet to be prepared to deal with. I'm quietly hoping Gimp folk will finaly release a stable 2.9 version (at long last: it's been 10+ years wait for more-than-8-bit-colour support).

I used to use Cinepaint but that's become source only too, with way too many annoying dependencies.

[Camelopardalis]

Yes and no...if the noise has higher values than the background sky (underexposed) then you can still stack a lot of subs and reduce the noise, it just takes more subs. The noise reduces with the square root of the number of subs, whereas the signal is additive. Exposed optimally, the noise is swamped by the values of the background sky.

Just to blur the boundaries, lookup lucky imaging...it's basically where small sensors suitable for planetary imaging (high sensitivity, very low read noise) are being used with fast scopes and taking lots (thousands) of short exposures and getting some really interesting results with DSOs :eyepop:

For testing GIMP development, look into virtualisation platforms such as Virtual Box, VMware Player or similar. You can then install one of the more cutting-edge distributions into the virtual machine, with all its dependencies, without messing up your running system ;)

[janoskiss]

Assuming the noise is completely random and independent of the signal, yes. Thank you Central Limit Theorem (https://en.wikipedia.org/wiki/Central_limit_theorem) (what a gem of a theorem!). But there is a garbage-in = garbage-out point where it becomes futile to do anything practical with short exposures. But why am I telling you this? You're the one who keeps emphasising the need to expose for long enough to get the background sky above the noise floor. :P


I watched a few talks on youtube last night on AP and heard "lucky imaging" mentioned several times. I thought I got the planetary vs deep sky thing but I'll look into it some more. I presume if you have an ultra low- (near no-) noise sensor and enough processing power you could in principle take very short exposures.

The ultimate form of this is a photon count detector, where each "exposure" is just a single photon hitting one detector pixel. For scientific/medical applications there is work under way on detectors that can not only log where and when each photon hits, but the energy of the photon as well (the precise colour). I understand that apart from the energy detection part there are CCDs available that come as close to no-noise as stray light and cosmic rays allow. But for the foreseeable future I'll have to make do with my cheap DSLR and all its inherent shortcomings.

I realise there are ways to get Gimp 2.9 going but unless it comes bundled and ready to go with a bleeding edge distro I don't think I have the time and/or patience to make it work.

I am very impressed with dcraw btw and that it provides direct access to each raw pixel value beneath the Bayer filter. :)

[Camelopardalis]

Glad to hear you've been listening ;) :D

While the perfect sensor doesn't exist, a good sensor exhibits read noise that is normally distributed, or thereabouts.

The latest sensors...like the IMX224 (in the ZWO ASI224MC) have <1 e read noise in high conversion gain mode.

Previously I mentioned that another factor in getting good results is to know the equipment and play to its strengths. Regardless of what the read (or thermal) noise behaviour is, once you know what it is you can expose accordingly and overcome it :thumbsup: There's a lot of fun, and some pretty amazing results, to be had from inexpensive equipment!