Ideal Number of Subs.

[PKay (Peter)]

Once again I am trying to put an understanding of difficult concepts into words.
Please feel free to contribute if I am in error.

I was reading PI documentation (in the Image Integration process) that is titled ‘How many images?’.
With regards to improved SNR.
It gives a graph that says 30 images is ideal, and beyond 50 there is little to be gained.

My understanding. There are two concepts involved here.
Firstly say a pixel can have a count from 1 to 10 (for simplicity).

Maximum signal:
Take any target for example an average star.
It is sending us photons at a certain rate (brightness) that are then captured in pixels.
Once a pixel is full (a value of 10) there is no more signal to be found.
For example a single image can capture a bright star. By taking another 20 images, no new detail will be revealed. After integration, the average or mean would will still be 10.

Atmospheric Noise:
Take the above example and call the ‘centre most’ pixel capturing the star Pixel01.
Move to the left (say 50 pixels) where there is no signal and call that Pixel50.
Pixel01 will capture photons on average faster than Pixel50.
(This is under the assumption that the target is bright enough to be above the average noise level).

Example: Take several exposures (say 100 sec).

Exposure 01:
Pixel01 = 10.
Pixel50 = 5.

Exposure 02:
Pixel01 = 10
Pixel50 = 5

If the same values were repeated for 10 or 20 or whatever images:
In this imaginary scenario, a single image would capture all the information that you can.
This is because the average or median values would be the same, for signal and noise.

Another example:
If pixel50 (the noisy one) varied about ie: 4,5,6,3,8,2 etc
It would still reach an average (or median) value very quickly, and the theory says that is after 30 images.

It can also be noted, at this stage, clever maths can recognise a pixel that varies greatly from the average (or median) value. This is how noise is isolated.

So in conclusion, what does this mean, and how do we use it?
I believe that once again, it is the choice of correct exposure time that rules, and that is more important than the number of subs going beyond 30 (50 tops).
Better to take 30 - 50 exposures at the correct exposure than 200 (at any setting).

[RickS (Rick)]

There is no magic number of subs beyond which there are diminishing returns. Provided you are taking sky limited subs the number you take is irrelevant and the only thing that matters is the total integration time. That's why people with low read noise cameras can get great results with hundreds of very short subs.

Cheers,
Rick.

[PKay (Peter)]

Thanks Rick

Will have to digest what you have said.


I am keen to understand why the chap that writes the code for PI, says that there is a limit.

The above essay was trying to answer that.

Maybe there is a link with the brightness of the target, as compared to the level of background noise?

I can feel some nasty maths coming on, something I was trying to avoid.

[RickS (Rick)]

Originally Posted by PKay

I am keen to understand why the chap that writes the code for PI, says that there is a limit.

Hi Peter, are you referring to this comment?

“There is, however, a law of diminishing returns. Statistically, the limit from which you can obtain significant benefit is thirty images.” – Inside PixInsight, Warren Keller

Warren knows his way around PI and produces some lovely images, but in this case I think he's promulgating a myth that originated in the days when astro cameras all had terrible read noise. I even gave a presentation debunking this myth to my astro club

Originally Posted by PKay

Maybe there is a link with the brightness of the target, as compared to the level of background noise?

There is a link to brightness. To get good SNR you need to collect lots of photons. A bright area doesn't need a lot of integration time to get to an adequate SNR because the photons are coming thick and fast!

The problem is that our images usually contain dim features as well. If we want to get good SNR in areas of faint nebulosity or the halo of a galaxy then a lot of integration time is required. This won't make much improvement to the bright features (which do suffer from diminishing returns) but it does matter for the faint stuff.

Originally Posted by PKay

I can feel some nasty maths coming on, something I was trying to avoid.

It is tough to understand without the mathematical basics. Our intuitive ideas about how this stuff should work are often wrong. The maths isn't very difficult but it would be tough to figure out without at least a little background in statistics and algebra.

[PKay (Peter)]

Good reply Rick

Just the sort of stuff that I, as a beginner need.

Will have a go at re-writing the essay incorporating your thoughts.

But straight off the bat, I can see that careful planning about your intended target won't be time wasted.

As an aside: I studied 7 units of advanced engineering mathematics, and understood none of it

[raymo]

I don't remember his name, but some 5 or 6 yrs ago there was a member
here who posted some terrific DSLR images resulting from 500 or more
subs of around 30 or 40 secs.
raymo

[Amaranthus (Barry)]

Beyond avoiding saturated stars and overwhelming skyglow (shorter subs = better) versus minimising the relative contribution of read noise (longer subs = better), an advantage of many subs is that it improves the ability of stacking routines, such as sigma clip or median, to remove artifacts. More, shorter subs, also means that you can toss out those that were affected by tracking errors, wind etc.

As Rick noted, if we had near-zero read-noise cameras (and a powerful computer for processing), then in theory, we should be taking 3600 x 1 second subs for every hour of integration. But until that happy day, we're left with a trade off.

[xelasnave]

Easy question for me...exposure times are determined by the mount and how well it is aligned as the first consideration.
Second in the city that pre determined exposue time is then tested at varying isos and a choice is made on how I feel...if I feel conservative I go 800 iso if I feel daring I go 1600 iso ..if I feel like experimenting I will try 400 iso or short shots at 6400 iso...fills the time...

ath the moment the 30 second trip works very well for me...firstly my polar is off because I move the mount for building this cube box...and irrespective of the mount etc the sky seems to say to my that 2 minutes will stop it ..so even if I could say do 30 at 5 minutes to fit the numbers the glow may kill it...
Up North its different..I have had a single wield field exposure of 1 hour 20 minutes...manually guided.
I would think the more exposures the better if for no other reason than percentage wise you should have more good ones...and you can always tell the stack program to select if you like to follow the 30 idea..limit its stack to 30.
I stacked hundreds of eta and it semmed better.
alex

alex

[Atmos (Colin)]

As Rick has mentioned, it is all about total integration time and there is an easy way of explaining this.
Let's say you take 49x60s luminance frames (I pretty much did this a few nights ago with my QHY163M [same as an ASI1600]), this will give 7x the SNR on any given area over a single 60s frame. You really only have 49 minutes of integration though.
If I was using an older noisy camera like the KAF-8300 sensor I would need 600s exposures to get the same amount above the read noise. So 49x600s is just over 8 hours of exposure.

49x60s vs 49x600s, well you don't have to think too hard to figure out which one is going to be better. What you need to consider is that in the fainter areas of your 60s images there are going to be parts that are above the noise floor BUT don't have a lot of signal.
Let's also say that some of the faint nebulosity only emits several photons per minute and only sits JUST above the noise floor. In a single 60s exposure it is going to look like it is just random noise. After 49 images it'll be 7x stronger but noise will still by and far be the dominating factor.

If images are above the noise floor, 10x60s will give more or less the same result as 1x600s. Technically the 10x60s will be a little better but it won't be noticeable at such short integrations.

So, in short, don't worry about the amount of exposures but only the total integration times. Remember, faint things don't emit much because they're faint! In shorter exposures you'll have photons that may NOT show up in every frame so what you need is a bucket load of them to properly sample the frames that do have that faint stuff.

[xelasnave]

Putting a high number to the test now.
Managed more eta last night. It travels nicely between two tree branches and I get a near two hour opportunity.
So I put 472 frames in Deep Sky Stacker, told it to select 85%.
That was at 2 am.
It is now 9.44am and says only another hour left to process the 361 frames it selected.
I wont be able to report the result as I now have to go out for the rest of the day...if my mate turns up as promised.
Alex

[Atmos (Colin)]

The effectiveness of stacking lots of subs also comes down to whether or not your swamping read noise.

If you’re not getting above the noise floor then you’ll get to a point where you’ll smooth data above the noise floor BUT never get fainter.
100x60s will not be beaten by 12,000x1s even though it has half the amount of total integration.

The quality of your calibrations will also determine stacking depth. If you don’t calibrate then Fixed Pattern Noise very quickly becomes the dominating noise. If your calibrations frames aren’t good quality then FPN will still be an issue. If you don’t dither sufficiently then FPN will be an issue.

Something else that more frames allows you to do is better sample the accuracy of data. Sometimes when pixel values vary it is due to noise but it can also be due to accuracy. Using the values you had in your first post: 2,3,4,5,6,8
I’d you median it will be 4/5 as it is an even number. If you average all of those you get 4.667. If you average while clipping the high and low values (3,4,5,6) you get 4.5.
Which one of these is correct? The more subs you have the better and more accurate that number becomes. This is where contrast comes in, the accuracy of pixel to pixel variation. With 30 subs it will be more accurate than 1 but with 100 it will be even better!

[xelasnave]

Well mate still has not turned up and when I checked the lap top all I had was the openning screen...maybe it was too much but I suspect maybe an update that interupts stuff ...so I started it all again...
Maybe a result 12 hours from now.
I hide the net connector in the hope that that will stop unwanted updates...
Alex

[cometcatcher (Kevin)]

Originally Posted by raymo

I don't remember his name, but some 5 or 6 yrs ago there was a member
here who posted some terrific DSLR images resulting from 500 or more
subs of around 30 or 40 secs.
raymo

You've forgotten me already?

I like to try for 1000+ 30sec subs. Works for me.

[xelasnave]

Back in and still 4 hours left before I know if it worked.
Alex

[xelasnave]

Disappointment.
Went right to end even went through "final picture" and then "out of memory"...so cut down to 248 light frames, 6 darks no flats and 5 bias.
It can get to work while I sleep...and if it doesnt work I guess I will try a few less frames...
Alex

[billdan (Bill)]

Alex, try and split your 470 subs into batches of 50 or 60, stack each batch and then stack the stacks,

[xelasnave]

Originally Posted by billdan

Alex, try and split your 470 subs into batches of 50 or 60, stack each batch and then stack the stacks,

Thanks Bill I may try that.
The last stack seemed to work as the image was on screen when I woke but when I left clicked it disappeared???
Anyways I wont be able to do much for a while as I have so many other things on my plate I wont have time for anything else.
Alex