#1  
Old 29-05-2016, 10:37 AM
codemonkey's Avatar
codemonkey (Lee)
Lee "Wormsy" Borsboom

codemonkey is offline
 
Join Date: Jul 2013
Location: Kilcoy, QLD
Posts: 2,058
What to do with them bad subs?

We've had some pretty average conditions here lately, even though the skies have been clear. Bad seeing some nights, very windy others, sometimes both simultaneously.

I'd captured a few good subs of my next target, but many more that were less than great and I wondered if I could use these less than great subs to reduce noise, whilst not having a negative impact on star sizes and finer detail.

I always weight my images prior to integrating them such that images with a lower FWHM and eccentricity have more weight than subs with higher values, and I still applied this to the examples attached.

This time I split up the subs. Anything less than or equal to 2.4" FWHM went into the good pile, everything else went into the bad pile. The bad pile contained subs up to 3.9". I then integrated the bad pile and good pile separately.

After integrating, I took the bad integration, cloned it and did a histogram stretch on it such that the background was very dark, but stars and the subject were well stretched.

I also cloned the good integration, then masked the image with the stretched bad integration (inverted).

From here, I used PixelMath to add the bad integration and the good integration together, weighted more strongly towards the bad one (which had many more subs, and thus less noise), and applied this to the masked good integration.

The result? The noise went down significantly, but I kept the nice stars and (most of) the details from the good integration. I believe I managed to retain 90% of the SNR when compared to an integration with the entire set of subs.

FWHM of the "all subs" integration was 2.46176", "good only" was 2.32288" and the merged one was 2.18288". That's not a typo, according to PI, the merged one had better FWHM. Not sure what to make of that.

Anyway, you could argue that it's not worth it to lose that bit of SNR, or that the marginal decrease in FWHM is simply not worth the effort, but I thought it was an interesting little experiment.

Of course, you could do many things differently. The masking in this case was entirely based on the intensity, but that may not always be ideal, you might have lots of dim, fine detail. You'd probably also be better integrating all the subs and using that instead of a manually weighted sum of the good/bad stacks for the lower signal areas.

Attached is a copy of one integration with all subs, another using the merged approach described here, and then a stretched copy of the difference which highlights the impact on star sizes.
Attached Thumbnails
Click for full-size image (all_subs.jpg)
91.4 KB56 views
Click for full-size image (merged.jpg)
95.4 KB51 views
Click for full-size image (diff.jpg)
114.1 KB50 views
Reply With Quote
  #2  
Old 29-05-2016, 06:05 PM
Shiraz's Avatar
Shiraz (Ray)
Registered User

Shiraz is offline
 
Join Date: Apr 2010
Location: ardrossan south australia
Posts: 4,918
well, that is something to think about - thanks for posting. Now to work out what your method does that reduces the FWHM?.
Reply With Quote
  #3  
Old 29-05-2016, 06:37 PM
PRejto's Avatar
PRejto (Peter)
Registered User

PRejto is offline
 
Join Date: Jan 2011
Location: Rylstone, NSW, Australia
Posts: 1,397
Yes, same lousy seeing here in Sydney, though a bit better tonight. I've been trying to get a decent amount of data on NGC4038 and had a very similar thought; could I somehow use the bad data together with the sharper data? I am more interested in using nebulosity from the bad data, but a smoother background also is tempting. Your experiment is most interesting!

Peter
Reply With Quote
  #4  
Old 29-05-2016, 07:02 PM
codemonkey's Avatar
codemonkey (Lee)
Lee "Wormsy" Borsboom

codemonkey is offline
 
Join Date: Jul 2013
Location: Kilcoy, QLD
Posts: 2,058
Quote:
Originally Posted by Shiraz View Post
well, that is something to think about - thanks for posting. Now to work out what your method does that reduces the FWHM?.
Not sure what's going on there. I did stick with Moffat 4 without analysing the MAD, so it's possible it might have changed the profile such that a different beta parameter might have given a better fit with a different outcome.

Quote:
Originally Posted by PRejto View Post
Yes, same lousy seeing here in Sydney, though a bit better tonight. I've been trying to get a decent amount of data on NGC4038 and had a very similar thought; could I somehow use the bad data together with the sharper data? I am more interested in using nebulosity from the bad data, but a smoother background also is tempting. Your experiment is most interesting!

Peter
Great minds eh Peter! I think careful masking could put some of that data to good use. I'm keen to hear about / see your results if you give it a shot.

I think my issue is actually a mount issue, rather than seeing I seem to be having problems with RA. I just disabled guide output and watched RA go almost 40" off in the space of a couple of minutes (was still going when I left it to gather data), whilst DEC drifted only 4". I'd been hoping it was just seeing, but now that I disabled guide output I think I'm going to have to be sending Luciano from Avalon an email. Man, I am so sick of hardware problems...
Reply With Quote
  #5  
Old 29-05-2016, 09:26 PM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
I have tried combining a best FWHM integration with a best SNR integration a few times in the past with a blend function favouring the FWHM integration for bright areas and the SNR integration for dim areas. Sounds very similar to what you've done, Lee.

I found that it works, but I haven't convinced myself that it gives a better result than applying a little careful noise reduction to the FWHM integration. If I was scrabbling really hard to find enough good quality subs I'd try it again, I guess.

I don't see why you'd get an actual reduction in FWHM. Sounds like you've just confused the measuring algorithm


Cheers,
Rick.
Reply With Quote
  #6  
Old 29-05-2016, 10:29 PM
Atmos's Avatar
Atmos (Colin)
Ultimate Noob

Atmos is offline
 
Join Date: Aug 2011
Location: Melbourne
Posts: 6,980
I had thought of doing something similar to that Lee in an attempt to salvage TERRIBLE data. Had really nice red and blue data but green was bad due to the auto guider going on the fritz. Wanted to use PixelMath and mask the green FWHM with the red and turn lines into fainter dots

Didn't work so well but what you've done sounds good. As for the lower FWHM, what you have done is increased the central region with the poor data which has increased the "half maximum". You've created an artificial cone
Reply With Quote
  #7  
Old 30-05-2016, 06:25 AM
codemonkey's Avatar
codemonkey (Lee)
Lee "Wormsy" Borsboom

codemonkey is offline
 
Join Date: Jul 2013
Location: Kilcoy, QLD
Posts: 2,058
Quote:
Originally Posted by RickS View Post
I have tried combining a best FWHM integration with a best SNR integration a few times in the past with a blend function favouring the FWHM integration for bright areas and the SNR integration for dim areas. Sounds very similar to what you've done, Lee.

I found that it works, but I haven't convinced myself that it gives a better result than applying a little careful noise reduction to the FWHM integration. If I was scrabbling really hard to find enough good quality subs I'd try it again, I guess.

I don't see why you'd get an actual reduction in FWHM. Sounds like you've just confused the measuring algorithm


Cheers,
Rick.
Yeah, there's definitely something not right with the FWHM measurements. For the record, I'm not suggesting it actually did go down in FWHM, that doesn't make any sense.

Fair point on doing the noise reduction instead. When you tried a similar thing, do you know what the approximate ratio of bad to good subs was? In this case it was roughly 2:1 in favour of bad subs, and I still need a lot more data on this subject, so I expect to be gathering more and applying noise reduction.

Quote:
Originally Posted by Atmos View Post
I had thought of doing something similar to that Lee in an attempt to salvage TERRIBLE data. Had really nice red and blue data but green was bad due to the auto guider going on the fritz. Wanted to use PixelMath and mask the green FWHM with the red and turn lines into fainter dots

Didn't work so well but what you've done sounds good. As for the lower FWHM, what you have done is increased the central region with the poor data which has increased the "half maximum". You've created an artificial cone
Cheers Colin. I'm not sure I follow you though: why would increasing the half maximum decrease the FWHM? I suppose it's likely that dimmer stars would have had their profiles altered to some extent due to it being an intensity-based mask, but I still don't follow how adding the bad data to the good data improves on the good data's FWHM.
Reply With Quote
  #8  
Old 30-05-2016, 06:45 AM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
Quote:
Originally Posted by Atmos View Post
I had thought of doing something similar to that Lee in an attempt to salvage TERRIBLE data. Had really nice red and blue data but green was bad due to the auto guider going on the fritz. Wanted to use PixelMath and mask the green FWHM with the red and turn lines into fainter dots
Have a look at the B3Estimator process in PixInsight. For black body targets it can synthesize a missing colour.
Reply With Quote
  #9  
Old 30-05-2016, 06:48 AM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
Quote:
Originally Posted by codemonkey View Post
When you tried a similar thing, do you know what the approximate ratio of bad to good subs was? In this case it was roughly 2:1 in favour of bad subs, and I still need a lot more data on this subject, so I expect to be gathering more and applying noise reduction.
.
I don't think my ratio would have been that high. Maybe 1.5:1.
Reply With Quote
Reply

Bookmarks

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +10. The time is now 07:50 AM.

Powered by vBulletin Version 3.8.7 | Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Advertisement
Bintel
Advertisement
Testar
Advertisement