Go Back   IceInSpace > Equipment > Astrophotography and Imaging Equipment and Discussions

Reply
 
Thread Tools Rate Thread
  #1  
Old 23-04-2015, 10:39 AM
Shiraz's Avatar
Shiraz (Ray)
Registered User

Shiraz is offline
 
Join Date: Apr 2010
Location: ardrossan south australia
Posts: 4,918
Using short subs to improve resolution

Hi

This idea arose in discussions with PRejto and was also recently raised in a post by Justin Tilbrook. The basic idea is that a form of lucky imaging can be applied to DSOs – you don’t need to image fast enough to freeze the seeing, but it can help if you image with short enough subs to enable you to deal with those occasions when seeing fluctuates over periods of minutes.

To illustrate with real data, the upper graph in the attached image shows the HFR for a sequence of 56 x 3 minute subs. (HFR is related to FWHM and these data correspond to FWHM of about 2-2.5 arc sec). The seeing varied fairly smoothly, but there were occasional bursts where it was worse. The lower data shows the same HFR data, but with each value being the average of a block of 7 shorter subs (ie this data represents 21 minute subs). Should I decide that I want to keep data with an HFR of say 1.2 pixels or less to maximise the resolution, I can keep 31 of the 56 short subs, but only 3 of the 8 longer subs (and one of those is marginal). Thus, I can keep > 40% more of the data if I use shorter subs – or alternatively I will get worse resolution if I choose to keep more of the longer subs. The basic problem with the longer subs is that they bundle good and bad data together and there is no way to weed out the bad without losing some of the good.

Of course, this method is not for everyone. You will need a sensitive system with low read noise to keep noise under control, but if you have a camera with high QE and low read noise, it might be worth seeing if you can get better resolution by using short subs. Even if you don’t get huge increases in resolution, you might get significantly better SNR, since you could possibly end up keeping much more data. And of course, it doesn’t cost a cent.

Thanks for reading. Regards Ray
Attached Thumbnails
Click for full-size image (subcomparo.jpg)
52.9 KB77 views

Last edited by Shiraz; 23-04-2015 at 11:09 AM.
Reply With Quote
  #2  
Old 23-04-2015, 10:57 AM
strongmanmike's Avatar
strongmanmike (Michael)
Highest Observatory in Oz

strongmanmike is offline
 
Join Date: May 2006
Location: Canberra
Posts: 17,152
Once again I love the way you verify and quantify more or less why I have found what I have through trial and error, a great explanation Ray

Mike
Reply With Quote
  #3  
Old 23-04-2015, 12:26 PM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
Interesting data, Ray. On a couple of occasions, when the data justified it, I have done two integrations of my luminance - one weighted for best FWHM and one weighted for best SNR. These are blended so that the high FWHM integration is used for the brightest parts of the image and the high SNR integration is used for the dim parts. It should be possible to do something similar with short subs for the high FWHM data, even with a camera that doesn't have incredibly low noise (like my KAF-16803...) Food for though
Reply With Quote
  #4  
Old 23-04-2015, 12:29 PM
multiweb's Avatar
multiweb (Marc)
ze frogginator

multiweb is offline
 
Join Date: Oct 2007
Location: Sydney
Posts: 22,060
That's interesting to see the numbers Ray. I too have had experience with shorter subs (3min) and found that the FWHM of the final was better, probably because you can be picky given the number of subs available. But read noise was an issue. I've never experienced long uninterrupted subs. Don't have an obs and there's always a wind gust, a cloud or plane that's bound to stuff one up. Want to make the most of your setup time.
Reply With Quote
  #5  
Old 23-04-2015, 04:18 PM
codemonkey's Avatar
codemonkey (Lee)
Lee "Wormsy" Borsboom

codemonkey is offline
 
Join Date: Jul 2013
Location: Kilcoy, QLD
Posts: 2,058
Great topic / work, Ray! I've seen some discussion elsewhere on identifying the shortest possible exposure that you can use while keeping the total integration time and resulting SNR the same and this is another great reason to do that. Short exposures for the win!
Reply With Quote
  #6  
Old 23-04-2015, 04:40 PM
tilbrook@rbe.ne's Avatar
tilbrook@rbe.ne (Justin Tilbrook)
JHT

tilbrook@rbe.ne is offline
 
Join Date: Jun 2007
Location: Penwortham
Posts: 3,039
Thanks Ray!

Thanks for doing the numbers, I'm glad to see there's some method in my madness!!

Cheers,

Justin.
Reply With Quote
  #7  
Old 23-04-2015, 04:49 PM
gregbradley's Avatar
gregbradley
Registered User

gregbradley is offline
 
Join Date: Feb 2006
Location: Sydney
Posts: 17,877
Nice work Ray. What does HFR stand for? ( I looked it up and it wasn't listed) Is it half full radius? Whats its definition?

Another factor here is longer subs average out guide errors as well. So if your guiding isn't super fabulous then you can see the stars getting a bit fatter with longer subs compared to shorter subs.

Of course an assumption here is a bright object or a low read noise camera where the short subs allow the read noise to be stacked out.

Doing this with higher read noise cameras or with narrowband would stop you from getting very faint detail above the noise floor of the camera.

So like anything you need to know when this would be a good approach and when it wouldn't be.

I was surprised recently comparing 5 and 10 minute subs from my CDK17 and SX694. It was hard to notice much difference in terms of the brightness of the image. Shorter subs tended to have slightly rounder stars (guide scope - yuck, but a necessary evil with that setup).

So in conclusion I think it fair to say as a general rule:

1. Shorter subs are better with variable seeing, windy or partially cloudy conditions or where you tracking is not 100%.

2. Its not good for faint maximum detail or for narrowband images.

3. If your tracking is top notch and your skies are clear and stable with no wind then longer exposures would be better.

How often do you think the seeing is variable like that? As a general trend I notice seeing often improves as the night progresses and possibly best at around 3am or so.

Greg.
Reply With Quote
  #8  
Old 23-04-2015, 05:54 PM
Shiraz's Avatar
Shiraz (Ray)
Registered User

Shiraz is offline
 
Join Date: Apr 2010
Location: ardrossan south australia
Posts: 4,918
Quote:
Originally Posted by strongmanmike View Post
Once again I love the way you verify and quantify more or less why I have found what I have through trial and error, a great explanation Ray

Mike
thanks Mike it's good to find agreement.

Quote:
Originally Posted by RickS View Post
Interesting data, Ray. On a couple of occasions, when the data justified it, I have done two integrations of my luminance - one weighted for best FWHM and one weighted for best SNR. These are blended so that the high FWHM integration is used for the brightest parts of the image and the high SNR integration is used for the dim parts. It should be possible to do something similar with short subs for the high FWHM data, even with a camera that doesn't have incredibly low noise (like my KAF-16803...) Food for though
thanks for that input Rick. Much food for thought indeed - have to think about what weighting for FWHM actually means, but the idea of using different stacks for different parts of the image seems really useful.

Quote:
Originally Posted by multiweb View Post
That's interesting to see the numbers Ray. I too have had experience with shorter subs (3min) and found that the FWHM of the final was better, probably because you can be picky given the number of subs available. But read noise was an issue. I've never experienced long uninterrupted subs. Don't have an obs and there's always a wind gust, a cloud or plane that's bound to stuff one up. Want to make the most of your setup time.
Hi Marc. The ability to throw out subs that are spoiled for whatever reason seems to be very useful. But, you are right that read noise limits what you can do.
Quote:
Originally Posted by codemonkey View Post
Great topic / work, Ray! I've seen some discussion elsewhere on identifying the shortest possible exposure that you can use while keeping the total integration time and resulting SNR the same and this is another great reason to do that. Short exposures for the win!
Hi Lee. would be great if we could get hold of cameras with really low read noise - would be nice to use 1 minute subs or less and that should be possible in future.

Quote:
Originally Posted by tilbrook@rbe.ne View Post
Thanks Ray!

Thanks for doing the numbers, I'm glad to see there's some method in my madness!!

Cheers,

Justin.
Hi Justin. the other possibility is that maybe we are both bonkers?? Anyway, have also been thinking about how you might optimise sub length for your camera - will PM you.

Quote:
Originally Posted by gregbradley View Post
Nice work Ray. What does HFR stand for? ( I looked it up and it wasn't listed) Is it half full radius? Whats its definition?

Another factor here is longer subs average out guide errors as well. So if your guiding isn't super fabulous then you can see the stars getting a bit fatter with longer subs compared to shorter subs.

Of course an assumption here is a bright object or a low read noise camera where the short subs allow the read noise to be stacked out.

Doing this with higher read noise cameras or with narrowband would stop you from getting very faint detail above the noise floor of the camera.

So like anything you need to know when this would be a good approach and when it wouldn't be.

I was surprised recently comparing 5 and 10 minute subs from my CDK17 and SX694. It was hard to notice much difference in terms of the brightness of the image. Shorter subs tended to have slightly rounder stars (guide scope - yuck, but a necessary evil with that setup).

So in conclusion I think it fair to say as a general rule:

1. Shorter subs are better with variable seeing, windy or partially cloudy conditions or where you tracking is not 100%.

2. Its not good for faint maximum detail or for narrowband images.

3. If your tracking is top notch and your skies are clear and stable with no wind then longer exposures would be better.

How often do you think the seeing is variable like that? As a general trend I notice seeing often improves as the night progresses and possibly best at around 3am or so.

Greg.
Hi Greg.

The seeing behaves like that quite often at this site, but ironically it is only possible to see fluctuations like this if you use short subs (if you use long subs you will never be aware of how much the seeing varies ). I used to get major seeing improvements as the night went on, but since I put up a ROR obs, the overall seeing is much more consistent (and consistently lower) throughout a night.

You are right to point out that what I call "seeing" also incorporates tracking error - but the technique works just as well on tracking error.

HFR is the Half Flux Radius (half of the Half Flux Diameter as in http://www.cyanogen.com/help/maximdl/Half-Flux.htm). It is used in Nebulosity and Sequence Generator Pro - which I use all the time.

Totally agree that this method is not appropriate for NB, except maybe for the brightest objects. However, I think that it can be used for getting maximum detail from broadband images in good conditions - provided that the subs are long enough to cover the read noise, I think that a lot of short subs will generally do slightly better than a few long subs.

Regards Ray

Last edited by Shiraz; 23-04-2015 at 06:05 PM.
Reply With Quote
  #9  
Old 23-04-2015, 06:24 PM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
Quote:
Originally Posted by Shiraz View Post
thanks for that input Rick. Much food for thought indeed - have to think about what weighting for FWHM actually means, but the idea of using different stacks for different parts of the image seems really useful.
Ray,

The ultimate weighting tool is to leave the worst subs out of the integration, but PI also has a script (SubframeSelector) which allows you to calculate a weighting for each sub based on a collection of parameters including FWHM, Eccentricity and SNR. This script adds a weight value to the FITS header that is used during image integration.

Cheers,
Rick.
Reply With Quote
  #10  
Old 23-04-2015, 08:10 PM
Shiraz's Avatar
Shiraz (Ray)
Registered User

Shiraz is offline
 
Join Date: Apr 2010
Location: ardrossan south australia
Posts: 4,918
Quote:
Originally Posted by RickS View Post
Ray,

The ultimate weighting tool is to leave the worst subs out of the integration, but PI also has a script (SubframeSelector) which allows you to calculate a weighting for each sub based on a collection of parameters including FWHM, Eccentricity and SNR. This script adds a weight value to the FITS header that is used during image integration.

Cheers,
Rick.
Thanks Rick - looked it up following your first post - much appreciated . I can understand weighting applied when adding noisy frames to a stack, but am still trying to understand what it means for resolution - do you understand it to mean that, if you only add a fraction of a dud frame, you can still get some SNR advantage from it without messing up the detail too much? I don't think that there is any resolution advantage in adding less well resolved data - seems to me that can only work to degrade resolution, even if weighted.

regards ray
Reply With Quote
  #11  
Old 23-04-2015, 08:45 PM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
Quote:
Originally Posted by Shiraz View Post
Thanks Rick - looked it up following your first post - much appreciated . I can understand weighting applied when adding noisy frames to a stack, but am still trying to understand what it means for resolution - do you understand it to mean that, if you only add a fraction of a dud frame, you can still get some SNR advantage from it without messing up the detail too much? I don't think that there is any resolution advantage in adding less well resolved data - seems to me that can only work to degrade resolution, even if weighted.

regards ray
Ray,

I'd reject poor FWHM subs (or set weight to zero) for the high res integration. I didn't mean to imply that there would be some benefit from including these except in the high SNR integration.

Cheers,
Rick.
Reply With Quote
  #12  
Old 24-04-2015, 09:19 AM
Shiraz's Avatar
Shiraz (Ray)
Registered User

Shiraz is offline
 
Join Date: Apr 2010
Location: ardrossan south australia
Posts: 4,918
Quote:
Originally Posted by RickS View Post
Ray,

I'd reject poor FWHM subs (or set weight to zero) for the high res integration. I didn't mean to imply that there would be some benefit from including these except in the high SNR integration.

Cheers,
Rick.
thanks for the clarification Rick - looks like a really neat idea and I must try it. regards Ray
Reply With Quote
  #13  
Old 24-04-2015, 09:35 AM
Paul Haese's Avatar
Paul Haese
Registered User

Paul Haese is offline
 
Join Date: Jan 2009
Location: Adelaide
Posts: 9,937
I agree that not everyone will agree on this. Though I also see its uses too. Recently doing long subs on Carina nebula and incorporating short subs to capture the Homunculus is one example that I think works. Also you could do this with other high dynamic regions.

My general opinion based on my own experience is that shorter subs fail to overwhelm the noise and in particular with a sensor such as the KAF8300, although as your point out the right sensor should help in certain cameras.

Overall though I think these theories are very contentious and the astro imaging community can be quite split on which is better. For me HDR objects means using shorter subs to masked in to much longer subs. I would not advocate using short subs overall, but that is just my opinion.
Reply With Quote
  #14  
Old 24-04-2015, 09:47 AM
Shiraz's Avatar
Shiraz (Ray)
Registered User

Shiraz is offline
 
Join Date: Apr 2010
Location: ardrossan south australia
Posts: 4,918
thanks Paul.

I appreciate that there are differing opinions and as I said, it is not going to work for cameras with low QE and high read noise.

However, the original post is not based on any theory or opinion at all - the data is real data from an imaging session and I just presented it to show what I had actually measured.

Regards Ray

Last edited by Shiraz; 24-04-2015 at 10:07 AM.
Reply With Quote
  #15  
Old 24-04-2015, 10:07 AM
Paul Haese's Avatar
Paul Haese
Registered User

Paul Haese is offline
 
Join Date: Jan 2009
Location: Adelaide
Posts: 9,937
Quote:
Originally Posted by Shiraz View Post
thanks Paul.

I appreciate that there are differing opinions and as I said, it is not going to work for cameras with low QE and high read noise. However, the original post is not based on any theory or opinion at all - the data is real measured data from an imaging session and I just presented it to show what I had actually measured.

Regards Ray
Sorry; use of the wrong wording Ray, but often theory is supported by evidence, so hence why I said theory. You present evidence with your findings. Nothing intended to deny your findings.

I like the way you present your evidence. I don't always agree with your findings but it always makes me think more on the issues. And; that is certainly good for any changes that might need to be made to my imaging techniques.
Reply With Quote
  #16  
Old 24-04-2015, 10:14 AM
Shiraz's Avatar
Shiraz (Ray)
Registered User

Shiraz is offline
 
Join Date: Apr 2010
Location: ardrossan south australia
Posts: 4,918
Quote:
Originally Posted by gregbradley View Post
Nice work Ray. What does HFR stand for? ( I looked it up and it wasn't listed) Is it half full radius? Whats its definition?

Another factor here is longer subs average out guide errors as well. So if your guiding isn't super fabulous then you can see the stars getting a bit fatter with longer subs compared to shorter subs.

Of course an assumption here is a bright object or a low read noise camera where the short subs allow the read noise to be stacked out.

Doing this with higher read noise cameras or with narrowband would stop you from getting very faint detail above the noise floor of the camera.

So like anything you need to know when this would be a good approach and when it wouldn't be.

I was surprised recently comparing 5 and 10 minute subs from my CDK17 and SX694. It was hard to notice much difference in terms of the brightness of the image. Shorter subs tended to have slightly rounder stars (guide scope - yuck, but a necessary evil with that setup).

So in conclusion I think it fair to say as a general rule:

1. Shorter subs are better with variable seeing, windy or partially cloudy conditions or where you tracking is not 100%.

2. Its not good for faint maximum detail or for narrowband images.

3. If your tracking is top notch and your skies are clear and stable with no wind then longer exposures would be better.

How often do you think the seeing is variable like that? As a general trend I notice seeing often improves as the night progresses and possibly best at around 3am or so.

Greg.
Hi Greg
re your question on how seeing varies, I did a couple of analyses on multi-night data from a couple of other targets (3 to 4 minute subs) - the attached data is HFR (in pixels as before) and shows how much and how quickly the seeing varies at this site. Hope it is interesting - it seems that short term fluctuations of around 0.2 pixels or more (or about 0.4 arcsec FWHM) are common, which is a fair bit really. Regards ray

edit: none of this data is normalised to remove the effect of elevation on seeing.
Attached Thumbnails
Click for full-size image (subs2.jpg)
133.4 KB23 views

Last edited by Shiraz; 24-04-2015 at 12:49 PM.
Reply With Quote
  #17  
Old 24-04-2015, 10:25 AM
Shiraz's Avatar
Shiraz (Ray)
Registered User

Shiraz is offline
 
Join Date: Apr 2010
Location: ardrossan south australia
Posts: 4,918
Quote:
Originally Posted by Paul Haese View Post
Sorry; use of the wrong wording Ray, but often theory is supported by evidence, so hence why I said theory. You present evidence with your findings. Nothing intended to deny your findings.

I like the way you present your evidence. I don't always agree with your findings but it always makes me think more on the issues. And; that is certainly good for any changes that might need to be made to my imaging techniques.
No need to apologise at all Paul . Ideas need to be challenged and defended - the more the merrier.

Last edited by Shiraz; 24-04-2015 at 11:07 AM.
Reply With Quote
  #18  
Old 24-04-2015, 04:10 PM
PRejto's Avatar
PRejto (Peter)
Registered User

PRejto is offline
 
Join Date: Jan 2011
Location: Rylstone, NSW, Australia
Posts: 1,397
Hi Ray,

A most interesting presentation that does back up some of what I measured when chasing my blue fringing issue a while back.

Sorry if this sounds dense! But, when you write:

"The lower data shows the same HFR data, but with each value being the average of a block of 7 shorter subs (ie this data represents 21 minute subs)"

I question whether taking an "average" of 7 subs actually represents a 21 min sub taken during the same interval. Wouldn't the 21 min sub be more heavily weighted towards whatever was the worst moment of seeing during the exposure? Taking an average would seem to make the sub better than it might be. I guess I'm thinking about an analogy towards guiding when I write this. It doesn't take much guide error, or very long, to destroy a sub that "averaging" wouldn't fix. I'm pretty sure you will adjust my thinking!!

Peter
Reply With Quote
  #19  
Old 24-04-2015, 05:14 PM
Bassnut's Avatar
Bassnut (Fred)
Narrowfield rules!

Bassnut is offline
 
Join Date: Nov 2006
Location: Torquay
Posts: 5,064
Quote:
Originally Posted by PRejto View Post
Hi Ray,

I question whether taking an "average" of 7 subs actually represents a 21 min sub taken during the same interval. Wouldn't the 21 min sub be more heavily weighted towards whatever was the worst moment of seeing during the exposure? Taking an average would seem to make the sub better than it might be. I guess I'm thinking about an analogy towards guiding when I write this. It doesn't take much guide error, or very long, to destroy a sub that "averaging" wouldn't fix. I'm pretty sure you will adjust my thinking!!

Peter
Im right out of my expertise here, but doesnt RMS guiding count more than average or PP for extended objects on long exposures?. OK, stars are destroyed by an outlier excursion, but in my experience short exposures (on extended objects) are more sensitive to bad guiding and seeing than long exposures!.

Edit: I might expand on that. More data on a longer sub (and lower noise) even if its not as sharp as a short sub, allows more application of decon and sharpening before noise becomes obvious and artifacts occur ?.

Last edited by Bassnut; 24-04-2015 at 06:04 PM.
Reply With Quote
  #20  
Old 24-04-2015, 06:13 PM
Shiraz's Avatar
Shiraz (Ray)
Registered User

Shiraz is offline
 
Join Date: Apr 2010
Location: ardrossan south australia
Posts: 4,918
Quote:
Originally Posted by PRejto View Post
Hi Ray,

A most interesting presentation that does back up some of what I measured when chasing my blue fringing issue a while back.

Sorry if this sounds dense! But, when you write:

"The lower data shows the same HFR data, but with each value being the average of a block of 7 shorter subs (ie this data represents 21 minute subs)"

I question whether taking an "average" of 7 subs actually represents a 21 min sub taken during the same interval. Wouldn't the 21 min sub be more heavily weighted towards whatever was the worst moment of seeing during the exposure? Taking an average would seem to make the sub better than it might be. I guess I'm thinking about an analogy towards guiding when I write this. It doesn't take much guide error, or very long, to destroy a sub that "averaging" wouldn't fix. I'm pretty sure you will adjust my thinking!!

Peter
Quote:
Originally Posted by Bassnut View Post
Im right out of my expertise here, but doesnt RMS guiding count more than average or PP for extended objects on long exposures?. OK, stars are destroyed by an outlier excursion, but in my experience short exposures (on extended objects) are more sensitive to bad guiding and seeing than long exposures!.
Hi Peter and Fred.

I also had some misgivings about using an average of the HFR, so I tested it by taking results from a highly variable block of seeing and:
1. averaged the HFR of 7 individual subs
2. stacked the subs and measured the HFR of the result (this should be pretty much equivalent to a longer sub)
I got the same result for the two processes (<2% difference), so decided to use the much easier averaging in the final analysis.

I agree that the actual effect of seeing on an image is almost certainly not fully indicated by HFR, but I wanted to show that this measure at least can be improved by shorter subs. I would be grateful if you have any ideas on how to better test this idea - I am at the limit for my pay scale.

regards Ray

edit: re deconvolution, although read noise will be lower for a few long subs, the fact that you can retain so much more good data with shorter subs (in this case 40% for the chosen rejection threshold) must compensate to some degree. FWIW, with my system, the total read noise is <10% of the total broadband noise with 3 minute subs so, by rejecting less data, I can actually get better SNR than I would with longer subs (broadband of course).

Last edited by Shiraz; 24-04-2015 at 07:42 PM.
Reply With Quote
Reply

Bookmarks

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +10. The time is now 09:21 PM.

Powered by vBulletin Version 3.8.7 | Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Advertisement
Testar
Advertisement
Bintel
Advertisement