#1  
Old 26-02-2017, 09:21 AM
Shiraz's Avatar
Shiraz (Ray)
Registered User

Shiraz is offline
 
Join Date: Apr 2010
Location: ardrossan south australia
Posts: 4,784
asi1600 data precision ?

Hi

Quick heads up. have been stacking large numbers of fairly short subs and stretching very heavily to go deep - it looks like the 32 bit floating point representation in PI does not have enough precision to do this properly (!!) and rounding errors show up in the histogram. Haven't investigated in any depth and am not sure if it is a major issue in most images (they are eventually converted to JPEG), but, since it is not hard to do, suggest that it may be a good idea to consider using 64 bit FP for "average" stacked data with this camera - everything else can stay as is and the histogram errors go away. EDIT: this is probably not necessary - see later post.

The other thing is that median stacking is not a successful option for deep imaging with this camera due to the bit filling process used to extend from 12 bit to 16 bit data - average stacking can fill in the histogram to any desired depth, but the median value of a stack will retain the structure of the original data - with the bottom 4 bits all zero..

Last edited by Shiraz; 27-02-2017 at 04:45 PM.
Reply With Quote
  #2  
Old 26-02-2017, 10:18 AM
Atmos's Avatar
Atmos (Colin)
Ultimate Noob

Atmos is offline
 
Join Date: Aug 2011
Location: Melbourne
Posts: 5,821
That's really good to know Ray. I cannot say I've done a large enough stack yet for this to be an issue but it's good to know for when I do
Reply With Quote
  #3  
Old 26-02-2017, 11:36 AM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,579
Quote:
Originally Posted by Shiraz View Post
Quick heads up. have been stacking large numbers of fairly short subs and stretching very heavily to go deep - it looks like the 32 bit floating point representation in PI does not have enough precision to do this properly (!!) and rounding errors show up in the histogram. Haven't investigated in any depth and am not sure if it is a major issue in most images (they are eventually converted to JPEG), but, since it is not hard to do, suggest that it may be a good idea to consider using 64 bit FP for "average" stacked data with this camera - everything else can stay as is and the histogram errors go away.
According to the doc, ImageIntegration uses 64-bit floating point internally even with the default 32-bit output so that's surprising... to me at least

Quote:
Originally Posted by Shiraz View Post
The other thing is that median stacking is not a successful option for deep imaging with this camera due to the bit filling process used to extend from 12 bit to 16 bit data - average stacking can fill in the histogram to any desired depth, but the median value of a stack will retain the structure of the original data - with the bottom 4 bits all zero..
Median stacking is never a good idea, IMO. Better to put some effort into careful rejection rather than throw away SNR.

Cheers,
Rick.
Reply With Quote
  #4  
Old 26-02-2017, 12:31 PM
codemonkey's Avatar
codemonkey (Lee)
Registered User

codemonkey is offline
 
Join Date: Jul 2013
Location: Kilcoy, QLD
Posts: 1,522
Interesting. How did you determine there's rounding errors via the histogram? Gaps? I integrated 136x36L frames (and then added to about 60 RGB to form a super L) and didn't see any sign of this.

I'd definitely be using averaging instead of median for integration as well, as Rick says, you'll get much better SNR that way.
Reply With Quote
  #5  
Old 26-02-2017, 06:39 PM
Shiraz's Avatar
Shiraz (Ray)
Registered User

Shiraz is offline
 
Join Date: Apr 2010
Location: ardrossan south australia
Posts: 4,784
Quote:
Originally Posted by Shiraz View Post
Hi

Haven't investigated in any depth .
Thanks folks. Have now thought about it further - looks like I was wrong. The camera can do a little less than 13 bits and 512+ subs takes the possible data range to ~22 bits. 32 bit FP can do 23-24 bits of precision, so, as Rick implied, it should cover the whole range of values from the camera with a couple of bits to spare - scratch the theory - 32 bits FP is good enough for up to about 2000 subs.

the effect is shown in the attached image showing the histogram of a heavily stretched 32 bitFP raw stacked fits file. It may be that this is what 22 bits look like when heavily stretched - there is no noise to hide the fact that data can only take discrete values and there will be values in the histogram that are not produced by the camera/stacking, even with 500+ subs.The other possibility is that the histo binning algorithm in PI does not take full advantage of the high internal precision and the effect is an artefact of it's limited resolution. Whatever... the initial suggestion to use 64 bits is not necessary and whatever the effect is, it is not something that can be (or needs to be) "fixed".

Sorry for the initial misinformation. Must keep thought bubbles under control in future - maybe that could be the guiding principle for all posts on IIS .

median stacking is far worse than just a bit of lost SNR with this camera - every sub has the bottom 4 bits padded with zeros, so the median from any stack will also have zeros in the bottom 4 bits. Since there is no way for noise to fill in the bottom bits, there is almost no way for median stacking to improve the SNR at all. Average works fine.
Attached Thumbnails
Click for full-size image (bin.jpg)
73.6 KB14 views

Last edited by Shiraz; 26-02-2017 at 10:12 PM.
Reply With Quote
Reply

Bookmarks

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +10. The time is now 01:04 PM.

Powered by vBulletin Version 3.8.7 | Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
Advertisement
Lunatico Astronomical
Advertisement
SkyWatcher WiFi Adaptor
Advertisement
Bintel
Advertisement
Interest Free Finance
Advertisement
OzScopes Authorised Dealer
Advertisement
NexDome Observatories
Advertisement
SkyWatcher 2018 Catalogue
Advertisement
Astronomy and Electronics Centre
Advertisement