Go Back   IceInSpace > Equipment > Astrophotography and Imaging Equipment and Discussions

Reply
 
Thread Tools Rate Thread
  #1  
Old 23-11-2018, 10:47 AM
Stonius's Avatar
Stonius (Markus)
Registered User

Stonius is offline
 
Join Date: Mar 2015
Location: Melbourne
Posts: 1,495
Help measuring CMOS performance

Hi there,

I've tried two methods of measuring my ASI1600's individual performance and can't get either to work.

I've tried following Craig Stark's tutorial, and the BasicCCDproperties Script in Pixinsight.

The PI script fails to execute, citing an error regarding the variable 'this.x0'. Some have suggested simply removing the variable from that line. I've tried that too, to no avail.

Doing it manually via Craig Stark's tutorial, I get stuck at the flats section.

So, I have to take pairs of flats where the only thing that changes is the number of photons hitting the sensor. Rodger that.

But the numbers are killing me here.

From what I can tell, I take one flat (presumably the brighter one) and read the mean value, say 38629. Then I multiply by 2 to get 77,258 (why? I don't know and he doesn't say)

Then I make a difference image with a 5000 pedestal value added to the first image to make sure everything comes out in positive numbers and no data is clipped. I don't have ImageJ, whatever that is, so I try pixelMath in PixInsight. I edit the expression to read "(image2+5000)-image 1"

At this point, every single pixel is maxed out to white. If I change the pedestal value to anything other than 0 or 1, it maxes out.

At this point I'm stuck because the tutorial is good on what to do, but not so good on why, so it's hard to extrapolate a solution when things go awry.

I can read off the values in PI statistics but is avgDev or MAD the same as a Standard Deviation?

I don't mind getting my hands dirty in a spreadsheet, but I don't really understand what he's doing. Surely the pedestal value thing means all your data will be wrong by 5000 if you don't subtract it out later on, which puts you in the same situation of clipping? If you need positive numbers, and neither flat frame has priority here, you can simply subtract the smaller value from the greater value with no need for a pedestal. And why does it have to be mean *2?

If someone could just tell me "take these values from PI statistics and do this with them in a spreadsheet" I'd be so grateful.

I also notice there is a difference in methodology between the script and the tutorial in that the script wants two flats that are taken under the same conditions, whereas the tutorial wants sets of two flats taken at the same settings with only the light level increasing between sets.

Any help gratefully received.

Best

Markus
Reply With Quote
  #2  
Old 23-11-2018, 11:13 AM
Merlin66's Avatar
Merlin66 (Ken)
Registered User

Merlin66 is offline
 
Join Date: Oct 2005
Location: Junortoun Vic
Posts: 8,904
Markus,
""I don't have ImageJ, whatever that is,""
It's well worth getting ImageJ as a good software tool for many of the things we do (or would like to do) in astronomy. I use it to assist in processing the solar spectroheliograph data.
https://imagej.net/Welcome

The review by Christian Buil may be helpful.
http://www.astrosurf.com/buil/atik_vs_zwo/
Reply With Quote
  #3  
Old 23-11-2018, 01:52 PM
Merlin66's Avatar
Merlin66 (Ken)
Registered User

Merlin66 is offline
 
Join Date: Oct 2005
Location: Junortoun Vic
Posts: 8,904
Markus,
Looking at the method described... the x2 comes from the formula used:
m = 2 x mean(Flat1)
Reply With Quote
  #4  
Old 23-11-2018, 02:01 PM
glend (Glen)
Registered User

glend is offline
 
Join Date: Jun 2013
Location: Lake Macquarie
Posts: 7,033
Might be worth dropping a message to Jon Rista over on CN. Jon did the 1600 beta testing for ZWO and seems to have numbers on everything.
Reply With Quote
  #5  
Old 23-11-2018, 02:04 PM
ChrisV's Avatar
ChrisV (Chris)
Registered User

ChrisV is offline
 
Join Date: Aug 2015
Location: Sydney
Posts: 1,737
You could cheat. The latest sharpcap pro has a routine that will work out all your camera specs. Or is this more an exercise is figuring out what it all means.

Edit: wow this is interesting. It's the same biological systems such as in channels in membranes - a poisson process hence you use the mean and variance to determine the gain of system

Last edited by ChrisV; 23-11-2018 at 02:42 PM.
Reply With Quote
  #6  
Old 23-11-2018, 02:10 PM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
Hi Markus,

I used the BasicCCDParameters script recently to characterise my ASI1600mm and ASI294. If you want to upload the raw files (2 flats, 2 bias, a short dark and a long dark) somewhere I'm sure I can get it to work. I know enough JavaScript to be dangerous and if there's a bug this would be a good time to fix it - 1.8.6 is being tested by PTeam members right now.

Cheers,
Rick.
Reply With Quote
  #7  
Old 24-11-2018, 10:08 AM
ericwbenson (Eric)
Registered User

ericwbenson is offline
 
Join Date: Sep 2009
Location: Adelaide, Australia
Posts: 209
Quote:
Originally Posted by Stonius View Post
Then I make a difference image with a 5000 pedestal value added to the first image to make sure everything comes out in positive numbers and no data is clipped. I don't have ImageJ, whatever that is, so I try pixelMath in PixInsight. I edit the expression to read "(image2+5000)-image 1"

At this point, every single pixel is maxed out to white. If I change the pedestal value to anything other than 0 or 1, it maxes out.

Hi, I think the problem here is that PI normalizes pixels values into the range 0..1. So that 5000 adu pedestal should be interpreted in PI speak as ~0.1. I think that would solve the maxing out to white you encountered.


Best,
EB
Reply With Quote
  #8  
Old 24-11-2018, 10:41 AM
Stonius's Avatar
Stonius (Markus)
Registered User

Stonius is offline
 
Join Date: Mar 2015
Location: Melbourne
Posts: 1,495
Quote:
Originally Posted by RickS View Post
Hi Markus,

I used the BasicCCDParameters script recently to characterise my ASI1600mm and ASI294. If you want to upload the raw files (2 flats, 2 bias, a short dark and a long dark) somewhere I'm sure I can get it to work. I know enough JavaScript to be dangerous and if there's a bug this would be a good time to fix it - 1.8.6 is being tested by PTeam members right now.

Cheers,
Rick.
Thanks Rick,

I'll see if I can upload them later on today. In the meantime, what numbers did you end up with, if you don't mind me asking?

Cheers

Markus
Reply With Quote
  #9  
Old 24-11-2018, 10:46 AM
Atmos's Avatar
Atmos (Colin)
Ultimate Noob

Atmos is offline
 
Join Date: Aug 2011
Location: Melbourne
Posts: 6,980
With my ASI1600 at Gain 76 I had a read noise of ~2.1e- and 12-bit of dynamic range.
By Gain 139 (unity) I was getting ~1.59e- and I cannot remember the bit depth.

With my QHY163M at Gain 80 I get 1.9e- and 12-bit dynamic range.
At Gain 140 I get about 1.55e- and cannot remember the dynamic range.

I don't often bother about calculating the dark current personally.
Reply With Quote
  #10  
Old 24-11-2018, 11:39 AM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
Quote:
Originally Posted by Stonius View Post
I'll see if I can upload them later on today. In the meantime, what numbers did you end up with, if you don't mind me asking?
Hi Markus,

A screen grab of my spreadsheet is attached. Set point was -20C.

PM me if you get a chance to upload the files.

Cheers,
Rick.
Attached Thumbnails
Click for full-size image (ASI1600mm.jpg)
185.6 KB36 views
Reply With Quote
  #11  
Old 26-11-2018, 04:38 PM
Stonius's Avatar
Stonius (Markus)
Registered User

Stonius is offline
 
Join Date: Mar 2015
Location: Melbourne
Posts: 1,495
Hey, Rick,

thanks so much for uploading your files, it's been invaluable.

With that as a guide I've managed to trouble-shoot a number of issues and my data is looking similar to yours. I've also learned out how to retrofit trend lines and derive the formulae in order to determine intermediate values, which is handy.

But, I'm a bit stuck with the Dark current.

According to Craig Stark, if you look at the mean for a bias frame and subtract that value from your darks, you will be left with the dark current, which should increase linearly the longer you expose.

So at any one gain setting the determining factor is time. More time = more dark current.

I'm getting a non-linear relationship. Seemingly at all gain levels, increasing exposure yields less dark current.

I can't figure out what I'm doing wrong. I integrated 250 Bias frames at the same temperature, gain, offset and exposure settings as the darks.

Hmmm.

I've included the data below. It's in 12bit values because I prefer to convert at the end.
Attached Thumbnails
Click for full-size image (Non linear dark current.JPG)
24.4 KB24 views
Click for full-size image (Exposure sec.png)
11.2 KB20 views
Reply With Quote
  #12  
Old 26-11-2018, 06:58 PM
Atmos's Avatar
Atmos (Colin)
Ultimate Noob

Atmos is offline
 
Join Date: Aug 2011
Location: Melbourne
Posts: 6,980
Some of that could be due to the non-linear way in which amp glow grows. You could try doing the same thing on the central third of the frame (same in all of them) to see if that makes a difference.
Reply With Quote
  #13  
Old 26-11-2018, 08:09 PM
Stonius's Avatar
Stonius (Markus)
Registered User

Stonius is offline
 
Join Date: Mar 2015
Location: Melbourne
Posts: 1,495
In which direction is amp glow non-linear? Does it compound with time?
Reply With Quote
  #14  
Old 26-11-2018, 08:41 PM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
Hi Markus,

I'm glad my data was helpful.

I don't have time for a more considered response tonight, but there also appear to be some issues with short frames on (some) CMOS sensors, including short darks and bias frames. I have also seen non-linear darks as a result of bias drift on CCD sensors.

Here's a post on the ASI294 that I'm still thinking about (and running tests to compare):

https://www.cloudynights.com/topic/6...-and-opinions/

The ASI1600 seems much better behaved but I suspect it does suffer from some of the same issues to a lesser extent.

Cheers,
Rick.
Reply With Quote
  #15  
Old 26-11-2018, 10:04 PM
Stonius's Avatar
Stonius (Markus)
Registered User

Stonius is offline
 
Join Date: Mar 2015
Location: Melbourne
Posts: 1,495
Quote:
Originally Posted by Atmos View Post
Some of that could be due to the non-linear way in which amp glow grows. You could try doing the same thing on the central third of the frame (same in all of them) to see if that makes a difference.
In terms of a regression line, Inotice that the exponential line isn't the best fit, power is, so instead of ;

Exponential
y=ab^x

It's more something along the lines of

Power
y=ax^b

is that in keeping with amp glow noise?

I am using the slowest USB transfer rate, though I am using the camera's inbuilt USB2 hub for the filterwheel. Maybe I should run it separately through the hub and see if the noise changes.

Best, M
Reply With Quote
  #16  
Old 29-11-2018, 03:01 PM
Stonius's Avatar
Stonius (Markus)
Registered User

Stonius is offline
 
Join Date: Mar 2015
Location: Melbourne
Posts: 1,495
Hey Rick,

Can I ask what Exposure times you were using for your darks to derive your dark current? Did you try several exposure lengths? And if you did, was the e-/sec the same for all exposures?

What I'm seeing in my data is a *fairly linear relationship between dark current and exposure length, but when I try to compare apples with apples by figuring out the actual e-/sec by dividing by exposure time, I get the dark current decreasing rapidly from 0.0274 e-/sec to a base of around 0.0096 e-/sec at Gain 139 (which is not too different from your data).

Once converted to e-/sec, shouldn't you get basically the same numbers for all sub lengths?

I can't think of kind of noise that *reduces rapidly as exposure increases. Usually it's the other way round.

That means without taking the sub-length into account my estimation of Dark Current may be out by as much as 186%.
I can only think of some residual thermal activity from the preceding sensor read.

I may try another series, leaving 10 minutes between reads to see if that is the issue.

It doesn't stop me imaging, of course. It just means that there is another variable to consider for determining optimum exposures, I guess. I'll also have to be very careful to get darks that are the exact same length as the subs (no scaling during calibration).

Cheers

Markus
Attached Thumbnails
Click for full-size image (Dark Current.JPG)
20.7 KB10 views
Click for full-size image (Exposure Time and Dark @139.png)
12.7 KB12 views
Click for full-size image (ADU.png)
9.3 KB14 views
Reply With Quote
  #17  
Old 29-11-2018, 03:39 PM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
Hi Markus,

I gave the BasicCCDParameters script a 30 sec and a 300 sec dark.

I'm just running a set of new calibration files at -15C since I can't get to -20C without stressing the cooling too heavily in the current heat wave. I'll do a bit of analysis when I'm done (and it's cloudy!) and report back.

The chap that did the ASI294 analysis found that adding dummy 1 second frames between each calibration frame removed some unexpected variation so I'm doing that this time around to see if it also helps with the ASI1600.

Cheers,
Rick.
Reply With Quote
  #18  
Old 29-11-2018, 04:48 PM
Stonius's Avatar
Stonius (Markus)
Registered User

Stonius is offline
 
Join Date: Mar 2015
Location: Melbourne
Posts: 1,495
I've also heard of people 'purging the sensor' between reads, but I don't know how to do that or if it's solely a CCD thing.

I didn't use the script in the end because a) I never got it to work and b) I wanted to know what goes on behind the scenes. I imagine taking two frames would not give enough information to determine how dark current reacts over time. Using that method, I get 0.0185e-/sec @Gain 139.

Cheers

Markus
Reply With Quote
  #19  
Old 29-11-2018, 05:12 PM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
Quote:
Originally Posted by Stonius View Post
I've also heard of people 'purging the sensor' between reads, but I don't know how to do that or if it's solely a CCD thing.
That's for CCD sensors that suffer from RBI. The camera has an IR source and floods the sensor and equally messes up all the pixels

The BasicCCDParameters script assumes linear increase in dark current which is usually a reasonable assumption for CCD sensors but CMOS sensors do a lot of weird stuff.
Reply With Quote
  #20  
Old 29-11-2018, 05:18 PM
Stonius's Avatar
Stonius (Markus)
Registered User

Stonius is offline
 
Join Date: Mar 2015
Location: Melbourne
Posts: 1,495
Ah, sounds like I found some 'weird stuff' :-)

Oh well, good to know it's there.

Best,

M
Reply With Quote
Reply

Bookmarks

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +10. The time is now 04:25 AM.

Powered by vBulletin Version 3.8.7 | Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Advertisement
Testar
Advertisement
Bintel
Advertisement