Image Processing Issue in Pixinsight

[mariner10 (Steve)]

Hi people,
Can I ask for some advice please for all those imagers that use Pixinsight to process?
I’ve just started the processing process on a second image after completing my first one.
I'm right at the start of the process doing image integration.
As I'm a little slack I'm using the same calibration files (darks, bias and flats) as I did on the first image (without any issues) and I got the following error message on the second image:

ImageCalibration: Global context

Loading master calibration frames:
Loading calibration frame image:
C:/Users/User/Pictures/QHY8L/Master Calibration Files/superbias.fit
Reading FITS image: 32-bit floating point, 1 channel(s), 3328x2030 pixels: done
Normalizing sample values: done
40 FITS keywords extracted
Loading calibration frame image:
C:/Users/User/Pictures/QHY8L/Master Calibration Files/Master Dark.fit
Reading FITS image: 32-bit floating point, 1 channel(s), 3328x2030 pixels: done
Normalizing sample values: done
29 FITS keywords extracted
Loading calibration frame image:
C:/Users/User/Pictures/QHY8L/Master Calibration Files/Master Flat.fit
Reading FITS image: 32-bit floating point, 1 channel(s), 3328x2030 pixels: done
Normalizing sample values: done
35 FITS keywords extracted

Applying bias correction: master dark frame ...

Dark frame optimization thresholds:
Td0 = 0.00000000 (189 px = 0.003%)
*** Warning: The dark frame optimization threshold is probably too high (channel 0).
Computing master flat scaling factors ...
*** Error: Zero or insignificant scaling factor for flat frame channel# 0. (empty master flat frame image?)
<* failed*>

As I used the files on the first image with no issue I’m perplexed as to why I would be getting an error message now.
I tried doing another calibration of a flat file and using that instead of my original one and that too came up with the same error message.
Any hints or help would be appreciated.
Cheers.

[RickS (Rick)]

The dark frame optimization process depends on the data you are calibrating (it attempts to optimize SNR in the calibrated frames by adjusting the scaling of the dark master) so you can get different results with different data sets.

You haven't done something like subtracting the bias from the master dark twice, have you? If you calibrated the darks with bias subtraction when you created the master dark and then calibrated the lights with the box to calibrate the master dark ticked then that would happen...

Otherwise, if you can upload the master files and a few lights somewhere I'd be happy to take a look at them.

Cheers,
Rick.

[mariner10 (Steve)]

G'day Rick,
Thanks for the reply, appreciate the time. I actually put this same question on the PI forum site and to date nobody has come back to me which sort of was suggesting to me that it might have been all a bit too hard.
Anyway, when I did the master calibration files I simply followed the process in PI. I haven't subtracted the bias from the dark twice. Also, the image I'm trying to process now is the Sombraro galaxy, before that I did M78 and I used the same calibration files on it with no problems so it's a bit odd.
If you want to email me at I can then reply to you with the masters and some lights if you're happy to do that.
Cheers.

Originally Posted by RickS

The dark frame optimization process depends on the data you are calibrating (it attempts to optimize SNR in the calibrated frames by adjusting the scaling of the dark master) so you can get different results with different data sets.

You haven't done something like subtracting the bias from the master dark twice, have you? If you calibrated the darks with bias subtraction when you created the master dark and then calibrated the lights with the box to calibrate the master dark ticked then that would happen...

Otherwise, if you can upload the master files and a few lights somewhere I'd be happy to take a look at them.

Cheers,
Rick.

[RickS (Rick)]

Hi Steve, I'll drop you an email. You might want to edit your post and remove the address before it gets scraped by a spammer

Cheers,
Rick.

[mariner10 (Steve)]

Thanks Rick, good idea. I get enough spammers as it is.

Originally Posted by RickS

Hi Steve, I'll drop you an email. You might want to edit your post and remove the address before it gets scraped by a spammer

Cheers,
Rick.

[RickS (Rick)]

Hi Steve,

Thanks for sending me some data to look at.

I started by looking at the calibration masters. I made a small representative preview in the master bias and copied it to the master dark and the master flat. Using the Statistics process I found that median values (as 16-bit integers) for my preview area were approx 4163 for the master bias, 12 for the master dark and 13724 for the master flat.

Now, this tells me that you have definitely created the master dark with bias calibration enabled (otherwise the median value of the master dark preview should have been greater than the 4163 of the master bias.)

Your error message was:

Originally Posted by mariner10

Applying bias correction: master dark frame ...

Dark frame optimization thresholds:
Td0 = 0.00000000 (189 px = 0.003%)
*** Warning: The dark frame optimization threshold is probably too high (channel 0).
Computing master flat scaling factors ...
*** Error: Zero or insignificant scaling factor for flat frame channel# 0. (empty master flat frame image?)
<* failed*>

So, you've calibrated the lights and told ImageCalibration to bias calibrate the master dark (Applying bias correction: master dark frame ...) which you already did when you created the master dark. So, as I guessed, you've subtracted the master bias from the master dark twice and that's the cause of the error.

I ran ImageCalibration without attempting to bias calibrate the master dark again (see first attached image) and it worked without error. I then debayered one of your 600s subs and it looks pretty good (see second attached image.)

I also checked the master flat and found some dust donuts. The calibration appeared to remove these nicely so all is well...

Let me know if any of this isn't clear.

Cheers,
Rick.

[mariner10 (Steve)]

Hey Rick,
As soon as I read your response it made sense so again my thanks. I'm about to give it another go.

Steve.

Originally Posted by RickS

Hi Steve,

Thanks for sending me some data to look at.

I started by looking at the calibration masters. I made a small representative preview in the master bias and copied it to the master dark and the master flat. Using the Statistics process I found that median values (as 16-bit integers) for my preview area were approx 4163 for the master bias, 12 for the master dark and 13724 for the master flat.

Now, this tells me that you have definitely created the master dark with bias calibration enabled (otherwise the median value of the master dark preview should have been greater than the 4163 of the master bias.)

Your error message was:



So, you've calibrated the lights and told ImageCalibration to bias calibrate the master dark (Applying bias correction: master dark frame ...) which you already did when you created the master dark. So, as I guessed, you've subtracted the master bias from the master dark twice and that's the cause of the error.

I ran ImageCalibration without attempting to bias calibrate the master dark again (see first attached image) and it worked without error. I then debayered one of your 600s subs and it looks pretty good (see second attached image.)

I also checked the master flat and found some dust donuts. The calibration appeared to remove these nicely so all is well...

Let me know if any of this isn't clear.

Cheers,
Rick.

[mariner10 (Steve)]

Hi Rick,
So I tried what you suggested and got a similar error message:

ImageCalibration: Global context

Loading master calibration frames:
Loading calibration frame image:
C:/Users/User/Pictures/QHY8L/Master Calibration Files/superbias.fit
Reading FITS image: 32-bit floating point, 1 channel(s), 3328x2030 pixels: done
Normalizing sample values: done
40 FITS keywords extracted
Loading calibration frame image:
C:/Users/User/Pictures/QHY8L/Master Calibration Files/Master Dark.fit
Reading FITS image: 32-bit floating point, 1 channel(s), 3328x2030 pixels: done
Normalizing sample values: done
29 FITS keywords extracted
Loading calibration frame image:
C:/Users/User/Pictures/QHY8L/Master Calibration Files/Master Flat.fit
Reading FITS image: 32-bit floating point, 1 channel(s), 3328x2030 pixels: done
Normalizing sample values: done
35 FITS keywords extracted

Dark frame optimization thresholds:
Td0 = 0.00000000 (3377920 px = 50.000%)
Computing master flat scaling factors ...
*** Error: Zero or insignificant scaling factor for flat frame channel# 0. (empty master flat frame image?)
<* failed *>

It appears to be slightly different to the original message.
I've attached an image of my settings for image calibration so you can see what I did.
I'll give BPP a go and see where that takes me.
Cheers.

Steve

Originally Posted by RickS

Hi Steve,

Thanks for sending me some data to look at.

I started by looking at the calibration masters. I made a small representative preview in the master bias and copied it to the master dark and the master flat. Using the Statistics process I found that median values (as 16-bit integers) for my preview area were approx 4163 for the master bias, 12 for the master dark and 13724 for the master flat.

Now, this tells me that you have definitely created the master dark with bias calibration enabled (otherwise the median value of the master dark preview should have been greater than the 4163 of the master bias.)

Your error message was:



So, you've calibrated the lights and told ImageCalibration to bias calibrate the master dark (Applying bias correction: master dark frame ...) which you already did when you created the master dark. So, as I guessed, you've subtracted the master bias from the master dark twice and that's the cause of the error.

I ran ImageCalibration without attempting to bias calibrate the master dark again (see first attached image) and it worked without error. I then debayered one of your 600s subs and it looks pretty good (see second attached image.)

I also checked the master flat and found some dust donuts. The calibration appeared to remove these nicely so all is well...

Let me know if any of this isn't clear.

Cheers,
Rick.

[mariner10 (Steve)]

Sorry Rick, missed the attachment.

[RickS (Rick)]

Hi Steve,

I checked my ImageCalibration settings and the only difference to yours was that I had the Master Dark optimization threshold at the default setting of 3.0. I set it to 0.0 as you had it and still didn't get the error. This is a bit mysterious...

The master flat you sent me has some areas around the borders where the pixel values are zero but it's certainly not an empty flat. Can you open the master flat in PI and see what the Statistics process prints for you?

If you have the original bias, darks and flats I think it would be worth running the lot through BPP. Let me know how it goes.

Cheers,
Rick.

[mariner10 (Steve)]

Hi Rick,
I played around with the optimisation threshold to see if it would make a difference and it didn't.
I also tried to run the BPP script and also got an error message, it didn't like something as well.
The statistics for the master flat is:

Reading 1 image(s):
C:/Users/User/Pictures/QHY8L/Master Calibration Files/Master Flat.fit
35 FITS keywords extracted
Reading FITS image: 32-bit floating point, 1 channel(s), 3328x2030 pixels: done
Normalizing sample values: done

I have tried to recreate another master light using the original lights and that didn't work either.
I'll try putting them through BPP as single files and see how that goes.

Cheers.

Steve.

Originally Posted by RickS

Hi Steve,

I checked my ImageCalibration settings and the only difference to yours was that I had the Master Dark optimization threshold at the default setting of 3.0. I set it to 0.0 as you had it and still didn't get the error. This is a bit mysterious...

The master flat you sent me has some areas around the borders where the pixel values are zero but it's certainly not an empty flat. Can you open the master flat in PI and see what the Statistics process prints for you?

If you have the original bias, darks and flats I think it would be worth running the lot through BPP. Let me know how it goes.

Cheers,
Rick.