PixInSight Issues

mariner10 (Steve) · #1 24-05-2016, 03:37 PM

Hi,
Anybody that is processing using PixInSight I have a couple of issues that somebody may be able to help me out with please.

Firstly, when I start the calibration and registration process and provide the lights (only 12 all up) and the master darks, bias and flats I get an error message (came up 13 times, assuming one message per light and one for one of the masters). The error message states "Parsing 32-bit floating point expression: conversion error: 1. #QNAN (line=59,column=160). The error message didn't seem to impact on the image in any way and it didn't stop the process from running.

Secondly, I'm having trouble calibrating and registering images I've taken using my Canon 550D DSLR (I now use a QHY8L OSC). The images from my DSLR are saved as CR2 files which apparently are a Canon RAW format. I select "DSLR_RAW" under the "Format Explorer" menu but I keeping getting an error message stating there's an issue with the file format.

Any help would be appreciated.

Steve.

RickS (Rick) · #2 24-05-2016, 03:45 PM

Quote:

Originally Posted by mariner10

Hi,
Anybody that is processing using PixInSight I have a couple of issues that somebody may be able to help me out with please.

Firstly, when I start the calibration and registration process and provide the lights (only 12 all up) and the master darks, bias and flats I get an error message (came up 13 times, assuming one message per light and one for one of the masters). The error message states "Parsing 32-bit floating point expression: conversion error: 1. #QNAN (line=59,column=160). The error message didn't seem to impact on the image in any way and it didn't stop the process from running.

Secondly, I'm having trouble calibrating and registering images I've taken using my Canon 550D DSLR (I now use a QHY8L OSC). The images from my DSLR are saved as CR2 files which apparently are a Canon RAW format. I select "DSLR_RAW" under the "Format Explorer" menu but I keeping getting an error message stating there's an issue with the file format.

Any help would be appreciated.

Steve.

Hi Steve,

I can comment on the first issue but not the second (I've only done mono CCD stuff in PI).

The conversion error looks like a file format problem. I could ask a bunch of questions but it would probably be easier if you posted a copy of the process console log for the whole calibration/registration process (select the appropriate range of text, right click and copy then paste into a reply.)

Cheers.
Rick.

rcheshire (Rowland) · #3 24-05-2016, 06:41 PM

Hello Steve.

This may be of assistance with your DSLR files.

Slawomir (Suavi) · #4 24-05-2016, 07:14 PM

Hi Steve,

Apparently you can convert your CR2 files into RAW files within the camera:

Press the "Menu" button at the top of the Canon camera to open the "Menu" display.

Locate the "Quick Control" knob to the right of the LCD monitor. Turn it until it's set to "Quality."

Press the "Set" button in the middle of the "Quick Control" knob. Locate the "Jump" button to the left of the LCD monitor. Press the "Jump" button.

Turn the "Quick Control" knob and select "RAW." Press the "Set" button to apply the "RAW" setting. Press the "Menu" button to close the "Menu" display. Your CR2 images are now converted to RAW images.

Otherwise, there are online CR2 to RAW converters.

Hope it helps

mariner10 (Steve) · #5 25-05-2016, 08:16 AM

Hi Rick,
Thanks for that, I'll see if the process console still has the relevant information and if it does I'll post it. I can't get to it until Sunday so it will take a few days.
Thanks for the reply.

Steve.

Quote:

Originally Posted by RickS

Hi Steve,

I can comment on the first issue but not the second (I've only done mono CCD stuff in PI).

The conversion error looks like a file format problem. I could ask a bunch of questions but it would probably be easier if you posted a copy of the process console log for the whole calibration/registration process (select the appropriate range of text, right click and copy then paste into a reply.)

Cheers.
Rick.

mariner10 (Steve) · #6 25-05-2016, 08:19 AM

Thanks Rowland, I'll have a read and see how I go.

Steve

Quote:

Originally Posted by rcheshire

Hello Steve.

This may be of assistance with your DSLR files.

mariner10 (Steve) · #7 25-05-2016, 08:23 AM

Hi Suavi,
Thanks for the information.
The problem is I have a whole bunch of images I took with the DSLR just waiting for processing and then I got the QHY8L.
So I didn't realise that the CR2 format would be an issue otherwise I would have saved them in RAW the first time. Oh well.
I don't use the DSLR now (although I may sometime in the future so your information will be valuable then).
I opening the images individually and tried to save them as RAW but PI didn't recognise the format then either.
Thanks.

Steve.

Quote:

Originally Posted by Slawomir

Hi Steve,

Apparently you can convert your CR2 files into RAW files within the camera:

Press the "Menu" button at the top of the Canon camera to open the "Menu" display.

Locate the "Quick Control" knob to the right of the LCD monitor. Turn it until it's set to "Quality."

Press the "Set" button in the middle of the "Quick Control" knob. Locate the "Jump" button to the left of the LCD monitor. Press the "Jump" button.

Turn the "Quick Control" knob and select "RAW." Press the "Set" button to apply the "RAW" setting. Press the "Menu" button to close the "Menu" display. Your CR2 images are now converted to RAW images.

Otherwise, there are online CR2 to RAW converters.

Hope it helps

mariner10 (Steve) · #8 29-05-2016, 01:09 PM

Hi Rick,
Unfortunately the script that I quoted the error message has been overwritten (I don't suppose the script goes on forever) but here is a scrip that I keep getting when I try to batch pre-process using either TIFF or GIFF images:

run --execute-mode=auto "C:/Program Files/PixInsight/src/scripts/BatchPreprocessing/BatchPreprocessing.js"

Processing script file: C:/Program Files/PixInsight/src/scripts/BatchPreprocessing/BatchPreprocessing.js

run --execute-mode=auto "C:/Program Files/PixInsight/src/scripts/BatchPreprocessing/BatchPreprocessing.js"

Processing script file: C:/Program Files/PixInsight/src/scripts/BatchPreprocessing/BatchPreprocessing.js

*********************************** **********************************
* Begin calibration of light frames
*********************************** **********************************

* Unknown exposure time frames and dark frame in calibration.
* Master bias: C:/Users/Steves/Pictures/Backyard EOS/QHY20160102/Master Bias Frames/superbias1.fit
* Master dark: C:/Users/Steves/Pictures/Backyard EOS/QHY20160102/Master Dark Frames/Master dark integration 1000 sec.fit
* Master flat: C:/Users/Steves/Pictures/Backyard EOS/QHY20160102/Master Flat Frames/Master flat integration.fit

ImageCalibration: Global context

Loading master calibration frames:
Loading calibration frame image:
C:/Users/Steves/Pictures/Backyard EOS/QHY20160102/Master Bias Frames/superbias1.fit
Reading FITS image: 32-bit floating point, 1 channel(s), 3328x2030 pixels: done
40 FITS keywords extracted
Loading calibration frame image:
C:/Users/Steves/Pictures/Backyard EOS/QHY20160102/Master Dark Frames/Master dark integration 1000 sec.fit
Reading FITS image: 32-bit floating point, 1 channel(s), 3328x2030 pixels: done
29 FITS keywords extracted
** Warning: The file format reports no CFA pattern for the master dark frame, but it is being forced as per process instance parameters.
Loading calibration frame image:
C:/Users/Steves/Pictures/Backyard EOS/QHY20160102/Master Flat Frames/Master flat integration.fit
Reading FITS image: 32-bit floating point, 1 channel(s), 3328x2030 pixels: done
35 FITS keywords extracted

Applying bias correction: master dark frame ...
Computing master flat scaling factors ...
s0 = 0.247739

Calibration of 9 target frames:
* Using 4 worker threads

Calibrating target frame 1 of 9
Loading target frame:
C:/Users/Steves/Pictures/Backyard EOS/07.11.2015/Pleiades/PLEIADES_LIGHT_180s_800iso_+18c_009 14stdev_20151107-23h15m43s782ms.gif

* Waiting for running tasks to terminate ...
*** Error: Incompatible image geometry
<* failed *>

*********************************** **********************************
* End calibration of light frames
*********************************** **********************************
Cheers.

Steve.

RickS (Rick) · #9 29-05-2016, 01:56 PM

Steve,

Quote:

Originally Posted by mariner10

Hi Rick,
Unfortunately the script that I quoted the error message has been overwritten (I don't suppose the script goes on forever)

If you go to Edit>Preferences>Miscellaneous Process Settings and change the Maximum Number of Console Lines to 0 then you'll get a log of unlimited length, which can be very handy!

Quote:

Originally Posted by mariner10

Calibrating target frame 1 of 9
Loading target frame:
C:/Users/Steves/Pictures/Backyard EOS/07.11.2015/Pleiades/PLEIADES_LIGHT_180s_800iso_+18c_009 14stdev_20151107-23h15m43s782ms.gif

* Waiting for running tasks to terminate ...
*** Error: Incompatible image geometry
<* failed *>

*********************************** **********************************
* End calibration of light frames
*********************************** **********************************

I don't think that calibrating a GIF file is going to do anything useful. GIF is an 8-bit RGB format and the calibration process will be expecting 16-bit or better monochrome data.

I'd be concentrating on getting PI to read the CR2 files directly. If you can put a copy of one of your CR2 files when it can be downloaded I'd be happy to see if I can convince PI to play nicely with it. PI implements DSLR_RAW with the DCRAW tool (http://www.cybercom.net/~dcoffin/dcraw/) which is widely used. I'd be surprised if it can't be made to work with a standard CR2 file...

Cheers,
Rick.

RickS (Rick) · #10 30-05-2016, 08:18 AM

Here's a DSLR workflow that uses BatchFormatConversion to convert CR2 files to TIFF. Might be worth trying?

http://synergous.com/flatpress/index...ata-long-hand/

Cheers,
Rick.

mariner10 (Steve) · #11 30-05-2016, 01:49 PM

Hi again Rick,
Thanks for the link and the information about changing the Maximum Number of Console Lines to 0.
I had a thought when I was coming to work this morning. I tried starting the processing routine again yesterday on a different target and set of images and came up with the same error message that should still be in the process console.
If I have a chance tonight when I get home I'll copy and paste it onto this string.
Thanks again.

Steve.

Quote:

Originally Posted by RickS

Here's a DSLR workflow that uses BatchFormatConversion to convert CR2 files to TIFF. Might be worth trying?

http://synergous.com/flatpress/index...ata-long-hand/

Cheers,
Rick.

mariner10 (Steve) · #12 30-05-2016, 01:51 PM

Forgot to add, the only reason I tried Giff and Tiff was to see if it made any difference and it didn't apart from that error message.
In one of the YouTube tutorials for PI the guy was using CR2 images so I know it can be done, how is the question.

Quote:

Originally Posted by mariner10

Hi again Rick,
Thanks for the link and the information about changing the Maximum Number of Console Lines to 0.
I had a thought when I was coming to work this morning. I tried starting the processing routine again yesterday on a different target and set of images and came up with the same error message that should still be in the process console.
If I have a chance tonight when I get home I'll copy and paste it onto this string.
Thanks again.

Steve.

mariner10 (Steve) · #13 30-05-2016, 02:02 PM

And I just had another thought and this may be really dumb of me to ask or a mistake I've made.
Could the problem be that I'm trying to pre-process CR2 files but my master dark, bias and flat files are a FIT extension?

Quote:

Originally Posted by mariner10

Forgot to add, the only reason I tried Giff and Tiff was to see if it made any difference and it didn't apart from that error message.
In one of the YouTube tutorials for PI the guy was using CR2 images so I know it can be done, how is the question.

RickS (Rick) · #14 31-05-2016, 09:04 AM

Hi Steve,

The file extensions shouldn't matter. You can calibrate a TIFF file with a FITS master. So long as the data is compatible it doesn't matter what type of file it is stored in.

I'd suggest you try the workflow in the link I posted converting the CR2 files to TIFF. If it works you win. If you get an error, post the message and we'll see if we can figure out what it means. As I said, I'm also happy to check out one of your CR2 files if you can put a copy in a place where I can download it.

Cheers,
Rick.

mariner10 (Steve) · #15 31-05-2016, 12:11 PM

Glad about that, I thought I was being really dumb there.
I've had a quick look at the link and I'll give that a go as soon as I can. I'll send you a link to Dropbox if you like for one of the CR2 files.

Quote:

Originally Posted by RickS

Hi Steve,

The file extensions shouldn't matter. You can calibrate a TIFF file with a FITS master. So long as the data is compatible it doesn't matter what type of file it is stored in.

I'd suggest you try the workflow in the link I posted converting the CR2 files to TIFF. If it works you win. If you get an error, post the message and we'll see if we can figure out what it means. As I said, I'm also happy to check out one of your CR2 files if you can put a copy in a place where I can download it.

Cheers,
Rick.

RickS (Rick) · #16 31-05-2016, 12:28 PM

Quote:

Originally Posted by mariner10

I'll send you a link to Dropbox if you like for one of the CR2 files.

That would be fine, Steve. If you want to include the calibration masters as well I'd be happy to see if I can get them all to play nicely together.

Cheers,
Rick.

mariner10 (Steve) · #17 01-06-2016, 08:07 PM

G'day Rick,
Here's the link for a couple of the CR2 files, it's been zipped if that's OK.
Cheers.

Steve

https://www.dropbox.com/s/3exm0xaogs...782ms.zip?dl=0

mariner10 (Steve) · #18 01-06-2016, 08:15 PM

Hi Rick,
Here's a link to a master flat file.
It would appear that I have inadvertently deleted my master dark and bias, I still have the individual images so I can recreate them but this is the best I can do at the moment.

https://www.dropbox.com/s/ls6tkqn8cw...ation.fit?dl=0

Cheers again.

Steve.

mariner10 (Steve) · #19 01-06-2016, 08:28 PM

Hi Rick,
Managed to get a copy of the error message from the process console as well, the error message is near the bottom and this is repeated for the entire batch preprocessing attempt:

* Limiting to 1000 brightest stars.
Matching stars: done
1000 putative star pair matches.
Performing RANSAC: done
1000 star pair matches in 23 RANSAC iterations.
Summary of model properties:
Inliers : 1.000
Overlapping : 1.000
Regularity : 0.992
Quality : 0.998
Root mean square error:
delta_RMS : 0.000 px
RMS error deviation:
sigma_RMS : 0.000 px
Peak errors:
delta_x_max : 0.000 px
delta_y_max : 0.000 px
Transformation matrix:
+1.0000 +0.0000 +0.0000
-0.0000 +1.0000 +0.0000
-0.0000 +0.0000 +1.0000
scale : 1.000
scale-X : 1.000
scale-Y : 1.000
rotation : +0.00 deg
dx : +0.00 px
dy : +0.00 px
Generating registered image
Homographic Projection / Lanczos-3 interpolation, c=0.30: done
Registration successful.
Writing output file: C:/Users/Steves/Pictures/Backyard EOS/QHY20160102/Rosette Neb Stockport 6.1.16/Calibrated/calibrated/registered/M78 lights -10c-2-480s-8-C0_c_d_r.xisf
Writing image: w=3328 h=2030 n=3 RGB Float32
72 FITS keyword(s) embedded.
Writing output file: C:/Users/Steves/Pictures/Backyard EOS/QHY20160102/Rosette Neb Stockport 6.1.16/Calibrated/calibrated/registered/M78 lights -10c-2-480s-8-C0_c_d_r.drz

Registering target image 5 of 5
Loading target file:
C:/Users/Steves/Pictures/Backyard EOS/QHY20160102/Rosette Neb Stockport 6.1.16/Calibrated/calibrated/calibrated/light/debayered/M78 lights -10c-2-480s-9-C0_c_d.xisf
Loading image: w=3328 h=2030 n=3 RGB Float32
*** Error: Parsing 32-bit floating point expression: conversion error:
1.⬛#QNAN (line=59,column=160)
51 FITS keyword(s) extracted.
Noise reduction: done
Structure map: done
Detecting stars: done
1504 stars found.

Cheers again.

RickS (Rick) · #20 01-06-2016, 09:23 PM

Steve,

I was able to download the master flat but I get a 404 not found error on the CR2 files link.

The error message points to a floating point format error in a .xisf file which looks like a bug to me. If you (or I) can isolate a test case that reproduces the problem and upload the files then I'm sure we can get Juan on the PI Forum to look at the problem.

Cheers,
Rick.