Go Back   IceInSpace > Equipment > Astrophotography and Imaging Equipment and Discussions

Reply
 
Thread Tools Rate Thread
  #1  
Old 10-07-2012, 12:01 PM
troypiggo's Avatar
troypiggo (Troy)
Bust Duster

troypiggo is offline
 
Join Date: Nov 2008
Location: Brisbane, Australia
Posts: 4,830
[PixInsight] Using BatchPreprocessing script on 32bit Win and having memory issues?

If you're having trouble on Win32 platform with memory and ImageIntegration (II), I believe the workaround is to reduce the Buffer Size from 16MB to, say, 1MB, and Stack Size from 1024MB to, say, 100MB. Can tweak the numbers, but these are a good starting point. This is pretty easy to do if you're manually using II, but if you're using the BPP script these variables are hard coded in the script.

The solution is to roll up your sleeves and manually edit the file. It's easy to do, it's just a plain text file. Using either Notepad or the inbuilt PI script editor, you want to edit the file BatchPreprocessing-engine.js. Around lines 1010 and 1011 you'll see this:

Code:
   var II = new ImageIntegration;

   II.inputHints             = this.inputHints();
   II.bufferSizeMB           = 16;                                     
   II.stackSizeMB            = 1024;
   II.images                 = frameSet.enableTargetFrames( 2 );
You just need to change the 16 to 1, and 1024 to 100, then save the file and close, and restart PI.

Code:
   var II = new ImageIntegration;

   II.inputHints             = this.inputHints();
   II.bufferSizeMB           = 1;                                            
   II.stackSizeMB            = 100;
   II.images                 = frameSet.enableTargetFrames( 2 );
Reply With Quote
  #2  
Old 10-07-2012, 12:13 PM
swannies1983 (Dan)
Registered User

swannies1983 is offline
 
Join Date: May 2006
Location: Adelaide
Posts: 781
Yeah I have already tried manually changing the buffer and stack size but I still get "out of memory" errors. I'll give your advice a go though.

I'm only integrating 7 images e.g. dark files. Is it because I'm stacking dslr images? Files are only ~8MB big so i wouldn't have thought this would be a problem. Oh well, I'll stick with PS.
Reply With Quote
  #3  
Old 10-07-2012, 12:40 PM
troypiggo's Avatar
troypiggo (Troy)
Bust Duster

troypiggo is offline
 
Join Date: Nov 2008
Location: Brisbane, Australia
Posts: 4,830
Is it only happening on ImageIntegration, or is it during other operations?
Is it happening when using the BatchPreProcessing script? The part of that script that does the integration is actually just calling the ImageIntegration, so I imagine if you're having trouble with one, you will with the other also.

Those figures of 1MB and 100MB are just starting points. They (the PI team) do recommend playing with those numbers to suit your system. I guess if those numbers don't work, start reducing until it does.

You know how to find them in II?
Reply With Quote
  #4  
Old 10-07-2012, 12:52 PM
swannies1983 (Dan)
Registered User

swannies1983 is offline
 
Join Date: May 2006
Location: Adelaide
Posts: 781
It's been a while since I tried. I did post a thread in the Pix forum but haven't received any useful replies.

This is what I posted:

I also tried to create my own master frames via ImageIntegration but I run into out of memory issues. I tried reducing Buffer size (8 and below) and stack size (100 and below) but I still run into issues when it reaches

MRS noise evaluation: —

*** Error: Out of memory
<* failed *>

All is ok up to the MRS noise stuff (whatever that is). I also had problems with the BPP operation. My other part of my post:

Loading master calibration frames:
Loading calibration frame image:
E:/Daniel/Astronomy/Images/Trifid Nebula/RAW Images/master/dark-BINNING_1-EXPTIME_300.fit
*** PCL Exception: PCL FITS Format Support: Unable to open FITS file:
E:/Daniel/Astronomy/Images/Trifid Nebula/RAW Images/master/dark-BINNING_1-EXPTIME_300.fit
CFITSIO error message stack:
01 : Extension doesn't start with SIMPLE or XTENSION keyword. (ffrhdu)
02 : END
03 : Error while closing HDU number 0 (ffchdu).
04 : Extension doesn't start with SIMPLE or XTENSION keyword. (ffrhdu)
05 : END
06 : Error while closing HDU number 0 (ffchdu).
07 : Extension doesn't start with SIMPLE or XTENSION keyword. (ffrhdu)
08 : END
09 : ffopen could not interpret primary array header of file:
10 : E:\Daniel\Astronomy\Images\Trifid Nebula\RAW Images\master\dark-BINNING_1-EXPTIM
11 : E_300.fit
12 : This does not look like a FITS file.
<* failed *>

Looking at the fit master file, it's only 3KB in size. My fits also are only 3KB in size if I convert my cr2 files to fit (BatchFormatConversion).

This error happens on two different laptops.
Reply With Quote
  #5  
Old 10-07-2012, 01:10 PM
troypiggo's Avatar
troypiggo (Troy)
Bust Duster

troypiggo is offline
 
Join Date: Nov 2008
Location: Brisbane, Australia
Posts: 4,830
1. Instead of 8 for Buffer Size, try 1MB.
2. Just check you have the latest version of BPP and II. I know there have been some updates recently. Assume you know how to update if it's not automatic on PI startup?
Reply With Quote
  #6  
Old 10-07-2012, 01:14 PM
swannies1983 (Dan)
Registered User

swannies1983 is offline
 
Join Date: May 2006
Location: Adelaide
Posts: 781
Quote:
Originally Posted by troypiggo View Post
1. Instead of 8 for Buffer Size, try 1MB.
2. Just check you have the latest version of BPP and II. I know there have been some updates recently. Assume you know how to update if it's not automatic on PI startup?

Yeah I have tried 1MB for each but still run into problems. I do watch the memory usage and notice that if I make the buffer size and stack size smaller, usage is lower but then spikes right at that MRS stuff.

PI should be up-to-date as it did update itself when I installed the trial version. I haven't looked since though. Probably run out of trial period anyway.

NOTE: I've got 2GB RAM
Reply With Quote
  #7  
Old 16-07-2012, 10:46 AM
TheDecepticon
Registered User

TheDecepticon is offline
 
Join Date: Feb 2008
Posts: 1,223
My two bobs worth.

It seems that to get the best from PI, you need a good 64 bit system with at least a 4Gb RAM and a 4 core processor. Then you can set it up to work really well. Windows 32 bit is limited in its RAM usage to 3.25Gb, so even if you stick another 2Gb of RAM in it, you will not get the full benefit of it.

I recently replaced the motherboard, RAM and processor in my machine and added a copy of Win 7 64 bit Home Premium. What a difference, everything works better!
Reply With Quote
Reply

Bookmarks

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +10. The time is now 12:05 AM.

Powered by vBulletin Version 3.8.7 | Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
Advertisement
Lunatico Astronomical
Advertisement
SkyWatcher Australia
Advertisement
OzScopes Authorised Dealer
Advertisement
Bintel
Advertisement
Meade Australia
Advertisement
NexDome Observatories
Advertisement
Celestron Australia
Advertisement
Astronomy and Electronics Centre
Advertisement