PDA

View Full Version here: : Asterism - image preprocessing in Linux


rcheshire
07-04-2018, 11:54 AM
This project has matured quite a bit, hence a new thread. It's a novel approach to image calibration but it works and produces comparable results. I believe it has been included in the 'Astronomy Linux' (https://www.cloudynights.com/topic/543184-new-open-project-%E2%80%9Castronomy-linux%E2%80%9D/) distribution produced by Oleg Astro - Cloudy Nights.

The latest stable version of Asterism (http://synergous.com/flatpress/fp-content/attachs/asterism.zip) has seen a good deal of change and is being maintained, until I am satisfied with it. It is ImageMagick 6 and 7 compatible, including IM compiled from source.

EDIT: New user manual (http://synergous.com/flatpress/fp-content/attachs/asterism_user_manual_new.pdf)

EDIT: Added histogram viewer

Why another preprocessing utility? Asterism is a product of curiosity. Mainly, coming to grips with the nuts and bolts of image processing to familiarise myself with methods of noise reduction / image calibration with tools widely and freely available.

Having no formal code writing or high end maths, it was a far better thing to use existing image processing programs. The capabilities of ImageMagick were quite a surprise, just scratching the surface. So, with a bit of bash scripting the project took shape and has become something of an obsession.

What does Asterism do? It does a lot of things. Some stand-alone, some integrated and some optional.

Asterism will run a full image calibration routine to a stacked linear image - including background extraction. Alternatively, processes may be run individually.

EDIT: There is no restriction to image input and output format, providing it is recognised by ImageMagick and for RAW files DCRAW. In many cases ImageMagick requires no extension. I have not tried video formats.

For most processing I use a super-bias or defect map and median combined flat (The default behaviour is bias subtraction and flat division of light frames, if bias and flat or master bias and master flat is selected, apart from any other master frame selection - this avoids truncating data. Alternatively, use a defect map and master flat). In-built noise reduction does the rest. Some terms, such as super-bias, have been borrowed from other programmes but the actual process may differ.

Asterism aims to produce a clean (high SNR) image from a set of DSLR RAW images (and other formats), whilst preserving detail. Drop into StarTools (or whatever) without difficult gradient removal or the need for excessive post processing noise reduction. Having said that - noise reduction does begin with image acquisition - particularly dithering.

Asterism also corrects for improperly exposed flats mixed with light frames of varying exposures - the process is configurable. It has a Files manager, an image format conversion utility, a blink utility (for inspecting and rejecting images before committing to processing), a stand-alone image alignment and stacking function (as well as the built-in version); and, if Hugin is installed, a panoramic stitcher and additional alignment and stacking utilities. Recently added an animated histogram viewer.

Manufacturer sRGB channel multipliers may be applied to images to give a little more spark to unmodified RAW data, prior to or post stacking.

System resources are configurable and the default settings are conservative - no freezes, at the expense of processing time.

To trial Asterism use a set of 3 bias 3 darks 3 flats and 3 lights (preferably from the same image set) to create a defect map or super-bias and master flat. Then select Lights and DeBayer Align Stack - and Extract background, if you wish - then select Process on the bottom toolbar.

Links to the latest version and the user manual are found under the Documentation tab.

To install - first install DCRAW ImageMagick, Panotools and Yad. 'bc' and 'find' if not installed by default; then

unzip the Asterism folder to the home directory. Open the folder. Open a terminal in the folder and type ./asterism (Enter) or, if you want to see the whole process, type bash -x ./asterism

It works on my system - but please let me know if it doesn't on another - it should...

(can be installed but best to run from a terminal as described)

Always open to suggestions and bash scripting tips...

Clear skies

Rowland

rcheshire
25-04-2018, 11:46 AM
If you have downloaded a previous version, I recommend downloading the latest (http://synergous.com/txtpat/files/asterism.zip). There's an updated user manual (http://synergous.com/txtpat/files/asterism_user_manual_new.pdf) ~4MB and a web version (http://synergous.com/txtpat/files/asterism_user_manual_new.html) - which has seen a number of rewrites as Asterism grew.
A searchable PDF (http://synergous.com/txtpat/files/asterism_user_manual_original_new.p df) ~20MB

EDIT: updated links

EDIT: 32 bit / HDRI on the way...

The latest version of Asterism includes a Develop function. This extends the earlier background extraction, adding auto white balance and a gamma curve, to see what you've got. The file can be saved but it's really just a look see, for now.

The Orion image is 60 minutes of data (an unfinished project) processed in Asterism and StarTools, with some final touches in ImageMagick. Noise was a problem because the image is stretched to the max to reveal detail.

Rather than apply noise reduction at post processing, it is more effective with the right tools to apply while the image is linear. Essentially, lift and suppress pixel values about the mean, rather than a median filter which reduces SNR.

EDIT: The 6yo Canon 1000D image of Antares / Rho Oph, is straight out of Develop with some touching up in IM - Laptop... The RAW files were captured at a regulated +5C - super-bias and master flat. Noise - radius 7; amount 3. Bilinear deBayer.

EDIT: Develop - background extraction, colour calibration (configurable) and stretched. It needs a little work. But not too shabby.

EDIT: Develop previews with saturation mask.

rcheshire
13-05-2018, 08:04 PM
Hopefully, the last update (http://synergous.com/txtpat/files/asterism.zip) for a while. Improvements and bug fixes.

I have added bit depth selections 8 through 64 - default 16 - to accommodate ImageMagick 7+ Q32. A little more work is required is to get 64 bit up and running.

To install IM 7 use one of distro binaries or install from source; as follows

Unzip and do
./configure --enable-hdri --with-quantum-depth=32
or to install to /usr (default is /usr/local)
./configure --prefix=/usr --enable-hdri --with-quantum-depth=32

make
sudo make install
To uninistall
sudo make uninstall

32 bit processing is a little slower than 16 bit.

rcheshire
09-06-2018, 06:12 PM
Basic master frame demo (http://synergous.com/txtpat/files/Asterism_Master_Frames.m4v)

Basic calibrate deBayer align stack and develop light frames demo (http://synergous.com/txtpat/files/Asterism_Light_Frames.m4v) - 3 frames only for testing.

rcheshire
15-08-2018, 05:07 PM
Asterism has been down for some time and is now UP (http://synergous.com/txtpat/files/asterism.zip) again...

rcheshire
16-08-2018, 10:32 AM
Asterism is on GitHub (https://github.com/geoland1/Asterism).