M7 Wide Field - Image Processing Challenge

[jase]

Hi All,

As the curse of all deep sky imagers is approaching (full moon), this is typically the time of the month I plan future deep sky acquisitions instead of collecting data (no point wasting my time in obtaining crap data). So during this period, I thought I’d share some data I acquired a while back for a stellar processing challenge. At the time, I could not think of a better challenge than a wide field version of the open cluster M7.

So, I have provided two versions of the R, G, B files (small and large).

The small is a crop of the cluster (16-bit TIFF files - around 5.2Mb in size each) and can be downloaded here;

Red (5.2Mb) (http://www.cosmicphotos.com/reprocessed/M7cropRED.tiff), Green (5.2Mb) (http://www.cosmicphotos.com/reprocessed/M7cropGREEN.tiff), Blue (5.2M) (http://www.cosmicphotos.com/reprocessed/M7cropBLUE.tiff)

The large is the STL11k full frame FIT files which are 42Mb in size. I’ve kept the data in FITs IEEE float format as some stars are greater than 65k pixel values (I think the R channel has a few 77k value stars). Providing the 16-bit FIT files would have possibly dropped the file size, but I'm not certain of your image processing routines so feel its best to give you the maximum dynamic range possible.

Red (42Mb) (http://www.cosmicphotos.com/reprocessed/M7-25m5s_R.fit), Green (42Mb) (http://www.cosmicphotos.com/reprocessed/M7-25m5s_G.fit), Blue (42Mb) (http://www.cosmicphotos.com/reprocessed/M7-25m5s_B.fit)

All frames are reduced (dark, flat, bias), registered and are a combine of 5x5min subs per colour channel (25min). They are ready for your processing pleasure. :P I would recommend a starting colour ratio of R:1.5,G:1.0,B:1.5 if you want to get close to the correct colour balance.

I have no intentions to use this data for anything else. I was simply going to trash it. Put bluntly, it’s not the best data I’ve acquired. Stars aren’t perfectly round. Those who process the full frame fit files will see stars in the top left corner have elongated and you’ve got a reasonable gradient to deal with. I did say it was a challenge! What? You wanted magnificent data present to you on a silver platter? If it was that good, I wouldn’t be giving it away so easily.:lol:

So, don’t be shy – have a go. :) There are no rules, just have fun and post your results and some info on what you did so we can check’em out.

Below you’ll find my first cut of the star field. These are low res because of the IIS upload limit. If you want to see a higher-res full frame version, you can click here (http://www.cosmicphotos.com/reprocessed/M7-Fullframe%28hires%29.jpg) (3.1Mb).

[2020BC]

Your hi-res looks great. Wonderful stuff.

[strongmanmike]

Forgot to comment on this image (Seem to remember it being posted before?).

Anyway, great vista not often imaged!

On dial up so projects like this are difficult for me :sadeyes:

Mike

[avandonk]

I'll give it a shot Jase. Should be enlightening.
No pun intended!
Bert

[Dr Nick]

Very nice, I would love to see the full-res but my dial up is a little too slow. The lower resolution is still great though! ;)

I particulary love the colours. ;)

[iceman]

I'll have a go next week - I don't often get to process deep sky stuff, especially such quality data. It will be a good experience for me.

[avandonk]

Here is an initial try.
Digital Development of fits files
Pixel math green x 0.66
Richardson Lucy on each colour fit 3X3 for ten cycles
Recombine with IP to colour tiff
Adjust in PS for levels and crop to get rid of black bits
Convert 16 bit tiff to LDR image with EasyHDR
Adjust levels in PS
Another cycle of RL as before
Convert to JPG 3.6MB

Here
http://avandonkbl.bigblog.com.au/data/2/13839/image/M7Jase5171820070728155807.jpg


Thanks Jase I will keep playing with the data.

Bert

[ballaratdragons]

Jase, here is the best I could get it. It was hard converting them from Black & White to colour before I added them together!

Not real good is it :lol:

[erick]

You got some colour into two stars Ken! What am I saying, much better than me - I wouldn't know where to start!!

[marc4darkskies]

Nice image Jase!

OK, I'll give it a go - I need the practice! Besides, not much else to do this Sunday arvo.

The following shows the same cropped area in your and my version. Stars are a bit smaller, but I dropped some colour depth. I didn't try to increase the star sizes in the cluster - that might make the image more appealing. I highlighted the darker areas a bit 'cause I like that.

http://www.pbase.com/gailmarc/temp

[jase]

Thanks Bill, Mike and Dr. Nick – though my intentions were not to display this image as a master piece, but I’ll acknowledge your compliments anyway.



Good stuff Bert. I find it interesting how you individually worked on each channel before combining them. I typically find it easier to combine the data before manipulating it primarily because if a routine changes the brightness of a channel more than the other channels, you’ve lost your colour weightings (sure you can get them back, but it easier to prevent).
Why did you increase the pixel values on the green channel with pixel math? The KAI-11000 chip is rather green sensitive so you it needs to be compensated by boosting up the other channel weights. Was this specifically done to provide more information for the deconvolution process?
How did you find the LDR process working with 16-bit data? In your opinion, are there greater benefits over 12-bit for this process considering the increase in dynamic range? The star centers aren’t white, but a subtle pink hue, was this caused by LDR process or incorrect colour weightings do you think?



Ken, your colours are nearly non-existent. What happened? If you want the quick and dirty way of doing things and you’re only using photoshop, open up the R channel, the convert the image to RGB. Then select the channels tab. Copy and paste the G, B images in to the G, B channels. This is a rather crude and nasty way of doing things (in my opinion) as it not always easy to get colour balance right. It’s all experience mate. Have a couple more goes if you’re having fun.



Looking good there Marcus. Well done. Emphasising the dark areas does enhance the image.

Don't hold back everyone, get in there and have a go.:poke:There is not right or wrong - just have fun.:)

[marc4darkskies]

OK, I enlarged the stars in the cluster to make them stand out more. I just did a quick & dirty - not knowing which stars belong to M7, I may have selected too many. http://www.pbase.com/gailmarc/temp

But it IS fun! :screwy: :nerd: Hey!! Maybe some artificial diffraction spikes next??!! :lol:

Cheers,
Marcus

[ballaratdragons]

Jase, I don't understand what you just said. I take it you are speaking in Computer talk (R Channels, Channel Tabs etc) :shrug:

It took me just on Two hours of frigging around to get them from Mono to colour and I almost gave up, until I tried using PS Actions.

I found that I could convert them one at a time by using 'Fake H-Alpha' for the Red, 'Fake OIII' for the green, and I forget the one I used for Blue.

Then I tried to layer them. I've never been able to work out how to do layers, and I still couldn't. I spent Four and a half hours working thru the tutorial and trying it as I read!!!! It still didn't work properly :doh:

In other words, I wasted a total of 8.5 hours to get my mess :lol:

This is the Main reason I don't want a Monochrome camera.

Colour will do me fine :thumbsup:

[jase]

Not too sure about the enlarged stars Marcus. This may work if you stopped the white clipping with specific curve before enlargement. Look like you being to look some colours associated with them as well. None the less, interesting way of processing a cluster. Wouldn't have thought to enlarge the stars only because if there is something else in the frame, things may not be proportioned as such. I guess with a cluster only frame it doesn't make much difference.



At least you gave it a go Ken. We learn by doing :thumbsup:

[avandonk]

Here is another go at Jases data

In images Plus
Digital Development of fits files as before
Pixel math green x 0.66 (same as mult R&B by 1.5) without blowing out either.
Convert each R,G&B Fits to three corresponding R,G&B Tiffs

In RegiStar
Align R,G&B Tiffs with RegiStar and simply sum.
Save as 16bit Tiff

In Photo Shop
Resultant colour tiff levels adjusted in PS
GradientXterminator used settings Coarse and Low
Levels adjusted again in PS

In Images Plus
Richardson Lucy for ten cycles 5x5
It is really important to carefully choose lower and upper levels when using RL enhancement otherwise you can enhance noise and introduce unwanted artefacts into the brighter stars.
Convert to JPG 90%

Large image 5.2MB
http://avandonkbl.bigblog.com.au/data/2/13839/image/M7Jase23692020070730075634.jpg


I think Registar does a better job of aligning images than any other method.
I could be imagining it but the faint backgroud stars are clearer.
GradientXterminator seems to have got rid of the gradient which often plagues wide fields.
Thanks again to Jase for letting us play with his data.

Bert

[jase]

Nice work Bert. When blinking your original and the latest, the stars might be a little sharper, but not by much. Certainly nothing really obvious. The colours look quite muted, but still looks good. Pleased you're making the most of the data and enjoying it in the process.

[RB]

I'm looking forward to having a go Jase, thanks for the opportunity.
Just been so busy lately.

I'll have a go soon.

:thumbsup:

[Tamtarn]

Never dabbled in RGB files before so this was a fun project

Processed in PS CS2

Thanks for the opportunity to use the data Jase

Here's my first try ........ Didn't know which colour to post so here's two.

I'll keep trying :)

30021 30022


Link to larger files http://barb.david.1.googlepages.com/barb%27sdemopage


Barb

[jase]

Hi Barb,
This is great. Really like the effort you've put in to bring out the colours. I enjoyed your second image more than the first. Vertical or horizontal flipping the image can add an extra twist. Keep at it.

Cheers

[Ingo]

Is it that yellowish orange color because of light pollution? Or is that natural?

[jase]

I can proudly exclaim that there is no light pollution in this data. It was collected from a rural dark sky location (as is most of my data). What you’re seeing is natural. Tens of thousands of stars from our rich MilkyWay. The yellowish/gold tint is produced from interstellar dust extinction.

[avandonk]

Here is a pseudo HDR image.

1 Digital development of fits files
2 Pixel math x0.66 for green
3 Recombine with images plus to produce colour tiff image

This was done three times to produce three images the only difference being the break point was doubled each time. This result is three tiff images a 'stop' apart.

These three were then cropped identically to get rid of non image edges with Images Plus under Image File Operations then choose Crop Files. This was done as otherwise you get floating point errors etc in EasyHDR.

These three images were then used to produce a LDR image from a HDR image generated in Easy HDR.
Levels adjusted in PS and GradientXterminator applied as before and RL in Images Plus.
Converted to JPG (90%) with IP and it is interesting the image is now 6.5MB.

Here
http://avandonkbl.bigblog.com.au/data/2/13839/image/M7Jase35348220070731091123.jpg
I think the colour is better.
Thanks again Jase. I have learned a bit more about this black art called image processing.

Bert

[iceman]

I think that's your best yet, Bert. The colours are probably still a bit muted (to my preference) but nice job.

[jase]

Incredibly sharp Bert. As Mike states, the foreground star colours a still a little muted. Background stars are looking great. You could do the trick I use when combining Ha and RGB data. Typically when you combine the two data sets, stars go crazy. So I copy the original RGB data that has been subtly stretched and saturated to bring out the star colours. Make this a new layer mask (hide all). Then simply highlight the stars on the mask layer. This will bring them out in their natural colour and dimensions without compromising the nebulosity or star field. This is one of many options. Certainly your best to date.:thumbsup:

[Ingo]

Very nice. I wish I had 0 light pollution :)

[jase]

Bert, I thought about your processing further. In particular the LDR/HDR process. I’m not too sure if this is suited to such a wide field vista that contains only stars. As the process is taking into consideration the entire frame, key features such as M7 are muted. I’m wondering if you would get a different result if you cropped M7 so there isn’t as much of the surrounding star field in the frame and ran the same LDR/HDR process. This may produce an improved image of M7. If the results are desirable, you could then blend the improved M7 image into the greater wide field vista.

[avandonk]

Jase the HDR process really needs exposures that really are stops apart. Not fudged like this is. But you can see how it helps. The brighter stars really need the shortest exposure to capture their colour. Our visual system has a logarhymic response and it it hard to simulate with what is essentially linear data cobbled together after distorting its linearity.
I am surprised it even came out a well as it did. Still it is interesting what can be done.

Bert

[jase]

Yep, I understand what you mean. You need to have differing exposures for the HDR process. Thus, CCD work does not lend itself to HDR work very well. Sure you could capture maybe 5min, 10min and 15min exposures, but this isn't the typical methodology used for long exposure CCD work. As CCD's response to light is linear, the DSO imaging trend is longer and longer subs. 15mins today, 30mins tomorrow. None the less, its good to try these processing tasks out to see what is capable.

[avandonk]

Just a few further thoughts.

Jase I also should mention that your camera has 16 bit dynamic range ie 65K odd levels whereas my Canon sensor is limited to 12 bits ie 4K levels by the A to D converter. To get the same dynamic range coverage with the Canon sensor as your camera I would have to take at least three images two stops apart. That is why I have been trying EasyHDR.
This of course assumes the target has this high dynamic range to make it worthwhile.

I think the real problem is trying to squeeze either 12bit or 16 bit linear information into an 8 bit log jpg and have the intensity distribution match our eyes logarhythmic sensitivity. All the while maintaining or attaining colour balance! This is what image processing is really about!?

Bert

[tornado33]

Heres my effort.
I used Iris (its freeware ), turned the .fit into.pic (the file format Iris uses)played with the colour balance till it looked okay, but only ended up cutting back the green a small amount, .66 seemed too much with Iris. Used the Dynamic Stretching function to bring out the background without blowing out the brighter areas too much. Finished it off with Levels in Photoshop .

If I won lotto Id have an STL11 with the Adaptive Optics acessory on my telescope and lenses in a flash :)
Scott

[jase]

Well done Scott. You've made the star cloud very rich looking. Though in the process have lost much of the star colours. The image appears to take on a mono-tone feel.

[tornado33]

Thanks
Heres another go but this time using the "colour stretch" function in Iris as well. Bit of colour in the bright stars now :)
Scott