HELP ME PLEASE: Raw files are doing my head in!

[Screwdriverone]

Hi All,

I'm becoming desperate now, this is frustrating me no end....

After setting up on Saturday night (after the whole day without clouds) I had to spend from 8:30am to 12:00am watching clouds slowly drift over my house.....grrrrr.....

Finally, it cleared so I turned the camera to M104 and then to M20 and got some 30 sec and up to 5 minute subs.....then the camera battery died, then the scope power tank died....but that's not the problem....

My RAW files seem to LOOK OK in DPP, but when I stack them in DSS as RAW or with Registax as TIFF, all I end up with is a washed out version which looks worse than a single sub....???? and it has no colour!

Nothing I seem to change in DSS like debayering or AHD or drizzle etc etc etc seems to help. I stretch the histogram and change the levels and curves and all I seem to get is a horribly noisy vignetted blob in the middle.......

Ive tried CS3, Startools, Registax, Nebulosity and have even resorted to surfing for instructions on what to do, to no avail.....

I must be horribly off the mark, considering I have sort of kinda OK processed an M42 by myself as RAW and my first three Astrophotos were in JPG with horrible coma and only 4.5 mins exposure....I have captured almost 30 mins of M20 Trifid (5 minute subs at ISO 400) and its WORSE than the 4.5 min effort.....

Its enough to make me want to throw my hands up and give it all away....

Is it the unmodded DSLR, the noise, or just the idiot behind the keyboard?

I have posted a smaller JPG version of one 5 minute sub as well as the stacked result so you can see what I mean....

Can anyone please point me to a tried and true "middle of the road" set of instructions that I can follow, with what I have so I can get some results without having to waste hours getting nowhere????

Please?

Hope someone can help me

Cheers

Chris

[mplanet62]

Honestly, I am not a great specialist in stacking business, but from your story I see one possible cause of it. What color space is your camera set to? It can be RGB, sRGB or Adobe RGB. If it's Adobe - colors will look washed off. I assume it will get worse when stacked. Adobe RGB offers wider gamut, but needs proper development in something like Lightroom to look good.

[Screwdriverone]

Hi Michael,

I just had a look at the 1000D and its definitely set for sRGB.....

Everything else looks normal, no ICNR, no long exposure noise reduction, shoot in RAW + L Jpg (large)....

Hmmm, dont think its the camera, although the data isnt all that good, surely 5 mins at ISO 400 should give me a nice bright? image of say M20????

Its easy to get some clean data on M42, but if I am going to struggle with this unmodded DSLR on anything other than globs or open clusters, then perhaps my camera of choice at the moment - a QHY8L or an Orion SSDSII Pro, then???

Just need the funds if I am heading down this path.....I just missed out on a QHY8 OSC the other day for a bargain $1K......grrrrrr

I would love to try out a QHY8 to see how much of an improvement there is......anyone.....???? Any shops care to let me do a review for them???? [crickets]....?

Chris

[DavidU]

Chris, watch this carefully and see what is done before registering & stacking. It is a DSLR.

http://www.youtube.com/watch?v=8hYB8gj4kqU

[Screwdriverone]

Thanks David,

Fingers crossed, I think the Bayer matrix FITS settings is the key here, I am stacking some now to see if its the case, hope so! It certainly makes sense considering my RAW stacks look monochrome more than colour....hmmmm.......:)

I will keep you informed of the results....I'm doing M104 at the moment, but I think I might just stack M20 here and show the unprocessed stack result for comparison if this fixes it....

Thanks for the nudge in the right direction (stay tuned)

Cheers

Chris

[Screwdriverone]

Well, the jury is still out, not much colour still, might have to RTFM.....

This is sooo annoying :(

Cheers

Chris

[Octane]

When taking RAW images, the colour space is irrelevant.

It only matters when saving images for print/web.

Always ensure that your images are converted to sRGB for web display.

H

[bojan]

There is nothing wrong with your results from DSS, Chris.
You just need a bit of processing in DPP - see here (this is how I do it):
http://www.iceinspace.com.au/forum/showthread.php?t=48651&highlight=Digital+photo+professiona l+DPP&page=2
Go to #37

Also, on this post-processed image it is visible now that you have vignetting issue (can be sorted out with flats) , and a bit of gradient (LP? could be removed with Iris maybe)

[Screwdriverone]

Thanks H, Thanks Bojan,

I have tried settings in DPP and the vignetting is there as you said and its horrible.

I think I might give up on this data set, I want to smash things now.

I fix the freaking coma corrector focus issue and now I get junk data, no doubt I will fix the vignetting with flats and something else will go wrong.

I spend friggin 9 hours in the cold and dark to try and capture SOMETHING and end up banging my friggin head against a wall because its crap, or I am crap or everything is crap.....

I dont think I will waste any more time on this POS camera or even entertain buying a $1400 qhy8.

I may as well just go to the ATM and withdraw some cash and then set it on fire......wont take as long to get the same result!

The irony is, I capture 4.5 minutes of CRUD with JPG the first time I try with NO tracking and get a good result, now I have GUIDING and MPCC and 5+ minute subs at lower ISO and I GET JUNK that I need a degree in photoshop to remove the rubbish in the photos!

FFS!

Thanks anyway for trying to help me, I think a hammer is my best bet from here.

Chris

[bojan]

Well, can you give me your gear and I will smash it (actually I will just tell you that I smashed it, for you it will be the same thing anyway, right? ;) )

Seriously now, you just need to try again, and what is most important, you need to know what you are doing at each step and why.

And yes, weather issue is frustrating.. but this is part of this hobby..

[Tandum]

Chris, You can improve vignetting in photoshop. I had a go and got this. Here's what I did :-

Open the image. Make a copy of it as a layer. Select this layer and pick filter/noise/median from the menus. Set radius to max and click ok. You now have a blurry image which is mostly the gradients. Now use the clone stamp tool to blot out any brightness from the target (M20), or any bright stars etc. Now select filter/blur/Gaussian and turn up the pixel size to smooth it out. Now blend this blurry layer as difference and adjust opacity to suit.

This works most of the time and a bit of trial and error gets you used to it, but flats really are a must have to start with :)

[Screwdriverone]

Hmmm, LOL Bojan, yes, I will keep at it, trouble is, I forget what I did last time, then fiddle and fiddle some more, and in the end stuff it all up. Better data would be nice, so I think I might start there.



Thanks Robin, did you do that to my single image or my stack? PS is another issue I have, seems some reading needs to be done.

Thanks for the tips on how to do it, I will give these a go when I have calmed down somewhat.

Dont know how processing is supposed to be fun, it drives me insane......:confused2:

Thanks

Chris

[Tandum]

I grabbed the one bojan posted. A light box and flats will make a big difference and knowing what to do in photoshop comes in time. Maybe look around for an e-book/dvd on astro processing.

[DavidU]

a quick one CS5

[Screwdriverone]

Thanks Robin,

A good suggestion. Might go have a look.

Cheers

Chris

[irwjager]

Chris,

Have you tried the vignetting preset in the Wipe module of StarTools? Make sure you get rid of stacking artifacts first and bump up the 'dark anomaly filter' setting if there are any small dark anomalies in your image (dead pixels for example).
Seems to work well on the images you posted.

Cheers,

[tlgerdes]

Chris,


I take you took these from Kellyville? Your real problem is light polution, 5mins and you are getting washed out with city glow, way too long an exposure.

I wouldnt do anything longer than 2mins for an un-flocked 8" newt and 3mins for a flocked 8" newt.

That is all I could do from my Sydney back yard with an 8" newt and a Canon 1000d.

What you really need is a dark site!

[Screwdriverone]

Right,

I have had my tanty now and I have untwisted my knickers....

Ivo, yes I have tried the vignetting preset in Startools, however, my results seem to give me a donut of haze on the edges of the middle blob....I might have to revisit and spend some more (read "little adjustments") time on this in your excellent program.....

Trevor,

Thanks for smacking me upside the head. Here I was *****ing and moaning and completely forgot that I was capturing these low in the east when Sagittarius couldnt have been more than 30-35 degrees up!

Damn fool I am, I completely got side tracked with debayering and stacking and ending up with a faded M20 covered in FOG like noise and didnt see the nose in front of my face.

The main thing that started me off was the fact that ONE single RAW 30 sec ISO 1600 sub viewed in DPP or converted to TIFF looks BETTER than the stacked 25 mins result of the 5 min subs. Having said that though, even the single RAW 5 miin sub looks better than my stacked result.

I just couldnt figure out why MORE data was looking WORSE than a 4.5 min JPG stack I did over two years ago with no guiding and no darks etc...its because I was STACKING more NOISE on top of each other.....

Just one thing (yes I know its more money) would a CCD CLS EOS clip help with the city/sky glow to give me better data or am I just polishing the proverbial excrement wasting $200 on this as I am using an unmodded camera?

OR, should I turf the underperforming 1000D and go for an Orion Starshoot V2 for $1450 or a QHY8L for about the same and simply get better data in shorter times to reduce this frustration level somewhat?

(BTW - thanks for the advice guys, and listening to me whinge and moan, yet again about my shortcomings....;) )

I'm gunna go and gently play with this and some eagle and M104 subs I also got that night now in Startools and see if I can scrub out some results.

Cheers

Chris

[rcheshire]

Frankly yes, Chris! They are a waste of time, unless you are guiding and can guarantee at least 6 - 7 minutes with round stars

Alternatively, iso800 and expose for 3 - 3.5 minutes under polluted skies - from Astropix exposure tables. StarTools Wipe will take care of light pollution.

And dither. A prerequisite for DSLR images. Modify your camera.

Have fun!

[bojan]

Chris,
Stacking ALWAYS reduces noise - it is the nature of noise, being a random signal, it averages out over the frame
The wanted signal, on the other hand, is not random - and the stacking process increases this signal.
The end result is increased Signal/Noise ratio, exactly what we want to achieve.
Now, the background illumination (LP) reduces contrast and shifts the colour balance. When you stretch the histogram, the consequence is reduced S/N ratio, plus increased readout and quantisation noise (this is because of limited dynamic range of the camera ADC ).
How to overcome all this?
Dark site, more light frames and adequate numbers of dark frames plus flats plus bias frames (the last two could be done once for the specific camera and lens).
There is no need to spend k$ for a decent result. Of course, if you have plenty of them, no worries.. But even the most expensive camera will have noise, vignetting, readout noise.. the only way to overcome all those limitations is patience and knowhow.

[tlgerdes]

The 1000d is still a good camera. I spent 2.5 years playing with mine before i "graduated". On a cold night in the country it performs very well. My issue was heat noise, when ambient temp is above 15deg you can really notice it. Things like dark/heat noise are only an issue when you have clear dark skies where noise is greater than signal. In the city with all the background light we have signal is always greater than noise.

A CLS filter or other light pollution filter will make a difference and if you dont get the EOS Clip variety it could be used on whatever you graduate to as well.

What you really need though is a night in a dark site, once you do this you wont want to do anything else. I used to image from my city back yard, but now I only test there and wait for the next new moon weekend at Wiruna (2.5hrs drive from your place).

If you dont want to drive, I am sure you can hitch a lift with Barry, he is only round the corner from you.

[Screwdriverone]

Thanks Trevor,

Last night out at Linden was a disaster, what with not being able to drift align properly, bugs and leaving my T ring at home, I got very frustrated and decided to stay at home for a while (then it rained for 3 months)....

Next time I get say $200 for an LP filter, I should be also to afford a QHY8L so I think I will save my sanity till then. Keeping in mind the LP from here, I should be right with better spectral response and lower thermal noise. My chip at 18 deg ambient was reporting 25 degrees constantly, although some subs were recorded early in the morning at 13 deg C....

Thanks for the suggestions!



Sage advice Bojan, a light box or some evening flats are on the to do list. Thanks for explaining the technical aspects, helps to understand why rather than just live with it.



Hmmm, OK, so some 120 sec subs at ISO 800 and LOTS of them is a better choice then. Cheers, I might go dig out the 2 min subs and give them a go.

Cheers

Chris

[bojan]

20 light frames is enough. Additional 100 more will do not much to the end result (1dB ? ). I am using usually 12 or 16.
Followed by the same number of darks.

[Screwdriverone]

Good to know, I will limit myself to 20 x say 2 min subs, at ISO 800....with corresponding darks and sooner rather than later, some flats.

Might have to go digging up on how to take the proper flats with the T shirt method, I vaguely recall something about 2/3 of the histogram and fastest shutter speed, or is that bias?

Thanks for your help Bojan :)

Cheers

Chris

[rcheshire]

It still depends on the sky condition.These articles are very good and speak to some of your questions.
http://www.pbase.com/samirkharusi/beginners
http://www.iceinspace.com.au/forum/showpost.php?p=673800&postcount=1

http://www.samirkharusi.net/filters.html

[Tandum]

Chris,
I built my light box out of foam board from office works, hot glue, spit and solder, but I saw one of Exfso's boxes (http://www.iceinspace.com.au/forum/showthread.php?t=49736) at pmrid's and it's built way better than anything I could ever knock up.

When shooting flats, I normally aim for the center of the hystogram bell curve to be around 20,000. As the maximum is 64K, 20K is close to 1/3. Some of us with slow shutters have problems getting the light dim enough to achieve this as we need to expose for 4 seconds at least to avoid shutter artifacts in the flats. Introduce mono cameras and binning and the problem gets worse. But with a one shot colour it shouldn't be an issue.

Here's a flat from the other night plus the image I used it on, rotated 90degrees CCW, I've been chasing eggy stars :( The black bit to the left of the flat is the prism from the off axis guider sticking too far into the light path and yes it needs a clean but they flatten out the image a lot :)

Also, if you are going to get an LP filter, get an Hutech IDAS filter not a CLS. The CLS will throw a colour cast over the image which I found near impossible to correct at the time, the IDAS does not do that but it costs more.

[Screwdriverone]

Update:

I surrender........for now

I have been stacking every which way from Sunday till now, using different bayer method options and tiff conversions, even reverting to stacking JPG to try and get something out of the data I captured and I have had enough of looking at my pc grinding its progress bar up and up.....

I just dont seem to be able to get any colour out of these shots. The RAW files have heat noise and LP all over them but I can still see colour in the stars in the straight RAW shots.

No matter what I try, once I stack, its all washed out. With OR without darks. (no bias or flats taken)

So, with this in mind, I have decided to bin this lot (pun intended) and give it a rest for a while, wait until I can capture some better data at higher altitude and hopefully less LP, and start fresh with fresh eyes and hopefully a better result.

Having said that, I will post this short stack of M104 (6 minutes - 3 x 2 min subs at ISO 400) just because I havent captured a galaxy yet, or posted one....

Its really quite crap, but hey, why not. As you can see, the three stars on the right (which are quite yellow in real life) look white and the galaxy itself is almost monochrome.....but hey, I can recognise it at least.

Nothing done to it (because I dont know how to anymore) in any processing, this is simply DSS result converted to JPG and sized to 1024 x 700 something to get under 200kb. - EDIT - I think I may have played with the curves slightly in Startools and used the gradient function...I cant really remember now, Ive done it so many times Ive lost count....

Ive posted M104 as a direct comparison to Robin's version to highlight what I mean. I have also posted a JPG version of one of the RAW subs showing how much colour there really is, BEFORE I start stacking....?

Hope you arent sick when you look at it, ;) apologies if you are :)

Thanks to those who offered advice and encouragement while I banged my head on the desk and had a tantrum. :sadeyes:
Cheers

Chris

p.s. I have decided to pursue a proper cooled CCD in another thread I've posted, just need to scrounge up the cash and leave this cruddy data be.