[Sali]

Originally Posted by irwjager

Please do send me a PM if you'd like to chat further about what you're doing technically - I don't want to bore everyone in this forum with technical talk.

I can just imagine someone in three months considering this same topic, finding this thread and having their research cut short. I'd be more than happy to PM you if that's just how it's done round here (learning heaps already - thanks) but I'd like to make at least one more post about this here first.

Originally Posted by irwjager

...I thought the JFIF scheme (as used for most JPEG compressed images, including those from the IPhone 3GS) do not have 'uncompressed' blocks .... Also, don't forget to take into account the debyaring algorithm which will likely have exacerbated noise as well .... Can you tell me what you mean by getting compressed 2 or 3 times? JPEG/JFIF is just a single pass compression scheme, no?

Also, what do you do if there are multiple stars within the same 8x8 pixel block?

Followed your links and googled some terms. What a mind-blowing adventure! Not possible without learning the true terminology. What I have so far called "uncompressed rasters" more acurrately refers to rasters which have retained their raw bayer matrix data. I don't know what process determines whether rasters are debayered/compressed or not. In my notes I just call this visual effect the "checkerboard matrix" because the red and blue pixels in those rasters pretty much always return no value at all - making the raster look like a green chess board.

The rasters which most likely contain stars all exhibit this appearance. Starsoup finds them based on the checkerboard pattern then examines each pixel: if the pixel fits neatly into the checkerboard pattern (in terms of colour and contrast) then it is probably not a star. If it exhibits huge variations in colour or contrast compared to its neighbouring pixels AND doesn't fit into the patern, then it is most likely a star and is recorded - even where there are two in one raster.

Some checkerboards (which Starsoup v1.0 cannot recognise) appear to have been filtered horizontally or horizontally+vertically (linear vs bilinear?); hence my 2 or 3 layers of compression observation. I guess that's a bunt observation?

Originally Posted by irwjager

...It is also important to realise that space is not empty, black nor fundamentally dark. There is an infinite amount of light coming from faint stars, galaxies, nebulae, etc, which will contribute to the light levels recorded by the CCD. Deciding to cut this light out completely is a rather arbitrary decision.

My observation is that the 3Gs camera is mostly blind to this "infinite light" because it is simply not sensitive enough to record it. There may indeed be light out there - but the camera can't see it and we only want to record what the camera actually sees. In such cases, pixels which failed to record a value should return "black" when read to a computer screen. But with the 3Gs they don't - they return various noises which appear to have nothing to do with the subject.

Fundamentally, Starsoup is based on my observation that the "noise" exhibited in the original picture is common across all images taken in near-no-light conditions with this camera. The only difference between a 3Gs picture of the night sky and a 3Gs picture taken inside a sealed box are those few rasters that have retained their checkerboard pattern (which I guess is the remnant bayer matrix). The noise (even those odd bright points which may "look" like stars at first) is present in both situations. It seems most likely the noise is not due to cosmic light and has no bearing on anything outside the hardware.

Originally Posted by irwjager

I attached a version of your input image of what it could look like, but as you say, the data is quite noisy. If you would have 2 or more frames (4 or more would be perfect) and would stack them, then you could get some real nice images, even with very short exposure, highly compressed JPEG data.

Without doubt, that is a cool picture. Thanks for showing such interest! Downside is, the image you posted mimicks the results I originally got when adjusting the image by hand. It makes unusually bright clumps of noise look like stars - and I'm pretty sure they're not, based on the observations I mentioned above. I may well be wrong in my observation - but it's pretty weird to see similar noise patterns in two different images and then label that noise based on the image context rather than its pattern.

I've attached another image to show what I mean - though I guess you figure it already. With Starsoup, the top-left raster would obviously record a star: note the checkerboard pattern and the unique traits of the pixels itself (this example also shows why we preserve the neighbouring pixels - in case of blurring).

Meanwhile, the bright point in the bottom right may indeed be a star - but the current version of Starsoup would not recognise it because its checkerboard has been filtered or compressed (how do we know which is responsible?). it looks linearly filtered to me - the new version should recognise this as a star.

But all the stuff inbetween is just soup. It's useless data that's made its way in at some point and which I sincerely doubt has any bearing on the image we tried to capture. It is noise that has been filtered and compressed - that's all. Manual non-biased adjustments cannot account for this.

Will your manually adjusted image compute in KStars? That might be really helpful for me - manually testing the new recognition algorithm.