Sharpening worms.

[alpal]

Sharpening artifacts.

We often mention sharpening worms when discussing a pic and
I agree that it can look awful when the image is expanded to full size.
Most of the time we talk about such artifacts in areas of
highlighted details but they can appear in backgrounds.


CHART 32 - a very popular site with the best high res pics -
has someone called Bernd Flach-Wilken who has managed
to turn all the backgrounds into worms with bad processing.
When you go to "expand to actual size" you can see millions of them -

see here:
http://www.chart32.de/index.php/recent

NGC 4216-4206-422 Galaxies in Virgo
and here:
NGC6820-3 Nebula + Cluster.


Yet - the ones processed by Johannes Schedler have
beautiful backgrounds with no worms - example -

NGC7550 Galaxy Group.

I wonder if CHART32 know about it?
What is generally accepted now when doing sharpening?
Any comments?
I know that Mike sometimes mentions this subject.


cheers
Allan

[strongmanmike]

Noooo, never! :shrug:.....:lol:

I'll wait before chiming in...I've said a bit on this topic in the past :rolleyes: :face: soooo, I'll wait a bit before commenting, interested to first hear/see what others think :)

Mike

[gregbradley]

Sharpening is best achieved at the point of acquisition with good seeing, good focusing, adjustments for temperature for the focus.

Then in processing by removing subexposures with FWHM that is too large or star bloats from a passing thin cloud etc.

Sharpening also works best on images with strong signal to noise ratio so long enough exposure comes into it.

I find deconvolution is handy sometimes when the red or blue subs are bloated compared to the other colours and can get rid of unwanted blue or red rings around stars.

Multi scale decon can work if only in the bright areas of an image. Worms appear first in the weak signal areas of an image. So selective sharpening works best. High Pass filtering is fairly non destructive when masked in only on the detail areas.

Other than that I haven't found too much that works well for sharpening. It easily goes south and poor sharpening is the mark of an amateur.

Greg.

[alpal]

Hi Greg & Mike,

I guess it's a matter of whether you want to show the full resolution picture
or a cut down small size version?
If you show full size after heavy sharpening then the worms will appear.
However there are many types of sharpening -
I like the one in Photoshop and the "smart sharpen" function.
I also have Fitswork4 which has:
Gaussian sharpening,
Iterative Gaussian sharpening,
Iterative PSF sharpening &
Deconvolution.


I find a mask is useful so that only areas of high signal strength are sharpened
otherwise the background will turn into blotches or worms
as per the example I have in the OP.


I have had interesting results using 3 x Drizzle in Deep Sky Stacker
on a small section of the picture.
That can bring out more detail when viewed at high resolution.



I don't think you can make everyone happy.
Sometimes I think that posters here need two pics full size -
one sharpened and one not?



cheers
Allan

[marc4darkskies]

Interesting to see even the CHART32 team isn't immune to processing artefacts and the need for sharpening.

Anyway, it's quite a complex topic Allan, as I'm sure you know. If discussed thoroughly, it would cover a significant number of processing techniques. So here are just a few of my quick thought bubbles and comments:


Worms are just enhanced noise - so don't sharpen your noise if you can avoid it!

Worms are more commonly seen in background and low signal areas where they are very easy to create

If you have noisy high signal areas - acquire more data!

Deconvolution is your friend when your stars are over sampled and larger than you want. If stars are well sampled or under sampled I'd consider it a crime to use decon :D. Decon is also good for carefully and selectively sharpening nebulous detail in high signal areas.

Decon readily produces enhanced noise artifacts (aka worms :lol:) unless your low signal areas have very little noise to begin with ie: you have megadata or very low noise cameras etc. Note: If you overdo decon in high signal areas they can also develop worms.

Various contrast / edge enhancing "sharpening" techniques and tools also produce noise artefacts - mostly ugly edge transistions and an "unnaturally" contrasty blobby look. Edge transitions in a good astro image should be smooth (mostly).

Noise mitigation techniques and tools typically DON'T produce worms. It's more likely they'll produce mottling. However, severe and full frame contrast enhancement will make existing worms very visible.

In any case, and I can't stress this enough, appropriate masking becomes your best friend when dealing with noise and sharpening. Having said that, over sharpening high signal areas and then masking them off from the rest of the image can yield an obviously fake differentially sharpened look.

As to what is generally accepted, I'd say good sharpening should be gentle and not be noticeable at full res to anyone other than the author. Of course, pixel peeping will reveal even the most minor issues and is a disease that afflicts many of us! We should resist the urge to pixel peep other people's images.

I use a multitude of techniques to varying degrees:


Decon at the initial calibration stage (CCDStack decon is quite good). It invariably produces sharpened noise because I don't create mega-data. This is especially apparent in the background and low signal areas of galaxy shots - so I often stretch the unsharpened one and then overlay the sharpened stars and high signal areas over that with masking

Noise mitigation and background smoothing. Always with masking. PS Reduce Noise, Neat Image Reduce Noise, my own background flattening Actions for background blotchiness (Coarse, Medium and Fine)

Contrast enhancement. I use Unsharp masking (my own PS action), Smart Sharpen, recursive high pass filtering (my own PS action), Astra Image wavelet (sparingly), APF-R (Absolute Point of Focus developed by Christoph Kaltseis - my own PS action implementation) and micro curve adjustments


Cheers, Marcus

[alpal]

Good post Marcus - I agree with you.
It's amusing to know that even the processing Gods at CHART32
are not immune from the mistakes of mere mortals like us.


cheers
Allan

[strongmanmike]

I think there might be different interpretations of what "worms" is referring too :question:

The pseudo detail, or dots and worms, that I have been known to bang on about :whistle: :bashcomp: appear in the medium to high signal areas, where the filter used (decon/wavelets etc) compress any hint of feature or structure, into thin unnatural looking bright curves and wavy lines, or make all or much of the detail in galaxies, appear as uniform circular dots. Both artefacts give the impression of resolution, when viewed at small screen size, but even a half R'sed close inspection reveals the awful fake news :scared:....provided the image posted is actually displayed at a reasonable size in order to be able to do this of course :)

The granular look seen in the low signal areas or the background, is a whole separate problem.

Mike

[that_guy]

looks like they removed the image from the site.

[alpal]

Yes Mike,
I know what you mean - I've seen it as well in high signal areas.
Just to illustrate the point -
I have made a sample picture from a bee shot that I did in January
and posted in the Terrestrial section here in Ice in Space:
http://www.iceinspace.com.au/forum/showthread.php?t=180548


I show below 3 cropped versions to have a full 100% size comparison.
the first is the original - not sharpened,
the 2nd shows the image at 2 pixel smart sharpening which is from memory
what I used for the photo I posted.
The 3rd is one that I have deliberately over-sharpened in Fitswork4
at 4 pixels deconvolution to show what happens as a demonstration only.
I believe it shows the sharpening worms and dots that you're talking about?


cheers
Allan

[alpal]

No - it's still there just moved to the galaxies section:
http://www.chart32.de/galaxies-m

the latest one in recent has the same problem.
http://www.chart32.de/index.php/recent


NGC5101 Galaxy in Hydra




cheers
Allan

[marc4darkskies]

Sorry Mike, my thought bubbles and comments didn't cover that - linear / crushed / merged detail in high signal areas. The worm-like artefacts I see and was mostly talking about are normally in low signal areas. See first example below.

The easiest way to reproduce the worms you're describing is using the minimum filter in PS (mainly used during star size reduction without masking) - see grossly exaggerated example below. I'm sure other software uses a similar mechanism.

Worms are NOT easily created using Wavelet sharpening - I've just tried and failed using gross Astra Image wavelet on a couple of my images. Decon and contrast enhancement can produce worm-like structures by virtue of the fact that they, when used in a ham fisted way, create bright blobs that may merge with other nearby bright areas to form a linear feature (per Allan's example).



Nope, that's more likely gross noise mitigation via smoothing. Minimum filter can also do that.

Cheers, Marcus

[strongmanmike]

Yes I agree the Minimum filters can indeed produce a similar type effect but the worms and dots I have described are different again. I think there are quite a few different implementations of decon, wavelettes etc across different platforms that can produce different looking effects, hence the confusion?

Mike

[strongmanmike]

That effect in the third shot looks like typical decon artefacts to me Al with the characteristic ringing around bright bits..? As opposed to dots and worms.

Mike

[marc4darkskies]

Hmmm - In that case, why don't you show us exactly what you mean because I'm only being confused by what you're trying to describe. In all my experience, wavelets have never produced worms during my processing - just dots.

In the interest of getting to the bottom of this though, see below. Normal on the left and downright vicious (read ridiculous) wavelet with everything dialled up to 100 on the right.

[codemonkey]

Mike, just show 'em a crop of one of my images and be done with it lol

[gregbradley]

Not entirely on-topic but somewhat relevant. The Fuji X series cameras with their 6 x 6 custom colour matrix (not Bayer) are renowned over the years for showing worms easily in their images. They are most common in foliage shots and show up easily in sharpening.

So perhaps resolution has a part to play in this as well. 6 x 6 would be lower resolution than your Bayer matrix of 2x2 pixels.

Greg.

[alpal]

To be scientific -

we would need to see an untouched pic & then the same pic sharpened.
Mike can't do that unless he used his own data.


cheers
Allan

[strongmanmike]

:scared: well... I couldn't bring myself to use another persons image and say.. do a roll over or anything, just to try and prove a point :rolleyes:...you can post it if you like though :D... but I'm not gunna do it man :shrug: :love2:...besides the effect is pretty muted in the odd image of yours when I maaaay have mentioned something :ashamed: I have seen muuuuch worse, where the image has been clearly interpreted as high res by a few viewers/commenters :shrug: :bashcomp: :lol:

Mike

[Camelopardalis]

Ahhh now we see who takes out the trash in this photo-mance :lol: :lol:

[strongmanmike]

Nice little bit of the Cats Paw there Marcus :thumbsup:...although the second image is ghastly and has worm like structures...it's still not what I am talking about though :)...man, there are certainly many ways to ruin an image though, huh? :lol:

Mike

[marc4darkskies]

Aaaaargh! :help: I tried my best! :bashcomp:

That probably means it's a combination of processing steps and should not be characterised as only decon or wavelet.

Maybe next time you want to critique an image using the term "worm", you should show the author (and the rest of us poor sods) what you're talking about with a small crop of the feature that bugs you? That's a perfectly acceptable thing to do if done in a respectful and constructive way. That would help all of us to understand and provide valuable advice on how to avoid them.

Cheers, Marcus

[codemonkey]

I realise my post might have come across as pissy... I was just stirring the pot because you're notoriously sensitive to them and I'm notorious for delivering images with them.

[alpal]

I agree Mike,
I know exactly what you refer to but I also would not be so rude
as to take a crop out of someone's picture and use
arrows to point to the sharpening worms.


Again - it would be nice if you could find a way to show some of your
own data in a small crop and then show such an artifact by creating it yourself in another modified pic
with whatever means you can such as deconvolution etc.


cheers
Allan

[strongmanmike]

He he yeah, I picked up your tongue in cheek post, it was funny :thumbsup: and I was confident you would take my light hearted response the right way :P...oh and your images are always most excellent...eeeeven when there is the odd hint of worms having slid through :D...you may occasionally run the worm gauntlet, shall we say but overall the result is usually most excellent anyway ;)

Mike