Uneven background [Deep Sky] Processing techniques · Steve Solon and Terry Chatterton · ... · 23 · 612 · 2

shootnmskies20 3.71
...
· 
Hi all,
Does anyone have information or suggestions as to why the background of images have a 'mottled' appearance? To some degree, it can be corrected in Photoshop using Image>Adjustments>Exposure and manipulating the offset and gamma, but this comes at a cost of loss of background data (stars, etc.)
This is a 10-hour composite CCD image of NGC 3198 taken with a QSI 683wsg-8 and an Optolong Luminance filter. All the calibration images have been correctly applied in PI, and has be run through HistogramTransformation

Many thanks for your thinkings.
 - - Steve3198 Lum.jpg
Like
astroswell 0.00
...
· 
·  1 like
Looks more like an artefact of heavy noise reduction, so called "orange peel" effect. Did you use noiseXterminator? To be honest I don't know a definite way to prevent it. You might try diffrent noise reduction techniques, e.g. TGVDenoise process.
Edited ...
Like
shootnmskies20 3.71
...
· 
·  1 like
Maxim:
Looks more like an artefact of heavy noise reduction, so called "orange peel" effect. Did you use noiseXterminator? To be honest I don't know a definite way to prevent it. You might try diffrent noise reduction techniques, e.g. TGVDenoise process.


Thanks Maxim. Yes, I ran through the usual PI workflow after ImageIntegration: DBE, BX, NX, and HT (carefully). I thought about possible over-stretching, but I don't think that's the case - I ran it through twice. I'll give TGV a try.
Edited ...
Like
astroswell 0.00
...
· 
·  1 like
Steve Solon and Terry Chatterton:
Maxim:
Looks more like an artefact of heavy noise reduction, so called "orange peel" effect. Did you use noiseXterminator? To be honest I don't know a definite way to prevent it. You might try diffrent noise reduction techniques, e.g. TGVDenoise process.


Thanks Maxim. Yes, I ran through the usual PI workflow after ImageIntegration: DBE, BX, NX, and HT (carefully). I thought about possible over-stretching, but I don't think that's the case - I ran it through twice. I'll give TGV a try.

Here's an old CN thread, maybe it'll help ;) https://www.cloudynights.com/topic/455608-pi-noise-removal-without-orange-peal/
Lately I get mottled look as well when I'm applying NX and stretching too heavily.
Edited ...
Like
andreatax 7.56
...
· 
·  1 like
Heavy noise reduction and not perfectly flat background. TGVDenoise won't help you here. Take a step back and see what you can do to avoid a bright backdrop.
Like
messierman3000 4.02
...
· 
too much noise reduction
Like
AstroRBA 1.51
...
· 
·  2 likes
Try HT *before* NX
Like
shootnmskies20 3.71
...
· 
andrea tasselli:
Heavy noise reduction and not perfectly flat background.


Thanks Andrea. Pretty sure the Flats are okay, but I have been applying NX at the default settings, so will experiment with that.
Like
shootnmskies20 3.71
...
· 
Try HT *before* NX
Many thanks. I'll experiment, although it's suggested that BX and NX are best applied to Linear images.
Like
shootnmskies20 3.71
...
· 
too much noise reduction

*Thank you. Too much noise reduction seems to be the consensus, so it's back to the PI drawing board!
Like
shootnmskies20 3.71
...
· 
·  4 likes
Hi all,
Well, your suggestions certainly improved things about 93.665 percent - many thanks. After some experimenting with all your ideas, this was my final attempt:
After running the usual Integration and DBE, I applied BX, Correct only (as usual), then BX normal at 70%. Then applied NX at 30%, ran the result through HT carefully, then applied NX again at 50%. This is the result - certainly googols better than before. A little further delving, and I'll be good. Lesson learned: do not be misled by default settings.

Thank you all, again for the wonderful thinks.
 - - Steve3198 Lum Raw DBE BX@70 NX@30 HT NX@50.jpg
Like
rotciv.greene 0.00
...
· 
Id like to add that it appears your image has not been cropped. The entire right edge of the image is a result of the stacking process and is visibly darker than other parts of image. I’d crop in a bit and give it another go. Uneven backgrounds will certainly lead to problems further down the line.
Edited ...
Like
messierman3000 4.02
...
· 
@Steve Solon and Terry Chatterton BTW, how long were your exposures? 5 minutes? 10 minutes?
Like
shootnmskies20 3.71
...
· 
·  1 like
BTW, how long were your exposures? 5 minutes? 10 minutes?


I always shoot 10-minute exposures with the 11".
Like
shootnmskies20 3.71
...
· 
Victor Greene:
Id like to add that it appears your image has not been cropped. The entire right edge of the image is a result of the stacking process and is visibly darker than other parts of image. I’d crop in a bit and give it another go. Uneven backgrounds will certainly lead to problems further down the line.


Hi Victor. Interesting idea. I usually save cropping for the very last process. Are you suggesting that the overlapped areas would pollute the HistogramTransformation readings? Hmm - never thought of that. Ah, always learning. Thank you for the idea. I'll give it a shot.
- - Steve
Like
jrista 8.59
...
· 
·  1 like
Steve Solon and Terry Chatterton:
Hi all,
Does anyone have information or suggestions as to why the background of images have a 'mottled' appearance? To some degree, it can be corrected in Photoshop using Image>Adjustments>Exposure and manipulating the offset and gamma, but this comes at a cost of loss of background data (stars, etc.)
This is a 10-hour composite CCD image of NGC 3198 taken with a QSI 683wsg-8 and an Optolong Luminance filter. All the calibration images have been correctly applied in PI, and has be run through HistogramTransformation

Many thanks for your thinkings.
 - - Steve

As others have mentioned, its a common consequence of overly-aggressive noise reduction. Noise is a part of signal, and whenever you try to obliterate noise, you tend to obliterate signal and other quality factors of an image. Noise can be reduced, but it generally shouldn't be obliterated. 

I also wonder about the periphery artifacts. It doesn't look like you cropped out the stacking artifacts (areas of the image border that have varying SNR due to shifts in the frames, either due to dithering or drift). I would crop out all that edge junk first, then I would try to reduce noise without obliterating it, and see if that not only helps with the mottling, but also helps preserve fine details.
Like
andreatax 7.56
...
· 
·  1 like
Best of all is to use LocalNormalization to produce normalization files and add them to the stacking images. If the frames are properly scattered around the final stacked frames shouldn't have any dark border at all.
Like
shootnmskies20 3.71
...
· 
andrea tasselli:
Best of all is to use LocalNormalization to produce normalization files and add them to the stacking images. If the frames are properly scattered around the final stacked frames shouldn't have any dark border at all.


Thank you, Andrea. A bit confused: Are the Normalization files to be included when Integrating the regular frames, doubling the number of frames to integrate, or are the Normalization files the only ones to Integrate because they are the corrected version of the regular frames? Also, as suggested, I will crop first.
Like
shootnmskies20 3.71
...
· 
Jon Rista:
I also wonder about the periphery artifacts. It doesn't look like you cropped out the stacking artifacts (areas of the image border that have varying SNR due to shifts in the frames, either due to dithering or drift). I would crop out all that edge junk first, then I would try to reduce noise without obliterating it, and see if that not only helps with the mottling, but also helps preserve fine details.


Thanks, Jon. As I mentioned to Victor, I usually do crop as the final step, but it appears that the uneven border edges of an uncropped image can have an overall effect, so I will crop first. Also, LocalNormalization, as Andrea mentioned, is a step I've not used before (one of PI's thousands), so I will give it a go, as well. Thanks again.
- - Steve
Like
andreatax 7.56
...
· 
·  1 like
Steve Solon and Terry Chatterton:
Thank you, Andrea. A bit confused: Are the Normalization files to be included when Integrating the regular frames, doubling the number of frames to integrate, or are the Normalization files the only ones to Integrate because they are the corrected version of the regular frames? Also, as suggested, I will crop first.


LocalNormalization is done after the single frames are registered and before they are stacked with ImageIntegration. In the latter menu, just below the Add Files button there is the button Add L.Norm. Files clicking which will bring a pop-up form to select these files. Usually PI takes care to switch all relevant entries so the final integrated image is properly accounted for re-normalization effects. After you get the final image THEN you might want to crop the black borders, if any, not before.
Like
jrista 8.59
...
· 
·  2 likes
Steve Solon and Terry Chatterton:
Jon Rista:
I also wonder about the periphery artifacts. It doesn't look like you cropped out the stacking artifacts (areas of the image border that have varying SNR due to shifts in the frames, either due to dithering or drift). I would crop out all that edge junk first, then I would try to reduce noise without obliterating it, and see if that not only helps with the mottling, but also helps preserve fine details.


Thanks, Jon. As I mentioned to Victor, I usually do crop as the final step, but it appears that the uneven border edges of an uncropped image can have an overall effect, so I will crop first. Also, LocalNormalization, as Andrea mentioned, is a step I've not used before (one of PI's thousands), so I will give it a go, as well. Thanks again.
- - Steve

The janky regions around the edges can definitely cause problems with various processes. Its best to crop them out as early as possible. I always do it right after integration. That'll improve things like STF even, as well as anything that evaluates things at various scales, etc. ABE can be affected by it, DBE could be if you placed any samples in that area, etc. etc.

Regarding LocalNormalization. That is a process that Juan mentions is a more complex process that should only be applied if it is truly needed. I used to know how to use it, but it was changed rather dramatically in newer versions of PI, and I am no longer sure how to produce useful results from it. I do often have thin cloudcover, and that is the kind of thing that LN is supposed to be able to help normalize out. In the past, it took a fair bit of trial and error. With the new tool, I've not been able to figure out how it works, and every time I've tried to use it, regardless of any adjustments I've made to its settings, I never see any difference with vs. without it after ImageIntegration. So, its not entirely a necessary step, and its primarily intended for when you have inconsistent backgrounds in your frames. It usually requires having at least one frame that is very good, good enough to be the reference frame, so that the variations in the background levels can be identified.

In any case, I think your primary issue is overly aggressive NR. Even with modern tools, that destroys smaller scale nuanced variations in signal level. The mottling is often an intrinsic characteristic of the stack, even...its not necessarily that it was introduced by NR, but more that it was revealed by NR. Without sufficiently aggressive dithering, integrating signals with a poisson/gaussian noise distribution, will often have a bit of that mottling inherent to the signal in the final integration. Dithering needs to be sufficiently aggressive (needs to move the frame enough) each time in order to produce an even noise distribution in the integration. This mottling is also the result of integrating, you won't see it in the individual subs, its an artifact that results from the nature of weakly dithered noise with a poisson or gaussian distribution. I always start with dithering enough that each light frame moves about 10 pixels or more, but you may need even more dithering than that to actually eliminate the mottling. 

If the mottling is indeed an intrinsic characteristic of the signal in your integrated light master, all that your NR is doing, is revealing it. Its not really adding the mottling. Some forms of NR can also add mottling, especially multi-scale forms of NR. At medium scales, some NR routines can introduce medium-scale artifacts, in which case then the NR IS in fact ADDING the mottling. A multi-scale NR tool can be a means of reducing the mottling. Since it usually is a medium-largish scale artifact, you could use something like MMT to target that/those scales, and try to even out the mottling effects. At the same time, reducing but not obliterating noise at finer scales, can help preserve smaller scale structures and help hide the mottling.
Like
shootnmskies20 3.71
...
· 
Hi all,
Wonderful advice from all of you, and certainly a learning session. What seems to work best, so far, is:
Pre-processing
Alignment
Experiment with LocalNorm, if needed
Integration
DynamicCrop
BX - correct only first, then regular application at 60-70%, depending on target
​​​NX at 40-50%
STF and HT
Carefully re-apply NX at 30-40%

Thanks to you all, this will be the workflow going forward, for Luminance, RGB composites, and CMOS with the other camera, adjusting values to taste. Your insights have been much appreciated.
 - - Steve
Like
frustratedphoton 0.00
...
· 
·  1 like
One thing I leaned about NoiseXterminator is that you can use it on linear images but the neural net was trained for non-linear images.

This went against my normal process flow of doing noise reduction before stretching but once I moved NoiseXterminator to non-lenear processing the blob/orange peel background went away.
Like
shootnmskies20 3.71
...
· 
·  1 like
James Hawks:
One thing I leaned about NoiseXterminator is that you can use it on linear images but the neural net was trained for non-linear images.

This went against my normal process flow of doing noise reduction before stretching but once I moved NoiseXterminator to non-lenear processing the blob/orange peel background went away.


Hi James,
Interesting that you mention that, because I don't really see too much difference in applying NX while linear or non. As mentioned above, I've been applying it before HT AND after - the results are pretty satisfying, and yes, as you mentioned, most of the orange peel goes away.
- -  Steve
Like
 
Register or login to create to post a reply.