NosieXterminator and Autostretch PixInsight Addicts · apo20232 · ... · 21 · 543 · 13

apo20232 0.00
...
· 
·  2 likes
Hi,

My trial version of NoiseXterminator is about to expire and I have a question before purchasing a permanent license. I'm really happy with how well it works, but I think I am using it incorrectly. I have been using it on OSC images early in my workflow (based on Croman's website recommendation) right after dynamic background extraction. However, once I perform the operation--using 24-bit lookup tables--and reset the screen transfer function and then reapply the STF, I get all sorts of color blotches all over the image. Am I not supposed to use autostretches after applying NoiseXterminator? Thanks for any help you can provide on what I hope is an easy question to answer.

Jim
Like
julianr 0.00
...
· 
·  2 likes
I think it is stretching it harder as the noise is lower, same thing happens when you do linear noise reduction in PI
Like
menardre
...
· 
·  1 like
I also am trying out NoiseExtermniator. I like the way it reduces noise, but the sample images that I have done it has impacted some of the stars. Some of the stars have developed a 'ring'.

I am not sure how to correct this since there are only 2 settings on the tool. Also..not sure how to setup for 24 bit lookup table.

Roger
Like
apo20232 0.00
...
· 
·  1 like
Roger,

I have not had any problems with "star rings".  24-bit lookup table is activated by clicking the "24" icon on the upper right of the ribbon near the STF reset buttons.

Jim
Like
teoria_del_big_bang 0.00
...
· 
I too am still on the trial and had some success and also some of the issues mentioned above.
I do not really do OSC but have been helping a few people lately with their OSC data and I seem to think it does work better with mono images than colour but with some careful settings does work well on colour as well.
Obviously if linear data do not forget to tick the linear checkbox and if wanting to both denoise and sharpen then I tend to first set the detail slider on zero, or close to zero and determine the amount of noise reduction I want to commit to, maybe on a preview to speed things up, then start moving the detail slider when happy with amount of denoise and if I get dark rings around stars (most noticeable on the brighter stars) then drop the detail slider back a bit as that's your limit.

Also like any denoise do not try to do too much denoising, all you are doing is blurring everything and the idea is not to make the background pixels all one colour.

Like I say I am still on trial, but probably will buy, especially with the reduced price if you already have StarXTerminator, so I cannot say this is true for all data as all images can act differently with noise reduction anyway so its never a one size fits all regarding what parameters to use.

Don't forget the discount if you already have StarXTerminator or other RC Astro product.
image.png
Edited ...
Like
talbotj 2.41
...
· 
·  3 likes
As many know there are many ways to do noise reduction in PixInsight.  NoiseXterminator is another wonderful tool and probably the easiest to use.  Since it uses a neural network that was trained on many types of images, its only as good as the images that it was trained on.  If you find an image that it does not work well on consider sending Russ Croman the image in xisf format.  He may add it to the next training run.  I talked to him last week at AIC and he was asking for those types of images. 

That said, the auto screen stretch stretches the image based on the statistics.  After you have run any type  of noise reduction the noise statistics change which is why "auto" gives a weird and sometimes aggressively nasty result.  I've find that its easiest to just use the screen transfer function and screen stretch your image to a point which you like it manually.  Auto gives a start then adjust from there.  You can also bring up the histogram transfer and move your screen stretch to the histogram tool and see what it looks like.  If you've clipped too many pixels the histogram tool will show you.  You can make an adjustment there and transfer that back to the screen transfer function tool by dragging the blue arrow down to the bottom of the tool.  Once you get the image looking like you want run noise exterminator.  As others have mentioned you don't want a totally smooth background.  Leave just a bit of graininess to it as it make the overall image look a bit sharper and better.  If you find noisexterminator knocks out some detail in an area of interest you can also always create a mask using something like the "Game" script and put the mask over the area to limit how much noise reduction takes place.  A simple luminance mask works well too, just remember to invert it.

Jon
Like
apo20232 0.00
...
· 
·  1 like
Jon,

You have several good suggestions that I will try, but as a relative newcomer to PixInsight it will take a few days to sort everything out .

Thanks!

Jim
Like
jerryyyyy 9.03
...
· 
Hi,

My trial version of NoiseXterminator is about to expire and I have a question before purchasing a permanent license. I'm really happy with how well it works, but I think I am using it incorrectly. I have been using it on OSC images early in my workflow (based on Croman's website recommendation) right after dynamic background extraction. However, once I perform the operation--using 24-bit lookup tables--and reset the screen transfer function and then reapply the STF, I get all sorts of color blotches all over the image. Am I not supposed to use autostretches after applying NoiseXterminator? Thanks for any help you can provide on what I hope is an easy question to answer.

Jim

I purchased the license.  Here is the STF trick I use.  Before I do the noise extermination on the linear image I get the STF reading.  After the extermination, I apply the STF taken BEFORE extermination to the exterminated image.  Seems to stretch correctly.... YMMV.

This is tricky as I have not developed the optimal workflow but I am in hook line and sinker to this ecosystem, even after buying $$$Topaz. 

I am now just doing a NB RGB image and in the RGB I am first stretching the image with ArcSin stretch, then I am getting the stars with StarX, then I apply NoiseX to the stars... again YMMV... I also try using Dust and Scraches at 1 or 2 on the stars to get rid of the fuzz... but it is minimal. 

image.png

Interesting, that when I denoise I get a warning that it thinks the ArcSin stretch image is linear... does nothing on the small scale crud..

Photoshop:

image.png

image.png
Hey, am sticking with PS on this noise reduction problem...

I'll have Sh-2 112 out soon... tune in for the NB_RGB image....
Like
talbotj 2.41
...
· 
(Here is the STF trick I use.  Before I do the noise extermination on the linear image I get the STF reading.  After the extermination, I apply the STF taken BEFORE extermination to the exterminated image.  Seems to stretch correctly.)

Jerry, if you manually adjust the STF, either after doing a auto STF or just doing it manually and run NoiseXterminator or any other pix tool, the STF doesn't just change unless you mess with it.  The noise statistics change.

The reason you are having difficulty getting the noise out of the stars using NoiseXterminator is that it was trained to go after the noise and leave the stars alone.  Usually stars are high S/N areas and you dont want them touched.  This is the big difference between Topaz and NoiseXTerminator.
Your better off just making a star mask and using ACDNR or other noise tool just on the stars if you want to.

Jon
Like
jerryyyyy 9.03
...
· 
Yes, I think the noise stats change after NoiseX, but the stretch(parameters)  should be the same.  The other writer was I think referring to so big changes if you try to run the Auto STF again after noise reduction.  I have had problems with that too.  I have recently run across Generalized Hyperbolic Stretch, which seems pretty powerful for the initial stretch but has a substantial learning curve.  I have never had much luck manually adjusting STF either.  GHS I sorta understand the theory at least. 

I see you point on running NoiseX on stars.  That is I am afraid where PS Dust and Scratches is coming in for me. The stars generated by StarX seem better than any other alternative despite the small speckles, which Rob Pfile thinks are a masking artifact.  I was unfamiliar with ACDNR but it seems to provide a similar clearing effect with small structure size. 

image.png
Like
menardre
...
· 
I am using the trial version of NoiseXterminator.

I have tried it on three different images that I previously processed and all three showed some signs of star 'ringing'. 

I start with an image that has gone through the following steps:
Flat Calibration
Flat Integration
Image Calibration
Debayer
Star Alignment
Local Normalization
Image Integration
DBE

I then invoke NTW.  I am not use what everyone is talking about relative to using STF. When I run NXT it does a great job of removing noise, but also adds some 'ringing' to some of the brighter stars.

I am doing something wrong in my process??

I currently use the EzDenoise script and it works well.
Roger
Like
apo20232 0.00
...
· 
@jerryyyyy , @Jon Talbot ,

Thanks for the tips. I got the STF readings from the pre-noise reduction image and then applied those readings to the image after NoiseXterminator and the stretch worked correctly.

@Roger Menard I use the same process and have not had any problems so far.

Thanks all for the quick responses,

Jim
Like
talbotj 2.41
...
· 
Roger,
Make sure you are not sharpening the image.  That is what typically creates the rings around stars.   If you are not adding any sharpening, then you can create a star mask first from the luminance part of your image, or run Starnet or starxterminator and output the stars. If the output is RGB stars, extract the luminance from this and stretch it to show decent looking stars.  Apply this mask to your image, invert it, and run NXT.

Jon
Like
menardre
...
· 
I am still having problems with NXT. It does a great job of removing noise, but creates artifacts around stars.
Here is an image of M85. It has been through the processes:
Flat cal, FLat integration
Image Cal, Image Debayer, SubframeSelector, Local Normalization, Star Alignment, Image Integration.
While still in linear mode I do DBE, then use NXT. I then went directly to ArcSinH stretch (I normally would od a PCC but I am just checking out noise so I skipped this step)

I then cropped the image to clearly show  the stars..  I previously processed the same image with the same steps using EzDenoise and did not get the star artifacts.

Is there an adjustment to NXT that I should be trying. I currently use both defaults. Clearly the NXT does a great job with noise... but does impact the stars.

M85 using NXT1.jpgM85 using Ez Denoise2.jpg
Like
janvalphotography 4.36
...
· 
·  1 like
Generally what I've seen during my own processing, that may contradict RC's instructions, is that doing NoiseXterminator before stretching does some horrible things to my images. At least on the color data, the luminance doesn't seem to matter that much. So I've resolved to doing it right after stretching the color data (before adding stars etc). 

I don't have any examples now but although STF is pretty agressive it does show what's really in the image, in my case I saw a lot of blotchy color patches all of a sudden if I applied the noise removal pre-stretching. After stretching it's been a lot more forgiving for me.

Sometimes I will run NoiseXterminator several times on the lum channel using various luminance masks to target specific noise, on the color data I just run it without masks - then I might do a final run using very modest values before I finish my image.


Edit: I only started noticing this issue around new years, before that I didn't see it happening the same way.
Edited ...
Like
cyendrey@gmail.com 6.15
...
· 
·  2 likes
I had some bad issues with SXT and a particular image and had an extensive email discussion with Russ Croman.  NXT should not be applied early in process, instead late if not the last thing done.

That is because it has a significant effect on the PSF of the stars/image which in turn effects the AI in BXT and SXT (and not in good ways).  The effects are exacerbated in auto stretch, particularly in dense star fields.

Russ is very reluctant to make any recommendations to anyone on workflow/workflow order since that is a topic that is a religion to some imagers.

However, I went from the early often camp on NXT to late/last and have had consistently good results, much better and consistently that I was having previously.  SPCC is another process that is intensely affected by anything that changes the PSF of the image, to the point that NXT can cause SPCC to fail in its plate solving attempt.  Where SXT is concerned, the effects on PSF from NXT can degrade SXT's ability to identify star vs other objects like galaxies.

BXT does work with SXT, so the order between the two isn't super critical, but I've been applying BXT prior to SXT with good results.

Another process that can negatively affect SPCC and SXT is drizzle in WBPP.  According to Russ, the 1x setting is manageable, but 2x or higher can degrade the performance of both.  I conducted several test WBPP/work flow process/order tests in the course of our discussion and gave Russ the results.  

My updated workflow after my exchange of information/images/discussion with Russ:
DBE - Color Correction/SPCC - BXT - SXT - the work flow splits here for the Stars images vs starless that result from application of SXT:
Stars - Histogram stretch to get the desired intensity, then Color Sat - CorrectMagentaStars script - optional SCNR.

Starless - STF (my tool of choice has become Bill Blanshards PixelMath stretching scripts) - the whatever the image requires in the way of masks, CRV tool - etc.  The last or next to last step is application of NXT.  

Then test runs of recombining the Stars + Starless frames, etc to get the final image.

Another reason for waiting until late/last in the processing to apply NXT is that almost every post processing step after STF has some effect on visual noise/graininess in the image - not in a good way.  Applying NXT at the end smooths that graininess/noise away.  

For context, my primary goal was always to get a clean, high quality Stars and Starless image separation.  I always process my Star field separately from that of the main target, even with images like M31 or M101.  Stretching and tweaking the stars image is quick and simple while the image of the nebula, etc can require a lot of steps that would adversly affect the quality of the star field part of the image.

Early application of NXT (excluding the other issues it creates) will mandate multiple applications of NXT to deal with the process generated noise in the later stages.  I've demonstrated to my satisfaction, that multiple applications can introduce artifacts around stars and nebula detail just as it does with other noise mitigation processes, including Topaz noise reduction outside PI.

 YMMV.

CS
Clayton
Edited ...
Like
janvalphotography 4.36
...
· 
·  1 like
That's sounds about right and is more or less the way I do things as well.

However I'm not sure I agree that multiple applications necessarily is a bad thing. It all depends on how you do it. If you apply NoiseXterminator to the whole image then I'm sure it would have negative effects, but the entire idea of doing several applications is highly dependant on masking out the noise to keep the nebula/details protected. I mostly do this on the luminance, then a general application on my color data, by comparing a masked Noise reduction process against a single application to the entire image - The masked option keeps more detail sharp IMO. Of course it comes down to the amount you apply. The last application for me at the very end is just about 10-20% or so across the entire image with no masks.

But I am intrigued to do a comparison between the methods by doing no noise reduction at all until the very end, against my normal process to see how that looks. I don't think I've tried that.
Like
cyendrey@gmail.com 6.15
...
· 
·  1 like
Jan,
I need to qualify my comment above very slightly.

Since June of last year my imaging has been solely monochrome (LRGB or narrowband).  Since going to monochrome image noise has gone from a significant issue (in my OSC imaging) to an almost negligible issue with the primary source being post processing induced noise.  Which, aside from the superior image detail, is another major reason why I won't go back to OSC imaging.

Which points out a step I didn't put in.  For monochrome, there is a channel combination between DBE and SPCC/ColorCal.

IMO - YMMV
Edited ...
Like
jerryyyyy 9.03
...
· 
·  1 like
I think this is a very important discussion about the sequencing of application of these AI tools.  The one missing here is BlurXterminator.  I get a lot of mileage out of that.  That is the first applied on the linear image in my workflow since I understand the AI is tuned/trained for that type of raw image.  Despite the concerns of the GHS guys, I then apply NoiseX.  If you watch the histogram in GHS you will see what these things do to noise... histogram gets tighter.  The GHS guys concern is that you clip out the detail in the darker areas... I switch back and forth in Pi to watch for this for this... but my big problem is light pollution so my subtle darks have already been lost... your millage may vary... they are there in my H-alpha images.  

Oh, and the other thing to watch in your linear images is FWHM Eccentricity (the script) follow FWHM and eccentricity before and after BlurX and star numbers after NoiseX...

This is an example of this processing working on some file H-alpha detail... I still need more subs...


Sh2 186 Sharpless Unusual Radio Source and IR Star Cluster


Just realized that Sh2 90 is also in progress and here are the three steps... Original Ha... to BlurX... to NoiseX (Top is one application of BlurX... did not realize but was applied twice in second row)

image.png
Edited ...
Like
cyendrey@gmail.com 6.15
...
· 
·  1 like
I have BXT (BlurXterminator) in my flow.  BXT does some pretty significant (usually good) things to images and it doesn't conflict/create insurmountable issues with the performance of SXT that I can tell.  According the the interviews and discussion with Russ Croman, BXT's AI "looks" at the entirity of the image so works best when channel combination has been completed.  ColorCalibration doesn't bother it either.

The auto function works well, but it works best when you determine the average PSF from the image in an area devoid of nebula/other objects.  Russ demonstrates this using the PSF function in PI while manually selecting stars.  However, VisibleDark demonstrated in his video a (for me) better/faster method.  
1) Create a preview pane of a small subset of the image, clear of nebula etc that has a fairly good star population but not any abnormally bright ones
2) Make sure that the preview is selected/has the focus in your image pane
3) Script/Render/PSFImage (it auto selects the image with the focus), execute with the defaults; its display will give you two FWHM values (x and y).  Split the difference and enter that value in the manual PSF value in BlurXterminator.
4) Select "Correct First" in the BXT options and execute.

You can use the BXT preview or create a preview image to apply it to while tweaking the other settings.  I tried that a few times and there may be some particular objects/images that need that increased tweaking, but I haven't found that it produces a result better than the defaults, so that is where I've left mine set other than the PSF input.

One of the reasons this works better in tuning for your image is that the PSFImage script is restricted to a specific clear/representative area without nebula features or super bright stars.  On the Auto setting, BXT's AI is looking at the entire image and trying to identify/exclude the things that would skew the result, but it will never be as 'focused' as result as human intervention can provide.  Russ Croman discusses the details of BXT and ways to get the best out of it in a video on the AstroImagingChannel YouTube channel.

He does point out that NXT changes the PSF of an image enough that BXT's AI maybe reduced to a simple sharpening routine because it cannot sufficiently differentiate stars for other objects.  Which is the same issue that SXT has.

I have found that with BXT, I really don't need drizzle to reduce star pixelization, nor do very minor tracking artifacts (stars very slightly oblong) persist into the final image.  I believe that Russ mentions this in one of the interviews/discussions on YouTube he participated in.
Like
jerryyyyy 9.03
...
· 
I think we are on the same page.  I understand using the manual PSFs but lazy me has not been able to discern a difference.  The effects on FWHM are substantial.  These are off an L image but HA is the same, original followed by Blur followed by noise... watch the FWHM and then the Stars

I had to tone back the settings to 12% on the Blur on stars or they get too small... think this effect is so profound it overwhelms any PSF issues.. but I am basically lazy....

image.png
image.png
image.png
Like
jerryyyyy 9.03
...
· 
If any of you are still following these issues made some comments in this recent image:


Sh2 90
Like
 
Register or login to create to post a reply.