Blurxterminator… a game changer? [Deep Sky] Processing techniques · Steeve Body · ... · 250 · 16732 · 28

​​​​​​If one likes the tool, awesome use it and have fun. That's what the name of the game is here in AP. If one doesn't like the tool, that's fine. Don't use it. More importantly though, also don't spread incorrect information about it, and don't shame those that do elect to use it.

Tim Hawkes:
At some point BlurXt will generate artifacts -- question is for any given image is that at 0.5, 0.7 or 0.9 etc sharpening and how can you tell?

Wei-Hao Wang:
Based on things I have seen thus far on the internet in the past few days and my own trial of it, I feel it is a good convenient tool in many ways. I probably will run it on every image from now on.  I think the most powerful part of it is the capability to shrink stars and to correct for minor aberration.  The fact that it does this differently in different areas of an image means it can deal with minor focal plane tilt and off-axis aberrations.  These can all be done with traditional methods, but those were highly tedious works.  Now it's basically just one click and a couple minutes of wait.  In some sense, you are upgrading your optics (and your collimation skills) with just a sub-$100 plugin.  For those who uses PI, this is a bargain.  (Sorry to those who don't use PI.  I really hope there is a Photoshop version.)

On the sharpening part, I am less excited than many other people.  (Still excited, but just not that much.).  My criterion is that I don't want deconvolution artifacts.  As a professional astronomer, I have seen enough images from HST and 10-m class telescopes.  I know what real high-resolution, sharp images should look like.  And previous amateur images undergone deconvolution are just not like that. There are either lots of artifacts, or just look unnatural (comparing to real high-resolution images). Because of this, I was never a fan of deconvolution for amateur astrophotography. So, the question I would like to ask for BXT is: when the strength of BXT is tuned down to a point where there are no clear indications of artifacts and the look remains natural, is BXT still better than a standard sharpening tool?  My answer to this question at this moment is: it's better, but not "gamechangingly" better.  I probably would say that BXT can do 20% to 40% better (don't ask me how to define this) than a traditional skillful sharpening.  This 20% to 40% difference alone may not justify the cost (plus the requirement of PI). However, to do that with traditional sharpening, a lot of steps are required and one really has to be careful and "skillful."  Now it's just one click and its 20% to 40% better.  I definitely will not say no to it.

These are my temporary conclusions thus far.  After I try more and see more, my thoughts may change.

Bogdan Borz:
Hi John,

This is impressive of course. I had the same result when applying BlurXT on my NGC7331, maybe even more impressive since my data is shot from the city with a seeing incomparable with that from the Chilean skies (and it took me around 30 seconds).  And while on your previous Top Pick version of the image I can clearly see that you used classical Deconvolution, it is impossible to tell that you applied BlurXT on the current version. 

The appropriate and interesting comparison would be between the BlurXT final sharpened version vs. your final classical processing. Comparing a raw unprocessed linear image with an image sharpened by an AI capable on working on linear data is always going to produce a difference, for all of our images. Agreed, it looks like a final processing, since that is what the AI has been trained to produce. 

My fundamental question is if your BlurXT version is more detailed than your manual version, how do you know that supplementary details that appear are from the information contained in your image and not inferred/produced by the AI model based on its Hubble training? And how can we tell? Especially when you don't have a Hubble image to compare it to. The difficult part would also be how do we analyse, since the difference appears for low scale details, pretty cumbersome to analyse correctly visually. 

In my case, BlurXT produced supplementary details in the galaxy core for my NGC 7331 (I have to look for the full project before presenting you guys the images). It does a better HDR, but since it is linear I risk maybe losing some data while stretching (or maybe it got lost for my processed version too). How do I tell if the new details are real or if they were actually in the information captured by my image?

And if the AI managed to sharpen the galaxy better than I did (or if you think it managed to sharpen it better than you were able to), do we need to replace our usual processing effort with the application of a BlurXT process ?

CS,

Bogdan

Die Launische Diva:
Hello John,

Your result is indeed impressive. Thank you and and all of the expert members who offer their experience in this sensitive but highly intellectually arousing matter.

I would like to hear your opinion about the lack of sharpening on the diffraction spikes. I understand that many astrophotographers are not fond of diffraction spikes but I am not sure if the effect on the diffraction spikes can be said to be the result of a deconvolution algorithm, classical or machine learning. Of course maybe I am nitpicking and/or I have a misconception of what the term deconvolution means.

I think my wetware neural network has already learned to identify BXT images just by looking at the diffraction spikes

John Hayes:
Bogdan,
My current image of NGC 1365 does not use BXT at all.  The example above shows the effect of BXT on my raw Lum stack and it is quite a bit sharper than what I could do with standard deconvolution.  The standard deconvolution method is limited by a number of issues including convergence control and ringing, which limit how much sharpening you can achieve.  In fact, in order to maximize sharpening on small details, it is entirely possible to actually increase the apparent size of the bright stars due to de-ringing control.  

I'm not sure how Russ trained the AI system used in BXT but using Hubble data makes perfect sense.  I'm not an expert on AI but I gained a pretty good understanding of how a neural net works when I wrote a scientific paper on the subject with Gaston Baudat about the AI technology used in his SkyWave product.  The one thing that surprised both of us was just how sensitive a neural net can be to small differences and I think that is one of the factors that makes BXT so effective.  The trick is to start with known data--such as Hubble data and then to apply a mathematically correct blurring function (taking into account both optical aberrations and seeing values) to load a large set of training data (like 400,000 to 1,000,000 images).  This is what enables the NN to pick a best estimate of the original details that created the blurred image within a given image patch.  The algorithm is then applied piecewise over the field to handle field aberration.  (This is called a non-shift invariant imaging system.)  The data has to be properly scaled and normalized to produce properly scaled output so there's a little "trickiness" to getting this all to work properly.  Like all algorithms, this method may produce errors.  Remember that the neural net is simply finding the best estimate of what the original scene looked like in order to produce the original blurred data.  It can't find an exact solution.  Even very small errors in the original data can cause the estimate veer off course.  So the better the input, the better the output.  That's why it's a good idea to closely examine the output to make sure that the sharpened details match features showing in the original image, which is what you should do with ANY sharpening tool.

John

John Hayes:
Bogdan,
My current image of NGC 1365 does not use BXT at all.  The example above shows the effect of BXT on my raw Lum stack and it is quite a bit sharper than what I could do with standard deconvolution.  The standard deconvolution method is limited by a number of issues including convergence control and ringing, which limit how much sharpening you can achieve.  In fact, in order to maximize sharpening on small details, it is entirely possible to actually increase the apparent size of the bright stars due to de-ringing control.  

I'm not sure how Russ trained the AI system used in BXT but using Hubble data makes perfect sense.  I'm not an expert on AI but I gained a pretty good understanding of how a neural net works when I wrote a scientific paper on the subject with Gaston Baudat about the AI technology used in his SkyWave product.  The one thing that surprised both of us was just how sensitive a neural net can be to small differences and I think that is one of the factors that makes BXT so effective.  The trick is to start with known data--such as Hubble data and then to apply a mathematically correct blurring function (taking into account both optical aberrations and seeing values) to load a large set of training data (like 400,000 to 1,000,000 images).  This is what enables the NN to pick a best estimate of the original details that created the blurred image within a given image patch.  The algorithm is then applied piecewise over the field to handle field aberration.  (This is called a non-shift invariant imaging system.)  The data has to be properly scaled and normalized to produce properly scaled output so there's a little "trickiness" to getting this all to work properly.  Like all algorithms, this method may produce errors.  Remember that the neural net is simply finding the best estimate of what the original scene looked like in order to produce the original blurred data.  It can't find an exact solution.  Even very small errors in the original data can cause the estimate veer off course.  So the better the input, the better the output.  That's why it's a good idea to closely examine the output to make sure that the sharpened details match features showing in the original image, which is what you should do with ANY sharpening tool.

John

Thanks John.  Yes, I know your current/published image did not use Bxt, but it seems to me you used the classical Deconvolution in PI on it, I was referring to that one. Correct me if I'm wrong. Hm, if I read you correctly, you think the Hubble image data set was blurred in order to train the NN? I was thinking this at first, but from the documentation and his very laconic replies, the Hubble was considered the "true deconvoluted" version of the ground image and the ground blurred images had to be deconvoluted by the NN in order to minimize the loss function compared to the final Hubble image. Of course, Russell won't reveal what kind of validation set or generalization set did he use, neither what was the accuracy of his model. Image recognition is increasingly applied in medicine (from what I've seen in the literature there seemed to be an interest in the mid 90's and now in the 5-10y), but the difference is that in medicine we usually use labels as outputs, or let's say continuous variables like age. I consider this a much more intelligible classification of "true result" compared to a "true deconvolved image" without unreal artifacts. If the neural network interprets your ECG as "arythmia" and you're in normal rhythm, it's false. Pretty clear. Some teams trained them to recognize implanted pacemakers or defibrillator brands based on your chest radiograph image, device identification. Well, they have an accuracy of 99% (for other tasks is much lower). But the outcome is the model and brand name. Much more simple to comprehend compared to what does it mean to say the resulting image is accurate, how much should it deviate from reality to be considered true or the "good outcome" of the model? Especially when it has so many structures displayed. 

I'm not sure that the training on 'deconvolved' Hubble images as reference is appropriate, since the reference has much higher resolution than the blurred system. But Russell thinks that this is actually an advantage: " [...] trained using extremely high-resolution images acquired by instruments such as the Hubble and James Webb space telescopes. It "understands" what astronomical structures actually look like at finer scales than can be resolved using amateur equipment."  A perfectly deconvolved C11 is a C11 without atmospheric interference or minimal, not Hubble, even downscaled. We don't have much choice (perhaps training blurred systems on images taken under the best planetary skies), but the NN seems to have been trained to produce outcomes above the capacity of the system. 

I totally subscribe to the principle of checking for matched features in the original, but my difficulty especially with this tool is that it sharpens details that I cannot see in the original. Not to talk if it is the raw image, the jump in detail is huge. I compared my NGC 7331 to Hubble and Bxt is clearly good in keeping the main details. The difference will be at very very high zoom and minute details and filaments, honestly really tough to discriminate.  If the AI does a good enough job tough, it will render our sharpening processing effort pretty superfluous.

Bill McLaughlin:
Comparison on a small galaxy with the same data. BX and NX used instead of LR and MMT. Also spectro color instead of standard so better color. Clearly Fewer artifacts.

Bogdan Borz:
Thanks John.  Yes, I know your current/published image did not use Bxt, but it seems to me you used the classical Deconvolution in PI on it, I was referring to that one. Correct me if I'm wrong. Hm, if I read you correctly, you think the Hubble image data set was blurred in order to train the NN? I was thinking this at first, but from the documentation and his very laconic replies, the Hubble was considered the "true deconvoluted" version of the ground image and the ground blurred images had to be deconvoluted by the NN in order to minimize the loss function compared to the final Hubble image. Of course, Russell won't reveal what kind of validation set or generalization set did he use, neither what was the accuracy of his model. Image recognition is increasingly applied in medicine (from what I've seen in the literature there seemed to be an interest in the mid 90's and now in the 5-10y), but the difference is that in medicine we usually use labels as outputs, or let's say continuous variables like age. I consider this a much more intelligible classification of "true result" compared to a "true deconvolved image" without unreal artifacts. If the neural network interprets your ECG as "arythmia" and you're in normal rhythm, it's false. Pretty clear. Some teams trained them to recognize implanted pacemakers or defibrillator brands based on your chest radiograph image, device identification. Well, they have an accuracy of 99% (for other tasks is much lower). But the outcome is the model and brand name. Much more simple to comprehend compared to what does it mean to say the resulting image is accurate, how much should it deviate from reality to be considered true or the "good outcome" of the model? Especially when it has so many structures displayed. 

I'm not sure that the training on 'deconvolved' Hubble images as reference is appropriate, since the reference has much higher resolution than the blurred system. But Russell thinks that this is actually an advantage: " [...] trained using extremely high-resolution images acquired by instruments such as the Hubble and James Webb space telescopes. It "understands" what astronomical structures actually look like at finer scales than can be resolved using amateur equipment."  A perfectly deconvolved C11 is a C11 without atmospheric interference or minimal, not Hubble, even downscaled. We don't have much choice (perhaps training blurred systems on images taken under the best planetary skies), but the NN seems to have been trained to produce outcomes above the capacity of the system. 

I totally subscribe to the principle of checking for matched features in the original, but my difficulty especially with this tool is that it sharpens details that I cannot see in the original. Not to talk if it is the raw image, the jump in detail is huge. I compared my NGC 7331 to Hubble and Bxt is clearly good in keeping the main details. The difference will be at very very high zoom and minute details and filaments, honestly really tough to discriminate.  If the AI does a good enough job tough, it will render our sharpening processing effort pretty superfluous.

John Hayes:
In order to train the neural-net (NN), you need to start with "ground truth" of what a sharp image looks like and that's why it makes sense to start with Hubble or JWST data. That's not "prefect" data but as far as small, ground based imaging systems are concerned it's close enough. The blurring that I referred to is required to train the NN so that it can find the nearest match in your input image to a blurred image in the training data. Once it finds that match, it can then replace the original data with a normalized version of the sharp (ground truth) data. To be clear, the algorithm does not actually contain Hubble data but it's set up to analyze the small variations in input data so that it can provide a "best guess" about the sharp features that would create the same blurry pattern in your data--all based on the training data set.

Tim Hawkes:

Wei-Hao Wang:
Based on things I have seen thus far on the internet in the past few days and my own trial of it, I feel it is a good convenient tool in many ways. I probably will run it on every image from now on.  I think the most powerful part of it is the capability to shrink stars and to correct for minor aberration.  The fact that it does this differently in different areas of an image means it can deal with minor focal plane tilt and off-axis aberrations.  These can all be done with traditional methods, but those were highly tedious works.  Now it's basically just one click and a couple minutes of wait.  In some sense, you are upgrading your optics (and your collimation skills) with just a sub-$100 plugin.  For those who uses PI, this is a bargain.  (Sorry to those who don't use PI.  I really hope there is a Photoshop version.)

On the sharpening part, I am less excited than many other people.  (Still excited, but just not that much.).  My criterion is that I don't want deconvolution artifacts.  As a professional astronomer, I have seen enough images from HST and 10-m class telescopes.  I know what real high-resolution, sharp images should look like.  And previous amateur images undergone deconvolution are just not like that. There are either lots of artifacts, or just look unnatural (comparing to real high-resolution images). Because of this, I was never a fan of deconvolution for amateur astrophotography. So, the question I would like to ask for BXT is: when the strength of BXT is tuned down to a point where there are no clear indications of artifacts and the look remains natural, is BXT still better than a standard sharpening tool?  My answer to this question at this moment is: it's better, but not "gamechangingly" better.  I probably would say that BXT can do 20% to 40% better (don't ask me how to define this) than a traditional skillful sharpening.  This 20% to 40% difference alone may not justify the cost (plus the requirement of PI). However, to do that with traditional sharpening, a lot of steps are required and one really has to be careful and "skillful."  Now it's just one click and its 20% to 40% better.  I definitely will not say no to it.

These are my temporary conclusions thus far.  After I try more and see more, my thoughts may change.

That is a very interesting point that you raise in the first paragraph.   As long as the geometric errors across an image are consistent for all the frames integrated together (e.g some tilt) then maybe software can compensate for slight alignment errors etc and correct accurately enough wrt star shape.

Similarly with deconvolution to recover image sharpening?  If the PSF for can be calculated accurately enough across the entire image accounting for all the local tilt distortions etc  ---and most importantly is consistent enough - then will it be possible to iterate to (almost)  just as sharp an image from a slightly misaligned telescope as from a perfectly optically aligned telescope?

Die Launische Diva:
Shouldn't the diffraction limit set a lower limit in the resolution uncertainty of the recovered image?