Arun H:
Miguel Morales:
Barry Schellenberg:
I find it fascinating to read this thread and how misunderstood this topic is.
Simply put, this topic is all about math. It's just that simple. It should not be ignored, or your end result will have the consequential flaws. They may not be discernible to some, but trust me and the others who have taken the time to study this topic, the flaws are there.
I would point to Arun's earlier post where he directs us to John Hayes' previous work where he and others have taken the time to study the math and outline it to us in layman terms. If you take the time to understand it, it will help you to appreciate why the calibration process is so important to your processing routine. https://www.astrobin.com/forum/c/astrophotography/deep-sky/lets-discuss-about-dark-bias-dark-flats/?page=1
The way I look at this and I think why the reason the OP has asked this question, is that we have spent so much time, energy and money collecting our data, why wouldn't we want to get the most out of what we collected? Afterall, a pretty picture isn't as pretty if it has mathematical defects within.
My 2 cents.
Yes, some people will disagree, and some other will argue until their heritage clearly shows. But math seems to work for nearly everything, even though nobody can explain exactly why.
I for one like to read what John Hayes has to say. He is the most knowledgeable person that I know of for the subject that we enjoy. If I were to believe anyone, it would be John Hayes, everyone else can go kick rocks.
Miguel 
.
I mean, there is a pretty good reason math works. All this is based on reasonably simple statistics that is covered in many college and even some advanced high school courses. More importantly, things like Poisson statistics and the central limit theorem show excellent agreement with observation. In a separate thread here, John showed how someone took hundreds of darks and the noise profile basically fell exactly in line with the theoretical prediction. What would be surprising is if it didn’t. Our world works on the basis of well understood laws of physics. Learn them, believe them, use them.
Yes, I understand that math works, and others understand all the complicated math that describes the universe and everything in it. But to try to make my point clearer, my question should be more succinctly stated as... why? Why does it work. Why math and not something else? Why is the universe so mathematical and not a jumble of unrelated and unexplainable chaos. That is what nobody really know the answer to.
I hope my explanation clarifies.
Miguel
.
|
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
|
This is very much off off topic but, to answer the previous question, if the universe was just a tad little different from what it is you wouldn't be here to type your question.
|
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
Miguel Morales:
Why does it work. Why math and not something else? Why is the universe so mathematical and not a jumble of unrelated and unexplainable chaos. That is what nobody really know the answer to.
Huh, yes they do....
|
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
Miguel Morales:
And the answer is?
the universe is the way it is because if it wasn’t, you would not be here to ask the question. A chaotic universe with constantly changing governing laws would certainly be inhospitable to sentient life and to astrophotography. For all we know, there could be or may have been or might be an infinity of other universes that are governed by very different rules and where math as we know it does not apply. However, we must understand our universe as it exists, has existed, and will exist in the future. And for this purpose, mathematics and physics seem to work very well.
|
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
The Answer to the Ultimate Question of Life, the Universe, and Everything is…. 42  |
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
Adel Kildeev:
The Answer to the Ultimate Question of Life, the Universe, and Everything is…. 42
Let update it to 42+Constant (where the constant is the number of the calibration frames that one need or/and want to use) 
And the rule of the towel translates to "keep the end of the optical train clean no matter what the constant is" ;)
|
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
20 Darks, 20 Flats, 50 Bias with my ASI183MC
20 Flats, 50 Bias with my ASI533MC
My Astrophotography buddy isn't worried about about really awesome results, so he doesn't do calibrations files at all with his ASI533 MC.
|
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
100 of each.
Built up a library of various exposures for the darks, and always keep dated archives of my flats.
|
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
|
You have to take the brightness of the sky into account. If you have a bright city sky, the sky noise overwhelms any noise in the dark current. An autodark may be good enough. In a pristine sky the shot noise is low so you have to take many more darks to average out the noise in the dark current. So you have to measure the sky noise at your site then do the maths to work out the number of dark frames that you need.
|
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
I and others in my club who use the 2600MC PRO do not use darks. Cuiv the Lazy Geek explains the math: https://youtu.be/0wB5PS3pGxA?feature=shared
Jeff
|
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
Miguel Morales:
Arun H:
Miguel Morales:
Barry Schellenberg:
I find it fascinating to read this thread and how misunderstood this topic is.
Simply put, this topic is all about math. It's just that simple. It should not be ignored, or your end result will have the consequential flaws. They may not be discernible to some, but trust me and the others who have taken the time to study this topic, the flaws are there.
I would point to Arun's earlier post where he directs us to John Hayes' previous work where he and others have taken the time to study the math and outline it to us in layman terms. If you take the time to understand it, it will help you to appreciate why the calibration process is so important to your processing routine. https://www.astrobin.com/forum/c/astrophotography/deep-sky/lets-discuss-about-dark-bias-dark-flats/?page=1
The way I look at this and I think why the reason the OP has asked this question, is that we have spent so much time, energy and money collecting our data, why wouldn't we want to get the most out of what we collected? Afterall, a pretty picture isn't as pretty if it has mathematical defects within.
My 2 cents.
Yes, some people will disagree, and some other will argue until their heritage clearly shows. But math seems to work for nearly everything, even though nobody can explain exactly why.
I for one like to read what John Hayes has to say. He is the most knowledgeable person that I know of for the subject that we enjoy. If I were to believe anyone, it would be John Hayes, everyone else can go kick rocks.
Miguel
.
I mean, there is a pretty good reason math works. All this is based on reasonably simple statistics that is covered in many college and even some advanced high school courses. More importantly, things like Poisson statistics and the central limit theorem show excellent agreement with observation. In a separate thread here, John showed how someone took hundreds of darks and the noise profile basically fell exactly in line with the theoretical prediction. What would be surprising is if it didn’t. Our world works on the basis of well understood laws of physics. Learn them, believe them, use them.
Yes, I understand that math works, and others understand all the complicated math that describes the universe and everything in it. But to try to make my point clearer, my question should be more succinctly stated as... why? Why does it work. Why math and not something else? Why is the universe so mathematical and not a jumble of unrelated and unexplainable chaos. That is what nobody really know the answer to.
I hope my explanation clarifies.
Miguel
.
This is an interesting and difficult question. I think it is important to realize that the Universe runs itself by its own rules, whatever those rules "really" are. We as scientists and mathematicians and philosophers interpret the Universe in our own way, with math, physics and philosophy. But it is we who are doing the math, the calculations, the philosphizing to explain the universe, which doesn't mean the Universe is math, only that we interpret it that way and for the most part that gives us reasonable answers and predictions. And note, we find new math to describe things in the Universe that we could not explain before: The Riemann curvature tensor to explain how gravity curves space, chaos theory to describe things that don't seem to be governed by math, but are in some ways, quantum theory to try to explain the unexplainable in atoms and their constituents, and as far as I know we don't know what yet to describe singularities (string theory, etc..), what is it really like in a black holes centre. So we make the math to fit the universe, but it is our construct, it is just our description.
Why should the Universe require rules? Well, the Universe has to get from one state to another, so it needs to "decide" how to get from there to here. It does use randomness for sure, but still needs to have some rules to evolve. Some have made the analogy that the Universe is like a computer, it computes itself to get from one state to the other. If so, there needs to be an algorithm for the Universe's computation, though it can and does apparently contain randomness as part of it. It seems unlikely we can ever really know what the Universe's full algorithm is, since that would require a computation as large as the Universe itself. But the algorithm is made up of rules, so we can model it the best we can with our calculations.
And as others here and elsewhere have pointed out, we would not be here to ask these wonderful questions, unless the Universe followed to some extent some rules that allow it to evolve to its current state, that involves us. For our Universe to be generated totally at random to get to our current state would be far too unlikely, and if it was random, it would just as quickly dissolve our current state into something unrecognizable.
There is a wonderful book by Martin Rees, "Just Six Numbers" showing that there are just 6 fundamental numbers that enable us to be here as we are now, six numbers that need to arise out of the Big Bang. Now those numbers need to be right, so one thought is there are many or perhaps even infinite universes, many of them non-functional, and we are in that "lucky" one that is functional for us. Or perhaps some day we will find there is some reason these 6 numbers are the way they are, so the Universe had to arise this way? Anyway, it is a very interesting book on our Universe's origins, and reasonably readable.
Rick
|
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
Arun H:
I’m reading some of these posts here and wonder - do people bother to actually learn what calibration frames are and why they are needed? Or just come up with some random protocol and claim, with no basis in fact or evidence, that is how it should be done. For example, claims that flats are not needed. Or darks are only needed if the camera has amp glow. Or one needs 10x the number of darks. If people bother to actually read the thread, there is excellent information with basis in science on what numbers of frames are practically needed. On the other hand, there is some joy to be had, I suppose, in coming up with one’s own concoction.
I agree with you here, there are far far too many differing solution here, so they can’t all work, all calibration frames are done for a specific purpose.
Darks to remove dark current noise and hot and cold pixels, which with some sensors are minimal but still there.
Flats to remove minor vignetting and dust bunnies in the images, which in my opinion can not be fully removed but keeping the filter clean, I have never heard such rubbish, it’s nigh on impossible to have filters in a filter wheel and not have dust on them that show in the images…or use a filter drawer where you are taking them in and out to change and so on…
Flats then need to be calibrated with either a flat dark, which is a ridiculous term in itself, as it’s just a dark at the same exposure time as the flat, but a dark all the same, or a bias frame, either or would work fine, but not both.!!
Now if you don’t dither like me then you need darks for sure, as it will remove any walking noise and hot and cold pixels, if you do dither, then with certain sensors, like the IMX571, you can get by without darks. but there is a proven reason to ALWAYS use calibrated flats…simple…
Cameras with amp glow is another story again, with these cameras darks and flats, well calibrated are a must.
Now some people have said in here, “my images are great with out them, so no need for darks or flats”, they may well be good, but they could be so much better, try and see…
|
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
Ivaylo Stoynov:
AstroShed:
Ivaylo Stoynov:
I don't use calibration frames, only dithering with ASI 2600MM which is cooled in the range -15C to -20C depending on the ambient temperature.
what about flats frames…surely you can’t keep the optical elements perfectly free of dust all the time..
In fact I keep the system clean. It is a permanent setup... Every time when there is a need to open something in the train reducer-wheel-camera am taking care to leave the things clean.
So far this approach is working well for me in last few years  as I don't see any artifacts in my images...
I keep one virgin sample of these (the first one of just opened box, without the handle of course) in a plastic bag. Taking it out only when it is used. Before putting two things together, very gently without any pressure, I am moving only the white part over the surfaces to be sure that there are no dust particles left.
Sorry but never heard such rubbish, and that will not keep filters clean, and even if it does as soon as you are closing the filter wheel or drawer back up. There will be dust all over the filters….
you are the creator of some superb software and am supprised at this post.
|
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
|
Not for the first time in my AP life I have cleaned the sensor (DSLR) squeaky clean only to find dust at the end of the session!
|
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
andrea tasselli:
Not for the first time in my AP life I have cleaned the sensor (DSLR) squeaky clean only to find dust at the end of the session!
Exactly, it’s impossible unless you do it in a special clean room and even then you will get some, it would be great if we could produce stunning images without the hassle of flats, but it’s just not the case….👍🏻
|
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
100 Bias
100 Darks
100 Flats
|
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
AstroShed:
Ivaylo Stoynov:
AstroShed:
Ivaylo Stoynov:
I don't use calibration frames, only dithering with ASI 2600MM which is cooled in the range -15C to -20C depending on the ambient temperature.
what about flats frames…surely you can’t keep the optical elements perfectly free of dust all the time..
In fact I keep the system clean. It is a permanent setup... Every time when there is a need to open something in the train reducer-wheel-camera am taking care to leave the things clean.
So far this approach is working well for me in last few years as I don't see any artifacts in my images...
I keep one virgin sample of these (the first one of just opened box, without the handle of course) in a plastic bag. Taking it out only when it is used. Before putting two things together, very gently without any pressure, I am moving only the white part over the surfaces to be sure that there are no dust particles left.
Sorry but never heard such rubbish, and that will not keep filters clean, and even if it does as soon as you are closing the filter wheel or drawer back up. There will be dust all over the filters….
you are the creator of some superb software and am supprised at this post.
Thank you for your kind words about APT!
I respect your opinion, but this is what I do and never have to take flats because of the dust bunnies. I have treated the wheel by the same way during the filters installation. Every filter is cleaned a second time after closing, moving the wheel position by position - from the camera side followed by camera attaching, position by position from the corrector side. After that the corrector+off-axis+wheel+camera are one piece. As it was said, the setup is permanent which simplifies things.
Maybe it is just luck... Anyway, I still think that keeping the end of optical train clean as much as possible, helps a lot no matter the imaging workflow and this simple trick does its job well |
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
Not sure if it is really possible to avoid flats, Ivaylo. I am sure about bias - you may not need them if you have darks and flats but no flats... Doubtful, my friend! Please do no fight over this issue, today anyway is a Fool Day  |
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
Adel Kildeev:
Not sure if it is really possible to avoid flats, Ivaylo. I am sure about bias - you may not need them if you have darks and flats but no flats... Doubtful, my friend! Please do no fight over this issue, today anyway is a Fool Day
My post was few days before April 1st.
What I say is that in some cases it is possible to avoid flats just because of dust - small sensor (or cropping because of small filters which is my case), good corrector, long focal length, etc. When I can avoid them, I do that as it saves me time for taking the other types of calibration frames which are needed in order the flats to work well.
Definitely, the flats have their undeniable role for correcting the vignetting or/and for photometric calibration |
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
Ivaylo Stoynov:
Adel Kildeev:
Not sure if it is really possible to avoid flats, Ivaylo. I am sure about bias - you may not need them if you have darks and flats but no flats... Doubtful, my friend! Please do no fight over this issue, today anyway is a Fool Day
My post was few days before April 1st.
What I say is that in some cases it is possible to avoid flats just because of dust - small sensor (or cropping because of small filters which is my case), good corrector, long focal length, etc. When I can avoid them, I do that as it saves me time for taking the other types of calibration frames which are needed in order the flats to work well.
Definitely, the flats have their undeniable role for correcting the vignetting or/and for photometric calibration
Flats do more than correct for mechanical vignetting! They also correct for cos^4 radiometric fall off with field angle and for fixed pattern noise (FPN), which is caused by PRNU in the sensor. PRNU is simply variations in responsivity of the pixels across the sensor. It is a multiplicative effect, which is why flats correct it and FPN can be a significant source of spatial "noise" in an image. FPN affects the brighter regions more than the dim regions in an image and it's possible for the spatial variations due to FPN to exceed photon noise.
Sure, you can ignore flat calibration but to do that, you have to be comfortable with the fact that your images won't be as good as they could be. Flat calibration is essential for maximizing image quality.
John
|
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
David Moore:
I have 2600 cameras and don't use calibration frames anymore.
Could you explain why you do that ?
Thanks
|
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
John Hayes:
Ivaylo Stoynov:
Adel Kildeev:
Not sure if it is really possible to avoid flats, Ivaylo. I am sure about bias - you may not need them if you have darks and flats but no flats... Doubtful, my friend! Please do no fight over this issue, today anyway is a Fool Day
My post was few days before April 1st.
What I say is that in some cases it is possible to avoid flats just because of dust - small sensor (or cropping because of small filters which is my case), good corrector, long focal length, etc. When I can avoid them, I do that as it saves me time for taking the other types of calibration frames which are needed in order the flats to work well.
Definitely, the flats have their undeniable role for correcting the vignetting or/and for photometric calibration
Flats do more than correct for mechanical vignetting! They also correct for cos^4 radiometric fall off with field angle and for fixed pattern noise (FPN), which is caused by PRNU in the sensor. PRNU is simply variations in responsivity of the pixels across the sensor. It is a multiplicative effect, which is why flats correct it and FPN can be a significant source of spatial "noise" in an image. FPN affects the brighter regions more than the dim regions in an image and it's possible for the spatial variations due to FPN to exceed photon noise.
Sure, you can ignore flat calibration but to do that, you have to be comfortable with the fact that your images won't be as good as they could be. Flat calibration is essential for maximizing image quality.
John
Many thanks, John! This is the beauty of our hobby - everyday can learn something new |
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
David Moore:
I have 2600 cameras and don't use calibration frames anymore.
Could you explain why you do that ?
Since it's still April 1 in some time zones, let me say this:
I more wish him to explain why he needs to have 2600 cameras. To me, having 5 good cameras is very sufficient.
|
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
Wei-Hao Wang:
I more wish him to explain why he needs to have 2600 cameras. To me, having 5 good cameras is very sufficient.
May be the logic is that if you average light frames from 2600 separate cameras, they are completely uncorrelated, so calibration is unnecessary. A sort of very rich man's version of dithering?
|
You cannot like this item. Reason: "ANONYMOUS".
You cannot remove your like from this item.
Editing a post is only allowed within 24 hours after creating it.
You cannot Like this post because the topic is closed.
1.51
7.56
4.67
11.91
2.41
9.58
0.00
0.90
0.90
6.65
1.81
0.00
22.40
0.00
11.57
