22.40
#...
·
·
3
likes
|
|---|
Linwood Ferguson: I've heard this argument before and I don't agree with it. Yes, calibration adds noise; however, stacking increases the SNR, which is what you really care about. A number of years ago I computed the effect of calibration on the noise contribution for stacked data and I've attached that slide. It shows how noise is increased for the size of the stack in both lights and in the calibration master. Remember that if a CMOS camera has very low dark noise (note: I said noise!) to start with, then subtracting the dark signal will add very little additional noise to the stacked result. Just because a sensor has low noise does not mean that it is pointless to remove the dark signal! To demonstrate that, I've also attached a screen shot showing how important it is to remove the dark signal even with your ASI16200MM Pro. This data is from my QHY600M-PH, which uses the same Sony sensor. All those white dots on the left are "warm" pixels. Those are pixels with higher than average dark current and they are common in all cameras. That is the dark signal that you want to remove when you calibrate your data--as shown on the right. By the way, the noise is much larger for those warm pixels but that's a very small price to pay to remove them. You'll never see any adverse effect in the stacked final image--particularly if you dither. John   |
12.30
#...
·
|
|---|
John Hayes: John: Thank you for correcting my poor choice of words. I completely agree with your correction and you were right to call me on it. If you notice, I do refer to bias "signal" in other parts of the post, possibly raising the confusion even more... Thank you again. And I never meant to suggest that the duration of the flat should match the duration of the light frame. Only that the duration of the master flat dark should match the duration of the flat and that the duration of the master dark should match the duration of the light subs. I think someone else has already pointed that out, but I just wanted to make sure that was clear. Regarding bias signal, it absolutely does vary based on conditions, and it varies differently in different cameras. As an example, look at the QHY600M that you have. The purpose of the Overscan Area on that camera is so that you can account for the bias shift that occurs when taking a bright image vs a dark image. There are ways to handle these changes in bias signal. Taking flat-darks (which was the question of the original poster) is one of them. I don't think that you and I disagree on the use of flat-darks, so I don't think we disagree at all, really. Again, thank you for pointing out my poor choice of wording. Best. Tim. |
|
22.40
#...
·
|
|---|
|
Tim, I completely agree that bias may vary between cameras; however, let's confine the discussion to how a single camera works. I'm not sure how you can tell that the bias signal varies with conditions. The overscan region is merely a region of the sensor that is masked so that light can't reach it. It contains bias offset, bias noise, as well as dark signal (and noise) along with read noise. It can be useful for calibration procedures but you have to be careful how you use it. You can indeed measure the bias offset and noise using the overscan region but only by using VERY short exposures--or maybe by using very deep cooling to reduce dark signal to a level below the bias level. Again, sensor calibration isn't going to work well if the characteristics of the sensor are non-linear or full of hysteresis effects. CMOS sensors are not immune to RBI and that can cause variations in the dark signal over time. Amp glow, which is due to RBI, is low (but probably not exactly zero) in these cameras and since they don't have a shutter, exposure to a bright light when the camera is cooled may affect the way a sub calibrates. Cooling the camera and taking flats before taking subs would be a very bad procedure. Since a lot of these CMOS sensors are back-illuminated (like the IMX455), I suspect that RBI effects are reduced--but again probably not to zero. I suspect that back-illumination may reduce the time constant for current flow from the bulk material considerably so that it disappears very quickly. Ultimately, the best way to measure the characteristics of these cameras is to run a full photon response curve. John |
0.00
#...
·
|
|---|
| One of the major issues of Flat Darks is that optimal exposure (50% Full Well) with a fixed light source will vary in duration from say a couple of seconds for L, a bit more for RGB and a lot more for narrowband. With Dusk/Dawn flats you may even have different exposures for the same filter. A software variable light panel and clever software that allows you to fix an exposure and adjust the panel brightness helps to an extent but you will still have to build a library of Dark Flats for different duration flats. For LRGB a short duration dark flat approximates well enough to a Bias. For Narrowband CCD imaging a 5 minute dark flat scales well enough but the “right” but “practical” way to do it with CMOS eludes me. CMOS darks do not seem to scale well. |
11.01
#...
·
·
1
like
|
|---|
|
I've ended up doing flats (using an illuminated tracing board set on its lowest brightness level) as follows: Lum: .03s Blue: .06s Green: .08s Red: 0.2s Ha: 5s OIII: 1.2s SII: 8s I guess this reflects the blue bias of the LEDs in the panel. I chose the exposures based on achieving the best balance of DN for each filter. Has anyone else ended up doing something similar or do you have a more elegant solution? i.e. is my use of a cheap LED tracing board fundamentally flawed? |
11.91
#...
·
·
4
likes
|
|---|
| I use a cheap LED tracing board all the time. I aim for a 4 second exposure and 20000-30000 ADU. The trick is to use sheets of white paper to dim the panel brightness adequately. |
|
12.30
#...
·
|
|---|
John Hayes: Hi John: What I was trying to say was that the WAY that bias CAN vary can different between different sensors. Not that, if for example, you looked at 2 different ASI1600's the bias signal would be different. That is true, but not what I meant. As far as the QHY600M is concerned, the Sony IMX455 bias signal will vary if you take an image that is bright (like a flat frame) as opposed to taking an image that is dark (like a light sub). This is not anything that I measured or verified empirically, rather this came to me directly from QHY. This quote is from the email that I received from Mr. Cha: "The on-chip calibration part of the cmos sensor will cause the drift of the whole image when the image is bright." Now, short of the other recent CMOS cameras out there, I have never heard of this behavior before. This is what I meant by saying that the way in which bias signal can vary can be different between different cameras/sensors. In any event... It is immaterial with respect to the original post, which was about should one take flat darks to calibrate flats. My contention remains that, for CMOS sensors, that is the best way to calibrate. BTW - So as not to take things out of context, here is the entire email from Mr. Cha if you are interested... The method for correcting this is slightly different than what he described when using PixInsight. I'm happy to describe how I use it if you are interested. I appreciate the discussion and hope that you are well. Tim. Email from Mr. Cha: Hi Tim |
|
22.40
#...
·
|
|---|
Tim Hutchison: Tim, Thanks for posting the note. I haven't discussed this with Cha so there are some things that I don't understand about his comments. First, why is he referring to "drift of light frame"? Accurate and repeatable calibration goes out the window if the light frame "drifts"--and I don't even know what that means. It is presented as something additive and perhaps he's talking about an offset to the dark level; but that still makes no sense for any given exposure and temperature. Second, when he talks about drift of the flat frame, he may be referring to an additive RBI term. Exposing to a bright signal will produce RBI and that will raise the level of the dark signal. The normal way to handle that is to simply expose the camera to the flat panel light, turn it off (or cover the camera) and then take the dark exposure. Then average N of those frames to compute the master dark-flat frame. That will take care of any offset generated by the flat panel due to RBI. If that's not what he's talking about then I need a better explanation of what "drift of flat frame" means. As I've said, having the responsivity, linearity, dark current, or bias levels "drifting around" is a very bad thing. John |
|
7.46
#...
·
|
|---|
This discussion piqued my interested to see where the argument for variable bias has any validity for the sensors I'm using. So, easy to test, I went for the Nikon D5100 (uses an Sony CMOS APS-C sensor with no amp-glow visible). Took 50 frames for each of the following expsoure settings (in the dark, obviously): 1/4000s, 1/400s, 1/40s, 1/4s and 4s, al at ISO200. Carried out the integration in PI 1.8.8-11 with linear fit clipping rejection. After that I used the SuperBais process to compute the superbias of each integration and retrieved the median values of each expsoure sets. The result is shown below: Visually they all look pretty much the same until you get to the 4s frames set, as the graph above bears out. This is where the effect of the dark current starts to appear and most likely already at 1s. The most obvious conclusion is that if you're exposing for more than 1/10s you should really take flat-darks. By looking at the absolute values of the median you can see how tiny these numbers are, even for 4s exposures. My flats fall in the 1/40s category so either using flat-darks or master/super bias would yield the same. One of the perks of using an OSC camera with a EL panel is that you never change your exposure for each camera/lens/telescope combo. |
|
22.40
#...
·
|
|---|
|
Andrea, that looks like a nice measurement but I think that the limit for using dark flat correction in the calculation might be better determined by how large the offset can be before it introduces, some limiting error (maybe somewhere in the range of 3% - 10%) into the calibrated result for the specific camera that's used. Just because the dark current starts to become noticeable at 1/10 second doesn't mean that that's where dark-flats are required. I don't produce photometrically accurate images and calibrating with up to 8 second raw flats (i.e. no dark correction) works just fine. Come to think of it, I've never even used flat-darks on NB flats exposed up to 90 seconds; although with that camera I do run at -25C and it's a CCD. A MUCH bigger problem can occur when taking flats using short exposures. Some types of flat boxes exhibit spatial variations at the line frequency that can print through into the calibrated results. In most cases, the spatial noise is random in phase so it is necessary to time average the results with exposures times many times larger than the inverse of the line frequency. That can make exposure times on the order of 3-10 seconds necessary--even with a bright flat panel. In that case, flat-darks may help a little, but I doubt that it will make a huge difference for most "garden-variety" images. John |
|
7.46
#...
·
|
|---|
| John, it was to show that bias values are rather constant for a wide range of typical times for a master bias to be produced, at least for this sensor. No doubt that the critical points in the calibratiion of an image are, as you wrote, in removing the dark signal and the correct creation and application of the flat frame. |
|
12.30
#...
·
·
1
like
|
|---|
John Hayes: Hi John: I believe what he is referring to is the difference in bias from a dark image to a bright image. I don't think he is suggesting that the sub is not repeatable, just that there is a bias shift when taking the brighter frame. From everything that I have read on the subject, BSI CMOS sensors do not suffer from RBI, per se. There is a phenomenon called "image lag" that looks an awful lot like RBI, but the mechanism is completely different than that of RBI. Personally, with this camera I have not experienced any image lag that I have been able to recognize. But, if I am being honest, I haven't done any real testing to see if I could demonstrate is presence or absence. So, we are back to his statement: The on-chip calibration part of the cmos sensor will cause the drift of the whole image when the image is bright. In subsequent emails between us (although I can't seem to locate them) he elaborated that the bias signal is higher when the camera takes an exposure that is bright vs an exposure that is mostly dark. So...looking to the documentation we see that the QHY600 has 2 regions on the edges of the sensor that do not respond to light. The optically black area runs along the left edge of the sensor and includes both the dark current and the bias. The Overscan Area runs along the bottom of the sensor and does not include the dark current, but does include bias (don't ask me how this is happens because I truly do not know. I only know what the documentation says and what Cha has explained). The way that I am using this (after many discussions with the folks at PixInsight) is to use the Overscan Region function in ImageCalibration as follows: 1. When creating master darks (and master flat darks) I simply stack them as usual. The Overscan settings are left unchecked. 2. When calibrating the flat subs, define the Overscan settings as follows. (These settings are for bin 1x1. Bin 2x2 would be different)  The "Image region" is where the actual image is on the sensor. The "Source region" is the Overscan area at the bottom of the sensor. The "Target region" is where that particular Source Region should be applied, which is our Image region (note that the values for both are the same). Place the master dark with the exposure that matches the flat frame (the master flat dark) in the box for the master dark and check the "calibrate" option. Use the calibrated flats to build the master flat. 3. When calibrating the lights, define the overscan region as above, place the master dark with the exposure that matches the light frame sin the box for the master dark and check the "calibrate" option. Place the master flat in the box for the master flat but leave the "calibrate" option for the flat unchecked (it has already been calibrated). Note that there is no master bias (or superbias) and the Master Bias section remains unused for all. I'm not sure if I answered any of your questions, John, but this is the best information that I have and it is how I have been using the QHY600m for over 1 year now. I hope this is helpful. Tim. |
|
22.40
#...
·
·
1
like
|
|---|
|
Thanks Tim! I'd like to understand more about what's going on with this stuff and I appreciate the explanation. John |
|
0.00
#...
·
·
1
like
|
|---|
|
You need EITHER bias or dark flats for calibrating flats, or they won't work well. The reason is in the math involved, flats are taken at large ADUs, you divide by them, not subtract. A few cameras don't work well with bias because they are non-linear at very short exposures. 1600s and 294s. I know of no others, chime in if you do. With anything else (CCD or CMOS) bias generally works fine. The only difference between bias and dark flats is thermal dark current, it's insignificant in the large majority of cases. Long narrowband flats with poor illumination can be an exception. With my Spike a Flat, even narrowband flats are a fraction of a second. That translates to maybe .001 ADU of dark current. Utterly insignificant. 1600s need dark flats because of non-linearity. That somehow morphed into "CMOS needs dark flats" which is not generally true. |
