Alan Brunelle:
Alan Brunelle:
The expectation being that one would alway, always go back and redo the PP manually.
i don't know about that. the suggestion was to go back and do the light integration manually, not the whole preprocessing flow. the point of BPP (and then WBPP) was to automate the drudge work of matching flats with lights, darks with lights, darks with flats, etc.
I am not able to go back to the source of my recollection, but it was a post on the PI forum not long after the WBPP was released. I tried, but there is just too much stuff about WBPP to sort through it all and the search function is not able to tease out that post. For what it is worth, it was a statement from one of the PI staff, and may well have been from one of the developers of WBPP. Just not sure. But it was a very strong, clear, philosophical statement made by someone in the organization. Whether the rest of the developers at PI agreed with that statement is irrelevant, since it was not quickly followed up by anyone else from PI refuting it within my recollection. But it was a clear enough statement that it has stuck with me through the years. And the point was that, in his words (my recollection), WBPP was not intended as a means to generate a final product, but to work out the issues to get the best conditions for preprocessing. However, the construction of WBPP then and now certainly is such that what you say makes much more sense to me. Except that I don't know what your first statement means about going back and doing the light integrations manually. Why wouldn't I just let WBPP do the light integration? Or why wouldn't I just use the product of integration if it meets my standard? Its probably the least problematic or complex part of the process.
in the forums, they were adamant about the fact that there are too many options involved in ImageIntegration for BPP to be able to automatically select the right ones, at least for light frame integration. in particular, how to normalize the input images and what pixel rejection algorithm to use. in fact, BPP used to pop up a warning dialog box saying exactly what i'm saying - that the recommendation was to go and re-do the light integration yourself. many people objected to this dialog box on the grounds you are saying here, that if i like the output, why not use it? but that's not how Juan and company roll. they believe in letting the math guide you - in this case, meaning that you should tweak the ImageIntegration controls to maximize the SNR of the final product. the fear being that BPP would select too aggressive or wrong rejection parameters and compromise the SNR of the result. in the end, the master light integration was felt to be an iterative process and BPP could not do the iteration for you.
you may wonder if this applies to bias, darks and flats, but the proper integration parameters for those types of frames are pretty cut-and-dried and there's not much need to tweak them. for instance with bias and dark you never use normalization and you don't need really fine-tuned rejection parameters as the outliers you are rejecting (cosmic ray hits) are usually super-outliers and easy to identify. for flats you always use multiplicative normalization. for panel flats at least, there's no need to do any pixel rejection.
really, there's no point in even having the BPP script if you are going to go back and manually run ImageIntegration on your bias/dark masters, ImageCalibration on your lights 4 times, once per filter, being sure to carefully select the right dark, bias and flats each time, etc. the process of matching dark durations, gains, offsets, flats, etc. is very error prone and matching these up properly is something that a computer is very good at doing and there's no risk that the results will be suboptimal.
there are lots of people who recommend doing the whole flow manually at least once so that you understand what WBPP is doing behind the scenes, but that's not an argument for not using BPP/WBPP for the preprocessing task. i think the warning that BPP used to produce about the master light and the recommendation to understand the flow have been conflated here. i'm certain the PI devs never said to only use BPP or WBPP to figure out the correct preprocessing pipeline. if they ever did, it is certainly not true now.
WBPP has become a beast of a script, and if you notice, two things have emerged since BPP was created so long ago. first is LocalNormalization, which the developers believe can run "unattended" as long as a good normalization reference image exists. the other is the ESD rejection method, which again is felt to produce good results without fine-tuning. unfortunately it is *very* processor and memory intensive. these two things together have relaxed the requirement to go back and re-do the light integration.
I completely understand and agree with what you state here. And the evolution of WBPP in action and in expectations is to be expected. To add to this, I guess what one user chooses to do with this script may well be very different than with what others choose, just related to experience and what each individual feels they need to get out of it. I fall into the group that is generally happy with what I get, though I have run into issues from time-to-time. Especially with artifacts that can crop up here and there. Mostly, as I stated above, I'll get a crash during a drizzle integration because my computer can be sensitive to the action of other applications running concurrently. But doing a drizzle integration after all the hardest work is completed is not that much of a burden. And with a "manual" drizzle, I just use the same settings I use in the WBPP script.
Best,
Alan
|
3.87
2.15
