0.00
#... |
|---|
|
I am curious to know what others use as their system for handling and in particular, storage of Raw data files. My process uses Pixinsight as my primary process software, however as part of the process PixInsight will create a folder of format converted and registered images which greatly increases the storage requirements. So I guess my question is whether you retain the original RAW files as taken on the camera as well as the .XISF files generated by PixInsight, or do you delete the RAW afterwards? As a side note, using any form of cloud type storage is not an option for me here in Australia, as our internet quality is crap and in particular, our upload speeds make a 3rd party cloud option un-workable Flint |
|
0.00
#... |
|---|
|
What I do might be a little goofi, but it works for me. Hard drives are pretty cheap, so I have a lot of hard drive space to start with. I run PI and let it do its thing, saving everything. Then, once a month I backup everything onto an external USB hard drive. After they're backed up, I prune the registered files out (I can always re-register them), but keep the lights (FITS off the camera) and calibrated files. I also backup my calibration files - just in case. Finally, any working/temp files I've created usually get deleted unless there's a reason I need them. |
0.00
#... |
|---|
| I've gone the route of getting a separate USB thumb drive for every image. It is nice because it keeps everything organized and is easy to go back to if you collect more data on a target. Also, nice to go back and see processing steps used to learn from later. Idea was gleaned from Adam Block's USB stick tutorials. It seemed great and worked well, so I copied it for my stuff. |
|
0.00
#... |
|---|
I use a similar way like Chris. I have a 4 terra HDD in the PC as main storage. All rawdata (and other stuff) will be stored there. Processing results are stored and used on an 2nd SSD disk (the 1st SSD is the program and Windows disk), and only the end Result goes back to the storage disk. A Mirror of the storage disk is done on a NAS, transferred a few minutes after changing a file automaticly. The NAS also acts as a personal cloud, so i have access everywhere to all the data. A weekly full backup of the NAS to 2 offline HDD prevents data loss in case of viruses or similar things, because i think, the whole PC and NAS could be affected then. Every week to the other HDD. Ok, this sounds perhaps a little overcautious, but I only use electronical storage for all my data and photos since the last 30 years and loosing this stuff.... hmmm no  |
0.00
#... |
|---|
|
This is a good question. I have a "generic" folder saved on my desktop. The folderstructure looks like this: Generic Folder Raw_Date Lights Darks Bias Flats Flat_Darks Integration Files Calibbrated_Flats Calibrated_Lights Debayered_Lights Registed_Lights ImageProcessing_Files Master_Files Every Time I collect data, I make a copy of this "generic" folder and name it like this: 16_4_23_RingNebula_LasCruces When I finish with my processing, I copy the whole folder onto my external hard drive. It is huge so I don't bother culling files. I like to keep everything, because as I learn new processing tricks, I often go back and reprocess everything from scratch. Bob |
|
0.00
#... |
|---|
|
This is a good question. I have a "generic" folder saved on my desktop. The folderstructure looks like this: Generic Folder Raw_Date Lights Darks Bias Flats Flat_Darks Integration Files Calibbrated_Flats Calibrated_Lights Debayered_Lights Registed_Lights ImageProcessing_Files Master_Files Every Time I collect data, I make a copy of this "generic" folder and name it like this: 16_4_23_RingNebula_LasCruces When I finish with my processing, I copy the whole folder onto my external hard drive. It is huge so I don't bother culling files. I like to keep everything, because as I learn new processing tricks, I often go back and reprocess everything from scratch. Bob |
11.57
#... |
|---|
|
I never delete raw images. They allow you to reprocess your images once your processing skill improves or once there is some brand new way of dealing with the data. Never delete the raw images. On the other hand, I always delete the intermediate files generated by PixInsight once I stack them and feel satisfied about the stacked image. Of course, it will take some time to process the stacked image to make sure it is good. So I may leave the intermediate files there for a few days, but I never want them to stay in my HD permanently. If for some odd reason that they are needed later, they can always be re-generated from the raw files. Not a big deal. |
|
0.00
#... |
|---|
|
same here ... keep the source (raws) , stacked stuff to try different processing, and don't forget the final pictures. Until ASTRO bin I tended to lose the finals...  of course you can big NAS with a few Tera bytes save everything  |
0.00
#... |
|---|
| Guess I use an old school approach. After a night of capturing, I immediately save the raw files to DVD, then do the same with supporting files (flats) as they're created. Darks are in a library on their own DVD. I don't save any of the processed files. I can usually get 3 targets' worth of data per DVD. |
