Suggested Workflow for Microscopy Data
At the microscopes
- Raw data is acquired at the microscope
- use the default vendor file format that preserves metadata in the best available manner.
- At the session level, zip-compress your session’s data, and name it with a meaningful and consistent naming scheme
- example : 2018-12-11_mutterer_by2_sync_assay_lsm780.zip
- Note: zip compression is lossless.
- Upload the zip-compressed data to a safe storage area, using a meaningful and consistent directory structure
Test run
- Typical confocal microscopy images, 1 folder, 25 *.lsm files, zipped
- 226.6 Mb -> 109.5 Mb (52% compression)
- Typical serial block face microscopy images, 1 folder, 1435 *.dm3 files, zipped
- 2.53 Gb -> 1.95 Gb (23% compression)
At your workstation
- Raw data is located on a safe server
- Retrieve the necessary zip files containing raw data you need to access
- Decompress it on your workstation
- Analyze/Use your data
- Keep all necessary intermediate analysis files, or scripts for reproducibility
- Note: if you have a workflow that allows getting reproducible results from a script and raw data in a reasonable amount of time, you do not need to store intermediate results.
- Upload important results and items needed for reproducibility to a safe storage area, using a meaningful and consistent directory structure
If you think of improvements for this page, please speak up.