Follow Us
|
"Best Reference" and "Used Reference" files for archival HST dataset retrievals
Summary:
We provide direct access to software for identifying and retrieving the best, most-up-to-date reference files for HST data from the new Calibration Reference Data System (CRDS), including instructions, examples, and an interactive web search interface:
Details:
When HST dataset retrieval requests are submitted to the archive, the user also has the option to request calibration reference files that have been produced and delivered to the archive by the HST instrument teams. Here we describe the system that has been in place since 2001, as well as relevant changes that are being implemented as of 2015.In 2001, the On-the-Fly Reprocessing (OTFR) system was implemented, which involves a full reprocessing and recalibration of each HST dataset that is requested as calibrated data. In addition to the calibrated HST datasets for a given observation, the system also provides the option to request calibration reference files for the datasets: either "Used reference files", namely those that were first used when the data was originally obtained, or "Best reference files", namely the latest reference files that were produced and delivered by the HST instrument teams. Note that, in the OTFR system, these options only determine which reference files are retrieved; the actual calibration of HST data in OTFR is always carried out using the best reference files. These options are documented along with all the other retrieval options at the following webpage: https://archive.stsci.edu/hst/help/retrieval_help.html
As of 2015, the OTFR system is being replaced with a new system, the HST Online Cache. This facilitates much faster access to calibrated HST datasets, which are now stored as static files online, and are not reprocessed automatically with each user request. As part of this transition, the option of retrieving "used" or "best" reference files is being discontinued; each dataset retrieved from the online cache will have been processed and calibrated with the most up-to-date reference files that were available when it was last reprocessed. If new calibration reference files are subsequently produced and delivered by the HST instrument teams, then the relevant online HST datasets are planned to be reprocessed and recalibrated using the new reference files. Given the amount of time required to reprocess all data that may be impacted by new reference files, there may be periods of time during which the current online version of a dataset might still be the version that was calibrated with the previous reference files, until the reprocessing has been completed using the new reference files.
Therefore, we are providing access to a python script which can be run on any given set of HST data, in order to determine if new calibration reference files are available for those data. The script can query the HST archives directly, even from external locations, and if new reference files are identified then the script can update the file headers, and can also retrieve the new reference files directly. This can then enable observers to recalibrate their HST data immediately using the latest reference files, without needing to wait for the online cache version of the data to be updated.
This script is currently available within the CRDS package, which is included as part of the standard STScI Ureka software distribution (http://ssb.stsci.edu/ureka/). Further details on the script, as well as examples on how to run it either on the command-line or via an interactive web interface, are available here:
Please send any questions or comments to "archive@stsci.edu", and for common questions about retrievals, please see our
FAQ page.
 |
|
|
|
|
|
|
|
|
|