Data Archiving Guidelines
The information below was written for projects intending to archive mission data sets within the Mikulski Archive for Space Telescopes (i.e., MAST). Recommendations are included regarding:
1. Data Set Formats
The astronomical community has adopted the Flexible Image Transport System (i.e. FITS) format as the default standard for the exchange of data between institutions. The FITS file format is platform independent, supported by many institutions, and endorsed by both NASA and the IAU. For these reasons FITS is the recommended file format for archiving data at STScI. A description of the FITS data format recommendations can be found in the MAST Data Format Guidelines document. A online version of the FITS Standard Document and the FITS User's Guide is also available.
It is recognized however that some archival data may be stored in other formats, particularly for those projects which preceded the recent developments in FITS. One example is the earlier-processed IUE data which is stored in a VICAR-based IUE "GO" format. In some other cases, projects have distributed data as ASCII text files or created auxiliary data sets as ASCII text files or postscript format. In these cases, no attempt will be made to reformat the data sets before being archived within MAST.
MAST, like other data retrieval systems, uses online database tables to search for requested data sets. In many cases, projects store the same information in both the catalog or database table and the FITS keywords. To simplify adding new data sets into MAST, it is helpful if either
Catalogs should contain those fields which would most help users locate the desired observation(s). Coordinates, observation date, exposure time, and target name are fairly essential (depending on the method of observation), while parameters needed for analyzing or interpreting archived data would be highly desirable. A target classification entry has been very useful for users interested in particular types of objects.
Although the MAST uses the Sybase Database Management system, tables can be exchanged between most database systems by copying them to ASCII table files (be sure to include a sufficient number of significant figures for representing floating point values.) A WEB page containing an observation list may be an adequate replacement for a database table. In either case, a description of the individual fields within the table or list should be provided as well. The description should also define the source of the entries. For example, it should state whether the coordinates were supplied by the observer, or obtained from an existing catalog.
Project-supplied documentation in the following categories should be made available for archive users:
Since MAST documentation will be accessed primarily from the WEB, documentation is most useful if it exists online. Most text processing formats such as LaTeX or Microsoft WORD (or standard ASCII files) can be fairly easily converted to HTML by staff members. Large documents such as user manuals or data analysis guides should be made available to users in several formats such as HTML for online access, and POSTSCRIPT and/or PDF for downloading.
Some projects have written software to analyze and interpret raw and/or processed data. These programs should be archived for future users. MAST will make project-supplied software available to requesters, although support for the software itself can not be provided. MAST currently maintains for example, the IUEDAC IDL software libraries, the UIT BDR software written in C and Fortran-77, and the EUVE EUV1.8 IRAF software.
A list of available
Fits software packages is available from HEASARC.
The list contains links to sites supporting general
FITS readers and writers written in a variety of programming languages.