spacer link to MAST page spacer logo image spacer
 
link to STScI page


NASA Data Center Annual Program Plan

Program Year: FY 2004
Data Center/Service: Multi-mission Archive at Space Telescope (MAST)
(Optical/UV Science Archive Research Center) 
Supporting Organization: Space Telescope Science Institute
3700 San Martin Drive
Baltimore, MD 21218

Overall Mission:  MAST supports active and legacy mission datasets and related catalogs and surveys, focusing primarily on data in the ultraviolet, optical, and near-IR spectral retions. Support includes curation of the data, providing expert support to users of the data, providing access to data-specific calibration and analysis software, providing user support for this software, and maintaining public access interfaces to the data. MAST works with new mission teams in the supported wavelength regions to assist in the development of data management plans, especially in the areas of data formats, descriptive metadata and standardization of keywords, in the development of data access and delivery plans, and assuring data quality control.

Total MAST Holdings by volume



MAST holdings without HST/GSC/DSS



MAST Data Holdings

Name Size Number of Observations Active Mission Duration
ACTIVE MISSIONS
FUSE 442.159 GB 3581 1999-
GALEX 8.25 GB (compressed) 10 2003-
HST 19.12 TB 494,505 1990-
LEGACY MISSIONS
ORFEUS: BEFS 4.1 GB 332 Sept. 1993; Nov. 1996
ORFEUS: IMAPS 0.3 GB 643 Sept. 1993; Nov. 1996
ORFEUS: TUES 0.2 GB 229 Nov. 1996
EUVE 96 GB 1377 1992-Jan. 2001
ASTRO: UIT 56 GB 1442 Dec. 1990; March 1995
ASTRO: HUT 0.6 GB 516 Dec. 1990; March 1995
ASTRO: WUPPE 0.1 GB 238 Dec. 1990; March 1995
IUE Final Archive 475 GB 103,552 1978-1996
IUE SIPS 125 GB 104,296 1978-1996
Copernicus 0.8 GB 551 1972-1981
CATALOGS & SURVEYS
VLA-FIRST 183.98 GB (compressed) 29,153 1993-
Digitized Sky Surveys 5 TB n/a 1950-58, 1975-99
GSC I, II 2 TB n/a 1950-58, 1975-99

Services Provided: MAST provides support for users seeking to understand the properties and instrumental signatures of all archived datasets and assistance with the interfaces to browse and retrieve these data. Access to non-HST mission and instrument specific calibration and analysis software and assistance in its use continues on a time-available basis. Full support for HST related software is provided by the MAST Help Desk and staff.

Non-HST Data Analysis Software Provided: IUE "RDAF" package (IDL-based), IUE Final Archive processing software (IRAF port), EUVE analysis software package (IRAF-based), Copernicus data analysis software (IDL-based), UIT data reduction/analysis software (FORTRAN, C, and IDL routines), WUPPE data analysis software (FORTRAN routines requiring the FITSIO library), and HUT data reduction software (IRAF-based) are available through MAST.

Mission Interfaces:

  • Last fiscal year the contract between MAST and FUSE ended, limiting MAST's role to continuation of data ingest. Nonetheless, the new DADS v10.3 build of June 2004 permits the delivery of CALFUSE data (with a new intermediate file to be ingested in late July 2004) with negligible extra effort from the STScI or MAST groups. MAST continues to respond to ad hoc requests from the Project, such as providing statistics and viewgraphs for the FUSE Users' Group, and to participate in periodic planning meetings to assess requirements pertaining to smooth ingest of FUSE data into DADS and to respond to requests from the Project for improved interface display and search tools.
  • The GALEX archive represents the largest mission dataset supported by the MAST contract (HST archiving is supported by the primary HST contract). In preparation for the first (Early Release) Observations, MAST has scheduled weekly meetings with Caltech and has exchanged personnel for training sessions and consultations. MAST's ERO made its debut in early 2004 and staged these data for Cycle 1 of GALEX's GI program. GALEX data usage (see figure) has a sustained usage of over 15000 hits/month since January. In addition to this release, MAST received an "Internal Release 0.1" dataset and is about to receive data for an IR0.9. These internal releases have permitted MAST to continue development of its database in preparation for the first large public release, DR1, scheduled for October 2004. During the period an Interface Control document (version 2.1) was written to document data interface protocols and arrangements between Caltech and MAST.

    Number of nonrobotic hits during 2004 (July incomplete) of MAST's GALEX website. (The darker red indicates the number of page errors.)

  • Over the last two years MAST has developed a GALEX database using Microsoft SQLServer and .NET technology. This technology permits the automated ingest of data, self-documentation, rapid browsing of the entire database structure, including a listing of tables, keywords and procedures used for qualified searches of object classes, and listings of all observations. An active helpdesk is maintained for user questions pertinent to the database.
  • The VLA-FIRST archive has been significantly enhanced, in part through the addition of coadded images. The FIRST database table and cross-correlation search tool was updated to include the new data.
  • MAST continues to interface with the CHIPS Project concerning their archiving plans. This project plans to transfer the CHIPS dataset and auxiliary tools approximately one year after mission completion, or about the middle of 2006. (The total data volume will be only 10GB.)

Interoperability Activities:

As part of a continuing effort to enhance archive mission interoperability, MAST held several discussions with the IRSA center to discuss integration of some of the IRSA datasets into the Scrapbook.

Committee Participation within the STScI: 

  • Levay chaired an ad hoc STScI-wide committee to investigate solutions to the computer bandwidth problems associated with difficulties users had in the summer and fall of 2003 in downloading data in a reasonable timescale. Initially, the need for increased bandwidth was documented by capturing network usage and summing all network activities for major subgroups at STScI. This documentation was used to justify to the HST project and to NISN the ISP, the increase in bandwidth from 11 Mbits/sec to 26 Mbits/sec. in addition, data going to current Internet 2 subscribers goes through an Internet 2 router at NISN. While this bandwidth is adequate currently, the projected growth in OPO services and deliveries of HST ACS data and High-Level Science Products (mostly related to the HST Treasury programs) Is expected to saturate the bandwidth during peak use times in about 18 months. Therefore, Levay is participating in a subsequent committee to investigate the use of another ISP either as a replacement or a supplemental service.
  • Shiao and Conti participated in an ad hoc Sybase Alternatives Committee to explore more economical and robust options to the Sybase database system. A report with recommendations was sent to Rodger Doxsey, and the announcement of an Institute-wide database technology is expected soon.
ACTIVITIES AND MAJOR ACCOMPLISHMENTS OF THE LAST YEAR

MAST Data Ingest & Retrieval Activity The combined active mission (HST, FUSE, GALEX) annual data ingest rate has maintained the steep increase from last year, raising the total holdings to 19.74 GB. The continued high rate from last year was due mainly to the addition of ACS and NICMOS instruments data from the HST. A jump in the number of retrieved datasets was due to the download of all HUT and Copernicus datasets in June by the Korean Astronomical Institute in preparation for the SPEAR mission (see below).

Date Ingest Volume (GB) - Active Missions Retrieval Volume (GB) - Active Missions Retrieval Volume (GB) - Legacy Missions Datasets Retrieved - Active Missions Datasets Retrieved - Legacy Missions
Jul  2003  559.845 1988.730 4.583 51926 7850
Aug  2003  501.606 1678.079 1.848 37638 2624
Sep  2003  454.652 1956.632 1.375 43187 1099
Oct  2003  520.291 2249.956 0.895 41774 1926
Nov  2003  499.103 1722.710 1.435 35745 1641
Dec  2003  455.190 866.008 2.315 21217 1393
Jan  2004  499.071 1519.304 2.123 47442 1314
Feb  2004  605.283 1799.266 15.725 57142 25866
Mar  2004  510.467 1397.013 19.631 48414 6184
Apr  2004  388.683 1550.157 13.737 42540 6753
May  2004  389.0021 1127.777 1.277 50575 819
Jun  2004  306.312 1136.748 39.661 44164 51966
Total 5689.503 18992.378 104.605 521764 109435
As MAST does not maintain retrieval statistics for DSS , only the number of searches is displayed in order to show the general interest level in these data.


Data Discovery and Search Tools:

Scrapbook and Pointings Table: FUSE was added to the Spectral/Imaging Scrapbook. New datasets are added to the scrapbook each month from the HST and FUSE observations. ACS and WFPC2 Associations Pointings tables were constructed and added to the pointings interface tool. Data for all the active HST instrument pointings tables were updated weekly.


High Level Science Products:

Several sets of High Level Science Products (HLSP) were delivered and implemented this year:

  • GOODS Version 1.0 data were delivered in late August 2003.  The associated catalogs were delivered in December 2003.
  • The stacked, composite image obtained during a search for Kuiper Belt objects was delivered in December 2003. 
  • The Hubble Ultra Deep Field data were released in early March 2004.  Anticipating high demand for these data, CADC and ST-ECF set up mirror sites for the UDF data.  The UDF data is comprised of 4 distinct sets of data: ACS-WFC, NICMOS,  ACS-HRC (parallels to the ACS-WFC), and NICMOS parallels to the ACS-WFC data. 
  • An atlas of FOS spectra of AGN and quasars was delivered in March 2004. 
  • Version 0.5 of the data from the Archival Pure Parallel program was delivered in May 2004.
The page describing the High-Level Science Products holdings was rewritten to include all HLSP in a single page and to include a new type of search to help users locate sets of interest.  After a couple of years of experience with HLSP, the Guidelines page for contributing HLSP was simplified and made more "user friendly".

A complete listing of HLSP hosted at MAST is below. Although MAST provides an interface to the WFPC2 Associations, the data are held at CADC. MAST distributes the data via a proxy.

High-Level Science Product Holdings
High Level Science Product Set Size
 Number of Files
10 Lac Spectral Atlas (HST/GHRS)
 5.3 MB
 67
AGN and Quasar Spectral Atlas
 73.8 MB
 451
Copernicus Atlas of 6 Selected Stars
 3.6 MB
 25
EUVE Spectral Atlas of Stars (EUVE)
 29.4 MB
 490
GOODS: The Great Observatories Origins Deep Survey
 96.3 GB*
 1342
Grayscale of Time Variation of gamma Cas Near SiIV Doublet
 5.0 MB
 7
Hubble Deep Field
 2.1 GB
 181
Hubble Deep Field South
 7.8 GB
 178
Hubble Helix Observations
 13.8 GB
 32
Magellanic Cloud Planetary Nebulae
 721 MB
 1620
OB Stars (Galactic): FUSE Spectral Atlas
 30 MB
 184
OB Stars (Magellanic): FUSE Spectral Atlas
 1.2 MB
 66
Pre-Main Sequence Stars: IUE Spectral Atlas
 10.7 MB
 733
Procyon (FV-IV) Spectral Atlas
 1.2 MB
 14
Quasar Spectrum HST/FOS
 .6 MB
 4
Search Field from a Search for Kuiper Belt Objects
 3.7 GB
8
The Medium Deep Survey
 11.9 GB
 4726
Ultra Deep Field
 30.7 GB
 2252
Ultraviolet Images of Nearby Galaxies
 728.0 MB
 334
WFPC2 Archival Parallels (Version 0.5)
 1.7 GB
 246
alpha Ori Spectral Atlas
 4.0 MB
 60
chi Lupi Spectral Atlas
 22.7 MB
 156
TOTAL
 165.94 GB
 13176

New plotting and graphical display tools:

  • Working with the author, Ivo Busko, MAST staff have beta tested and participated in further development of Specview (a spectral data analysis tool). This tool is now mature enough for use on MAST pages, but for most efficient use should (and will) be developed as a web service.
  • Another new tool provides the ability to plot results from any MAST mission or VizieR catalog search with a new JAVA-based graphical display tool called VOPlot (http://vo.iucaa.ernet.in/~voi/voplot.htm). This new tool allows users to plot the search results from any numerical field as a histogram, or as a scatter plot against any other field, as well as to display statistics on the results. (VOPlot was developed as a part of the Virtual Observatory - India initiative.)

New protocols and IVOA-related services:

  • During the past year MAST increased activity in the Virtual Observatory effort.  All mission searches are now available as a web service utilizing the VO Simple Cone Search Protocol.  HST previews, UIT data and previews, VLA-FIRST coadded images and selected High-Level Science Products (GOODS and UDF) are available as a web service utilizing the VO Simple Image Access Protocol (SIAP).
  • A search form has been written to allow MAST users to access SIAP web services available through the IVOA. The search form operates on either target name or coordinates.
  • MAST began work on a web-based image cutout service in conjunction with the GOODS team. This service will have ``spin-off" applications to other projects.
  • The GALEX archive website has been implemented as a SkyNode in the Skyquery portal system developed by JHU as part of the IVOA project. Several HST catalogs (HDF-N, HDF-S, and GOODS) have also been registered on Skyquery. In addition, MAST has entered the GALEX cone search and Simple Image Access Protocol in the US-NVO (IVOA) registry.

New interface pages and search tools:

  • A new page entitled ``MAST and the Virtual Observatory" was developed to document ways in which MAST is adopting IVOA protocols and services to aid users in data discovery and to integrate IVOA services and tools into its website and applications.
  • A new VizieR Catalog Search and Cross-correlation tool allows MAST users to search the entire set of 4,000 VizieR catalogs and cross correlate the results with any MAST mission. This web-based search tool accesses one of the several VizieR catalog servers (e.g. at CfA) as a web service and retrieves the search results in a machine-readable VOTable XML format.
  • A search interface was added for the AGN catalog of Padovani, the High Level Science Products, and the WFPC2 Associations.
  • The user interface to the GALEX database is a web site that provides the capability of querying the data by object name, sky position, or physical property by a "MAST-style" query form or by detailed queries written in SQL. Results pages (see figure) are "active." This means that objects anywhere in the field of view can be "clicked," permitting image and spectra to be displayed with identifying information. This interface was cited in NASA's Senior Review report as representing the "state of the art" and an example of how other missions and archives should represent and deliver data to the community. A tutorial was written for ease of navigation. A syndicated (automated) What's New page has been developed on the home page.


Enhancements of User Interface Pages and Tools:

  • The MAST search interface has been rewritten in PHP to permit a common set of functions to be shared in searches across all MAST missions. This enhancement improves the maintenance of interfaces for existing missions, facilitates the construction of new mission search pages, as well as existing search page "tools" such as the Cross Correlation Pointings Table, and Scrapbook. The PHP rewrite also incorporates increased functionality, for example, the inclusion of lists of target names as well as the storage of input parameters for later use in assembling statistics on usage.
  • ACS and WFPC2 Associations have been added to the Pointings Tables search. This addition allows users to determine what regions of the sky these instruments have observed and to do sophisticated searches based on the observational parameters.
  • FUSE mission data was added to the MAST Scrapbook.
  • The MAST Spectral Coplotter has been enhanced to allow users to provide averaged spectra of "like" constituent datasets, that is of the same mission, and spectroscopic configuration.


Outreach to the user community:

  • MAST conducted a comprehensive survey of user preferences, attitudes, and search practices (315 users responded). The results were presented at the October 2003 MUG meeting and have since been used as a guide to prioritize development of web pages and data search/retrieval tools.
  • MAST wrote and distributed biannual newsletters to its user community. The February 2004 newsletter was the first to be "published" on-line.
  • MAST staff participated in several conferences (AAS, ADASS, and SpaceObs) during the year by presenting posters on new MAST services and the new GALEX website.


The Astrophysical Data Centers Executive Council (ADEC): 

ADEC representatives Rachel Somerville and Paolo Padovani participated in ADEC meeting in September 2003. The principal topic, the standardization of dataset identifiers, has been taken as an action item during the reporting period. MAST has completed its part by creation of a web service that verifies that a data ID is a valid MAST dataset. This in turn is part of a coordinated service to be hosted at ADS. ADEC members are working with various publishers of astronomical journals to encourage or possibly require authors to submit a complete list of the data IDs used in the paper.

MAST representatives also participated in several discussions to coordinate the ADEC role in the Senior Review.  A common appendix was added to the proposal of each data center.


MAST Literature Links:  

The publications database and the links between scientific papers and the referenced MAST datasets are regularly updated as new citations become available through the ADS. The journal database tables are now updated by an automated monthly procedure.

Senior Review Proposal:

With its funding continued only through September 2004, MAST participated in NASA's Senior Review proposal process. This consisted of a 4-year proposal outlining the organization's program and goals for this period with a comprehensive budget covering the slated staffing, software, and hardware needs. This proposal was defended by Marc Postman and Rick White in April. The reviewers of the proposal gave MAST very positive grades, ranking it first among NASA's data archiving centers.

STAFFING CHANGES AT MAST

Rick White became the new Chief of the Archive Branch (in October 2003) and MAST, (in May 2004). During the year Mike Corbin, Megan Donahue, Paolo Padovani, and Antonio Volpicelli left MAST, while Inga Kamp and Bernard Shiao joined the staff. (Dr. Dorothy Fraquelli will start work with MAST on August 2, 2004.) Of the staff listed, only Shiao and Volpicelli are, or have been, salaried from the MAST contract.


Press Releases from MAST Data (HST-archival only):

  • PR03-17 "Brighter Neptune Suggests a Planetary Change of Season" - L. Stromovsky
  • PR03-20 "Celestial Fireworks" - Hubble Heritage Team
  • PR03-21 "Too Close for Comfort" - Hubble Heritage Team
  • PR03-29 "Hubble Photographs Turbulent Neighborhood Near Eruptive Star" - Hubble Heritage Team
  • PR03-30 "Firestorm of Star Birth Seen in a Local Galaxy" - Hubble Heritage Team
  • PR04-13 "Hubble Sees Stars as Numerous as Grains of Sand in Nearby Galaxy" - Hubble Heritage Team

PLANS FOR COMING YEAR

Future datasets:

  • MAST will ingest GALEX Data Release 1 (DR1) in late fall of 2004. MAST will begin hosting GALEX/DR1 data (400-500 GB) for archival Guest Investigators and other interested users in the fall of 2004. In early 2005 MAST will serve Cycle 1 proprietary data to Guest Investigators via password-protected accounts.
  • MAST will continue to archive new sets of HLSP.  Expected during the coming year are data from the GEMS and COSMOS projects, and additional data from the GOODS and Archive Parallels projects.
  • MAST has recently learned of a second diffuse UV spectrum project, SPEAR, ("Spectroscopy of Plasma Evolution from Astrophysical Radiation"), which wishes to archive its data with MAST. MAST initiated contacts with J. Edelstein to establish a rough timeline (2006-08) and data volume (500 GB) for this mission. In late 2004 a timeline for data delivery will be determined, pending an identification of funding for archiving of this dataset.
  • Following NASA's acceptance of the Preliminary Design review of the Kepler Project, MAST's role in the Data Management System will begin ramping up in FY2005. The roles of each of the data pipeline (STScI) and archive teams (DADS, MAST) will be defined during this period.

New services for archival research:

  • Coincident with the anticipated surge in usage following the DR1 release, MAST will enhance its GALEX website services with new functionality and tools. These services will include: new registered web services (footprint and SSAP services) entered in the IVOA registry, a fully functional GMAX cross-correlator that links GALEX data to data from other missions over requested regions of the sky, cutout services or various data types (including fits files), upgrades to the web interface forms (including several of the project's "20 Questions" written in SQL), and a prototype of "IDL on the Network" (ION) for quick evaluation and analysis of data on our website.
  • MAST will continue work with GOODS team members to adapt a cutout service developed for team usage, to a more general web based application for use with GOODS, UDF, and other image data.

Coordination activities:

  • MAST will continue to work with IRSA to integrate MAST and IRSA data into a joint (archive center-wide) Scrapbook.
  • Work continues on delivering recalibrated data products from CADC on the GHRS and FOC instruments.
  • Specview will be ported for use on appropriate MAST applications such as the Scrapbook.

Continued Services for Archival Researchers:

  • MAST continues to acquire and make available High-Level Science Products, such as spectral and image atlases, composite-data spectra and images, and products from the HST Treasury and Large programs.
  • MAST will continue to provide assistance for archival researchers with questions about searches, retrievals, data characteristics, analysis software, and other issues related to MAST data.
  • MAST will continue to send out biannual newsletters to its user community.
  • MAST/GALEX will conduct a poll of its GI users to determine their preferences for enhancements of its website pages and data mining tools.