NOTE: Most of the High Level Science Products are unavailable while unscheduled maintenance is being performed. They will be incrementally restored over the course of this week. We apologize for any inconvenience.
The Multimission Archive at STScI Newsletter
Table of Contents
GALEX Release (GR2) Data (Images) at MASTMAST is happy to report two portions of the GR2 were released to the public in early May and early June. The current release subsumes the previous GR1 release made 1-1/2 years ago. The data volume and sky coverage is now increased by a factor of 3-4. The primary component of the release, the AIS (All Sky Imaging Survey) now covers 7077 sky regions called tiles (generally circular areas of 1.2 square degrees). The Project reports that the AIS is now about 21% complete (note that when completed the AIS will cover about 3/4 of the full sky). The GR2 deliveries so far also include the Medium and Deep Imaging Survey data as well as the Nearby Galaxy Survey.
Over the past year, MAST has received 4 imaging and 1 spectroscopic
deliveries of Guest Investigator programs. Note that MAST learns which programs are
coming only upon their delivery. The MAST GI website informs potential
archival users of GI program information, including data, abstracts,
and in the case of still-proprietary programs, release dates to the public.
MAST expects to archive the first GALEX High Level Products Data this summer. Gil de Paz et al. plan to provide an image atlas of nearly 1000 galaxies in both near and far UV bands. Watch the "What's New" box for more updates!
XMM Optical Monitor Data at MASTThe X-ray Multi-Mirror (XMM) Telescope, launched by ESA in December 1999, carries a 30-cm optical/UV telescope with a microchannel-plate pre-amplified CCD detector in its focal plane. The Optical Monitor (OM) spatial pixel size in normal operation is 1 arcsecond with a 17 arcminute field of view, and the limiting sensitivity is B=24 for a star viewed with the detector in unfiltered light. Filters cover the 160-600 nm wavelength range. The OM telescope is coaligned with the X-ray telescope, and most XMM X-ray observations have accompanying OM images. MAST now distributes the XMM-OM images (http://archive.stsci.edu/xmm-om/). These will be of interest to many of our users, since they are typically more sensitive and higher resolution than comparable GALEX images. Most fields have images through multiple UV and optical filters (see the figure for examples.)
Community Contributed High-Level Science ProductsHigh-Level Science Products (HLSPs) are fully processed (reduced, co-added, cosmic-ray cleaned etc.) community provided images and spectra that are ready for scientific analysis. HLSPs also include object catalogs and spectral atlases. The Multimission Archive at the Space Telescope Science Institute (MAST) archives HLSPs, which have MAST missions as an integral part of the products. The data are accessible through the MAST web pages and available via ftp download. We have made most HLSP images available through the Virtual Observatory (VO) and its common tools such as Aladin. See a complete list of currently held HLSP at http://archive.stsci.edu/hlsp/index.html.
Five sets of HLSP have been archived since the last archive newsletter, summarized as follows:
ACS imaging of HUDF NICMOS parallel field (PI:Stiavelli)
The Catalog and Atlas of Cataclysmic Variables (PI:Downes)
You may download the images through your browser by clicking on the filename or via wget.
Most data is also available via anonymous ftp (ftp archive.stsci.ed cd /pub/hlsp/cosmos).
Eta Carinae (PI:Davidson)
Kris Davidson (U. Minnesota) is the PI for an HST Treasury Program that observed Eta Carinae.
The program was designed to obtain data during an anticipated spectroscopic event in mid-2003.
Observations were obtained using the HST STIS/CCD, STIS/MAMA, WFPC2 and ACS images.
The initial release of HLSP contains only the STIS/CCD observations.
The team has an extensive website at http://etacar.umn.edu/.
They provide in addition to the actual data also the teams detailed view of the observation
including a preview and a display showing the slit placement.
The MAST archive web pages for the Eta Car HLSP are at
Archive Pure Parallel Program (PI:Casertano)
These data can be used to address a wide range of science topics: measuring the cosmic shear on scales from 20" to 2'; discovering ~ 50 starforming galaxies at z ~ 4; finding optical counterparts to AGNs in wide-area radio and X-ray catalogs; improving the determination of the scale length of the Galactic disk; and studying stellar populations down to 1 solar mass for about 50 separate lines of sight in the Magellanic Clouds.
We are interested in archiving your High-Level Science Products!HLSPs are provided by experienced users and legacy teams, who make their calibrated data available to the community after a careful and sophisticated data reduction process. Our archive retrieval statistics over the past year reveal that the HLSPs get downloaded ~10 times more often than the average HST pipeline processed dataset. This will certainly reflect in the citation index of associated papers.
If, as part of your research,
you have produced such science ready data products based on data from any of the MAST missions
and you are interested in contributing them as HLSPs to MAST,
please see our page of guidelines
and contact us at email@example.com.
Recent Changes to the Digitized Sky SurveyThe Digitized Sky Survey (from scans of the Palomar Schmidt and AAO UK Schmidt all-sky photographic surveys) is one of the most heavily used resources available from the MAST archive, with an average of 5000 downloads per day (http://archive.stsci.edu/cgi-bin/dss_form). Even though these data were created and made available to help support HST operations, they are valuable for many other uses and have become part of the the underlying infrastructure of modern astronomy. (See http://www-gsss.stsci.edu/SkySurveys/SkySurveys.htm for more detailed information on the available surveys). A color composite image derived from the 3 passbands of the POSS-II is shown as an example of the usefulness of the DSS data.
One of the limitations of the data has been the precision of its absolute astrometry. All of the astrometric information in the headers was tied to the HST Guide Star reference frame which was constructed back in the 1980's when there was no single unified all-sky reference frame. This resulted in potential astrometric errors of 1-2 arcseconds when compared to observations in other coordinate systems. In addition, when initially conceived, there was no standard FITS world coordinate system, so the astrometric keywords contained in the FITS headers are not standard and require one to either convert the headers or use a tool that understood this particular implementation.
The Catalogs and Surveys group at STScI have created an updated version of the Guide Star Catalog (GSC-II) that beginning with HST cycle 15 replaced the GSC-I as the guide star reference frame (see http://www-gsss.stsci.edu/Catalogs/GSC/GSC2/GSC2.htm for additional details). This is based on the IAU recommended International Celestial Reference System (ICRS) and is the reference frame for many other catalogs (e.g. SDSS, 2MASS etc).
In coordination with this change, the astrometry of the DSS has also been updated to use the GSC-II
reference frame in order that HST observers can measure coordinates for observation planning.
We also took this opportunity to update the headers to reflect the current FITS standard.
In addition to preserving the GSC-I astrometric solutions in the FITS header, we have added the more
complex GSC-II astrometry (including a position-dependent correction as an extension table) for completeness.
However, since this is not supported by most display tools, the code that returns the subset of the image
from the selected plate will dynamically compute a simple FITS WCS for the extracted image and inserts
these keywords into the returned FITS header that is useable by most image analysis tools.
EUVE Extracted Files
NED Search Interface at MAST
During a lively discussion at the April MAST Users Group, a new search interface was conceived. This interface permits a user to search the NASA/IPAC Extragalactic Database (NED) and cross correlate the results of the NED search with observations from the MAST archive. The interface is now available at http://archive.stsci.edu/ned.php. Like the VizieR search tool (http://archive.stsci.edu/vizier.php) the NED interface uses Virtual Observatory (VO) web services to submit parameters from the MAST search form to NED. The returned results can then be displayed in a standard MAST search results page. The interface utilizes NED's "Near Position" and "Advanced All-sky" search scripts, which are both accessible at http://nedwww.ipac.caltech.edu. Current search parameters include target name, coordinates, search radius, redshift constraints, and "included" and "excluded" object classifications. Alternate output formats are also available including VOTable, Excel spreadsheet, and comma-separated values (CSV).
HST Preview ImprovementsHST preview images and spectra have undergone some improvements.
The CADC, which creates the previews, has increased the sizes of preview images. WFPC2 images, formerly mosaicked, binned and converted to integer, are now actual-size mosaics. (The datatype of the pixels is still integer.) The new preview images include a full set of header keywords and are cosmic-ray rejected. Older previews have not been redone in the larger size.
We've added some new stretches (which for ACS will be automatically selected) which eliminates
a percentage of the brightest and darkest pixels.
These improve ACS preview GIF images in particular.
You may wish to view the preview with a different pixel correction. To do this, click on the link "More preview format options" just under the initial preview image. You may then choose from a number of pixel correction options. You may also choose to see the full size version of the gif file.
A User/MAST Collaboration - HST Pointings Interface and Table ImprovementsFor a number of years, MAST has maintained a set of tables that summarize available data for regions of the sky as well as an interface that allows users to query these tables. We called this the "HST Pointings Interface". Early last winter, Phil Marshall, who wanted to use the pointings interface to select data to be retrieved for the HAGGLeS archival program wrote to us. The pointings interface and summaries we had developed were almost what he needed! He had some specific needs and suggestions on how to make the pointings interface more useful in general. The team was looking for sky regions that had more than 2000 seconds of total exposure time and that had data in two or more bands (not filters). The team was doing reiterative searches and needed a constant reference (pointing number). He wanted some additional meta-data returned in the search results and he wanted to be able to search via pointing but have all the exposures displayed in a single return. He also noted that it seemed that the algorithm we were using to define regions was perhaps too conservative and thus defined as separate regions, sky areas that could actually be combined. We reviewed his suggestions and decided to implement several of them.
The algorithm for selecting the observations to be grouped together was made more selective, resulting in fewer "regions". The selection processes was redefined so that once a sky region has been defined, it maintains the arbitrary "pointing number" assigned to it. As new observations are acquired they are assigned to an existing sky region when possible or used to define a new region if appropriate. Now separate pointings are maintained for different instrument modes (e.g. ACS-WFC and ACS-HRC) while previously they were combined.
The interface was ported to our standard format. You may now search for pointings, or for the exposures assigned to those pointings. The new interface allows you to select the instrument mode and the output format (html, comma separated value (CSV), VOTable or Excel).
Check out the new HST Pointings Interface and answer such questions as How many high galactic latitude observations exist for RA between 9 and 18 hours with dec > -20 where there have been at least two observations and where at least one of those observations was in the I-band, and at least one of those observations was in any other filter?
Set these parameters in the interface:
You may also access the pointings interface as a web service running the query from a GET request. For more information about running MAST queries as a web service see the web page http://archive.stsci.edu/vo/mast_services.html. You may also send mail to firstname.lastname@example.org.
For example, you may want to search the ACS Pointings table for the target "M 81", output format = "CSV",
and more than 11 exposures in the U band:
If you search the pointings for exposures, you will be able to make quite large requests. PLEASE be careful not to exceed the DADS operational capacity. Please read the information on constraints for making large requests from DADS.
MAST Website Navigation ChangesThis month we modified some of the links in the top navigation bar to improve site navigation. These changes are in response to some suggestions received from the MAST User Survey asking that better navigation to the HST search form be provided from the main MAST page. The "missions" pulldown menu has in the past been used to provide links to the main mission pages; although these were also provided in the mission links in the lower right navigation section of the top MAST page. The "missions" pulldown is now labeled "Mission Search" and contains links to the main mission search forms. This allows you to access search forms directly from the main MAST page. Since this navigation bar also appears on all search forms, users can now move from one search form to another with one click. In a parallel change, the "Searches-Tools" link is now labeled "Tools", and contains links to programs and search forms which are not specific to any one mission. For example, the link to the new NED search form is now here. A new link called "Abstracts" was added to help users find proposal abstract search forms. Since the links in the pulldown menus contain only the main MAST search forms, both "Tools" and "Mission Search" now have a "more" link which links to a page that displays all the available MAST search forms. This page contains links to all seven HST-related search forms, and all seven GALEX search forms. To suggest further ways to improve access to the MAST web site, please send comments to email@example.com.
MAST Virtual Observatory (VO) ActivitiesBesides offering new interfaces using Virtual Observatory (VO) services (see article on new NED interface), work at MAST in support of the VO has progressed in mainly three areas: the spectral container project, documentation, and validation of existing services. Much of this work is transparent to the user.
Unlike image data, projects have historically archived spectral data in a variety of FITS file formats. Wavelengths, fluxes, and sigmas have been stored in separate files, as non-homogeneous primary arrays and in a variety of binary table formats using scalar, vector, or variable length array fields. In addition, the metadata stored in the FITS headers varies considerably between missions with common parameters stored using a variety of keyword names. VO application developers (for example the people developing the spectral browser tools) would have to write mission-specific code to extract data from each type of file.
As a result of these problems and following the FITS serializations recommended in the VO Spectral Data Model paper, files were created using a FITS binary table format containing fixed length vector fields with a core set of standard keywords defined in the binary table extension header (the original mission-defined primary header keywords were preserved). This format has been used to generate new files for the following missions (special modifications, if any, are described on a separate page. Work is also progressing on defining a set of standard keywords that will use consistent names and definitions for all missions.
Documentation: We are in the process of updating our web page describing available MAST web services. The page has been expanded to include examples of how scripts can be written to access web services using different programming languages, and to provide more information on the parameters that may be specified in an HTTP GET request. See http://archive.stsci.edu/vo/mast_services.html
Validation of Existing Services:
A web site is now available from the National Center for Supercomputing Applications (NCSA)
for VO developers to validate their registered
Cone Search and Simple Image Access Protocol (SIAP) VO services.
The web site runs a series of tests using both valid and invalid search parameters,
then evaluates the returned VOTable files for errors.
Before this tool was available, SIAP and cone search services were based solely on
each data provider's interpretation of the VO standards.
As a result of running the validator on the registered MAST services,
changes were made to several MAST services to:
The plot to the left contrasts the holdings as of April 2006 with the total downloads from April 2005 through March 2006.
For almost every group of data, a value equal to the of the total volume of data was distributed during that past year, with the community contributed HLSP being the most popular group of data.
On June 13, 2006, DADS serviced its 100,000th request! Jay Anderson of Rice University submitted the request for HST/ACS data. Over the past year, users retrieved a total of about 29 Tbytes, an average of nearly 80 Gbytes/day. The typical retrieval time for HST data is usually under 1 hour, with on-the-fly-reprocessing (OTFR) requests taking a median time of about 50 minutes and non-OTFR requests taking about 10 minutes.
The MAST Users Group (MUG) met in April 2006. This year's members were Ann Zabludoff (Chair), Anton Koekemoer, Leonidas Moustakas, Jeff Newman, Sandyha Rao, Aki Roberge, and Richard Wade. The MUG provides us with important feedback from the user's perspective, helps to set priorities and suggests directions for future enhancements to the archive. You can see the presentation made at this year's meeting and at previous years at http://archive.stsci.edu/mug/.
The 2006 MAST User Survey drew a record 364 responses.
Thanks to all who participated for taking the time to give us feedback!
We have already implemented some changes in response to your comments.
For example, the algorithm used to create the display of ACS previews has been greatly improved so that the browser
images are much more useful. More changes will appear in the coming months.
The results are summarized at http://archive.stsci.edu/surveyresults/2006/survey_mar2006.html.
We welcome additional suggestions at any time using the MAST Suggestion Box
or through email to firstname.lastname@example.org.
How Can MAST Improve and Expand to Serve You Better?MAST is continually looking for ways to help you find the data you want and we would really like to hear your ideas. We have provided a "suggestion box" at http://archive.stsci.edu/suggestions.html and encourage you to post suggestions, comments and concerns through this interface. A link to the suggestion box is found on the second line of the top menu on most MAST web pages.
Some past users have allowed us to post their suggestions. Those suggestions and the MAST responses can be found at
Let us hear from you!
To Subscribe or Unsubscribe to the NewsletterIf you would like remove yourself from the mailing list send mail to email@example.com and put the single word unsubscribe in the BODY of the message. Information on the subject line will be ignored.
If you would like to subscribe to the mailing list to receive notification of future Archive Newsletters, send mail to firstname.lastname@example.org and put the single word subscribe in the BODY of the message. Information on the subject line will be ignored. You will be asked to confirm the subscription.