Follow Us
|
Data Retrieval Help Page
To retrieve IUE data, mark the rows in the search results page of the desired exposures (or click on the "more retrieval options" page and enter camera and image numbers in the list under "image IDs"). Once the desired files are marked, several options are available for formatting and compressing the data. Generally, MAST files are most efficiently (and quickly) downloaded as a bundled tar or zip file. This means however, that the user is left with the task of extracting the files. The descriptions below list the programs available on different operating systems for reading the various file formats.
Clicking the "Download NEWSIPS MX Files as a .tar file" from the IUE search results page will download a tar file of NEWSIPS FITS files. Users desiring a zip file, or a gzipped tar file, should click the "More retrieval options" button after selecting the desired files from the search results list.
Other options available from this page include downloading NEWSIPS, IUESIPS-GO, or IUESIPS-RDAF format files, selecting various NEWSIPS or IUESIPS file types, or requesting files by entering the IUE camera names and image numbers. Note the current retrieval options allow only one data type and one file format to be requested at a time and the two selected options must agree (e.g. don't select NEWSIPS files and an IUESIPS file type). If both IUESIPS and NEWSIPS files are desired, two separate file downloads are required.
- NEWSIPS - FITS files produced by the IUE New Spectral Image Processing System,
- IUESIPS_GO - IUE GO format files produced by the original IUE Spectral Image Processing System,
- IUESIPS_RDAF - IUESIPS-processed files in RDAF-format.
- .tar - uncompressed files bundled in a tar file. (Note, prior to 10/21/99, the individual files were also gzipped by default.)
- .tar.gz - files are bundled into a tar files which is then compressed using gzip.
- .zip - files are bundled (and compressed) into a zip file,
Users know when the request is completed when the browsers "Download window" disappears. Requests for a few files are normally completed in a couple of minutes.
It should be pointed out that the "time Left" gauge in the pop-up window will not accurately show the remaining download time. The file transfer speeds will eventually be displayed, but the download is basically not complete until the pop-up window disppears.
In addition to the requested files, users will also receive an ASCII text file called filetypes.txt which describes the various IUE data types, and, an ASCII text file called status.txt describing the status of the users data request. More information on status.txt is provided below.
UNIX
For unix users, the downloaded tar files can be extracted using the command:Gzipped tar files can be read the same way, after first running the command:
Requested ZIP files can be opened using unzip:
If FITS files were requested, the uncompressed files can be read with a standard FITS reader (although it must support binary table extensions). If IUESIPS GO format files were downloaded, they can either be read using custom software, or converted to rdaf format using the gotordaf routine from the IUEDAC IDL library. RDAF format files can also be requested directly from MAST.
VMS
Since gzipped, tar, and zip file formats are not as common on
VMS systems, users may first need to
download the software necessary to extract the downloaded files.
GUNZIP, TAR2VMS, and UNZIP are all public domain software available from DECUS. To
see if these programs are already available on your system, type:
Depending on the format you select for downloading the files, if the program you want is undefined, (and assuming it doesn't exist under another name) they can be downloaded as follows:
Download the TAR2VMS executable by clicking
on the appropriate OpenVMS system below:
tar2vms executable)
(15 blocks)
tar2vms executable)
(20 blocks)
Once you have downloaded the executable, type the following commands
(note the first command defining tar2vms could be added to your login.com file):
VAX:
or ALPHA:
where "disk" and "account" are the names of the disk and directory where the executable resides.(Note the "$" in front of the disk name!) The file "iue.tar" listed above is an example of the file name downloaded from MAST. The actual name will depend on the file format requested.
The gunzip program (as well as unzip for ZIPed files) can be dowloaded directly from
Digital.
Be sure to select the appropriate
executable for your system (i.e., *-vax for vaxes, *-axp for alphas).
To install gunzip
or unzip, follow the directions above for tar2vms. That is, define a
symbol for the command name (with the "$" added to the file path) and then just type
(for example):
(Note after downloading and unpacking RDAF format IUESIPS files, VMS users will need to convert their files with a program such as CON_RDAF (i.e., using the SPARC to VMS mode option).
The above commands will create a subdirectory called IUE containing all the requested files. NEWSIPS files will have names in the form swp12345.mxlo, while IUESIPS files will have names such as swp12345llg.dat & .lab, or swp12345.melol.
IUESIPS files can be requested from the "More Retrieval Options" page
in either GO format, or RDAF format.
After extracting, GO format files will have names such as swp12345.melol,
while RDAF format files will have names such as swp12345llg.dat
and swp12345llg.lab.
The main differences between the 2 formats is that the RDAF format stores
the VICAR label in a separate file from the data and, the data stored in the
RDAF .dat files are assumed to be converted to a format compatible with the
local operating system. Because of the latter difference,
VMS users planning to use the IUERDAF IDL software
should run the program con_rdaf on all rdaf-format files.
(i.e., using the SPARC to VAX mode option).
(Since the MAST archive is stored on a unix computer, no conversion
is needed when the files are downloaded to unix or Macintosh computers.)
Alternatively, users may request GO format files and convert
them to RDAF format using the IUERDAF program GOTORDAF.
For example, to convert the file described in the
example above from GO to RDAF format, simply type (from IDL):
This will produce a standard .dat and .lab file in the format appropriate for the host computer. (The RDAF IDL routines can be downloaded from the IUEDAC web page.)
Windows
If you are using a PC running Win3.x, Win95, or WinNT, you can request
the data in either tar or zip format and use the shareware program
WinZip
to both uncompress and extract the files. Extracting the tar or zip file
from WinZip
will create an IUE subdirectory containing
the images requested.
If you don't have WinZip, the programs
pkzip/pkunzip
should also work, but they have not been tested.
The tar.gz format can also be read on a pc, but in our experience with WinZip, the system renames the downloaded file to be iue_tar.gz (instead of the original iue.tar.gz), and therefore the file may need to be renamed to iue.tar to be recognized as a tar file after unzipping.
IUESIPS files can be requested from the "More Retrieval Options" page
in either GO format, or RDAF format.
After extracting, GO format files will have names such as swp12345.melol,
while RDAF format files will have names such as swp12345llg.dat
and swp12345llg.lab.
The main differences between the 2 formats is that the RDAF format stores
the VICAR label in a separate file from the data and, the data stored in the
RDAF .dat files are assumed to be converted to a format compatible with the
local operating system. Because of the latter difference,
Window users planning to use the IUERDAF IDL software
should run the program con_rdaf on all rdaf-format files.
(i.e., using the SPARC to MIPSEL mode option).
(Since the MAST archive is stored on a unix computer, no conversion
is needed when the files are downloaded to unix or Macintosh computers.)
Alternatively, users may request GO format files and convert
them to RDAF format using the IUERDAF program GOTORDAF.
For example, to convert the file described in the
VMS example above from GO to RDAF format, simply type (from IDL):
This will produce a standard .dat and .lab file in the format appropriate for the host computer. (The RDAF IDL routines can be downloaded from the IUEDAC web page.)
Macintosh
Mac users can now use
a free file-decompressor program called
MindExpander
from MindVision, to extract .zip and .gz files. Alternatively, the
standard (but not free-ware)
Stuffit Deluxe
utility can also be used. Other utilities include:
UnZip,
ZipIt,
and UnTar. Since most utilities only support zip or gz files,
Mac users may want to request zip format rather than
tar from the "more retrieval options" page. Tests run with Stuffit
Deluxe however, showed that tar files
can be read without any trouble.
IUE Data Types
The available IUE data types are displayed on the "More Retrieval Options" page and are
described below.
Normally users request the merged extracted spectral files (i.e., NEWSIPS MXLO or MXHI files, or
IUESIPS MELO or MEHI files). Additional documentation is available on both the
NEWSIPS and the (GO-format)
IUESIPS data
products.
NEWSIPS Files
- MX - The merged extracted spectral files containing the absolutely-calibrated fluxes, wavelengths, data quality flags, and sigmas (or noise estimate for MXHI files) in a FITS binary table extension. The high dispersion files are named MXHI and the low dispersion files are MXLO. Note that although low dispersion double aperture exposures are listed as two separate entries on the search results page, both spectra are contained in one MXLO file.
- EXTRACT - This group contains the MXLO, MXHI files described above, plus the resampled image files (SILO & SIHI) from which the fluxes were extracted. The SI files contain the photometrically-corrected and rotated image as a primary array, wavelength information in a binary table extension, and an image extension containing an array of data qualiy flags. These files are useful for identifying camera artifacts such as cosmic ray hits, etc.
- All-FA - All the archived NEWSIPS data products for a given observation can be requested by selecting this group. In addition to the files above, the RILO, RIHI raw image files (a 768x768 primary array image) and the LILO, LIHI linearized image files (a 768x768 primary array file with associated data quality flags as a 768x768 array image extension) will be downloaded.
- SI - Select this group if only the SILO, SIHI resampled image files are to be downloaded.
- RI - This group allows only the RILO, RIHI raw image files to be downloaded.
- LI - This group allows only the LILO, LIHI linearized image files to be downloaded.
IUESIPS Files
- ME - This group includes the merged extracted spectral files containing the absolutely-calibrated fluxes, wavelengths, and data quality flags in a project-defined data format based on VICAR. The high dispersion files are named MEHI and the low dispersion files are MELO.
- EXTRACT - This group contains the MELO, MEHI files described above, plus the low dispersion resampled line-by-line image (ELBL) files from which the fluxes were extracted (note no IUESIPS high dispersion resampled images were generated). The ELBL files contain the photometrically-corrected image data, wavelength information, and data qualiy flags. These files are particularly useful for identifying cosmic ray hits, reseau, microphonic noise, etc.
- ALL - All the IUESIPS archived data products for a given observation can be requested by selecting this group. In addition to the files above, the RAW image file (a 768x768 array) will be downloaded.
- ELBL - This group allows only the low dispersion resampled image (ELBL) files described above to be downloaded.
- RAW - This group allows only the RAW image files to be downloaded.
Status.txt File
The status.txt file is downloaded along with the users requested
files to provide information on the users request. This file may
be useful for explaining why an expected file was not received.
The following information is currently included
in these files:
- Observations requested - The number of camera name/image sequence number entries specified by the user.
- Requested data types - A list of the IUE data types (e.g., melo, sihi, etc.) to be retrieved from the MAST archive as specified by the user in the data type menus on the "more retrieval options" page, or as specified on the search results page if downloading MX files was requested,
- Compression type - The method used to compress/bundle the requested
files. The possible values are:
- tar - files will be bundled as a .tar file.
- zip - Files bundled and compressed as a zip file with no further compression applied to individual files.
- tar.gz - Files bundled as a gzipped tar file
- Data format - Describes whether NEWSIPS, IUESIPS-GO, or IUESIPS-RDAF format files were requested,
- File list - The list of downloaded files showing the (uncompressed) file name and the uncompressed file size in kilobytes. A message will appear next to any entries either "not found in the database" or, more rarely, "found in the database but not located on the jukebox". (Users should report any missing files to archive@stsci.edu.)
- Total number of found files - The total number of files found in the database that were requested by the user.
- Total file size - The total amount of disk space (in kilobytes) required to uncompress the requested files. Note that RDAF format IUESIPS file sizes are approximate.
 |
|
|
|
|
|
|
|
|
|