next previous contents

SECTION 8 - GUEST OBSERVER DATA PACKAGE

8.1 OUTPUT PRODUCTS DECRIPTION

8.1.1 PHOTOWRITE HARDCOPY IMAGES

8.1.1.1 Processed Data

Every IUE image (excluding Fine Error Sensor (FES) images) which is treated by the image processing system results in a photographic hardcopy generated by an Optronics Photowrite film writing device. This device produces photographic representations in 256 discrete gray levels (i.e., in an 8-bit format) of the IUE two-dimensional spectral images on 8 x 10 inch sheet film, using as input magnetic tape files containing the images in digital form. The input magnetic tapes are generated by the Sigma-9 image processing computer and contain, in addition to the images themselves, ancillary information which is also transmitted to the photowrite film to facilitate the identification and interpretation of the images. This information includes an 8-step DN-labeled gray scale, a tick-marked border, and selected lines of the IUESIPS label associated with each image. The typical processed-data photowrite film sheet is arranged to display three images side by side at a scale of 100 microns per pixel: the raw image, the raw image with a wavelength-scale overlay showing the location of the registered dispersion relation, and an intensity-scaled version of the photometrically corrected image. These films are contact prints of master negatives generated directly by the Optronics equipment.

The image with the wavelength scale overlay is particularly significant in that it provides a permanent record of the placement of the extraction slit used to compute the spectral intensity. The extraction slit follows each order, centered on the dispersion relation marked by the overlay. If for any reason the dispersion relations do not adequately track the orders (for example, because of bad geometric compensation or improper registration), the extracted intensities will be inaccurate. The overlaid image will make such circumstances immediately apparent. Note that the wavelength scale overlay is not corrected for velocity effects or vacuum-to-air conversion, as mentioned in section 7.1.4. Such corrections are applied only to extracted spectral fluxes.

The photometrically corrected image is presented in a scaled form because the 8-bit photowrite format will not accept the 16-bit photometrically corrected pixel values. The portion of the image outside the region of photometric correction is left as is, i.e., raw DN from 0 to 255. The scaling of the photometrically corrected portion of the image is done by first adding 2000 to the 16-bit FN values to move most of any negatively extrapolated FN values (Section 5.3.2.2.1 ) into the positive range, and then dividing by 70, so that most scaled pixels will fall in the 0 to 255 range. Sixteen-bit numbers larger than 17850 are scaled to the value 255, and 16-bit values less than 0 are scaled to the value 0, although the offset of 2000 mentioned above should suppress most negatives.

8.1.1.2 Raw Data

In addition to the photowrite hardcopy contact prints generated for processed data, a single large-scale positive film of every raw image (excluding FES images) available on a telescope operations archive tape will be produced before any computer processing is performed. This photowrite hardcopy image is produced for quick delivery to the Guest Observer and is an original rather than a contact print. Because it is generated prior to computer processing, this image contains none of the ancillary data such as gray scales, label data, etc., described above. Since it is generated at twice the scale (200 microns per pixel) of the processed-data photowrites, it is particularly suitable for detecting image flaws such as radiation events, "hot pixels," cosmic ray tracks, reseau marks, etc., which can affect the extracted spectral intensities. The Guest Observer is advised to examine these images carefully to avoid misinterpretation of the data.

8.1.1.3 Photometric Considerations

It cannot be too strongly emphasized that photowrite hardcopy images are in fact merely convenient visual representations of the digital data. They are by no means intended to support photometric measurements, although reasonable care is taken to provide a stable, useful output product. Thompson, Turnrose, and Bohlin (1982b) have discussed this situation in some detail and have described the methods used to determine, implement and maintain the overall system response function relating input 8-bit digital exposure values to the diffuse photographic densities on film. Figure 8-1 illustrates the standard desired system response function. The typical 1s tolerance level for adherence to the standard is ±0.15 photographic density units. On the processed-data photowrite films, the 8-level labeled grey scale can be used to monitor the effective response function actually achieved.

Figure 8-1: Photowrite System Response Function (Desired Photographic Density vs. DN).


8.1.2 CALCOMP PLOTS

At the option of the Guest Observer, extracted IUE spectral fluxes are plotted as a function of wavelength on 10-inch full scale, continuous-roll, gridded paper by an offline CalComp plotter. The Guest Observer must indicate in the "PROCESSING SPECIFICATIONS" section of the observing script that plots are desired, otherwise by default no plots will be generated. The plots provide a convenient representation of the extracted spectra, but they are not intended to be the definitive data display medium, because of the inherent limitations discussed below. The primary source of reduced data remains the magnetic tape files given to each Guest Observer.

8.1.2.1 Scaling of Plots

Although the horizontal scale (wavelength axis) of the plots is fixed as described below, the vertical scale (flux axis) is determined by the software according to the data being plotted. The vertical scale is chosen so that the maximum unsaturated flux point is at least one half of the 10-inch full scale. By observer option, SWP plots may be treated in such a way that the region around 1216 Å (Lyman alpha ) is ignored when the flux scale is set. This option is particularly useful for faint, long exposures dominated by geocoronal emission at Ly alpha .

8.1.2.2 Low Dispersion Plots

Low dispersion spectra are plotted with a horizontal scale of 50 Å/inch. Three distinct and separately scaled plots are provided: (1) the gross, background, and filtered background spectra plotted on a common axis, (2) the absolutely calibrated net spectrum (gross minus filtered background, multiplied by the inverse sensitivity function (see Section 7.2.4.2)), and (3) the common logarithm of the absolutely calibrated net spectrum.

Exceptional flux points (i.e., those affected by saturation, extrapolation, reseaux, microphonics, etc.) are flagged by special plot symbols explained in a key which is placed on each plot. As described in Section 7.3, such flagging is suppressed in plots of the filtered background fluxes.

Each plot is identified by an excerpt from the IUESIPS label for the spectrum. Whenever possible, such information has been kept to a minimum to reduce plotting time. In particular, only the second plot (net spectrum) displays all of the image processing history records added to the image label; the first and third plots carry abbreviated headers.

8.1.2.3 High Dispersion Plots

In the high dispersion case, two distinct and separately scaled plots are provided: (1) the gross and smoothed background spectra, plotted on a common axis at 10 Å/inch; and (2) the net spectrum (gross minus smoothed background), after correction for the echelle ripple function, plotted at 2 Å/inch. In creating the first plot at 10 Å/inch, a two-point running average filter is used; i.e., each plotted point represents the mean of two adjacent extracted flux points. This filtering is not needed for the second plot, done at 2 Å/inch. In the second plot, the wavelength range for each order is restricted as described in Section 7.1.4.

As in low dispersion, exceptional points are plotted with special symbols (except on smoothed backgrounds) and IUESIPS label excerpts are used to identify the plots.

8.1.2.4 Plot Accuracy and Registration

Guest Observers are reminded that the CalComp plots are of only finite accuracy. The plotter hardware is limited to a precision of approximately ±0.5 percent, which means that scale or registration errors of one small grid unit (0.05 inch) could be expected over distances of 10 inches. This tolerance is the limit of precision guaranteed in all delivered plots, although most plots will exceed this precision. As an aid to the Guest Observer in assessing the internal registration precision of each plot, special registration symbols are plotted before and after the data are plotted, as follows. Before the axes are drawn, a "+" symbol is plotted just above and to the left of the origin; after the horizontal axis is drawn, and before the paper is rewound to begin the data plotting, a second "+" symbol is plotted just above and slightly beyond the end of the horizontal axis. When the paper is rewound to the origin to start the plotting of flux points, an "x" symbol is overplotted at the location of the first "+" mark plotted. As long as registration has not been lost since the start of the axis plotting (the first "+"), an asterisk symbol will result from the overplot; lost registration will result in a symbol without a unique intersection point (e.g., "*"). Similarly, a second "x" overplot is made at the location of the second "+" mark after all the flux points have been plotted. As before, an asterisk symbol signifies good registration between the axes and plotted data.

8.1.3 MAGNETIC TAPES

Each Guest Observer is given magnetic tapes (GO tapes) containing the raw and reduced data for each of his or her images. These tapes are in 800 bytes per inch (bpi), 9-track, odd parity NRZ (No Return to Zero) format. The various files associated with each image represent the most complete rendering of the data delivered to the Guest Observer and are intended to be the primary data source for further astronomical analysis.

Each GO tape begins with a tape header file identifying the GSFC tape inventory number. This is used for internal accounting purposes and may generally be ignored by Guest Observers, although its format is given in Section 8.2 along with the formats of all other GO tape files. In the following subsections the file types and sequences which normally appear on GO tapes are explained.

8.1.3.1 Low Dispersion Files

For single-aperture extractions, the sequence of four files written to tape is listed below. The associated mnemonic abreviation identifying each file type used in tape-contents listings (Section 8.1.4) is given parenthetically.
  1. Raw image (RI)
  2. Photometrically corrected image (PI)
  3. Line-by-line spectra (LBLS)
  4. Merged extracted spectra (MELO)
In the case of double-aperture extractions, six files are written to tape. Note that since the inception of the new software, the large-aperture files have preceded the small-aperture files in the standard sequence.
  1. RI
  2. PI
  3. LBLS (large aperture)
  4. MELO ( " " )
  5. LBLS (small aperture)
  6. MELO ( " " )

8.1.3.2 High Dispersion Files

The sequence of three files is as follows:
  1. RI
  2. PI
  3. Merged extracted spectra (MEHI)

8.1.3.3 Other Files

For images from which neither high nor low dispersion spectra are extracted, the file(s) written to tape differ from those listed in the above subsections, according to data type. Images designated by the GO as "Do Not Process" are copied to tape in raw form (RI), as are Fine Error Sensor Images (FES) which are archived at the time of observation. Floodlamp calibration exposures from which reseau positions are determined are copied in raw form (RI) and also give rise to an associated file of derived reseau positions (RES).

8.1.4 COMPUTER LISTINGS

Each GO tape is accompanied by a computer-generated listing of the IUESIPS label of each file on the tape and an overall tape-contents summary. Such listings are called "labelprints" and are generated by reading the entire GO tape on a drive other than the drive which wrote the tape. This process not only verifies the logical contents of the tape but also certifies the physical acceptability of the tape.

8.1.4.1 Label Displays

Details of the format and structure of the IUESIPS labels, which contain much documentary information pertinent to the scientific instrument status, data acquisition, and data processing, are contained in Section 9. Here we point out only that the IUESIPS labels are written partly in EBCDIC characters and partly in binary-integer format. In the labelprint listings, the labels are interpreted as if all data were in EBCDIC format, so that those portions of the label actually in binary format are interpreted as either nonsense or undefined characters.

Since the information in lines 1-100 of the labels of files associated with a given image is identical, the complete label data are listed only for the first file (RI) of the sequence of files for that image. For successive files associated with that image, lines 6-100 are suppressed, and only lines 1-5 and the file-unique processing-history records in lines 101 et. seq. are printed. Note that this abbreviated format applies only to the printed listings; all label lines are physically present on the magnetic tape.

8.1.4.2 Tape Contents Summary

At the end of the label listings, the labelprint concludes with a tape contents summary sheet which serves to recap the file identifications and verify the physical error status of each record on the tape. Figure 8-2 illustrates a typical tape summary sheet, and Table 8-1 provides an explanatory key. The files pertinent to a particular image are enumerated in consecutive lines, with a space left between images. The information on the number of 360-byte label records may be of particular convenience to users with certain hardware configurations. The error count must be zero on all tapes released to GOs.

Figure 8-2: Tape Contents Summary Sheet


Table 8-1: Key to Figure 8-2
Field No.Contents
1The number of physical records in the label portion of the file
2The length in bytes of each physical record in the label. Any number not equal to 360 is indicative of an error condition.
3The number of physical records in the data portion of the file
4The length in bytes of each physical record in the data portion of the file
5The number of irrecoverable read errors encountered within each file as the tape was read back to produce labelprint
6Normal tape termination messages


8.2 Magnetic Tape File Formats

As is discussed in other sections, each output file generated by IUESIPS consists of a set of label records followed by a set of data records. Schematically, a GO tape consists of series of files as shown in Figure 8-3.

Figure 8-3: Schematic GO Tape Structure


For the ith file, the number of label records, Ni, the number of data records, Mi, and the length in bytes of each data record, Bi, all depend on the type of data in the file. The detailed label and data formats for each type of file which might appear on GO tapes are given in Sections 8.2.1 and 8.2.2.

8.2.1 LABEL RECORDS

8.2.1.1 Normal Labels

Normal IUESIPS file labels consist of between 20 and 30 physical tape records, although the maximum allowable number of records is 42. Each physical record is 360 bytes in length (one byte = 8 binary bits), being a concatenation of 5 72-byte logical records. A 72-byte logical record corresponds to one line in the labelprint listing described in Section 8.1.4. Thus, lines in the image label are blocked 5 at a time to form 360-byte physical records (blocks) on a tape.

Raw image labels are 20 physical records (blocks) long. As the image proceeds through the processing system, additional label information is appended. Since the information added at any given step may or may not fill one or more entire block(s), a continuation character at the end of each logical record is used to flag the end of the label as follows. If any logical record is followed by at least one other, the EBCDIC character "C" is placed in byte no. 72 of that logical record to signify a continuation. The last logical record of the whole label contains the EBCDIC character "L" in byte no. 72. Note that the end-of-label flag need not occur on a block boundary; any logical records which appear after the "L" in the last block are undefined (they generally contain core garbage). The overall label-record structure is shown in Figure 8-4.

As explained in Section 8.1.4, the label records are in a mixture of EBCDIC and binary formats. Observers using computers with non-EBCDIC characters (e.g., ASCII) are reminded that a character format conversion will be required to display the EBCDIC label portions correctly.

Figure 8-4: Standard IUESIPS Label Record Structure


8.2.1.2 Nonstandard Labels

Two special types of files have nonstandard labels. (1) The Tape Header file which begins each GO tape has only 1 block, of which only the first logical record is filled; see Section 9. All information is in EBCDIC format. The Tape Header File label has no "size parameters" field (see Section 9) as all other labels do. (Note further that the Tape Header File has no data records.) (2) Reseau-position files generated from calibration images have only several blocks in their labels. Unlike the Tape Header file labels, reseau-position labels do have the standard size parameters in the first logical record, and generally several other logical records are presented containing identification information for the source of the reseau data.

8.2.2 DATA RECORDS

The length, number, and format of the records in the data portion of each file depend on the file type and are described below. In all files, the data records are unblocked, i.e., logical record = physical record. Except for reseau-position files, all entries are binary integer quantities, in either one-byte or two-byte format.

8.2.2.1 Image Files

8.2.2.1.1 Raw Images

There are 768 physical records, each containing the pixel values along one scan line of the image. Each record is 768 bytes long; each pixel value is represented by one 8-bit byte (range 0-255). See Figure 8-5(a).

8.2.2.1.2 Photometrically Corrected Images

There are 768 physical records, each containing the pixel values along one scan line of the image. Each record is 1536 bytes long; each pixel value is represented by two 8-bit bytes, i.e., one halfword binary integer (range +/- 32767, with negatives represented in two's-complement form). See Figure 8-5(b); see also Section 5.3.3 for information on the coding of the halfword values.

Figure 8-5: (a) Data Record Structure for Raw Image File (RI)
(b) Data Record Structure for Photometrically Corrected Image File (PI)

8.2.2.2 Extracted-Spectral Files

The extracted spectral data are presented in a scaled-integer form. The number of records depends on the dispersion and file type (LBLS, MELO, MEHI), although the overall format is common to all three file types.

Figure 8-6: Data Record Structure for Spatially Resolved Low Dispersion Spectral File (LBLS)

Table 8-2: Format of Scale Factor Record (Record Sequence Number Zero)
Item (16-bit halfword)Quantity
1 * Zero (for record 0)
2 * 1022 (Maximum number of halfword entries in remainder of record 0)
3 * Minimum wavelength (truncated to nearest Å)
4 * Maximum wavelength (rounded to nearest Å
5 * Number of orders present
6 * Camera number
7 * Image number
8 * Number of records per group (i.e. per order)
9 Yearof midpoint of observation (GMT)
10 Day Number
11 Hour
12 Min
13-16 Date as above for time of image processing (GMT)
17 Target aperture (1 = large, 2 = small)
18 Total line shift (pixels × 1000)
19 Total sample shift (pixels × 1000)
20 *** THDA × 10 (°C) used for reseau correction (normally at the time of read)
21 * Scaled minimum flux for Gross
22 * Scaled maximum flux for Gross
23 * J for Gross where actual FN j data on tape × J × 2-K
24 * K for Gross
25-28 * as in 21-24 for Background
29-32 * as in 21-24 for Net
33-36 * as in 21-24 for Absolute Net (Low) or Ripple Corrected Net (High)
37-41 * Spares
42-44 Min, sec, ms of exp in target aperture (not implemented)
45 Hours Right ascension of target
46 Minutes
47 Seconds × 10
48 Degrees Declination of target
49 Arc Minutes
50 Arc Seconds
51-53 ** Vx (earth), Vy (earth), Vz (earth) Velocity of earth in celestial coordinates (km s-1 × 10)
54-56 ** Vx (IUE), Vy (IUE), Vz (IUE) - same as 51-53 for IUE with respect to earth, at midpoint of exposure
57 ** Net velocity correction applied (km s-1 × 10)
58 Omega angle (degrees × 10) - (zero in high dispersion)
59 Wavelength scaling factor (=5 for low dispersion, = 500 for high dispersion where actual lambda = ( lambda on tape)/(scale factor) + lambda o
60 Background slit height Low dispersion only (pixels × 100).
61 Background distance from dispersion line
62 Dispersion constant shift mode (0 = no shift, 1 = auto shift, 2 = manual shift)
63 Bright spot removal threshold DN, for weak, long exposures (not implemented)
64 THDA × 10 for dispersion constant correction (normally at the time of the end of exposure)
65-70 * Spares
71-102 * For use of IUE Regional Data Analysis Facilities
103-202 * lambda o , offset wavelengths for each order
203-302 * m, order number for each order
303-402 * Number of extracted data points in each order
403-502 Slit height for each extracted order (pixels × 100)
503 Sign and first 4 digits after decimal of dispersion constant A1
504 Sign and second set of 4 digits after decimal of dispersion constant A1
505 Sign and third 4 digits after decimal of dispersion constant A1
506 Exponent (including sign) of dispersion constant A1 where:
A1 = [item(503)×10-4 + item(504)×10-8 + item(505)×10-12]×10**(item(506))
507-538 As above, for dispersion constants A2 through A9
539-574 As above, for dispersion constants B1 through B9
575-1024 Spares
* Existing quantity under old software.
** High dispersion only
*** Currently not used to correct reseau positions for the LWR or LWP cameras


Figure 8-7: Data Record Structure for Merged Low Dispersion Spectral File (MELO)



Figure 8-8: Data Record Structure for Merged High Dispersion Spectral File (MEHI)

8.2.2.3 Other Files

8.2.2.3.1 Fine Error Sensor (FES) Images

There are M physical records, each containing the FES pixel values along one scan line of the FES image. Each record is 2M bytes long; each pixel value is represented by two 8-bit bytes (range ± 32767, with negatives in two's- complement form). M is determined by the telescope operations procedures. See Figure 8-9.

8.2.2.3.2 Reseau-Position Data Sets

There are four physical records, each of which is 1400 bytes long. The data represented are the precise locations of reseaux output from the program FNDRES. A data set for camera n (1 = LWP, 2 = LWR, 3 = SWP, or 4 = SWR) has blanks in all records except the nth, which contains the line and sample coordinates, in pairs of real numbers for each reseau mark, beginning at the upper left of the camera faceplate and proceeding left to right by rows. Each coordinate is written as a positive fullword (4-byte) floating-point number (R*4). Since each line and sample coordinate pair thus occupies 8 bytes, the number of bytes containing meaningful data in the nth record is 8 times the number of reseau marks identified; remaining bytes out to 1400 are zeroed out. See Figure 8-10 where a camera 3 (SWP) reseau set is illustrated, assuming a total of N reseau marks are identified. Note that N is always less than or equal to 169.

Figure 8-9: Data Record Structure for Fine Error Sensor Image File (FES)

Figure 8-10: Data Record Structure for Reseau-Position Data Set File (RES)

next previous contents