spacer link to MAST page spacer logo image spacer
 
link to STScI page


FUSE Tools in C

Introduction

This is a guide to the data analysis tools that are distributed along with the CalFUSE pipeline pagkage. CalFUSE (pipeline processing) and the C-tools use the same libraries of supporting routines and are packaged together. We begin with instructions for performing common tasks: combining data from multiple exposures of a bright target, combining data from multiple exposures of a faint target, extracting night-only data from day+night IDF files, and dividing an IDF file into multiple time segments. We conclude with descriptions of each program.

Chapter 8 of the FUSE Data Handbook, Retrieving and Analyzing FUSE Data, provides further description and examples of the appropriate use of these tools.

Combining Spectra from Bright Targets

For bright targets, the goal is to maximize spectral resolution, so it's important to align the spectra from individual exposures precisely before combining them. A procedure to cross-correlate and combine spectra uses the IDL routine FUSE_REGISTER, but it becomes clumsy if you have more than a few exposures to combine.

To combine a large number of spectra for a moderately bright target, you can use cf_xcorr to determine their relative offsets, then cf_combine to shift and combine them. 


  prompt> ls P1010101*1alif4ttagfcal.fit > list_of_files
  prompt> cf_xcorr list_of_files list_of_shifts
  prompt> cf_combine list_of_shifts output_file.fit

The two programs are described in detail below, but note that cf_xcorr works only on detector segments that span the 1000-1100 Å waveband. The shifts are the same for sides A and B of a given detector, so you can, for example, use the shifts computed from LiF 1A spectra align the corresponding LiF 1B files.

Note: Due to fixed-pattern noise in the detectors, it may not be possible to achieve a S/N ratio greater than 30 simply by increasing exposure time, no matter how many exposures you combine, unless the data were obtained in the FP split mode. See the FUSE Data Handbook for details.

Combining Spectra from Faint Targets

For faint targets, the goal is to maximize the fidelity of the background model by increasing the signal-to-noise ratio on unilluminated regions of the detector. To do this, one combines the IDF files before extracting the spectra. This is easily done with cf_edit, but the program runs slowly if you feed it too many data files. A faster alternative is idf_combine.

Step 1: Combine the IDF files. The files need not be listed in chronological order, but all must live in the current directory. 

  prompt> idf_combine combined_idf_file input_idf_file1 \
  prompt> input_idf_file2 input_idf_file3 ...

Step 2: CalFUSE v3.2 produces a bad-pixel mask (BPM file) for each exposure. If you've just run the pipeline, you should have the BPM files. If not, and you have the jitter and housekeeping files, you can create new BPM files easily: 

  prompt> cf_bad_pixels input_idf_file1
  prompt> cf_bad_pixels input_idf_file2
  prompt> cf_bad_pixels input_idf_file3

To combine the bad-pixel masks, use bpm_combine. File names and wavelength shifts are read from the header of the combined IDF file. All files must live in the current directory. 

  prompt> bpm_combine combined_bpm_file combined_idf_file

Step 3: Extract LiF and SiC spectra from the combined IDF file. To modify the wavelength range or binning of the resulting spectra, edit the parm*.fit files. 

  prompt> cf_extract_spectra combined_idf_file

Note: Because the extraction routine uses time-dependent effective-area and background calibration files, we do not recommend that you combine data taken more than a few months apart into a single IDF file. Instead, perform the steps listed above for each visit separately, then combine the resulting extracted spectral files as described above.

Modifying Wavelengths within an IDF

If you are in the position of having multiple exposures of a faint target, but knowing their relative wavelength offsets with exquisite precision, you can optimize both spectral resolution and background subtraction by shifting the spectra within the individual IDFs before combining them. For example, you can extract the individual spectra, determine the appropriate shifts for each, apply them to the IDF files, then run idf_combine. John Grimes has written a tool to shift spectra within an IDF. See the FUSE IDL Tools reference page. If you use John's tool and wish to combine your BPM files, you will have to modify the WOFFL* and WOFFS* keywords in the combined IDF by hand. These keywords, which list the shift in Ångstroms applied to the target spectra in the LiF and SiC apertures, are readily modified using the program modhead. See the instructions below.

Extracting Night-Only Data from an IDF

The CalFUSE pipeline does not discard undesirable photon events, but merely flags them as bad. This makes it possible for the user to change the screening applied to the data without re-running the pipeline. An interactive tool for doing this is the IDL routine cf_edit, but the C program idf_screen provides the same capabilities and runs considerably faster. To select only night-time events, type 

  prompt> idf_screen M10101010011attagfidf.fit \
	   M10101010011a_night_ttagfidf.fit DAY GOOD

Here, "DAY" refers to the day/night flag, and "GOOD" selects only the night-time photons. A complete description of the program is provided in the IDF Cookbook.

Note: Strictly speaking, if you modify the good times in an IDF file, then you should generate a new bad-pixel mask with the same good times. To do that, use cf_bad_pixels, described above. If you don't care about correcting your spectra for bad pixels, you can skip this step. The program cf_extract_spectra will complain that it can't find the bad-pixel file, but it will run anyway.

Dividing an IDF into Multiple Time Segments

The program idf_cut allows you to chop an IDF file into several smaller files: 

  > idf_cut idf_file RefTime Period Nout

Designed for time-resolved spectroscopy, the program sorts the records of the input file according to their phase, but you can turn this off by using a reference time of zero and a period that exceeds the total exposure time. For example, if EXPEND - EXPSTART = 1900 s, set the period to 2000 s. To slice such a file into four 500-second chunks, the command is 

  > idf_cut idf_file 0 2000 4

Complete instructions are provided in the IDF Cookbook.

Descriptions of Individual Programs 

bpm_combine combined_bpm.fit combined_idf.fit

Combines bad-pixels masks from multiple exposures into a single file. For details, see the IDF Cookbook. 

cf_arith file1.fit - file2.fit difference.fit

Performs a variety of arithmetic operations on two CalFUSE calibrated spectrum files. The second operand can be either a file or a numeric quantity. Supported operations are +,  -,  *,  /,  shift, boxcar, bin, avg_t, avg_s. If the second operand is a scalar, the operation is performed only on the FLUX column; otherwise, it is performed on the FLUX, COUNTS and WEIGHTS columns, and the statistical errors are propagated. The shift operation shifts the contents of the columns in the file by the number of pixels specifed by the second operand relative to the WAVE column (which is not shifted), the boxcar operation performs a boxcar smoothing of the flux and counts columns with a width specified by the second operand, and the bin operation bins the data by the number of pixels specified by operand 2. The avg_t and avg_s operations average two spectra: avg_t weights the spectra by their respective exposures times, and avg_s weights each pixel by its statistical weight.

Note that if you use the multiply operation you MUST enclose the asterisk in quotes ("*"); otherwise, the shell will replace the asterisk with a listing of all the files in the current directory before passing the command line to the program!

A more detailed description of the operation of cf_arith is given in the header of the source code.

Examples:

# shift spectrum 4 pixels; positive shift moves spectrum to higher pixel numbers; WAVE column is NOT shifted.
cf_arith P11122330011blif2ttagfcal.fit shift 4 P11122330011blif2ttagshft .fit

# smooth spectrum with a 6-pixel boxcar:
cf_arith P11122330011blif2ttagfcal.fit boxcar 6 P11122330011blif2ttagsm6 .fit

# bin spectrum by 4 pixels:
cf_arith P11122330011blif2ttagfcal.fit bin 4 P11122330011blif2ttagfcalb4 .fit

# subtract airglow template from a spectrum
cf_arith P11122330011blif2ttagfcal.fit - airglow_lif1b_day.fit P11122330011blif2_agsub.fit 

cf_combine [-ahk] [-v level] file_list output_file.fit
  Options:
    -h:  this help message
    -v:  verbosity level (=1; 0 is silent)
    -a:  ignore EXP_STAT keyword
    -k:  Change EXP keywords to OBS

Combines a list of calibrated spectral files into a single output spectral file. If the file list has a second column listing the requested shift (in pixels) for each input file, cf_combine will shift and sum the files all at once. Note that a positive shift moves a spectrum to longer wavelengths. Input files for which the header keyword EXP_STAT != 0 are not included in the sum unless the -a flag is set.

The header of the output file is copied from the first input file, except that the EXPTIME keyword is updated with the sum of the exposure times of the input files, and some HISTORY lines are added.  The WAVE column is copied from the first input file. The output FLUX column is the average of the fluxes in the input files, weighted by their exposure times, and the output ERROR column contains the propagated errors from the input files.  The WEIGHTS and COUNTS columns are summed. 

cf_xcorr input_file_list output_file_list

Computes shifts necessary to align multiple extracted spectral (*fcal.fit) files. The file with the greatest EXPTIME becomes the template. Each file is then compared with the template. We don't actually cross-correlate; instead, we shift the data file by -20, -19..., +19, +20 pixels and compute chi-squared between the data and template spectra. The wavelength range is hard-wired into the program: for point sources, it's 1045-1070 Å; for extended targets, it's 1030-1039 Å (the O VI doublet). If you prefer a different range of wavelengths, consider using get_shift, described below.

The output file contains six columns:
input file name
optimal shift
quality flag (-1 = bad, 0 = reference spectrum, 1 = good)
EXPTIME
EXPNIGHT
exposure number

Note: while this routine is not elegant, it does a good job on most targets, which is our definition of success for pipeline software. 

get_shift file1.fit file2.fit wave1 wave2

Returns the offset in pixels between two extracted spectral (*fcal.fit) files. The program performs a cross-correlation within the wavelength window defined by wave1 and wave2. Note that returned value is the offset of file 2 with respect to file 1. To align the two spectra, you must shift file 2 by the NEGATIVE of this value. 

idf_combine [-ahbc] [-v level] output_idf_file \
	input_idf_1 input_idf_2 input_idf_3 ...

Combines IDF files from multiple exposures into a single file. For details, see the IDF Cookbook. 

idf_cut [-hm] [-v level] idf_file RefTime Period Nout

Chop an IDF file into several smaller IDF files. The records of the input IDF file can be sorted according to their time phase. For details, see the IDF Cookbook. 

idf_screen [-h] [-v level] input_idf output_idf \
  timeflag value

Alters the screening performed by the pipeline by modifying the photon flags within an IDF file. For details, see the IDF Cookbook. 

mjd2hjd [-h] [-v level] mjd ra_h ra_m ra_s \
  dec_d dec_m dec_s

Given the modified Julian date and a target's RA and DEC, program computes the heliocentric modified Julian date. 

modhead filename.fit[ext] keyword new_value

Return or modify the value of a header keyword in any FITS file. The extension number is 0 for the primary header-data unit (HDU), 1 for the first extension, etc. If omitted, the primary HDU is assumed. If filename.fit[ext] returns an error (modhead: No match), put this string in quotation marks ("filename.fit[3]") or use an alternate construction (filename.fit+3).

If keyword is not found and new_value is specified, the keyword is added to the requested header. If new_value is not specified, the current value is returned. New_value may be a multi-word string, but modifications to the comment field of a keyword are not supported. 

remove_target_orbital_motion

If you have a TTAG observation of a source in a close binary system, you might be interested in removing the orbital motion that smears out the photospheric lines. For a circular orbit, you can use remove_target_orbital_motion. For details, see the IDF Cookbook. 

ttag_lightcurve
ttag_lightcurve_channel_sum
ttag_lightcurve_combine
ttag_lightcurve_mjd2hmjd
ttag_lightcurve_periodogram

These routines, written to facilitate time-resolved photometry of bright stars, are described in the IDF Cookbook. 

listhead any_fits_file.fit

This utility program produces an ASCII file containing a listing of the headers for each extension in the input FITS file. The output file is written to a file in the current directory. Note: listhead is provided as an executable, so runs only on Sun workstations.