Reducing ACSIS data

A quick guide – for all details see the Cookbook .

Contents

1 Data
2 Software
3 Raw Data
4 Quick Reduction
5 Data Products
6 Visualising Data
7 Configuration files
8 Polarization-separated reduction (Nāmakanui specific)
9 Combining Images
10 Other useful commands
- 10.1 Nāmakanui Receptors – fitsheaders
11 Conversion to CLASS format

Data

JCMT continuum data obtained with ACSIS are written to disk and available at the JCMT or via CADC.

Software

For information on downloading and installing Starlink suite of data reduction and analysis software click here.

Raw Data

Raw JCMT heterodyne data obtained with ACSIS data files follow a fixed naming convention. For instance, within the file name “a20070420_00014_01_0001.sdf“, the sub-sections have the following meaning:

“a”: indicates data has been obtained by ACSIS.
“20070420”: Indicates the UT date on which the observation was take.
“00014″: An index that uniquely identifies the observation within the night in question. So for instance, this file holds data for the 14th observation taken on the 20th April 2007.
“01”: instruments with two sidebands and/or observations using multiple sub-systems this number identifies the sideband and/or sub-system.
“0001“: The sub scan index i.e. if the are too much data to fit into one raw file.
“.sdf”: Indicates the file holds data in the STARLINK NDF format.

* for `Ū`ū, a 2SB instrument you get a minimum of two files – one containing LSB data and another containing USB data. HARP is a SSB instrument so there is only one sideband.

Quick Reduction

To invoke the ORAC-DR software:

oracdr_acsis

This sets up ORAC-DR to run in your current working directory. You will also need to specify where your raw data is stored. For tcsh users we do:

setenv ORAC_DATA_IN folder/

or for bash:

 export ORAC_DATA_IN=folder/

For help with ORAC-DR simply do:

oracdr -help

To reduce your raw Heterodyne data using the ORAC-DR software run:

 oracdr -loop file -file mylist.lis

where mylist.lis is a list of raw files (either for a specific wavelength, single/multiple observations/sub arrays) that should be located in the “folder” directory as specified by ORAC_DATA_IN. the raw data will be automatically reduced using a per-defined reduction recipe.

oracdr -loop file -file mylist.lis -nodisplay -log sf

The above is another way of running the ORAC-DR pipeline. In the above example the output will be written to screen and logged in a .oracdr_* file (as specified by -log sf s = screen f = file) and not displayed in an xwindow (as specified -nodisplay).

To really understand what is happening to your data it is advised to also (for the first few times/reductions) run with -verbose. This will print messages from the Starlink engines (rather than just ORAC-DR messages):

oracdr -loop file -file mylist.lis -nodisplay -log sf -verbose

If you have HARP data and one or two receptors are bad it is possible to exclude them (i.e H02 and H08 in this example) from a reduction by using a bad receptors mask by running:

oracdr -loop file -file mylist.lis -nodisplay -log sf -verbose -calib bad_receptors=\"H02:H08\"

Please note sometimes this will mean you are required to update the default Quality Assurance parameters as fewer pixels will remain in your map.

Data Products

Data that have been reduced by the pipeline have T_sys and RMS for all receptors calculated, the pipeline also find and fits baselines, locates regions of emission using Clumpfind, creates moments maps, velocity maps and integrated intensity images. The output of the pipeline includes the following:

.oracdr_*.log – The ORAC-DR log file
log.group – The files contributing to each group
a20140103_00043_01_cube001.sdf – baselined cube
a20140103_00043_01_integ.sdf – Integrated intensity image
a20140103_00043_01_rimg*.sdf – Representative image (same as integ file), used to form rimg PNG
a20140103_00043_01_sp001.sdf – Spectrum taken from position of peak intensity in the integ file
a20140103_00043_01_rsp*.sdf – Representative spectrum (same as sp001), used to form rsp PNG
a20140103_00043_01_iwc.sdf – Intensity weighted co-ordinate image
a20140103_00043_01_noise.sdf – Noise map
a20140103_00043_01_reduced001.sdf – Final trimmed, baselined cube of the 1st (of n) file.
a20140103_00043_01_rmslo.sdf – Low-frequency noise
a20140103_00043_01_rmshi.sdf – High-frequency noise
log.qa – Quality assurance reports
log.noisestats – Noise statistics for each observation and group

Files beginning with prefix a are individual reductions of raw observations, the a simply identifies the data as being an ACSIS obervation. Files beginning with the prefix ga are coadded reductions of a number of observation (as determined by the files included in mylist.lis). The group files produced are as outlined above with the difference of the ga prefix and the omission of leading zero’s.

ga20140103_43_1_reduced001.sdf – Combined baselined cube
ga20140103_43_1_integ.sdf – Combined integrated intensity image

For larger cubes the reduced file is split into several subcubes with the numbers 001, 002, …, which can be combined using the KAPPA command PASTE.

paste g*.sdf out=fullmap

or then they can be combined into a single co-added map with the PICARD recipe MOSAIC_JCMT_IMAGES:

picard -log sf MOSAIC_JCMT_IMAGES *files.sdf

Visualising Data

Images can be viewed using GAIA (cubes; GAIA-3D)– in interactive image and cube visualisation tool created by the STARLINK project for viewing NDFs and FITS files.

gaia ga20140103_00043_01_integ.sdf

For greater versatility, the STARLINK KAPPA package contains many commands for visualising images in many different ways, including displaying multiple pictures in a grid, overlaying masks and contours, scatter plots, histograms, etc, etc.

Data can also be viewed in SPLAT (single spectra):

splat a20140103_00043_01_sp001.sdf

Configuration files

ORAC-DR is a pipeline that calls several STARLINK commands in order to reduce your data. the precise behavior of the pipeline is governed by the recipe file. You can find out which recipe is set in the data header via the FITS header RECIPE keyword in any of your raw files. For example both of these options will return the same result:

>> kappa

>> fitsval ga20140103_43_1_reduced001.sdf RECIPE

>> fitslist ga20140103_43_1_reduced001.sdf | grep RECIPE

It is possible to run with a different recipe (i.e. REDUCE_SCIENCE_GRADIENT). Links to specific reductions can be found in Appendix B – Main Recipes and are executed by entering the following:

>> oracdr -loop file -files myfiles.list REDUCE_SCIENCE_GRADIENT

You can tailor the recipe parameters by supplying a .ini file (called myparams.ini in the following example). This file contains the recipe name (which must match the one assigned to your data, whether from the header or any different one specified on the command line) followed by the options you wish to specify in the following format:

[REDUCE_SCIENCE_NARROWLINE]
MOMENTS_LOWER_VELOCITY = -30.0
MOMENTS_UPPER_VELOCITY = 155.0 
PIXEL_SCALE = 6.0,10.0
SPREAD_METHOD = gauss
SPREAD_WIDTH = 9
SPREAD_FWHM_OR_ZERO = 6
REBIN = 2

For more information about running ORAC-DR click here.

Polarization-separated reduction (Nāmakanui specific)

The Nāmakanui instruments produce data from both sidebands and polarizations. It is strongly recommended by the observatory that users check the data for each polarisation separately before combining them to use the total intensity spectra. To do this, you can make use of the bad receptors mask, running: (instructions are for a bash-type shell)

# Invoke the ORAC-DR software
oracdr_acsis

# reduce only P0 data
mkdir p0
ORAC_DATA_IN=$(pwd)
ORAC_DATA_OUT=$(pwd)/p0
# define P1 as "bad receptor" 
# in the line below, NU** is for Uu data, NW** is for Aweoweo, and NA** is for Alaihi;
echo "NW1L NW1U NU1L NU1U NA1L NA1U" >> $ORAC_DATA_OUT/bad_receptors.lis
# call the reduction pipeline
oracdr -loop file -batch -file "$ORAC_DATA_IN"/filelist.lis -nodisplay -log sf -verbose -calib bad_receptors=FILE

where the filelist.lis is a text file containing raw data files to be reduced (see Section “Quick Reduction” above for details on this and the other parameters). Similarly, to reduce P1:

# reduce only P1
mkdir p1
export ORAC_DATA_IN=$(pwd)
export ORAC_DATA_OUT=$(pwd)/p1
# define P0 as "bad receptor"
# in the line below, NU** is for Uu data, NW** is for Aweoweo, and NA** is for Alaihi; 
echo "NW0L NW0U NU0L NU0L NA0L NA0U" >> $ORAC_DATA_OUT/bad_receptors.lis
# call the reduction pipeline
oracdr -loop file -batch -file "$ORAC_DATA_IN"/filelist.lis -nodisplay -log sf -verbose -calib bad_receptors=FILE

and you will find your spectra in P0-only and P1-only under the folders p0/ and p1/, respectively. Note, for the receptors, N stands for Nāmakanui, the next letter is either A for Alaihi, U for ʻŪʻū or W for ʻĀweoweo. The number corresponds to polarization 0 or 1, and the final letter is L or U for Lower or Upper sideband.

Important note: These instructions to reduce each polarisation separately will not work for standard calibrator sources. ORAC-DR assigns the reduction recipe “REDUCE_STANDARD” to any observation with a ‘standard’ flag in it (i.e., standard calibrators). The REDUCE_STANDARD recipe, by default, ignores the “bad_recptors” file supplied to ORAC-DR. The way to fix this is to specify the recipe as “REDUCE_SCIENCE_NARROWLINE” when running the reduction as follows:

oracdr -loop file -batch -file "$ORAC_DATA_IN"/filelist.lis -nodisplay -log sf -verbose -calib bad_receptors=FILE REDUCE_SCIENCE_NARROWLINE

If you want to produce the polarization-combined spectra simply remove the parameter “-calib bad_receptors=FILE”:

# reduce both pols combined
mkdir combined
export ORAC_DATA_IN=$(pwd)
export ORAC_DATA_OUT=$(pwd)/combined
oracdr -loop file -batch -file "$ORAC_DATA_IN"/filelist.lis -nodisplay -log sf -verbose

and you will obtain the spectra with both polarizations combined inside the combined/ folder.

Combining Images

If you have reduced several Heterodyne files that you wish to combine into a single co-added map, this can be done with the PICARD recipe MOSAIC_JCMT_IMAGES:

picard -recpars mypar.lis MOSAIC_JCMT_IMAGES *files.sdf

where the “parameter file” mypar.lis looks like the following, with a mosaic method specified (in this example makemos):

[MOSAIC_JCMT_IMAGES]
MOSAIC_TASK = makemos
MAKEMOS_METHOD = mean

Other useful commands

Header information can be checked with

kappa
fitslist a20070420_00014_01_0001.sdf

Fitslist is a powerful tool and with it you can check particular attributes: Dates of observations

fitslist file | grep DATE-OBS
fitslist file | grep DATE-END

Observing conditions:

fitslist file | grep TAU225ST TAU225EN
fitslist file | grep WVMTAUST WVMTAUEN

System temperatures:

smurf
gettsys -statistics myfile #This command works for raw data only.
hdstrace a20090718_00024_01_0001.more.tsys nlines=all
fitslist file | grep MEDTSYS

Tracking receptor (HARP only):

fitslist file | grep INSTAP

Observing mode

fitslist file | grep SAM_MODE
fitslist file | grep SW_MODE
fitslist file | grep CHOP_FRQ CHOP_PA CHOP_THR

Bandwidth and resolution

fitslist file | grep BWMODE
fitslist file | grep SUBBANDS
fitslist file | grep NCHNSUBS

reference position

fitslist file | grep SKYREFX
fitslist file | grep SKYREFY

Instrument calibration

fitslist file | grep SW_MODE
fitslist file | grep CHOP_FRQ CHOP_PA CHOP_THR

Integration time

fitslist file | grep INT_TIME

Reduction history

hislist out

Backend

fitslist file | grep BACKEND

The positional information can be found with (add fullframe for even more info) ndftrace which displays the attributes of the data structure. This will tell you the units of the data, pixel bounds, dimensions and axis assignations.

kappa
ndftrace a20070420_00014_01_0001.sdf
ndftrace file fullframe fullwcs

other information in fits extensions can be accessed with

hdstrace a20070420_00014_01_0001.sdf

or in more detail for e.g. T_rx with

hdstrace a20070420_00014_01_0001.more.acsis.trx nlines=all

Files contributing to a coadd can be examined via:

provshow out

Nāmakanui Receptors – fitsheaders

To look at what receptors were working at the time – a particualrly useful trick for `Ū`ū or ʻĀweoweo, one can use fitsval or fitslist to look at the specific fitsheader “RECPTORS” (not a typo) e.g. All examples given below are for ʻŪʻū – Note that for ʻĀweoweo, NU*L and NU*U -> NW*L and NW*U (N stands for Nāmakanui, the next letter is either U for ʻŪʻū or W for ʻĀweoweo. The number corresponds to polarization 0 or 1, and the final letter is L or U for Lower or Upper sideband.)

fitslist a20201203_00124_01_0001.sdf | grep RECPTORS

RECPTORS= 'NU0L NU0U NU1L NU1U ' / Active FE receptor IDs for this observation

or by using fitsval e.g.

fitsval a20201203_00124_01_0001.sdf RECPTORS

NU0L NU0U NU1L NU1U

and also look at fitsheaders OBS_SB and TRACK_SB e.g.

OBS_SB	= 'USB	   '	       / The observed sideband (this subsystem)
TRACK_SB= 'USB	   '	       / The tracking sideband (primary subsystem)

after a reduction has been made one can look for the fitsheader RECPUSED to find out what receptors are in the reduced file:

RECPUSED= 'NU1U NU0U'	       / Used receptor IDs

Conversion to CLASS format

It is possible to convert the spectra to CLASS format, or the cubes to GDF files, readable by the IRAM GILDAS software package, see here.