Data Retrieval and Calibration

Data Retrieval and Calibration#

Calibrating Data#

If the sdfits program was run with -mode=raw as recommended, the data in the SDFITS file are uncalibrated. The observer can calibrate the data in GBTIDL. If the data were taken in one of the GBT standard observing modes supported by GBTIDL, then there is a GBTIDL command that can be used to retrieve and calibrate the data. GBT calibration can be complex. These procedures typically give results accurate to 10% - 20%. If you require higher precision, refer to the document Calibration of Spectral Line Data in GBTIDL.

Todo

Retrieve the document “Calibration of Spectral Line Data in GBTIDL and add it to GBTdocs.

For spectral line data, the following calibration procedures are available:

Table 27 Calibration procedures#
Frequency Switched	`getfs`
Total Power Position Switched	`getps` or `getsigref`
Total Power Nod	`getnod`
Beam Switched	`getbs`

Each of the calibration procedures except getsigref takes one required parameter: the M&C scan number. In the case of getfs, the calibration and retrieval pertains to a single scan. For getps and getnod, the data comes in scan pairs and the scan parameter can be either scan in the observing procedure. For example, if an OffOn() observation comprises scans 9 and 10, then the following commands give the same result:

getps,9

and

getps,10

The procedure getsigref takes two required parameters: the scan used for the “signal” and the scan used for the “reference”. So getsigref offers some flexibility to use mismatched sig/ref pairs or data from non-standard procedures.

getsigref, 14, 21    ; Get and calibrate data with signal scan 14 and reference scan 21

Each of the calibration procedures takes optional parameters. A few of the data selection keywords are listed below:

Table 28 Keywords related to data selection#
Keyword	Description	Default Value
`ifnum`	spectral window index	0
`intnum`	integration number	all integrations averaged
`plnum`	polarization index; 0=LL or XX, 1 = RR or YY	0
`fdnum`	feed number	0
`sampler`	sampler name; alternative to `ifnum`, `plnum`, `fdnum`	unused unless given explicitly

So, for example, to retrieve the polarization LL data from the second IF, third integration in scans 12-13, which make up a total power NOD observation, one could use:

getnod, 12, ifnum=1, plnum=0, intnum=2

Unlike some data processing packages, GBTIDL does not automatically average the two polarizations associated with a scan. So, you must be sure to average polarizations by hand where appropriate.

Table 29 Keywords to control details of the calibration#
Keyword	Description	Default Value
`smthoff`	smooth the off spectrum by smthoff channels	no reference smoothing
`tsys`	system temperature	Tsys derived from the data
`tcal`	cal temperature	taken from the data header
`eqweight`	apply equal weighting to integrations when averaging	0 (false) - weight by Tsys
`tau`	zenith opacity	get_tau(freq)
`ap_eff`	aperture efficiency	get_ap_eff(freq)
`units`	units of ‘Ta’, ‘Ta*’, or ‘Jy’	‘Ta’

Table 30 Other keywords#
Keyword	Description	Default Value
`quiet`	suppress messages printed to the screen	False (0)
`keepints`	save the individual integration results to the keep file	False (0)
`useflag`	use all or some of the flag rules by id string (see flagging)	use all flag rules
`skipflag`	skip all or some of the flag rules by id string (see flagging)	do not skip any rules
`instance`	for multiple occurances of the same scan, choose this instance	0 (the first instance)
`file`	for multiple occurances of the same scan, find it in this file (relevant with `dirin` only)	first file
`timestamp`	for multiple occurances of the same scan, find the one with this timestamp

Retrieving Individual Records#

For most users, the data retrieval and calibration procedures discussed in the previous section will be sufficient. Others may need to access data in its raw, uncalibrated form. There are two commands for accessing uncalibrated data, get and getrec. The getrec command is also useful for retrieving calibrated data from a keep file (a file which contains data already calibrated in GBTIDL and stored using the keep command).

get: The get procedure can be used to retrieve individual data records from the input data file based on the scan number, feed, IF, integration, polarization, cal state, and sig/ref state of the data. If these parameters are not sufficient to uniquely identify a single row in the SDFITS file, only the first matching row is returned and a warning message is printed. The get procedure might be used as follows to calculate a system temperature from an uncalibrated data file:

get, scan=10, pol=’LL’, ifnum=1, fdnum=1, int=1, sig=’T’, cal=’T’
calon = getdata()
get, scan=10, pol=’LL’, ifnum=1, fdnum=1, int=1, sig=’T’, cal=’F’
caloff = getdata()
tcal = !g.s[0].mean_tcal
tsys = caloff/(calon-caloff)*tcal
print,’Mean Tsys = ’,mean(tsys)

Table 31 Complete list of parameters for the `get` procedure#
Parameter	Description
`index`	record number
`project`	project ID
`file`	SDFITS file names (only relevant if the input data set is specified with dirin rather than filein)
`timestamp`	scan timestamp as YYYY MM DD HH:MM SS
`extension`	SDFITS extension number
`row`	SDFITS row number
`source`	source name
`procedure`	procedure name
`procseqn`	procedure sequence number
`scan`	M&C scan number
`polarization`	polarization, e.g. ‘LL’, ‘RR’, ‘XX’ or ‘YY’
`plnum`	polarization index, zero-based
`ifnum`	IF (i.e. spectral window) index number, zero-based
`feed`	feed name (e.g. B1)
`fdnum`	feed index number, zero-based
`int`	Integration number
`numchn`	number of channels in the spectrum
`sig`	‘T’ or ‘F’ to identify SIG state
`cal`	‘T’ or ‘F’ to identify cal-on or cal-off
`sampler`	backend sampler name
`azimuth`	antenna azimuth
`elevation`	antenna elevation
`longitude`	longitude-like axis, e.g. RA
`latitude`	latitude-like axis, e.g. DEC
`lst`	LST
`centfreq`	center frequency in Hz
`restfreq`	rest frequency in Hz
`velocity`	source velocity in km/s
`freqres`	frequency resolution in Hz
`freqint`	frequency interval (channel spacing) in Hz
`dateobs`	date-time value
`bandwidth`	bandwidth in Hz
`exposure`	exposure time
`tsys`	system temperature
`nsave`	nsave index
`trgtlat`	latitude coordinate of source
`trgtlon`	longitude coordinate of source
`obsid`	observation ID
`subref`	subreflector state (subref state); 0=moving, 1=first position, -1=second position

getrec To retrieve an individual record, use the getrec procedure. This procedure takes one parameter, the record number. The record number is equivalent to the row number in the SDFITS file. Like all indices in IDL, the record number is a zero-based index. So, for example, the fifth record can be retrieved and displayed as follows:

getrec,4

Getting Scan Header Information#

After data have been loaded into the PDC using one of the GBTIDL calibration procedures, get, or getrec, the header command can be used to show header information for that scan. For example:

GBTIDL -> header

returns

--------------------------------------------------------------------------------
Proj: TREG_050627 Src : W3OH Obs : Jim Braatz
Scan: 79 RADec : 02 27 04.1 +61 52 22 Fsky: 1.667696 GHz
Int : 0 Eqnx : 2000.0 Frst: 1.667359 GHz
Pol : YY V : -44.0 OPTI-LSR BW : 50.000 MHz
IF : 0 AzEl : 379.232 16.105 delF: 3.052 kHz
Feed: 1 Gal : 133.948 1.064 Exp : 26.2 s
13
Proc: Track UT : +04 10 20.0 2005-06-28 Tcal: 1.45 K
Sub : 0 LST/HA: +17 16 29.4 -9.18 Tsys: 28.38 K
--------------------------------------------------------------------------------

The header command shows information for the PDC by default, but headers for other data containers can be displayed by specifying the desired buffer index, or by specifying the IDL variable name explicitly. The following two commands are equivalent, and show the header for the data stored in buffer 2.

GBTIDL -> header, 2
GBTIDL -> header, !g.s[2]