Data Analysis#
Using the Stack#
The stack is a list of indices that can be used to gather scan numbers or record numbers to be used in a later operation, such as averaging. The stack system is modeled closely after the UNIPOPS commands.
To add entries to the stack, use the addstack command or the appendstack
command. The addstack command adds a sequence of entries using parameters that
describe the first entry, last entry, and increment. The appendstack command appends an array
of indices to the stack. For example:
emptystack ; clears the contents of the stack
addstack, 15 ; Add only index 15
addstack, 18, 21 ; Add indices 18 through 21
addstack, 22, 26, 2 ; Add indices 22 through 26 with an increment of 2. (22, 24, 26)
appendstack, [29, 35] ; Adds indices 29 and 35 to the stack
The tellstack command lists the indices currently contained in the stack. The GBTIDL
global variable !g.acount contains the total number of entries in the stack. The power of the
stack will become more evident in the discussion on averaging data. For now, here is a simple
example of using the stack to show spectrum headers for scans 6, 8, 10 and 12:
emptystack
addstack, 6, 12, 2
for i=0,!g.acount-1 do getnod, astack(i) & header & end
The following procedure gives an example of one way the stack could be put to use. The procedure averages Nod data identified by scan numbers listed in the stack. To use a procedure like this one, first populate the stack with the appropriate scan numbers then call the procedure.
pro myavg,_extra=extra
freeze
for i=0,!g.acount-1 do begin
getnod,astack(i),plnum=0,units=’Jy’,_extra=extra
accum
getnod,astack(i),plnum=1,units=’Jy’,_extra=extra
accum
endfor
ave
unfreeze
show
end
The following stack commands are available.
Command |
Purpose |
|---|---|
Adds a sequential list of indices to the stack, using addstack,begin,end,increment syntax |
|
Adds a single index or array of indices to the stack |
|
Returns the value of a specific stack entry, given an index, |
|
Averages the records associated with the stack entries |
|
Removes a stack entry from the list |
|
Removes indices from stack based on criteria such as source, polarization, and integration number. |
|
Clears the stack |
|
Runs a list on records identified by the stack. |
|
Adds indices to stack based on criteria such as source, polarization, and integration number. |
|
|
Shows the indices in the stack or returns all of the stack entries if no index is specified |
Removing Baselines#
GBTIDL uses “general orthogonal polynomials” in a least squares fit to determine baseline models. GBTIDL does not support sinusoid or Fourier component models.
To remove a spectral baseline, you must first identify a line-free region of the spectrum to be fit. The
region can be specified with either the nregion command, which allows you to specify the range by
typing the beginning and ending channels for each range, or with the setregion command, which allows
you to select the baseline region on the plotter, using the mouse cursor.
You can specify the order of the polynomial with the nfit procedure, or provide
it as a parameter in the baseline fitting routines. In either case, the value is stored and
becomes the default for later baseline fits. You can view the baseline without subtracting
it via the bshape procedure. When the baseline appears satisfactory, the
baseline procedure can be used to subtract it. A typical baseline fitting session
might then look like this:
nfit, 5 ; Specifies that a 5th order polynomial baseline will be fit
setregion ; Specify baseline regions using the mouse
bshape ; View the fitted baseline, but don’t subtract it yet
nfit, 4 ; Specifies that a 4th order polynomial baseline will be used
bshape ; View the new baseline fit, but don’t subtract it
baseline ; Subtract the most recent baseline fit
When a baseline is fit with either bshape or baseline, the baseline model
itself can be stored in a global data container by setting the modelbuffer keyword. You can
view the baseline model separate from the data as follows:
baseline, modelbuffer=5 ; Subtract the baseline and store the model in buffer # 5
show, 5 ; Show the baseline model in buffer # 5
and the data could be restored to its original form by:
add, 5, 0 ; Add baseline back to original spectrum to undo subtraction
After a baseline region is specified using the setregion procedure, a box is displayed
indicating the region to be used in a baseline fit. The height of the box is twice the RMS of the
data within the box, centered at the mean of the data within the box. These boxes can be removed
using the showregion, /off command (the regions remain set, but are not displayed).
If you wish to subtract from spectrum B a baseline model derived from spectrum A, use this method:
getfs, 1 ; Get spectrum A
baseline, modelbuffer=5 ; Fit and subract the baseline
getfs, 2 ; Get spectrum B
subtract, 0, 5 ; Subtract the old model
Here is a more sophisticated example of using various baseline features and commands.
getnod, 32 ; Get some data
setregion ; Set a region to be fit
bshape, nfit=10 ; Fit a 10th order polynomial
bmodel, nfit=2, modelbuffer=5 ; Use 2 coefficients to generate a new model
bmodel, nfit=5, modelbuffer=6 ; Use 5 coefficients
bmodel, nfit=10, modelbuffer=7 ; Use all 10
oshow,5, color=!yellow
oshow,6, color=!cyan
oshow,7, color=!green ; Plot all three for comparison
subtract, 0, 6 ; Subtract the 5th order fit
Averaging Data#
GBTIDL uses an accumulator to average data. For example:
sclear ; Clears the default global accumulator
get, index=1 ; Get record # 1
accum ; Put the data in the accumulator
get, index=2 ; Get record # 2
accum ; Adds the data to the accumulator
ave ; Averages data in the accumulator and places result in PDC
The sclear command clears the accum buffer to ensure it starts empty.
The result of the average is then stored in the PDC unless otherwise stated.
The above example uses the default accumulator buffer. There are 4 accumulator buffers numbered 0, 1, 2, and 3 so you can perform up to 4 different averages simultaneously. These are useful, for example, when accumulating data from two polarizations simultaneously, as shown in the following script:
sclear, 1 ; Clear the 1st accum buffer
sclear, 2 ; Clear the 2nd accum buffer
for i=10,15 do begin
getfs, i, plnum=0
accum, 1 ; Put data in 1st buffer
getfs, i, plnum=1
accum, 2 ; Put data in 2nd buffer
end
ave, 1 ; Average data in 1st buffer
copy, 0, 10
ave, 2 ; Average data in 2nd buffer
copy, 0, 11
show, 10
oshow, 11
Note that the IDL code in the above example works only if it is stored as a script, not interactively, because the for loop is split over several lines without the IDL line continuation characters & and $.
When the ave command is issued, the contents of the accum buffer are cleared unless the noclear
keyword is set. So, if you wish to view intermediate results in an ongoing average, you must specify that
the buffer should not be cleared:
sclear
get, index=1
accum
get, index=2
accum
ave, /noclear ; The accum buffer is NOT cleared here
get, index=3
accum
ave ; The accum buffer IS cleared here
It is also possible to use the stack when averaging data by using the avgstack command.
In the following example, the stack is used to identify records in the data file, and these are averaged.
addstack, 25 ; Add index 25 to stack
addstack, 30, 39 ; Add indices 30 through 39 to stack
avgstack ; Average the stack (data in records 25, 30-39)
In the following example, we select some data associated with the “LL” polarization and average them.
emptystack ; Start with an empty list
select,source=’W3OH’,scan=[177,178],pol=’LL’,cal=’F’
tellstack
liststack
delete,4 ; Remove record 4 from the list
avgstack ; Average the three scans in the stack
show
Averaging Data not Aligned in Frequency#
Suppose you wish to average spectra that overlap in frequency but are not exactly aligned. You must
use fshift to determine the shift needed to align the spectra, apply that shift using
gshift, and then add the spectra to the accumulator and average.
For example:
getps, 30
accum ; Accumulate first spectrum, no alignment needed yet
getps, 32
fs = fshift() ; Determine the shift to align scan 32 with the spectrum in
; the accumulator
gshift,fs ; Apply the shift to scan 32 in the PDC
accum ; Add the result to the accumulator
getps, 34
gshift, fshift() ; All in one line, shift 34 to align with the accumulator
accum
ave
It is also possible to align spectra on the basis of velocity using vshift, or using
the current x-axis units using xshift.
Smoothing Data#
GBTIDL provides users with 3 different smoothing options: boxcar, Gaussian, and hanning. In each
case it is possible to use ”decimation”, which means that every n-th channel will appear in the
smoothed spectrum, n being determined by the smoothing parameters. Boxcar smoothing requires a
parameter to specify the width of the boxcar. The gsmooth feature convolves the data
with a Gaussian of width \(\sqrt{newres^2 − origres^2}\), where newres is the new resolution
given by the user in units of channels. The hanning smooth uses a 3-channel hanning filter.
Examples:
getps, 25 ; Get some data into the PDC
boxcar, 4 ; 4-channel boxcar smooth, no decimation
getps, 25 ; Get some data into the PDC
boxcar, 2, /decimate ; 2-channel boxcar with decimation (keeps every other channel)
getps, 25 ; Get some data into the PDC
gsmooth, 4, /decimate ; Smooth to 4 channels & decimates (keeps every 4th channel)
getrec, 1 ; Get some data
hanning ; Apply hanning smooth and show the result
Fitting Gaussian Profiles#
The procedure fitgauss is used to fit Gaussian profiles to spectral line data.
Since a Gaussian function approaches zero away from the line center, you get the best
results by subtracting a baseline from the data prior to using fitgauss. In
general the procedure for Gaussian fitting is as follows:
Subtract a baseline from the spectrum of interest.
Using the plotter, zoom in to a region near the lines to be fit.
-
- Run the fitgauss procedure
-
Mark the line to be fit using the left mouse button. Only the channels selected will be included in the fitting algorithm. By selecting carefully, it is possible to have the procedure ignore any nearby lines or even fit one among blended lines.
Using the middle mouse button, click first on the peak of the line to be fit, and then middle-click again on the half-power point. These two clicks specify the initial guesses for line height, width, and center used by the Gaussian fitter. To fit multiple profiles simultaneously, continue to click the middle mouse button to mark additional lines.
When all lines have been marked, click the right mouse button to do the fit.
To retain the continuum level in a fit of absorption lines, the following recipe can be applied: * Determine the continuum level of the source. * Fit and subtract a baseline. * Fit the absorption line with a Gaussian and save the model using the modelbuffer parameter. * Add the continuum as a bias to both the data and the model.
For example, suppose we wish to fit an absorption line on a 1.5 Jy continuum source, and display the fit as an overlay.
setregion ; Set the baseline region
nfit, 3 ; Plan to fit a 3rd order polynomial baseline
baseline, modelbuffer=3 ; Fit and subtract the baseline. Continuum is also subtracted.
fitgauss, modelbuffer=10 ; Fit the Gaussian and store the model in buffer 10
bias, 1.5 ; Add the continuum level back to the data
copy, 0, 5 ; Store the data in buffer 5
copy, 10, 0 ; Copy the model to buffer 0
bias, 1.5 ; Add the continuum level to the model
copy, 0, 10 ; Return the model to buffer 10
copy, 5, 0 ; Return the data to buffer 0
oshow, 10, color=!orange ; Overlay the model on the data
Flagging and Blanking Data#
RFI and other faults that cause intermittent or frequency-dependent bad data make it necessary to be selective when operating on a data set. Bad data can be addressed with a combination of flagging and blanking. Flagging is the process of assigning a set of rules for marking bad data. Blanking is the process of applying these rules to the data, and replacing the flagged data with a special blanking value. See More about Flagging Data for more information.
The most common purpose of flagging and blanking is to identify data to be excluded from a calibration or averaging operation. As such, flagging is usually applied to raw data and data that have not yet been averaged.
In GBTIDL, the special value for blanked data is the IEEE not-a-number (NaN). Many native IDL
procedures already recognize that value and treat it appropriately. So, operations such as fitting and
averaging will ignore NaN values. As an example of the special handling of blanked values, consider
the show command. It handles the special values by putting gaps in the plotted spectrum at the
locations of blanked data. The stats procedure simply ignores any blanked channels in computing
the statistics. The hanning procedure blanks channels in the smoothed spectrum whose constituent
channels are themselves blanked. In general, procedures know how to take the appropriate action when
they encounter blanked data, and this action varies depending on the procedure.
Blanking is automatically applied to data when it is read into memory using the calibration or I/O
procedures such as get, getfs, getps, etc. Blanking can also be
applied by using the replace command.
As an example, suppose you have a spectrum displayed in the plotter and you would like to blank bad data in channels 500 to 525. The following command will perform the task.
replace, 500, 525, /blank ; Blanks the range of channels from 500 to 525
Flagging is different from blanking in that flagging does not change the data in a data container. Instead, flagging commands are associated with data on disk, and describe which of those data should be blanked when it is read with the GBTIDL I/O routines. The flagging commands are stored in a separate file from the data file, so you can unflag data or selectively ignore or apply certain flagging rules without changing the data on the disk or in memory.
Examples of setting flag rules, changing flag rules, and blanking data:
You know your data are bad in channels 500 to 525 and 1000 to 1100 for scan 11 but just in plnum=1 and ifnum=2. However, the data in the two channel ranges are bad for different reasons. The flags would be set and the data blanked like this:
flag, 11, plnum=1, ifnum=2, chans="500:525", idstring="rfi" ; Flag and label "rfi"
flag, 11, plnum=1, ifnum=2, chans="1000:1100", idstring="acs_glitch" ; Flag and label "acs_glitch"
getfs, 11, plnum=1, ifnum=2 ; Flagged data are now blanked in the PDC
Notice the use of idstring to document the reason a particular flag is being used.
If you have set up flagging rules but wish to ignore them when reading the data, the following command will retrieve the data without blanking:
getfs, 11, plnum=1, ifnum=2, /skipflag ; No flags are applied
If you want to flag the first integration of a range of scans because you suspect the telescope was still settling and not on target:
flag, scanrange=[6,10], intnum=0, idstring="first int" ; Flags integration 0 of scans 6-10
To view all the flag rules that have been set, use the listflags command:
listflags ; Produces a list of all the flag rules established
To remove a flag rule, use the unflag command. This works by either providing the idstring
attached to a flag or an integer matching an ID number as shown by listflags:
unflag, "first int" ; Unflags the rule with the id string "first int"
More about Flagging Data#
This section provides more information and examples about flagging. When data requires flagging, an iterative approach to reduction is often useful. Here is one approach:
Calibrate the raw data.
Examine the calibrated data and determine whether any flagging is required to improve calibration.
If necessary, flag the offending data and return to step 1.
Write a new SDFITS file with calibrated data. In general, the new SDFITS file should contain an entry for each integration that will be considered as a candidate for the average.
When all data are calibrated and written to disk, specify the calibrated data file as the new source of input.
Again examine the data and use the flagging procedures to mark residual bad data to exclude from the average.
Average the data.
Examine the average and, if necessary, return to step 1 or step 5 and modify the flagging commands as necessary.
Proceed with analysis of the averaged spectrum.
Because of the iterative nature of the process, it is common to set and then unset flagging commands for a given data set. It is important to emphasize that blanked data are not recoverable without going back to data retrieval, but flagged data are recoverable. Flagging (setting flag rules) allows you to iteratively decide which data should be blanked during processing.
Data can be flagged either by specifying scan number, integration number, polarization number, IF number, feed number, and channel number, or by specifying the record number (location within a file) and channel number. It is permissible to mix these two methods in a single flag file, if desired. The data I/O system in GBTIDL applies the flags, blanking data as appropriate (some control over which flags are applied is possible, as described later in this document). Averaging, analysis, and display procedures in GBTIDL take the appropriate action when blanked data are encountered.
Flagging is intended mainly for uncalibrated and pre-averaged data. However, it is not forbidden to flag calibrated, averaged data. Use caution in such cases because the header parameters used in the parametrization of flags can be changed during averaging operations. For this reason, when flagging averaged data it is generally best to flag by record number. Flagging by record number also offers a finer level of detail. The select procedure can be useful in conjunction with flagging by record number when the normal flag procedure isn’t sufficient (this is described in more detail later in this section). In the iterative flagging scheme outlined earlier in this section, flagging in Step 3 should be parametrized by scan, polarization, etc. while flagging in step 6 should be parametrized by record number.
Using Flags in GBTIDL#
Flag rules (flags) can be set from the command line with the procedures flag
and flagrec. These procedures generate entries in the flag file associated
with the current SDFITS file. The flag procedure has the following syntax:
flag, scan, intnum=intnum, plnum=plnum, ifnum=ifnum, fdnum=fdnum,
sampler=sampler, bchan=bchan, echan=echan, chans=chans,
chanwidth=chanwidth, idstring=idstring, scanrange=scanrange, /keep
and the flagrec procedure has the following syntax:
flagrec, record, bchan=bchan, echan=echan, chans=chans, chanwidth=chanwidth,
idstring=idstring, /keep
One uses idstring to associate with a rule an identifying string that is typically
a reminder of the reason for the flag.
Examples:
The following example shows how to flag a channel range for a small number of scans and integrations. Note that either the scan parameter or scanrange keyword is required but both can not be used at the same time. For the other parameters, if they are not specified, “all” is assumed. So in the first example, all polarizations are flagged. Also, notice that the integration numbers specified are 1 AND 3, not 1 through 3. To select a range, use intnum=[1,2,3] or intnum=seq(1,3) (the first example specifies all of the integrations to be flagged as integers, the second generates that sequence of integers using the ”seq” function).
flag, [18,19,20], intnum=[1,3], bchan=512, echan=514, idstring="RFI"
Equivalently, using the scanrange keyword:
flag, scanrange=[18,20], intnum=[1,3], bchan=512, echan=514, idstring="RFI"
To flag all channels for a given integration in one scan:
flag, 15, intnum=3, idstring="spectrometer glitch"
To flag all data for the given three scans:
flag, [101,105,107]
To flag a record in a processed data file (a keep file):
flagrec, 15, idstring="Glitch", /keep
To flag two channel ranges in a given scan you could do this:
flagrec, 16, bchan=0, echan=10, idstring="Two RFI Spikes"
flagrec, 16, bchan=100, echan=110, idstring="Two RFI Spikes"
or abbrieviate it like this:
flagrec, 16, bchan=[0,100], echan=[10,110], idstring="Two RFI Spikes"
The next example flags uses chans and chanwidth to flag the same channels:
flagrec, 16, chans=[5,105], chanwidth=11, idstring="Two RFI Spikes"
The select procedure can be used along with flagrec to provide even more flexible flagging. In this example, the “RR” polarization of IF number 3 for all data with the source name “Orion” is flagged in channels 500 to 520:
emptystack ; Clear the stack first
select, source=’Orion’, polarization=’RR’, ifnum=3 ; Populate the stack
flagrec, astack(), bchan=500, echan=520, idstring=’RFI-Orion’
Using Flags in Data Retrieval and Averaging Procedures#
Flags are applied by the data I/O subsystem when data are retrieved from disk. All of the data retrieval
procedures in GBTIDL (including calibration procedures such as getnod and getfs
that do data retrieval as part of their operation) use the I/O subsystem, so flags are applied whenever
you get data from disk.
All of these procedures allow you to fine tune which flag rules are actually applied via the useflag
and skipflag keywords. The default is to use /useflag, meaning that all flag rules are applied.
You can turn off all flagging by using /skipflag. In that case, no data will be blanked by the data
retrieval process. You can also apply or not apply some of the flags by referring to them by their
idstring. You can not use both the useflag and skipflag keywords in the same call. Unlike
unflag, the data retrieval commands do not allow you to skip or use flags based on their ID
number - only the idstring can be used as an argument to these keywords.
Examples:
getnod, 15 ; Apply all flags
getnod, 15, /skipflag ; Do not use any flags
getnod, 15, useflag="RFI" ; Only use the "RFI" flag
getnod, 15, useflag=["RFI","wind"] ; Use "RFI" and "wind" flags only
getnod, 15, skipflag="RFI" ; Use all flags EXCEPT "RFI"
All of the standard procedures in GBTIDL that in turn use these procedures also have the useflag
and skipflag keywords.
Listing Flags#
Use listflags to list all of the flags for the current data file, or only those
flags having a specific idstring. The default listflags output shows all flags
in their entirety, but the format sometimes is difficult to read. Appending the /summary
keyword to listflags aligns the columns but in order to do that, it may truncate
the information in a particular column and so not all information may be shown.
Examples:
listflags, ’RFI’ ; Shows the flag information associated with the ’RFI’ idstring
listflags, /summary ; Shows all flags with the information aligned by column
To list all of the unique idstring values in the flag file use the listids command. Example flag lists:
If one executes the flagging command:
flag, [35,36,37], intnum=[1,3], bchan=512, echan=514, idstring="RFI"
the listflags output will look like this:
#ID, RECNUM, SCAN, INTNUM, PLNUM, IFNUM, FDNUM, BCHAN, ECHAN, IDSTRING
0 * 35:37 1,3 * * * 512 514 RFI
The first line of the output identifies the contents of each column. Most of these fields are
self-explanatory. The first field is an ID number that is assigned dynamically and is simply
the location of that flag rule in this list. The ID number can be used in the :unflag
procedure to remove a flag rule.
Flagging a few more scans, not in a nice sequence:
flag, [40,42,44,47,48,50,56], intnum=[1,3], bchan=512, echan=514, idstring="More RFI"
adds one new line to the listflags output:
#ID, RECNUM, SCAN, INTNUM, PLNUM, IFNUM, FDNUM, BCHAN, ECHAN, IDSTRING
0 * 35:37 1,3 * * * 512 514 RFI
1 * 40,42,44,47,48,50,56 1,3 * * * 512 514 More RFI
And listflags, /summary truncates the output and produces the following:
#ID, RECNUM, SCAN, INTNUM, PLNUM, IFNUM, FDNUM, BCHAN, ECHAN, IDSTRING
0 * 35:37 1,3 * * * 512 514 RFI
1 * 40,42,44,+ 1,3 * * * 512 514 More RFI
Notice how the scan information is truncated. Fields that contain more information than
shown end in a plus sign, while asterisks indicate all values for that parameter are flagged
(as in the unformatted listflags output).
The second column, RECNUM, is set when flagrec is used. For example:
flagrec, 15, bchan=0, echan=8, idstring="bad channels"
listflags
#ID, RECNUM, SCAN, INTNUM, PLNUM, IFNUM, FDNUM, BCHAN, ECHAN, IDSTRING
0 * 35:37 1,3 * * * 512 514 RFI
1 * 40,42,44,47,48,50,56 1,3 * * * 512 514 More RFI
2 15 * * * * * 0 8 bad channels
Undoing Flags#
If you would like to remove all the flags associated with a given SDFITS file, you can simply remove the
associated flag file and restart GBTIDL. Alternatively, flags can be unset using the unflag
procedure. The unflag procedure takes a single parameter, id, and it removes all flagging
commands that have that id, where id can either be a string matching an idstring value or an
integer matching an ID number as shown by listflags.
unflag, id
If you want to re-flag that same data, you have to reissue the flag or flagrec
commands. The id parameter can be either a scalar or an array, to unflag multiple entries at once.
Unflagging by ID number is simple and appealing but users should be familiar with the following very
important feature. Since the ID number is generated dynamically, it changes after each flagging-related
command, including the unflag command. Users should always use listflags before
each use of unflag to be sure that they are using the appropriate ID value. Consider this
example:
listflags
#ID, RECNUM, SCAN, INTNUM, PLNUM, IFNUM, FDNUM, BCHAN, ECHAN, IDSTRING
0 * 35:37 1,3 * * * 512 514 RFI
1 * 40,42,44,47,48,50,56 1,3 * * * 512 514 More RFI
2 15 * * * * * 0 8 bad channels
If you want to unflag the last 2 IDs, so you might (mistakenly) try the following:
unflag, 1
unflag, 2
% FLAGS::UNFLAG_ID: ID could not be found to unflag: 2
The error happens because the first unflag causes the remaining two flag rules to be renumbered to 0 and 1, and so there is no ID 2 to unflag any more. This would have been a more dangerous, silent error had there been more than 3 rules to begin with. The correct way to unflag the entries:
listflags
unflag, 1
listflags
unflag, 1
or:
listflags
unflag, [1,2]
Weighting Issues not Addressed by this Flagging Scheme#
You should be aware of some potential issues with the weights when averaging flagged data.
Consider two reduced spectra, A and B, which resulted from an average of flagged data. In each of the two spectra, the individual channels have been flagged to different extents, so the final noise in each channel differs depending on how much of the raw data were flagged going into the average. For example, channels 0-10 in A may have been heavily flagged prior to averaging, and so they contain a higher noise than the other channels in A. If the observer then wishes to average A and B, the weighting in the average will be wrong because relative weights have not been stored for these spectra on a channel-per-channel basis.
Statistics#
Statistics are available from the stats procedure. If stats is given
no parameters, you must specify the range over which statistics are calculated using the
mouse cursor. Otherwise, stats can take two parameters, indicating the begin
and end values for the range, in units currently displayed on the plotter. The /chan
keyword can be used if you want to give the range in channels regardless of the current plotter
units. For example:
getrec, 1 ; Get some data
stats ; Prompt user for the range using the mouse
stats, 1420.0,1420.1 ; Show stats over the specified range
stats, /full ; Show stats over the full spectrum
stats, /full, ret=mystats ; Return statistics to the IDL data structure called mystats
print, mystats.mean ; Prints the mean value stored in the mystats data structure
print, mystats.rms ; Prints the rms value stored in the mystats data structure
stats, 0, 99, /chan ; Show stats for the first 100 channels
Here is an example of the output of the stats command:
GBTIDL -> stats
Click twice to define stats region
Chans bchan echan Xmin Xmax Ymin Ymax
13661 10692 24352 1.6103 1.6155 -4.0177 14.259
Mean Median RMS Variance Area
-0.14350 -0.051825 0.55484 0.30785 -0.00074783
Using the Select and Find Features#
Select#
The select procedure in GBTIDL is used to search and select records from
the input data set and add indices of the matching entries to the stack. To locate
the relevant records, select uses the contents of the GBTIDL index file.
The parameters for the search procedure are the same as those for the
get procedure (see Retrieving Individual Records). The procedure
listcols can be used to list all parameters available for searching. Note
that in this procedure, like all IDL procedures, the parameter names do not need to be
typed in their entirety, only enough characters to uniquely identify the parameter
are necessary.
To select all records associated with a given source name:
select, source=’3C286’
Multiple parameters are combined with a logical AND, so the following command selects all 3C286 records between scans 100 and 119:
select, source=’3C286’, scan=seq(100:119)
To select specific integer values, use an array as follows:
select, source=’3C286’, scan=[100,102,104,106]
The syntax for selections depends on the data type that is being selected, as shown in the following examples.
Integer Searches
select, index=10 ; Selects one index
select, index=[10,14,17,18] ; Selects a list of indices
select, index=’10:15,20:23’ ; Selects the given ranges
select, index=’:30’ ; Selects indices less than 30
Float Searches
select, tsys=’33.26’ ; Selects values between 33.255 and 33.265
select, tsys=’33.0:38.0’ ; Selects the range 33-38 K
select, tsys=’:45.0’ ; Selects based on Tsys < 45.0K
select, tsys=33.26 ; Selects values that are exactly 33.26, rarely useful
String Searches
select, source=’NGC1068’ ; Select based on single string value
select, source=[’NGC1068’, ’NGC1069’] ; Select from a list of strings
select, source=’NGC*’ ; Wildcards allowed at beginning and end of string
Find#
The find procedure and the related procedures setfind, clearfind,
and listfind (each described below) use select in a way that has been designed
to mimic some of the features of the CLASS find command. The find command is
particularly useful if you want to repeat the same or slightly modified selection. Each use of
find first clears the stack (unlike select) unless the /append keyword is used.
setfind: Used to set specific selection criteria. Once set, they remain set until cleared using clearfind.find: Used to place the entries specified by thesetfindcommand into the stack.clearfind: Used to clear the current setfind selection criteria.listfind: Used to list a specified selection parameter or all selection parameter values used byfind. This allows you to tell the value of one or all of the selection parameters used byfind.
Examples:
First define the initial selection criteria:
GBTIDL -> setfind, ’scan’, 80, 82 ; Select scans 80 through 82
GBTIDL -> find ; Add the selection to the stack (See 8.1)
Indices added to stack : 288
GBTIDL -> listfind ; Show current selection parameters
All set FIND parameters for LINE mode
SCAN 80:82
Then refine them:
GBTIDL -> setfind, ’polarization’, ’XX’ ; Select only the XX polarization
GBTIDL -> find ; Update the stack so it only contains scans 80-82 with
Indices added to stack : 144
GBTIDL -> listfind ; Show current selection parameters
All set FIND parameters for LINE mode
SCAN 80:82
POLARIZATION XX
Refine them again:
GBTIDL -> setfind, ’int’, 3 ; Select only integration 3
GBTIDL -> find ; Update stack to only contain indices that satisfy all
Indices added to stack : 24
GBTIDL -> listfind ; Show current selection parameters
All set FIND parameters for LINE mode
SCAN 80:82
POLARIZATION XX
INT 3
Change your mind and decide to include integration 4 also:
GBTIDL -> setfind, ’int’, 4, /append ; Use the /append keyword to add data
GBTIDL -> find ; Add the 4th integration indices to stack
Indices added to stack : 48
GBTIDL -> listfind ; Show new selection parameters
All set FIND parameters for LINE mode
SCAN 80:82
POLARIZATION XX
INT 3,4
Mapping#
GBTIDL does not support mapping. There is a mechanism for exporting SDFITS data into classic AIPS. Contact your GBT support person for details.
Other Analysis Procedures#
The following table lists additional analysis commands that may be useful.
Procedure |
Action |
|---|---|
|
Truncate spectrum to a min and max data value |
Decimate the spectrum by paring channels |
|
Convolve the spectrum in the PDC with an array |
|
FFT or inverse FFT the spectrum |
|
Interpolate across blanked channels |
|
HI profile fitting procedure |
|
Caclulate first 3 moments |
|
Flip the data end-to-end |
|
Show molecular transition frequencies on the plotter |
|
Compute power spectrum |
|
Plot the H alpha, beta, gamma; He alpha, beta, and C alpha recombination lines |
|
Compute and plot frequencies of Carbon recombination lines |
|
Compute and plot frequencies of Helium recombination lines |
|
Compute and plot frequencies of Hydrogen recombination lines |
|
Compute and plot frequencies of Nitrogen recombination lines |
|
Compute and plot frequencies of Oxygen recombination lines |
|
Replace bad data values |
|
Resample the spectrum in the PDC at the new interval |