1. Zenith opacity will be implemented as a vector, where tau is a function of frequency (tau[freq]). In addition, the database will allow time tags to be associated with opacity entries so that, in effect, tau is made a function of both frequency and time.

1. Zenith opacity will be implemented as a vector, where tau is a function of frequency and time (tau[freq, time]).

The calibration scheme described in that document has known limitations. In this document we describe several enhancements that address those limitations.

The calibration scheme described in that document has known limitations. Here we describe several enhancements that address those limitations.

1. Application of vector Tcal, where Tcal = Tcal(f), to obtain vector Tsys = Tsys(f)
2. Application of vector aperture efficiency, where ap_eff = ap_eff(f,elv)
3. Application of vector opacity, where tau = tau(f,t)

1. Tcal and Tsys will be changed from scalar values to vector quantities, where both Tcal and Tsys are functions of frequency (Tcal[freq] and Tsys[freq])
2. Aperture efficiency will be implemented as a 2-D vector, where ap_eff is a function of frequency and elevation (ap_eff[freq,elv])
3. Zenith opacity will be implemented as a vector, where tau is a function of frequency (tau[freq]). In addition, the database will allow time tags to be associated with opacity entries so that, in effect, tau is made a function of both frequency and time.

Default values for these vector calibration quantities must be generated by the sdfits program and stored in a database. The user should be able to inspect and replace the calibration values. Additionally, the user should be able to revert to the defaults.

Default values for these vector calibration quantities must be generated by the sdfits program and stored in a database that is accessible to GBTIDL. Opacity values will be taken from weather conditions. Aperture efficiency and Tcal entries will be filled from lookup tables. The user should be able to inspect the database and replace the calibration values. Additionally, the user should be able to revert to the defaults.

The sdfits program writes a database containing vector Tcal, vector ap_eff, and vector tau for all data contained in the sdfits file. The database can be stored either as an extension in the SDFITS file or as an external file associated with the SDFITS file. GBTIDL calibration procedures will access this database by default to improve flux calibration. Users will have tools to inspect and modify values in the calibration database.

The sdfits program will write a database containing Tcal, ap_eff, and tau as vector quantities for all data contained in the sdfits file. The database can be stored either as an extension in the SDFITS file or as an external file associated with the SDFITS file. GBTIDL calibration procedures will access this database by default when generating spectra. Users will have tools to inspect and modify values in the calibration database.

are used to enter new calibration information into the calibration database. Each of these parameters is optional, but at least one must be supplied unless the /default switch is set. Each parameter can either be entered as a scalar value, in which case it is applied to all frequencies, times, and elevations, or it can be supplied as a structure, as follows:

are used to enter new calibration information. Each of these parameters is optional, but at least one must be supplied unless the /default switch is set. Each parameter can accept either a scalar value, in which case the value is applied to all frequencies, times, and elevations, or it can accept a structure, defined as follows:

tau.time specifies the starting time for which the specified tau values will be applied. tau.freq specifies an array of frequencies over which tau will be specified. tau.value specifies the opacity values, one for each frequency. Intermediate frequency values will be interpolated from this curve by the calibration procedures.

tau.time is the starting time for which the specified tau values will be applied. tau.freq is an array of frequencies for which zenith opacity is specified in tau.value. tau.value is the array of opacity values, one for each frequency. Opacity values used in calibration will be interpolated from this curve by the calibration procedures.

ap_eff.elv specifies an array of elevations for which the ap_eff will be specified. ap_eff.freq specifies an array of frequencies for which the ap_eff will be specified. ap_eff.value specifies the 2-d array of aperture efficiencies, one for each frequency

ap_eff.elv is an array of elevations for which the ap_eff is specified. ap_eff.freq is an array of frequencies for which the ap_eff is specified. ap_eff.value is the 2-d array of aperture efficiencies, one for each frequency

tcal.freq specifies an array of frequencies for which the tcal will be specified. tcal.value specifies the tcal values, one for each frequency, feed, and polarization.

tcal.freq is an array of frequencies for which the tcal is specified. tcal.value is the array of calibration temperatures, one for each frequency, feed, and polarization.

number. The function returns a structure with the appropriate fields to be fed directly into set_cal.

number to the get_cal function. The function returns a structure with the appropriate fields to be fed directly back into set_cal.

high resolution observations are unnecessary. It should be acceptable to observe the flux

high resolution observations are unnecessary. It should be acceptable to observe flux

they should be sent as structures, as in the values used by the procedure set_cal.

they should be sent as structures, using the same field names as the tau and ap_eff parameters defined above under the procedure set_cal.

; the following frequency range should cover all observations

Note that in this example, it is unclear what get_cal should return for tau.time.

Note that in this example, it is unclear what get_cal should return in tau.time.

set_tcal, my_tcal

set_cal, tcal=my_tcal

Note in this example, scalar tau and ap_eff are used.

Note in this example, scalar values are used for tau and ap_eff.

• Measure Tcal values from an observation of a calibrator, and apply to all spectral scans in the SDFITS file. Vector tau and ap_eff are used this time.

• Measure Tcal values from an observation of a calibrator, and apply to all spectral scans in the SDFITS file. Vector values for tau and ap_eff are used this time.

set_tcal, my_tcal

set_cal, tcal=my_tcal

set_cal, my_tcal

set_cal, tcal=my_tcal

; Plot tau

; Retrieve tau and plot it

• Retrieve hourly zenith opacity for a given day, and plot it.

opac = findgen(24)
for i=0,23 do $
    opac[i] = get_wx_tau(53510.0+float(i)/24.0, 22235.080)
plot, opac

• This document does not address the details of how vector calibration qunatities will be implemented in the GBTIDL calibration routines.
• Ron plans on modifying the cleo weather forecast app to allow a CLI from which we could make GBTIDL calls.
• Reduction of tipping scans will happen in GFM initially, and eventually a routine will be incorporated into GBTIDL.
• For frequency-dependent calibration parameters Tcal, Tsys, ap_eff and tau, we need to decide whether the default frequency spacing will be fixed or user-configurable. If the latter, then the sdfits program will need a parameter to set the frequency spacing.

• This document does not address the details of how vector calibration quantities will be implemented in the GBTIDL calibration routines.
• Ron plans on adding a CLI to the cleo weather forecast application so that we can access it from GBTIDL.
• We plan to add the capability to process tipping scans. Initially the capability will be re-implemented as part of GFM and eventually a GBTIDL procedure will be added.
• For frequency-dependent calibration parameters Tcal, Tsys, ap_eff and tau, we need to decide whether the default frequency spacing will be a fixed or user-configurable value. If the latter, then the sdfits program will need a new parameter to allow setting the frequency spacing.

GBTIDL spectral line flux calibration capabilities, current to version 2.1, are described in the Calibration document. This document describes the design of several enhancements to the calibration scheme.

Spectral line flux calibration capabilities for GBTIDL version 2.1 are described in the Calibration document. The calibration scheme described in that document has known limitations. In this document we describe several enhancements that address those limitations.

The calibration scheme needs to be enhanced by adding three new capabilities:

The existing calibration scheme needs to be enhanced by adding three new capabilities:

1. Application of vector aperture efficiency, where Ae = Ae(f,elv)

1. Application of vector aperture efficiency, where ap_eff = ap_eff(f,elv)

When these enahncements are implements, the observer will get better default calibration using exactly the same syntax as is already familiar in GBTIDL. Observers requiring higher precision flux calibration will be able to attain it, but in order to maintain flexibility, the syntax of some of these operations is somewhat more complex than the standard GBTIDL syntax (e.g. the user will have to gain some familiarity with IDL structures).

Upon implementation of these enhancements, observers will get better default calibration using exactly the same syntax as is already familiar in GBTIDL, and observers who require high precision flux calibration will have better tools to attain it.

The sdfits program writes a database containing vector Tcal, vector ap_eff, and vector tau for all data contained in the sdfits file. The database can be stored either as an extension in the SDFITS file or as an external file associated with the SDFITS file.

The sdfits program writes a database containing vector Tcal, vector ap_eff, and vector tau for all data contained in the sdfits file. The database can be stored either as an extension in the SDFITS file or as an external file associated with the SDFITS file. GBTIDL calibration procedures will access this database by default to improve flux calibration. Users will have tools to inspect and modify values in the calibration database.

• The sdfits program will be modified to determine appropriate Tcal, ap_eff and tau, and write these values to a database. The program will remain unchanged from the user's perspective. All new functionality is "under the hood".

• The sdfits program will be modified to determine appropriate Tcal, ap_eff and tau, and write these values to a database. The program will remain unchanged from the user's perspective. No new parameters are required in the sdfits program. All new functionality is "under the hood".

This procedure writes new values into the calibration database. Each parameter is optional, but at least one must be supplied. Each parameter can either be entered as a scalar value, in which case it is applied to all frequencies, times, and elevations, or it can be supplied as a structure, as follows:

This procedure writes new values into the calibration database. The parameters tau, ap_eff, and tcal are used to enter new calibration information into the calibration database. Each of these parameters is optional, but at least one must be supplied unless the /default switch is set. Each parameter can either be entered as a scalar value, in which case it is applied to all frequencies, times, and elevations, or it can be supplied as a structure, as follows:

tau.time specifies the starting time for which the specified tau values will be applied. tau.freq specifies an array of frequencies over which tau will be specified. tau.value specifies the opacity values, one for each frequency. Intermediate frequency values will be interpolated from this curve by the calibration procedures.

tcal : tcal.freq[], tcal.value[n_feed, n_pol, freq]

2. get_cal(scan, time=time, /tau, /ap_eff, /tcal)

ap_eff.elv specifies an array of elevations for which the ap_eff will be specified. ap_eff.freq specifies an array of frequencies for which the ap_eff will be specified. ap_eff.value specifies the 2-d array of aperture efficiencies, one for each frequency and elevation.

This procedure returns calibration values. Exactly one of the (tau, ap_eff, or tcal) switches should be set. Either a scan number or MJD time must be specified only when the user requests tau values, since tau is the only time-dependent calibration quantity within a given SDFITS file.

tcal : tcal.freq[], tcal.value[n_feed, n_pol, freq]

3. show_cal, scan, time=time, /tau, /ap_eff, /tcal, freq_range=freq_range, elv_range=elv_range

tcal.freq specifies an array of frequencies for which the tcal will be specified. tcal.value specifies the tcal values, one for each frequency, feed, and polarization.

This procedure plots calibration values. Exactly one of the switches for tau, ap_eff, or tcal must be set. Either a scan number or MJD time must be specified if the user requests tau values be plotted. Frequency range and elevation range (which applies only to tau) can be given as options, and are 2-element arrays with begin and end values of the range.

==> This might need to be broken into parts.

2. get_cal(scan=scan, time=time, /tau, /ap_eff, /tcal)

4. calc_tcal(scan, ap_eff=ap_eff, tau=tau)

This function returns calibration values from the database. Exactly one of the switches (tau, ap_eff, or tcal) should be set. If the user requests tau values and there are tau values in the database for more than one time, then the user must also specify either a time or a scan number. The function returns a structure with the appropriate fields to be fed directly into set_cal.

3. calc_tcal(scan, ap_eff=ap_eff, tau=tau)

spectrum. The observer should integrate on one of the accepted flux calibration sources (3C48, 3C286, ...) using the same number of beams and polarizations as in the target observations. The frequency range of the calibration scans should at least cover the range of frequencies observed in the target sources. Wide-bandwidth, low-resolution observing modes are fine for the calibration observations. The Tcal measurements are smoothed the about 1 MHz, so high resolution observations are unnecessary. It should be acceptable to observe the flux calibrators using several scans, if necessary, to cover the frequency range of the target sources.

spectrum. The observer should integrate on one of the accepted flux calibration sources (3C48, 3C286, ...) using the same number of beams and polarizations as in the target observations. The frequency range of the calibration scans should at least cover the range of frequencies observed in the target sources. Wide-bandwidth, low-resolution observing modes are fine for the calibration observations. The Tcal measurements are smoothed to about 1 MHz anyway, so high resolution observations are unnecessary. It should be acceptable to observe the flux calibrators using several scans and combine these measurements in the calibration database using several calls to calc_tcal, if necessary.

A good starting place for the development of the procedure

A good starting place for the development of this procedure

5. get_wx_tau(time, freq)

4. get_wx_tau(time, freq)

• Set tau to a scalar value for all data after time 53850.5 MJD

• Set tau to a single value which will be applied for all frequencies, for all data after time 53850.5 MJD

• Set tau to a scalar value for all data after scan 100

• Set tau to a single value which will be applied for all frequencies, for all data from scan 100 forward

• Set tau to a vector value for data starting at a given time

• Set tau as a function of frequency, for all data starting at the given time

* Set tau to a vector value for all scans between 150 and 170

* Set tau for all scans between 150 and 170. Tau is a function of frequency in this example.

In this example, it is unclear what get_cal should return for tau.time. So, I've explicitly set the time in the my_tau_170 structure before applying the second set_cal.

Note that in this example, it is unclear what get_cal should return for tau.time. Is it the time at the start of the scan, or the time that defines the start period over which the returned tau values are applied? To avoid the ambiguity, the user should explicitly set the time in the my_tau_170 structure before applying the second set_cal.

• Measure Tcal values from an observation of a calibrator, and apply to later spectral scans

• Measure Tcal values from an observation of a calibrator, and apply to all spectral scans in the SDFITS file.

• Measure Tcal values from an observation of a calibrator, and apply to later spectral scans. Vector tau and ap_eff are used this time.

• Measure Tcal values from an observation of a calibrator, and apply to all spectral scans in the SDFITS file. Vector tau and ap_eff are used this time.

• Tweak a few Tcal values from the default calibration database

• Tweak a few Tcal values from the default calibration database and apply the tweaked calibration values.

• Erase all user-supplied calibration values and revert to the defaults

• Erase all user-supplied calibration values and revert to the defaults.

• Set a new gain curve

• Set a new gain curve.

• Plot the existing calibration factors

• Plot data from the calibration database using standard IDL plot tools.

show_cal(/ap_eff,freq_range=22235.080, elv_range=[0,90] ; shows ap_eff for one freq

; Plot tau my_tau = get_cal(/tau) ; this works only if the database has tau set once for all times plot, my_tau.freq, my_tau.value, xstyle=1, xtitle='Freq (MHz)', ytitle='Tau (nepers)'

; Retrieve tau for a given scan and plot it my_tau = get_cal(/tau, scan=50) plot, my_tau.freq, my_tau.value, xstyle=1, xtitle='Freq (MHz)', ytitle='Tau (nepers)'

; Plot tcal for each of two feeds and two polarizations my_tcal = get_cal(/tcal) plot, my_tcal.freq, my_tcal.value[0,0,*],xstyle=1, xtitle='Freq (MHz)', ytitle='Tcal (K)' oplot, my_tcal.freq, my_tcal.value[0,1,*], color=!white oplot, my_tcal.freq, my_tcal.value[1,0,*], color=!green oplot, my_tcal.freq, my_tcal.value[1,1,*], color=!yellow

; Plot tcal over a range of frequencies for a given feed and polarization my_tcal = get_cal(/tcal) plot, my_tcal.freq, my_tcal.value[0,0,*],xstyle=1, xtitle='Freq (MHz)', ytitle='Tcal (K)', $ xrange=[22000.0, 23000.0]

; Plot aperture efficiency as a function of elevation for each frequency given in the calibration database

my_ap_eff = get_cal(/ap_eff) n_freqs = n_elements(my_ap_eff.freq) plot, my_ap_eff.elv, my_ap_eff.value[*,0] for i=1,n_freqs-1 do $ oplot, my_ap_eff.elv, my_ap_eff.value[*,i], color=!red+i

Questions:

Issues and Plans:

• Has Ron already worked out how vector Tsys values will be implemented in the calibration routines?
• How do we get tau values into the database from the cleo weather forecast app? Either we (1) modify the cleo app so that it can be called by the sdfits program or (2) we regenerate the calculations in GBTIDL starting from the weather data itself.
• Should we leave reduction of tip scans to GFM?
• For frequency-dependent calibration parameters Tcal, Tsys, ap_eff and tau, should the frequency spacing be a user-configurable parameter or should defaults be instituted?
• For frequency-dependent parameters, do we have a standard spacing (e.g. 1 MHz?) or do we require the user to configure the frequency spacing?
• How do we handle multiple and overlapping calibration measurements of Tcal calibrators?

• This document does not address the details of how vector calibration qunatities will be implemented in the GBTIDL calibration routines.
• Ron plans on modifying the cleo weather forecast app to allow a CLI from which we could make GBTIDL calls.
• Reduction of tipping scans will happen in GFM initially, and eventually a routine will be incorporated into GBTIDL.
• For frequency-dependent calibration parameters Tcal, Tsys, ap_eff and tau, we need to decide whether the default frequency spacing will be fixed or user-configurable. If the latter, then the sdfits program will need a parameter to set the frequency spacing.

GBTIDL Calibration Design - Initial Thoughts

Design of Flux Calibration Enhancements in GBTIDL

Jim Braatz

GBTIDL calibration needs to be enhanced by adding three new capabilities:

GBTIDL spectral line flux calibration capabilities, current to version 2.1, are described in the Calibration document. This document describes the design of several enhancements to the calibration scheme.

The calibration scheme needs to be enhanced by adding three new capabilities:

Default values for the vector calibration quantities must be generated by the sdfits program and stored in a database. The user should be able to inspect and replace the calibration values. Additionally, the user should be able to revert to the defaults.

Default values for these vector calibration quantities must be generated by the sdfits program and stored in a database. The user should be able to inspect and replace the calibration values. Additionally, the user should be able to revert to the defaults.

When these enahncements are implements, the observer will get better default calibration using exactly the same syntax as is already familiar in GBTIDL. Observers requiring higher precision flux calibration will be able to attain it, but in order to maintain flexibility, the syntax of some of these operations is somewhat more complex than the standard GBTIDL syntax (e.g. the user will have to gain some familiarity with IDL structures).

• The sdfits program calculates and attaches a scalar Tcal value to each record's data header. The Tcal scalar is taken as an average of the Tcals over the frequency range observed. No information is stored in the SDFITS file to describe tau or Ae. The user can supply scalar tau and Ae values to the calibration routines (getnod, getps, ...).

The sdfits program calculates and attaches a scalar Tcal value to each record's data header. The Tcal scalar is taken as an average of the Tcals over the frequency range observed. A scalar Tsys is then calculated from the central 80% of the observed band, and applied in the flux calibration. No information is stored in the SDFITS file to describe atmospheric opacity (tau) or aperture efficency (ap_eff). The user can supply scalar tau and ap_eff values to the calibration routines (getnod, getps, ...). When the user does not specify tau or ap_eff, GBTIDL applies default values, as described in the calibration document.

• The sdfits program writes a database containing vector Tcal, vector Ae, and vector tau for all data contained in the sdfits file. The database can be stored either as an extension in the SDFITS file or as an external file associated with the SDFITS file.

The sdfits program writes a database containing vector Tcal, vector ap_eff, and vector tau for all data contained in the sdfits file. The database can be stored either as an extension in the SDFITS file or as an external file associated with the SDFITS file.

Note the situation can get complicated if the observer changes receivers during a run. (Do we want to enforce writing a new SDFITS file when there is a RX change, or just keep track of the RX change in the software and allow a single SDFITS file for the entire dataset?).

Note the situation can get complicated if the observer changes receivers during a run. As an implementation issue, we might consider enforcing each receiver's data for a given project being written into its own SDFITS file.

• The sdfits program will be modified to determine appropriate Tcal, Ae and tau, and write these values to a database. The program will remain unchanged from the user's perspective. All new functionality is "under the hood".

• The sdfits program will be modified to determine appropriate Tcal, ap_eff and tau, and write these values to a database. The program will remain unchanged from the user's perspective. All new functionality is "under the hood".

This procedure writes new values into the calibration database. Each parameter is optional. Each parameter can either be entered as a scalar value, in which case it is applied to all frequencies, times, and elevations, or it can be supplied as a structure, as follows:

This procedure writes new values into the calibration database. Each parameter is optional, but at least one must be supplied. Each parameter can either be entered as a scalar value, in which case it is applied to all frequencies, times, and elevations, or it can be supplied as a structure, as follows:

This procedure returns calibration values. Either a scan number or MJD time must be specified. Exactly one of the switches for tau, ap_eff, or tcal must be given.

This procedure returns calibration values. Exactly one of the (tau, ap_eff, or tcal) switches should be set. Either a scan number or MJD time must be specified only when the user requests tau values, since tau is the only time-dependent calibration quantity within a given SDFITS file.

This procedure plots calibration values. Either a scan number or MJD time must be specified. Exactly one of the switches for tau, ap_eff, or tcal must be given. Frequency range and elevation range (which applies only to tau) can be given as options, and are 2-element arrays with begin and end values of the range.

This procedure plots calibration values. Exactly one of the switches for tau, ap_eff, or tcal must be set. Either a scan number or MJD time must be specified if the user requests tau values be plotted. Frequency range and elevation range (which applies only to tau) can be given as options, and are 2-element arrays with begin and end values of the range.

==> This might need to be broken into parts.

spectrum. A good starting place is ~rmaddale/mypros/getscalquad.pro. We can drop the parts of the routine that determines the non-linearity calibration factors.

spectrum. The observer should integrate on one of the accepted flux calibration sources (3C48, 3C286, ...) using the same number of beams and polarizations as in the target observations. The frequency range of the calibration scans should at least cover the range of frequencies observed in the target sources. Wide-bandwidth, low-resolution observing modes are fine for the calibration observations. The Tcal measurements are smoothed the about 1 MHz, so high resolution observations are unnecessary. It should be acceptable to observe the flux calibrators using several scans, if necessary, to cover the frequency range of the target sources.

A good starting place for the development of the procedure is ~rmaddale/mypros/getscalquad.pro. We can drop the parts of the routine that determine the non-linearity calibration factors.

ap_eff and tau can be scalars or vector functions of frequency. If they are vectors then the they should be sent as structures, as in the values used by set_cal.

ap_eff and tau can be scalars or vector functions of frequency. If they are vectors then they should be sent as structures, as in the values used by the procedure set_cal.

<nop>

Ron notes: it may be necessary for the observer to specify the smoothing factor. Also, for Ka band, a sampler name may be necessary.

my_tcal.value[0,0,10] = 4.55 my_tcal.value[0,1,10] = 2.97

my_tcal.value[0,0,10] = 4.55 ; feed 0, pol 0, freq 10 my_tcal.value[0,1,10] = 2.97 ; feed 0, pol 1, freq 10

• Plot the existing calibration factors

Use two frequencies as boundaries for interpolation, and three elevations.

filein,'mydata.fits'

show_cal, /tcal, ret=tcal_array          ; plots tcal vs. freq and returns values

my_tcal = calc_tcal(101, 0.62, 0.05)     ; returns a tcal spectrum in a structure that has
                                         ; my_tcal.freq[*] and my_tcal.temperature[2,*].
                                         ; scan 101 must be a spectral line observation
                                         ; of an accepted flux calibrator

calib, tcal=my_tcal                      ; sets new tcal values for entire SDFITS file

calib, ap_eff = 0.70                     ; apply a single, scalar aperture efficiency to all
                                         ; data in this sdfits file
<nop>

my_ap_eff = get_cal(/ap_eff) ; get the ap_eff structure

getnod, 10 ; this uses all the newly added calibration information

my_ap_eff.freq = [22000.0, 23000.0] my_ap_eff.elv = [0.0, 50.0, 90.0] my_ap_eff.value = [[0.51,0.45],[0.62,0.58],[0.61,0.57]]

calib, /default ; get rid of all the user-supplied entries in the ; calibration database

set_cal, ap_eff=my_ap_eff

getnod, 10 ; this now uses the default calibration database ; as originally filled by the sdfits program

• Plot the existing calibration factors

show_cal(/ap_eff,freq_range=22235.080, elv_range=[0,90]  ; shows ap_eff for one freq

tau : tau.time, tau.value[freq]

tau : tau.time, tau.freq[], tau.value[freq]

4. calc_tcal(scan, ap_eff, tau)

4. calc_tcal(scan, ap_eff=ap_eff, tau=tau)

ap_eff and tau can be scalae or vector functions of frequency.

ap_eff and tau can be scalars or vector functions of frequency. If they are vectors then the they should be sent as structures, as in the values used by set_cal.

• Set tau to a scalar value for all data after time 53850.5 MJD

my_tau = {time = 53850.5, value = 0.022}
set_cal, tau=my_tau

my_tau = {} come back to this .... I think it might be better to set the time as a parameter in set_cal.

gettp, 100 my_time = !g.s[0].mjd my_tau = {time=my_time, value = 0.085}

• Set tau to a vector value for all scans in a SDFITS file

• Set tau to a vector value for data starting at a given time

my_time = 53850.5 freq_vector = [22000.0, 22100.0, 22200.0, 22300.0, 22400.0] tau_vector = [0.05, 0.06, 0.075, 0.063, 0.055 ] my_tau = {time=my_time, freq=freq_vector, value=tau_vector} set_cal, tau=my_tau

gettp, 150 time_150 = !g.s[0].mjd

gettp, 170 time_170 = !g.s[0].mjd

default_tau = get_cal(170, /tau) freq_vector = [22000.0, 22100.0, 22200.0, 22300.0, 22400.0] tau_vector = [0.05, 0.06, 0.075, 0.063, 0.055 ] my_tau_150 = {time=time_150, freq=freq_vector, value=tau_vector}

my_tau_170 = default_tau my_tau_170.time = time_170

set_cal, tau=my_tau_150 set_cal, tau=my_tau_170

In this example, it is unclear what get_cal should return for tau.time. So, I've explicitly set the time in the my_tau_170 structure before applying the second set_cal.

my_tcal = calc_tcal(10,ap_eff=0.65,tau=0.05) set_tcal, my_tcal

my_tau = get_cal(10,/tau)     ; get the tau structure for scan 10
my_tau.freq =   [1400.0, 1410.0, 1420.0, 1430.0, 1440.0]
my_tau.value =  [0.015,  0.017,  0.019,  0.021,  0.024 ]

my_ap_eff = get_cal(/ap_eff)  ; get the ap_eff structure
my_ap_eff.freq = [1400.0, 1440.0]
my_ap_eff.elv = [0.0, 30.0, 90.0]
my_ap_eff.value = [[0.61,0.60],[0.71,0.70],[0.73,0.73]]

my_tcal = calc_tcal(10,ap_eff=my_ap_eff,tau=my_tau)

set_tcal, my_tcal
<nop>

my_tcal = get_cal(/tcal) my_tcal.value[0,0,10] = 4.55 my_tcal.value[0,1,10] = 2.97 set_cal, my_tcal

set_cal, /default
<nop>

• How do we handle multiple and overlapping calibration measurements of Tcal calibrators?

ap_eff and tau can be scalae or vector functions of frequency.

See my getscalquad.pro for what I'm finding useful in the call. Need IFNum, PLNUM, FDNUM, and, for high resolution spectra, a smoothing factor. For Ka, we needed to add in sampler name. Ap_eff and tau can be vectors or scalars.

See my getscalquad.pro for what I'm finding useful in the call. Need IFNum, PLNUM, FDNUM, and, for high resolution spectra, a smoothing factor. For Ka, we needed to add in sampler name.

and freq can be either a scalar or a vector. If freq is a vector, the funtion returns tau for each

and freq can be either a scalar or a vector. If freq is a vector, the function returns tau for each

• Set tau to a scalar value for all data after scan 100

my_tau = {}
come back to this .... I think it might be better to set the time as a parameter in set_cal.
set_cal, tau=my_tau

• Set tau to a vector value for all scans in a SDFITS file

* Set tau to a vector value for all scans between 150 and 170

• Measure Tcal values from an observation of a calibrator, and apply to later spectral scans

• Tweak a few Tcal values from the default calibration database

• Set a new gain curve

• Plot the existing calibration factors

• The sdfits program to be modified to determine appropriate Tcal, Ae and tau, and write these values to a database. The program will remain unchanged from the user's perspective. All new functionality is "under the hood".

• The sdfits program will be modified to determine appropriate Tcal, Ae and tau, and write these values to a database. The program will remain unchanged from the user's perspective. All new functionality is "under the hood".

tau : tau.time[*], tau.value[*,*] ap_eff : ap_eff.elv[*], ap_eff.freq[*], ap_eff.value[*,*] tcal : tcal.freq[*], tcal.value[n_feed,2,*] (n_feed feeds, two pols)

tau : tau.time, tau.value[freq] ap_eff : ap_eff.elv[], ap_eff.freq[], ap_eff.value[freq, elv] tcal : tcal.freq[], tcal.value[n_feed, n_pol, freq]

This procedure returns calibration values. Either a scan number or mjd time must be specified.

This procedure returns calibration values. Either a scan number or MJD time must be specified.

This procedure plots calibration values. Either a scan number or mjd time must be specified.

This procedure plots calibration values. Either a scan number or MJD time must be specified.

• The sdfits program to be modified to determine appropriate Tcal, Ae and tau, and write these values to a database.
• A GBTIDL procedure to reduce an observation of a flux calibrator in order to determine the Tcal spectrum. A good starting place is ~rmaddale/mypros/getscalquad.pro. We can drop the parts of the routine that determines the non-lineariy calibration factors. RJM
• A GBTIDL procedure to return (and plot?) the current and default Tcal/tau/Ae values stored in the database.
• A GBTIDL procedure to set the Tcal , Tau and efficiency vectors using an IDL array.
• A GBTIDL procedure to retrieve tau from Ron's weather database, for an arbitrary time and frequency. Probably dynamic scheduling has the same need for a command-line front-end to my forecast program. It'll take me a good part of a week tocreate this. RJM

• The sdfits program to be modified to determine appropriate Tcal, Ae and tau, and write these values to a database. The program will remain unchanged from the user's perspective. All new functionality is "under the hood".

What will it look like?

• sdfits (the program) will remain unchanged from the user's perspective. All new functionality is "under the hood".

• New GBTIDL procedures:

• The following new GBTIDL procedures and functions will be developed:

1. calib, tau=tau, ap_eff=ap_eff, tcal=tcal, /default

1. set_cal, tau=tau, ap_eff=ap_eff, tcal=tcal, /default

<nop>

2. get_cal(scan, time=time, /tau, /ap_eff, /tcal)

This procedure returns calibration values. Either a scan number or mjd time must be specified. Exactly one of the switches for tau, ap_eff, or tcal must be given.

2. show_cal, tau=tau, ap_eff=ap_eff, tcal=tcal

3. show_cal, scan, time=time, /tau, /ap_eff, /tcal, freq_range=freq_range, elv_range=elv_range

3. calc_tcal(scan, ap_eff, tau)

This procedure plots calibration values. Either a scan number or mjd time must be specified. Exactly one of the switches for tau, ap_eff, or tcal must be given. Frequency range and elevation range (which applies only to tau) can be given as options, and are 2-element arrays with begin and end values of the range.

4. calc_tcal(scan, ap_eff, tau)

This procedure will reduce an observation of a flux calibrator in order to determine the Tcal spectrum. A good starting place is ~rmaddale/mypros/getscalquad.pro. We can drop the parts of the routine that determines the non-linearity calibration factors.

4. get_wx_tau(time, freq)

5. get_wx_tau(time, freq)

<nop>

Returns tau values from Ron's weather-based calculations. The input for time is a scalar value in MJD and freq can be either a scalar or a vector. If freq is a vector, the funtion returns tau for each element of the vector.

So an example use might look like so:

Use Cases

tau : tau.time[*], tau.value[*]

tau : tau.time[*], tau.value[*,*]

tcal : tcal.freq[*], tcal.value[2,*] (two pols)

tcal : tcal.freq[*], tcal.value[n_feed,2,*] (n_feed feeds, two pols)

• Has Ron already worked out how scalar Tsys values will be implemented in the calibration routines?

• Has Ron already worked out how vector Tsys values will be implemented in the calibration routines?

• For frequency-dependent parameters, do we have a standard spacing (e.g. 1 MHz?) or do we require the user to configure the frequency spacing?

• A GBTIDL procedure to reduce an observation of a flux calibrator in order to determine the Tcal spectrum

• A GBTIDL procedure to reduce an observation of a flux calibrator in order to determine the Tcal spectrum. A good starting place is ~rmaddale/mypros/getscalquad.pro. We can drop the parts of the routine that determines the non-lineariy calibration factors. RJM

• A GBTIDL procedure to set the Tcal spectrum using an IDL array
• A GBTIDL procedure to retrieve tau from Ron's weather database, for an arbitrary time and frequency.

• A GBTIDL procedure to set the Tcal , Tau and efficiency vectors using an IDL array.
• A GBTIDL procedure to retrieve tau from Ron's weather database, for an arbitrary time and frequency. Probably dynamic scheduling has the same need for a command-line front-end to my forecast program. It'll take me a good part of a week tocreate this. RJM

<nop>

<nop>

Where freq can be a vector or scalar.

GBTIDL Calibration needs to be improved by the addition of three new capabilities:

GBTIDL calibration needs to be enhanced by adding three new capabilities:

1. Application of vector system temperature determined from vector Tcal, where Tcal = Tcal(f)

1. Application of vector Tcal, where Tcal = Tcal(f), to obtain vector Tsys = Tsys(f)

1. Application of vector atmospheric opacity, where tau = tau(f,t)

1. Application of vector opacity, where tau = tau(f,t)

Default values for the vector calibration quantities must be generated by the sdfits program and stores in a database. The user should be able to inspect and replace the calibration values. Additionally, the user should be able to revert to the defaults.

Default values for the vector calibration quantities must be generated by the sdfits program and stored in a database. The user should be able to inspect and replace the calibration values. Additionally, the user should be able to revert to the defaults.

• The sdfits program calculates and attaches a scalar Tcal value to each record's data header, taken as an average of the Tcals over the frequency range observed. No information is stored in the SDFITS file to describe tau or Ae. The user can supply scalar tau and Ae values to the calibration routines (getnod, getps, ...)

• The sdfits program calculates and attaches a scalar Tcal value to each record's data header. The Tcal scalar is taken as an average of the Tcals over the frequency range observed. No information is stored in the SDFITS file to describe tau or Ae. The user can supply scalar tau and Ae values to the calibration routines (getnod, getps, ...).

• The sdfits program writes a database containing vector Tcal, vector Ae, and vector tau for all data contained in the sdfits file. The database can be stored either as an extemnsion in the SDFITS file or as an external file associated with the SDFITS file.

• The sdfits program writes a database containing vector Tcal, vector Ae, and vector tau for all data contained in the sdfits file. The database can be stored either as an extension in the SDFITS file or as an external file associated with the SDFITS file.

• The sdfits program modified to determine appropriate Tcal, Ae and tau and write the values to a database.

• The sdfits program to be modified to determine appropriate Tcal, Ae and tau, and write these values to a database.

• A GBTIDL procedure to return (and plot?) the default Tcal/tau/Ae values stored in the database.

• A GBTIDL procedure to return (and plot?) the current and default Tcal/tau/Ae values stored in the database.

Questions:

What will it look like?

• sdfits (the program) will remain unchanged from the user's perspective. All new functionality is "under the hood".

• New GBTIDL procedures:

  1. calib, tau=tau, ap_eff=ap_eff, tcal=tcal, /default

     This procedure writes new values into the calibration database.  Each parameter is optional.
     Each parameter can either be entered as a scalar value, in which case it is applied to all
     frequencies, times, and elevations, or it can be supplied as a structure, as follows:

     tau    : tau.time[*], tau.value[*]
     ap_eff : ap_eff.elv[*], ap_eff.freq[*], ap_eff.value[*,*]
     tcal   : tcal.freq[*], tcal.value[2,*]  (two pols)

  2. show_cal, tau=tau, ap_eff=ap_eff, tcal=tcal

  3. calc_tcal(scan, ap_eff, tau)

  4. get_wx_tau(time, freq)

So an example use might look like so:

filein,'mydata.fits'

show_cal, /tcal, ret=tcal_array          ; plots tcal vs. freq and returns values

my_tcal = calc_tcal(101, 0.62, 0.05)     ; returns a tcal spectrum in a structure that has
                                         ; my_tcal.freq[*] and my_tcal.temperature[2,*].
                                         ; scan 101 must be a spectral line observation
                                         ; of an accepted flux calibrator

calib, tcal=my_tcal                      ; sets new tcal values for entire SDFITS file

calib, ap_eff = 0.70                     ; apply a single, scalar aperture efficiency to all
                                         ; data in this sdfits file

getnod, 10                               ; this uses all the newly added calibration information

calib, /default                          ; get rid of all the user-supplied entries in the 
                                         ; calibration database

getnod, 10                               ; this now uses the default calibration database
                                         ; as originally filled by the sdfits program

Questions:

• How do we get tau values into the database from the cleo weather forecast app? Either we (1) modfiy the cleo app so that it can be called by the sdfits program or (2) we regenerate

• How do we get tau values into the database from the cleo weather forecast app? Either we (1) modify the cleo app so that it can be called by the sdfits program or (2) we regenerate the calculations in GBTIDL starting from the weather data itself.
• Should we leave reduction of tip scans to GFM?
• For frequency-dependent calibration parameters Tcal, Tsys, ap_eff and tau, should the frequency spacing be a user-configurable parameter or should defaults be instituted?

%META:TOPICINFO{author="JimBraatz" date="1178652240" format="1.0" version="1.1"}% %META:TOPICPARENT{name="JimBraatz"}%

GBTIDL Calibration Design - Initial Thoughts

Overview

GBTIDL Calibration needs to be improved by the addition of three new capabilities:

1. Application of vector system temperature determined from vector Tcal, where Tcal = Tcal(f)
2. Application of vector aperture efficiency, where Ae = Ae(f,elv)
3. Application of vector atmospheric opacity, where tau = tau(f,t)

Default values for the vector calibration quantities must be generated by the sdfits program and stores in a database. The user should be able to inspect and replace the calibration values. Additionally, the user should be able to revert to the defaults.

Current situation with GBTIDL calibration:

• The sdfits program calculates and attaches a scalar Tcal value to each record's data header, taken as an average of the Tcals over the frequency range observed. No information is stored in the SDFITS file to describe tau or Ae. The user can supply scalar tau and Ae values to the calibration routines (getnod, getps, ...)

Proposed situation:

• The sdfits program writes a database containing vector Tcal, vector Ae, and vector tau for all data contained in the sdfits file. The database can be stored either as an extemnsion in the SDFITS file or as an external file associated with the SDFITS file.

Note the situation can get complicated if the observer changes receivers during a run. (Do we want to enforce writing a new SDFITS file when there is a RX change, or just keep track of the RX change in the software and allow a single SDFITS file for the entire dataset?).

New tools required

• The sdfits program modified to determine appropriate Tcal, Ae and tau and write the values to a database.
• A GBTIDL procedure to reduce an observation of a flux calibrator in order to determine the Tcal spectrum
• A GBTIDL procedure to return (and plot?) the default Tcal/tau/Ae values stored in the database.
• A GBTIDL procedure to set the Tcal spectrum using an IDL array
• A GBTIDL procedure to retrieve tau from Ron's weather database, for an arbitrary time and frequency.

Questions:

• Has Ron already worked out how scalar Tsys values will be implemented in the calibration routines?
• How do we get tau values into the database from the cleo weather forecast app? Either we (1) modfiy the cleo app so that it can be called by the sdfits program or (2) we regenerate

-- JimBraatz - 08 May 2007