NRAO Home  >  Green Bank  |  Wiki Topic:    GB > Data > TWikiUsers > RonMaddalena > SpecLineCalibration > CalibrationStatus

Changes | Index | Contents | Search | Statistics | Jump to Topic

Status of the Calibration Project for the GBT

The following outlines where we currently are in the development of calibration memos, software, documentation, etc. Plus descriptions of the steps that are in progress or are yet to begin.

Astronomical SCAL

Actually there are two aspects of this project. Determining SCAL over the full band of a receiver and the determining of SCAL over a narrower, single band. The first is more complicated and should be done by a staff member. The latter should be something we should make easy for the observer.

Full Receiver passband

Status:

• GLISH scripts completely implemented -- since December, algorithms augmented to handle high frequencies where efficiencies are unknown and depend upon frequencies; and where opacity is frequency dependent.
• Reduction requires hand massaging of the data for RFI, edge affects, ... (i.e., tools are staff-oriented) so as to provide something useful for the engineer's TCAL data base.
• Draft technical memo written

To Do

• GLISH scripts need to be converted to IDL
• Staff training on how to use them
• We don't have astronomical measurements for C-, Ku-, Ka-, Q-band, PF1 (4 bands), and PF2. And as needed for other receivers as they are altered. C-, Ku-band and PF2 were done in August 2005 - RJM 10/19/05; PF1 800 MHz were done in January 2006 - RJM 3/1/06
• R&D: Although Tsys are a byproduct of the scripts, we still have DC offsets that are hardware induced that need to be explored by the engineers. The high resolution Tsys plots we currently provide have been critical in some observing projects to help determine instrumental affects, determine frequencies to avoid, etc.
• Need development of rudimentary plotting and IO in IDL for developing routines for continuum SCAL observations

Observer's passband

To Do:

• Can use the same GLISH scripts (and IDL scripts, once the GLISH scripts are translated).
• Need user-oriented ASTRID and GBTIDL documentation.
• User documentation for the various interface for our SCAL/TCAL/Tsys/Trx tables and graphs

Engineer's TCAL values

These are still necessary if we want to determine telescope efficiencies. But, we now only need a handful of values across the passband of a receiver instead of a fully-sampled data set. However, the trade off is that these few engineer's values must have a high degree of accuracy.

My draft memo with Chelen describes how one goes from astronomical determined SCAL values, and the handful of engineer TCAL values to a high frequency resolution, high accuracy table of astronomically determined TCAL values.

Weather Information

Air Mass

1/sin(el) approximation starts to break down below 20 deg for high accuracy calibration. I have developed a better approximation than 1/sin(el) that is good to 5 deg. But, this used only 6 months of my archived weather data.

To Do:

• Completed 2/2006 - RJM 3/1/06 Now that I have a full year, I need to redo the fit. And, for the community, I can now extend the fit down to 0 deg elevation. This has been done and the algorithm now extends down to 1 deg elevation. Below that, the function I'm fitting doesn't work as well as I would like but what I have is more than sufficient for GBT data
• Need a way to impliment the new air mass estimates into GBTIDL/GFM.

Opacity

The current AIps++, GBTIDL calibration algorithms already accept scalar opacities. This isn't sufficient for high frequencies where opacity can double across our wide bandpasses. We thus need to alter our algorithms to use opacity vectors/tables/function (see vector data bases below).

I now have an alpha version of a GUI frontend for my opacity forecasts that anyone can now use to determine opacities within the last year at frequencies of 5 to 110 GHz. But, we need a way to get the output of this program into a opacity vector that GBTIDL can use.

Users may want to measure opacities using the GBT. We need the tipping plugin for GFM for low frequency, narrow band observers. Wide band, high frequency spectral-line observers should use multiple DCR-AFR samplers for their tippings so as to sample multiple narrow band frequencies that are scattered across their full spectral-line observing bandwidth. Line tippings aren't practical due to the limited dynamic range of our backends.

I've pretty much given up on the concept of using the 86 GHz tipper as a way to reliably determine opacities.

To Do :

• GFM Tipping implemented
• Continuum analysis in GBTIDL including Tipping fits in GBTIDL
• ASTRID and GBTIDL documentation; config-tool examples of multi-frequency AFR-DCR setups.
• Add calculation of opacity from hydrosols Beta version of an opacity algorithm is now in place but it might need some tweaks as I learn more - RJM 10/19/05 Current algorithm is satisfactory - RJM 3/1/06
• Need a way to impliment using model/forecasted opacities in GBTIDL & GFM.
• R&D : Combine the in-situ opacity at a single frequency from a tipping with the CLEO results to extrapolate opacities to other frequencies.

Extended opacity modeling to 230 GHz - RJM 3/5/06

Tatmosphere

Values for the effective temperature of the atmosphere are needed before one can fit a tipping curve. My CLEO forecast application can now provide users with a value suitable for their observing frequency, date, etc.

Found a way to use a lookup table to provide Tatm to an accuracy of 5 K, better than that required for 1% calibration accuracy - RJM 3/5/06

To Do:

• Need a way to impliment lookups for Tatmosphere into GBTIDL & GFM.

Non linearities & Calibration

The assumption that Pout = B * Pin fails for the GBT for high accuracy calibration. We've been exploring the Taylor expansion of Pout = f(Pin) where f is an arbitrary function. Using On-off data of a known calibrator, and using the noise diode, we can determine up to the second-order terms in the Taylor expansion. [Note: This work is orthogonal to Toney's efforts on baseline shapes in the presence of non-linearities.]

Although one can very reliably measure the existence of non-linearities, and even estimate their affects on data that's been processed as if the system were linear, it's not clear that one should actually correct the data for non linearities. We've had mixed results doing this and more work is needed.

A draft memo exists explaining the measurements and data reduction.

To Do :

• Convert our GLISH scripts into GBTIDL
• Continuum data reduction in GBTIDL
• R&D : See if one can actually correct for non-linearities.

Aperture Efficiencies

Two aspects of this project. Yuri/Frank's efforts are meant to determine efficiencies as a function of elevation at a single frequency within a receiver's band. The absolute accuracy of their results are compromised since they are using the engineer's TCAL that have an accuracy of ~10%. Since the inaccuracy of Tcal and efficiencies wash out in their analysis, their results don't compromise the quality of an observer's calibration if the same Tcal tables, exact observing frequencies, etc. are used.

My efforts with Chelen of combining high accuracy engineer TCAL values with astronomical measurements provide efficiencies (frequency-dependent at the higher frequencies) that have a higher absolute accuracy but only for a single elevation.

To Do:

• Use Yuri/Frank's data to measure the relative change of efficiency with elevation, and Chelen/my efficiencies at a single elevation to derive efficiencies with high absolute accuracy. How to do this is explained in my calibration thesis.
• Yuri/Frank have not measured all of the receivers. Probably Ku and above need to be done. Likewise, I need PF1, Pf2, C-, Ku-, Ka-, and Q-band SCAL data.As stated above, PF2, C- and Ku-band data are now available - RJM 10/19/05
• A GBTIDL function that will take a SCAL and TCAL value, plus opacity, and return efficiency.
• R&D : We've seen for both K- and Q-band efficiencies that are feed dependent.

Beam Efficiencies

Every pointing measurement we do has the unexploited benefit of providing the conversion factor to derive beam from aperture efficiency. This is discussed in my calibration memo.

• Augment GFM to provide the ratio of beam/aperture efficiency.
• Continuum analysis in GBTIDL to derive this same value from pointing data.
• Continuum & line analysis in GBTIDL to derive cruder estimates from on-off measurements of planets, moon, etc.
• Use Yuri/Frank's data to derive beam efficiency as a function of elevation for the high frequency receivers. Probably everything below K-band will have beam efficiency curves that are essentially flat (beam efficiency curves are always much flatter than aperture efficiency curves).

Other calibration factors

Except for some theoretical prediction, many that uses an optical setup that's different from the final telescope, we do not have measurements of eta_l, eta_fss and Tspill(elevation). We may never be able to measure these directly. The accuracy of our calibration, the accuracy of opacities from tippings, ... are somewhat compromised by this lack of good measurements.

To Do:

• R&D : derive ways we can measure these factors with the GBT
• R&D : Use the as-built optics to derive theoretical estimates of these quantities (Roger/Sri?).

Calibration Algorithms

The current state of the calibration memo has been sufficient for us to implement the rudimentary, scalar calibration now done in GBTIDL. The GBTIDL algorithms are robust for some classes of experiments and are typical of what one finds in all single-dish analysis packages.

We've built into GBTIDL the concept that our calibration routines will always act as templates for people who want to go beyond what we've developed so far. Although our GBTIDL algorithms will be useful for the majority of our users, there's a number of classes of GBT experiments where we can do better and will need to go beyond what has, up to now, been typically provided by observatories.

To Do:

• Finish the calibration memo. The sections still to be done are:
  • Calibration algorithms for the baseline methods that Toney has developed since I last worked on the memo.
  • How to resample the engineer's TCAL table to produce a TCAL vector.
  • Finish the vector Tsys, Gain, etc. sections for about three remaining calibration methods
  • Minor revisions to the air mass, opacity sections
  • The section on calibration in the presence of RFI and strong lines.
  • Appendices that are essentially the draft SCAL and non-linearities memos Chelen, Hogg, and I are writing
• A science data model that would abstract out the calibration. The draft memo has enough in it that developing the SDM need not wait for the memo to be finished.
• A decision-tree one can use to pick, a priori, a calibration methods that's reasonable for the combination of an observer's object, expected line width, integration time, desired accuracy, frequency, receiver, observing method, .... The choice will usually involve compromises. We need to begin developing the decision tree for basic observing methods. But a lot of the branches will be developed by observers so we can expect the tree to mutate and evolve from our original versions.
• Implementation of aspects of the SDM & the memo in the way data files are created from our engineering FITS files . In particular, storing vector Tsys, Tcal, opacities, ... values is a paradigm shift that we've not yet fully thought through how we'd implement. Bob and I have been considering using a database file, separate from the SDFITS file. This needs to be fleshed out over the next few months.
• Modify GBTIDL to take advantage of the not-yet-implemented methods in the calibration memo, in light of the SDM, for vector Tsys, etc.

Pipelining/E2E/UFO

Once the above infrastructure exists, the next step will be how one can specify the desired calibration when pipeline processing data. The number of options to specify the desired calibration is large so the UI problem will be interesting to solve. The above mentioned decision tree will be critical for helping the user decide.

Since calibration involves compromises, we can expect that a user may want to specify multiple calibration options and methods, implying that the pipeline will fork producing multiple calibrated data sets. Forking pipelines will also add interest to the E2E efforts.