Auto... scripts : Selecting "Gold Standard" sources.

Modification Request 2C807 : November 26, 2007

1. Introduction

The Auto... observing scripts (AutoPeak, AutoFocus, and AutoPeakFocus) in Astrid have the capability of choosing a suitable calibration source. They select sources from a predetermined source catalog, usually called "pointing.cat", located in the directory /home/astro-util/astridcats/.

Auto... scripts select sources based on two criteria:

distance from the current GBT tracking position, and
source flux density.

This MR adds an additional selection criterion based on a new catalog column appearing as of version PCALS4.1.

2. Background

We have been using a pointing calibration list, version 4.0; JimCondon has recently produced a revised list, "PCALS4.1", which has improved data for the higher frequencies. A new category of sources, designated "gold standard" calibrators, have improved position accuracy ( < 0.1") and no sign of extended structure. These "gold standard" sources are included in the new catalog and are indicated by a "P" in column 12.

The improved calibrator catalog is described in PTCS project note 58.0.

3. Requirements

A new parameter is requested for the Auto... scripts in Astrid called "gold", and specified as "gold=True" or "gold=False", the default being false.

We would like the option of telling the Auto... scripts to use only the "gold standard" calibrators, by specifying the parameter "gold=True". If set to True, source selection would only consider catalog sources with a "P" in the "gold" column, column 12. This capability would be of great importance for PTCS observing runs, and also for any high frequency observing where accurate pointing is a high priority.
Astrid must robustly support this extra "gold" column in the standard pointing catalog; in fact, it should be as flexible as the Catalog utility in being tolerant of extra columns. If it happens that the pointing catalog version does not have a "gold" column (which we don't expect to happen), Astrid should not crash or generate errors, but should ignore the "gold" parameter.
If the 'source' parameter is specified, the 'gold' parameter will be ignored. If any of the other parameters are specified, the 'gold' parameter will be incorporated.
Calculation of flux density - When checking that the flux density of a source is above a certain threshold, a calculation is done to interpolate or extrapolate from the frequencies given in the catalog to the desired frequency. As part of this MR, verify that this calculation conforms to the latest version of JimCondon's "calfind2.f" program.

MarkClark has indicated that the requested changes would probably take about a week.

4. Design

This design modifies the AutoProcedure software module of Sparrow's gbt / api / turtle sub-system, effectively adding a new “gold” parameter to all three Auto* scripts, since they all inherit from this module. PointingCatalog software module will be modified to read catalogs files that either have the gold standard "GOLD" column or not, and to provide for the selection of calibration sources from either type of catalog. If the gold_standard parameter is specified by the user for any of AutoPeak, AutoFocus, or AutoPeakFocus scripts in Astrid, then PointingCatalog CalFind() method will return those calibration sources which first have the “gold standard” column value of “P” that also otherwise satisfy the existing selection criteria for flux and proximity.

4.1 Modified / New Sparrow software modules

In gbt/api/turtle/src/proc, modify the following modules:

Module	Method / Data member	Notes
AutoProcedure.py	`DoesSourceSelectionAllowScanToContinue()`	Add "gold" parameter value to method invocation
	`Execute()`	Modify calls to SelectCalibrator() and DescribeCalibrator() methods to pass along "gold" parameter value
	`__init__()`	Add "gold" parameter with default value of `False`

In gbt/api/turtle/src/user, modify the following modules:

Module	Method / Data member	Notes
PointingCatalog.py	`GOOD_HEAD_GOLD`	NEW - Constant set to ['GOLD']
	`CheckKeywordValues()`	Check for header that matches either pre-4.1 (without GOLD) or 4.1 (with GOLD column)
	`GenerateDictionary()`	Add gold standard field to each calibrator source dictionary entry. If no GOLD column, add empty value
	`SelectCalibrator()`	Pass user's "gold" parameter value specification to the selection method, CalFind()
	`CalFind()`	Add the "gold" parameter as the first selection criterion to this algorithm
	`IsSource()`	Add the "gold" parameter to this method determination of 'source-hood'
	`GetGoldStandard(name)`	NEW - Returns "gold" column value for named calibrator
	`DescribeCalibrator(...)`	Add description of "gold" status to the description string for the named calibrator

4.2 Modified / New Sparrow unit test software

A new pointing catalog will be added to the unit test scripts directory, which is a copy of the catalog default.cat, but with the new "gold" column added and purely notional indicators added for each entry. This new catalog is named defaultWithGoldStandard.cat.

Unit tests that will be modified and supplemented to test this MR's functionality:

Location	Unit Test	Notes
`gbt/api/turtle/test/proc`	AutoObsTest.py	Add 18 unit tests corresponding to table in Section 6.1
`gbt/api/turtle/test/user`	PointingCatalogTest.py	Add unit tests to test reading gold standard and non-gold standard sources from catalogs with and without gold standard sources
	CatalogTest.py	Modify to add name of new catalog in `scripts` directory

4.3 Modified / New Configuration files

system.conf (in /home/gbt/etc/config or /home/sim/etc/config/) is where the name of the pointing catalog file is specified. Currently, for release 7.7 of the sparrow software set, the entry in this file reads:

     PointingCatalog := "/home/astro-util/astridcats/pointing.cat"

Although it's most likely that the new version of the pointing catalog will keep the same name, and therefore not require a change to the release version of this file, changing this entry will facilitate testing between two different versions of the pointing catalog file co-located in the same folder, before overwriting the pre-4.1 version with the 4.1 version.

5. Deployment Checklist

What has to get done to integrate this completely into the system. This checklist must be completed before Cycle Integration Testing begins.

Communication with Computing group needed:
- Not required.
What documentation needs to be updated?
- Release Notes, if this feature is held until the next release. Otherwise, a Release Notes like memo will be sent to the same audience as Release Notes when the feature is patched into the current software release.
Training Needed? Is this being released to staff astronomers or everyone right now?
- Training is not required, although a reference to the J. Condon PTCS Memo will be included in the release notification for explanatory purpose. This feature is being released for general use.
Notification Needed?
- Yes, when patched or released.

6. Test Plan

6.1 Internal Testing

Unit tests will be written to exercise the several combinations of catalogs with and without the "gold standard" column, and using the Auto... observation scripts in Astrid either specifying or not specifying the gold_standard parameter:

Obs script	gold standard column	no gold standard column
AutoPeak(gold=True)	Consider only calibrator sources with value of "P"	Consider all calibrator sources
AutoPeak(gold=False)	Consider all calibrator sources	Consider all calibrator sources
AutoPeak()	Consider all calibrator sources	Consider all calibrator sources
AutoFocus(gold=True)	Consider only calibrator sources with value of "P"	Consider all calibrator sources
AutoFocus(gold=False)	Consider all calibrator sources	Consider all calibrator sources
AutoFocus()	Consider all calibrator sources	Consider all calibrator sources
AutoPeakFocus(gold=True)	Consider only calibrator sources with value of "P"	Consider all calibrator sources
AutoPeakFocus(gold=False)	Consider all calibrator sources	Consider all calibrator sources
AutoPeakFocus()	Consider all calibrator sources	Consider all calibrator sources

Written	- FrankGhigo - 27 Nov 2007
Checked	- MarkClark - 13 Dec 2007
Approved by Sponsor	- FrankGhigo - 18 Dec 2007
Approved by CCC	- AmyShelton - 07 Jan 2008
Accepted/Delivered by Sponsor	- FrankGhigo - 14 Jan 2008

Symbols:

Use %X% if MR is not complete (will display )
Use %Y% if MR iscomplete (will display )

CCC Discussion Area

-- FrankGhigo - 26 Nov 2007

Attachment:	Action:	Size:	Date:	Who:	Comment:
calfind2.f	action	12693	21 Dec 2007 - 14:33	RonGrider

Topic ModificationRequest2C807 . { Edit | Attach | Ref-By | Printable | Diffs | r1.16 | > | r1.15 | > | r1.14 | More }

Revision r1.16 - 14 Jan 2008 - 01:25 GMT - FrankGhigo	Content copyright © 1999-2007 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.