NRAO Home  >  Green Bank  |  Wiki Topic:    GB > Software > ReviewExecutiveSummary2004

Changes | Index | Contents | Search | Statistics | Go

This GBT Software Review is being held to provide information exchange, and an update on GBT software status and initiatives, in the context of scientific and software development priorities. The GBT is a unique instrument, with wide frequency coverage (100 MHz to 115 GHz), diverse and highly complex instrumentation, substantial point source sensitivity, and a clean beam. Single-dish observing with the GBT is often very experimental, requiring real-time bidirectional feedback between the observer and the telescope, and enabling a variety of scientific studies such as bi-static radar, ionospheric investigations, and focal plane array development. Because of the sensitivity and high-frequency capabilities of the telescope, and the scarce availability of high-frequency observing weather, dynamic scheduling is a particularly vital component of optimizing GBT use.

Development of the GBT to produce unique science is the highest priority. In direct support of this goal are the Precision Telescope Control System (PTCS) project, which aims to “provide the pointing, collimation and surface accuracy required to allow the GBT to operate effectively at 3mm,” the high-frequency development program which includes the Caltech Continuum Backend and Penn Array Receiver, and the spectrometer spigot card and associated backends. These top development priorities focus on delivering new science capabilities. Other priorities include development of an observer’s interface which enables interactivity and scripted, queue-based observing, a scalable interactive data reduction system with flexibility and extensibility, and pipeline analysis capabilities for advanced imaging. Pipeline capabilities are just beginning to come into focus for the GBT, as the delivery of the Penn Array (for which this will be valuable) nears.

Development Team and Strategic Goals

The GBT Single-Dish Development (SDD) integrated product team consists of 2 astronomers and 8 software engineers, and is responsible for emergency/operational support, maintenance of existing software, and development of new software. Applications within the group’s domain include the GBT control system and its interfaces, observation processing software, data capture, quick look data displays, and data analysis. In 2004, science time and other responsibilities occupied 10% of the group’s time, with an additional 8.4% on inefficiencies for a total of 18.4% overhead. In 2005, the group aims to improve this to 10%. Operational support occupied 12% of the time in 2004, and the goal for 2005 is to have this down to 10% also. Maintenance was only allocated 5% of the time, but a substantial increase is necessary in the upcoming year to prevent operational support needs from becoming prohibitive. New development received 64.6% of the group’s effort in 2004. The group works on GBT’s 6-week development cycle, and regularly receives 25% of the time of two scientists, plus additional sporadic support from other staff scientists on an as-needed basis.

The strategic software goals of the SDD are:

• Enable high-frequency operation of the GBT antenna
• Provide capabilities to generate new science from the GBT
• Increase scientific productivity for observers and staff scientists by making the GBT easier to use
• Conserve vital high-frequency weather by supporting automated queue-based dynamic scheduling and remote observing
• Provide reliable data analysis for all standard GBT observing modes

Pipelines and archiving have not been key drivers for the GBT to date, but are gaining significance as the higher frequencies are enabled. Without instrumentation that reliably and consistently produces raw data, downstream applications are of limited use.

Goals and Objectives for 2005-2006

In CY 2005 and 2006, the SDD aims to support the development of new instrumentation such as the Caltech Continuum Backend and 3mm receiver, installation of these instruments on the telescope, and related commissioning exercises. The group will support the PTCS project, in particular, working with PTCS scientists and engineers to upgrade the existing telescope control system to support inter-device communications. Another goal is to complete a design study for the long-term data analysis solution early in the period, so that a prototype pipeline can be installed for use with Penn Array data. The goal is to adapt the ALMA solution for GBT use through limited customization. To achieve this, development of data capture and calibration subsystems is crucial. A major goal is to eliminate the need for visiting observers to use engineering applications in the course of their observing, and future phases of the Ease of Use project address this specifically. The group must also reengineer software for the IF system and the spectral processor since degradations in performance have been recently noted. GBT seeks to implement a data quality diagnostics system in 2005 thanks to the addition of a student resource. A very significant goal is to move to entirely Scheduling Block based observing by the end of 2005, and implement dynamic queue-based scheduling by the end of the following year.

Precision Telescope Control System (PTCS) and New Scientific Capabilities

The PTCS Project in its current form has been in execution since November 2002. A key requirement identified at the Conceptual Design Review was to deliver concrete, intermediate performance improvements on the way to providing full 115GHz operation. To this end, PTCS development has been following a dual-track approach; on the one hand combining traditional astronomical approaches with simple technology (e.g. temperature sensors) to deliver immediate performance improvements, while in parallel performing system design, instrumentation studies and other activities (e.g. the Engineering Measurement System) aimed towards the complete metrology solution. To date, these activities have consumed ~ 2-3 ftes of software development effort. Currently, the PTCS project is heavily focused on developing improved astronomical techniques, and prototyping new instrument hardware. For this reason, and also because of the pressure from other areas, we have decided to ramp down PTCS software work to ~ 1 fte through winter 04/05. This will need to increase to ~3 ftes from Spring 2005, for at least a year, if we are to meet our existing goals and timescales.

As well as PTCS development, the GBT has an extensive instrument development program. Two key components of this include the Penn Array (64 pixel bolometer array operating at 3mm) and the Ka-band (26-40GHz) receiver and Caltech continuum backend (CCB). The Penn Array and CCB are highly leveraged projects that take advantage of detector development and other instrumentation capabilities available from our University partners. However, in neither case could our partners provide the required instrument control and common-user data analysis facilities. GBT staff have written the majority of the control software for the Ka-band receiver and CCB. Contingency plans are in place to complete at least engineering commissioning of the Penn Array, using laboratory control software and prototype data analysis software. Converting both of these to full common user status will require considerable GBT software resources. These instrumentation projects, and their successors (3mm receiver, wideband spectrometer, focal plane arrays) could easily consume ~2-3 ftes for the foreseeable future.

In summary, the active GBT development program requires at least ~5 ftes of software effort if it is to proceed according to plan. Although there may be limited overlap with other NRAO developments (e.g. the use of ALMA common software for horizontal communications), the majority of this work is extremely GBT-specific. Since this work is all aimed at realizing the unique scientific capabilities of the GBT, we consider it our highest priority.

Data Analysis

The GBT supports a wide variety of users with different data reduction needs. A simple solution is required now to support users, and also to enable development staff to work on longer-term objectives.

In 2004, SDD staff completed analysis on GBT’s standard observing modes and processes for data reduction. This fall, the analysis will be used to generate IDL modules which can serve as the stopgap solution for basic offline spectral line data analysis. The exercise will also be used to generate a first draft of the GBT science data model. In 2005, the analysis deliverables will serve as a basis for longer-term design and development, to focus on learning Penn Array data analysis techniques already conceived and blending continuum and spectral line data models into one cohesive model, influenced by solutions that have already been generated at NRAO or elsewhere (SDFITS, MBFITS, etc.) The fundamental technology to be used is CASA, which has just been constructed by the Interferometry Software Group.

Because the architecture of CASA is similar to the architecture chosen for GBT’s quick look data display, and experimental Penn Array analysis software, this provides a sound basis for unifying all the developments within a scalable framework for pipeline processing within the next few years at the GBT. We anticipate a reinvention of the DISH package on the new framework that will integrate these other developments that have arisen over the past two years for use with GBT data.

Observation Processing

Significant advancements have been made in observation processing at the GBT. A Scheduling Block system modeled closely after ALMA is being released for internal testing and training in September. This will enable offline observation preparation (Phase II), automated dynamic scheduling, remote observing, and data mining on observations made with the GBT (which will be useful for proposal review and selection). All GBT standard observing modes are covered, except those involving near-earth objects such as satellites and comets. This functionality, plus the integration of source catalogs and IF power balancing interfaces, is desired in 2005.

By the end of 2004, software regression tests and “all-sky” pointing observations for the PTCS project will be managed entirely as groups of Scheduling Blocks, serving as the first prototypes for automated dynamically scheduled observations in the future.

Conclusion

In the midst of the new development, operational support and maintenance requirements cannot be overlooked or given less billing than new development work. Sufficient maintenance time must be allocated each development cycle to prevent cascading bugs which can have large and noticeable impacts on downtime and telescope utilization. Given past experience, we know that 5% of the SDD’s effort (the measured amount of maintenance time spent in 2004) is insufficient to effectively prevent excessive downtime, and a shift must be made to achieve operational dynamic stability. The intended target is 25%, which proved to be a very useful allocation in 2003, when several key operational issues with the spectrometer and LO were resolved once and for all.

The SDD is routinely overbooked, and tasks must be chosen parsimoniously with the critical path for all projects in mind. As compared to the amount of work that would be desirable to accomplish, the SDD is 68.8% overbooked with respect to work hours required for task completion, testing, and releases. Scientific resources are also spread thinly. Typical tradeoffs considered are allocating effort to maintenance tasks vs. new development, and spending effort on ease of use and observation processing vs. data analysis.

-- NicoleRadziwill - 22 Sep 2004