Grail: A Generic Programming Interface to the GBT

Grail Client | About SOAPpy | Current Open Issues | Code Review Results | Current Status

*WARNING:* Grail is not recommended for general use by astronomers! Instead, please use one of the APIs (Configuration, Observing, or Data Preprocessing). Grail is designed for use by programmers to help them write APIs and applications which are required to directly interface with the control system. Please notify RamonCreager and NicoleRadziwill if you are an astronomer who wishes to use Grail for application development.

Introduction

Grail (which is neither Holy nor an acronym) is an Ygor service that provides a Simple Object Access Protocol (SOAP) interface to the M&C system. Using this interface, clients will be able to control all aspects of M&C operations. The Generic Interface will provide full access to Managers, their parameters and their samplers.

Why a SOAP based interface? SOAP provides a language neutral, OS neutral and architecture independent messaging and Remote Procedure Call (RPC) facility. To date, SOAP libraries are available for a variety of languages, including (but not limited to) C, C++, C#, Visual Basic, Delphi (Borland Pascal), ADA, Java, JavaScript, Python, Perl, Ruby, Tcl, SmallTalk and PHP. Implementations of these languages and SOAP libraries for them are available for all major operating systems: Win32, Unix (Linux, Solaris etc.), and Mac OS/X. Finally, because the SOAP protocol is implemented in XML using ASCII, it is completely architecture neutral (no byte alignment/endian issues, etc.) and human readable.

Development Plan

Grail in its current state (October 2003) has the following capabilities:

You can access it using any programming language that supports SOAP. For the purposes of this discussion, Python will be used as the client language. The server itself is written in C++.
It exposes full Manager interface
It keeps a cache of all parameters for a given manager. If a new value is sent, it is only passed on to the manager if the new value differs from the old one.
If you specify a bad M&C device, parameter or value for that parameter, you will get an error message
Connections from the client to the SOAP adapter are made on a per-request basis: that is, when the client makes a request, the TCP connection is made. When the request is served, the connection is closed. There is no continuous connection between a client and the SOAP adapter.
Returns device and sub-devices, retaining hierarchy between devices and sub-devices.
Provides a polling-only interface to samplers.

The following capabilities may be added in the future. Some may not be appropriate for enhancements to this program.

Full interface to Samplers
Maintain connection with all devices, even after the latter have been reset or shut-off and have come back on-line.
Provide callbacks for event notification (parameter changed, etc.)
Show all parameter details: type, range of values etc.
Document, through the interface, the SOAP adapter's services.

Prerequisites:

A client to Grail is easy to write in Python. Grail clients written in Python should run equally well on Windows, Linux, or Solaris. The only requirement is an appropriately configured Python 2.x installation that includes the SOAPpy (Python SOAP library) module. Nothing else is needed. (For the die-hard Perl fanatics out there, it should be just as easy to use Perl with SOAP::Lite to write Grail clients. Most of what follows on installing and running Python also applies to Perl).

Linux/Solaris:

The general Python installation currently does not include SOAPpy. Until this deficiency is rectified users wishing to run Grail clients have two choices: install their own copy of SOAPpy and use the global Python installation, or use the M&C Python installation that does include SOAPpy. To do the latter users must source /home/gbt7/igkya/igkya.bash every time prior to running Python.

Windows:

Windows installations generally do not include Python at all. Thus, the first step is to instal Python 2.2 or above. ActiveState's ActivePython is a good choice. Next, install SOAPpy.

Python usage (Windows or Linux):

Python can be used interactively, or on the command line to run a script. To start up the Python interpreter, type python in the shell (ActiveState users can also use PythonWin, a nice Python interactive environment). Or, a Python script can be written in your favorite editor and executed later: python yourscript.py will work on both platforms.

The Grail daemon:

Two Grail TaskMaster processes run on Vortex. Both should be up and running at all times. One, accessed through TCP port 18000, gives complete control to the client. The other, accessed at port 18005, is a read-only version.

NOTE: Grail is capable of controling any GBT M&C manager. Misuse of Grail can cause interference with legitimate use of the telescope! Always use either the read-only Grail on vortex, or your own copy of Grail in simulation mode, or your own copy on a telescope simulator (see below) when experimenting with its capabilities.

Those who wish to experiment with Grail but not use it to control telescope devices should checkout the code from CVS, compile it and run a separate copy of it in simulation mode on a solaris machine. Grail is located in the ygor source tree at ygor/user/Grail. It requires ygor libraries to link properly, so make sure your YGOR_ROOT is set appropriately.

Once compiled, you can use a couple of command line parameters to customize Grail's behavior:

-s: Simulation mode. This runs Grail in a read-only mode. All parameter and sampler values can be read, but any call to modify anything on a manager will result in the call merely being echoed on stdout. When starting Grail with the -s option, Grail will print a message stating that it is in simulation mode. Be sure to look for this message if you do not wish to control a manager! In the following example, the ToyTelescope manager 'Undulator' was turned on by a client, but Grail only prints out Undulator.on() without actually turning the manager on:

     [rcreager@titan sparc-solaris]$Grail -s
     NOTE: Grail is in simulation mode
     Socket connection successful: master socket = 3
     1: accepted connection from IP = 192.33.116.175 socket = 4
     Undulator.on()
      request served

-p port: The port number you wish to have Grail listen for connections. Using this option allows you to avoid interference with a port already in use, and allows multiple copies of Grail to run on the same machine. NOTE: To find a free port, use the command netstat -a and look for a free area, or netstat -a | grep port, if you have port in mind. If the port does not show up in the netstat output, it is probably OK to use it.

Grail will look for the environment variable YGOR_TELESCOPE, so an alternative way to experiment with Grail is to set YGOR_TELESCOPE to something other than the GBT. I use Mark Clark's ToyTelescope to test Grail during development.

Sample Python Client Examples:

Set up a Grail SOAP client object (assumes that the Grail service is running on titan at port 18000):

>>> from SOAPpy import *
>>> cl = SOAPProxy("http://titan.gb.nrao.edu:18000", namespace="urn:grail")

Show the names of all managers on the system:

>>> for i in cl.show_managers():
...    print i
...    
QDDataAcquisition.QDDataAcquisition
ScanCoordinator.ScanCoordinator
Archivist.Archivist
SwitchingSignalSelector.SwitchingSignalSelector
DCR.DCR
Holography.Holography
SpectralProcessor.SpectralProcessor
SpectralProcessor.SpStorage
SpectralProcessor.SpectralProcessorA
(etc.)

Find out names of all parameters with a manager (note the ./, convention when referring to managers; see section "Services" for an explanation)

>>> a = cl.show_params("LO1.LO1A")
>>> for i in a:
...     print i
...    
{u'name': 'asap', u'description': 'controls whether the manager starts as soon as possible'}
{u'name': 'requestedStartTime', u'description': 'target start time of next scan if asap flag is clear'}
{u'name': 'requestedStopTime', u'description': 'target stop time of next scan'}
{u'name': 'startTime', u'description': 'expected start time of the next scan'}
{u'name': 'scanLength', u'description': 'expected duration of next scan'}
{u'name': 'nextScanNumber', u'description': 'scan number of next scan'}
{u'name': 'scanNumber', u'description': 'scan number of current or last scan'}
{u'name': 'projectId', u'description': 'project identifier string < 16 char'}
{u'name': 'source', u'description': 'source identifier string < 32 char'}
(etc.)

Get a parameter value:

>>> cl.get_value("LO1", "restFrame")
'Local'

Set a parameter value:

>>> cl.set_value("LO1", "restFrame", "Local")
'OK'
>>> cl.send_values("LO1")
'OK'
>>> cl.reg_change("LO1")
'OK'

Show all of a manager's samplers:

>>> s = cl.show_samplers("TemperatureMonitor")
>>> for i in s:
...    print i
... 
{u'name': 'StructureTemp', u'description': 'Temperature Sensor readings'}

Poll a sampler on a manager (Note the empty string provided to reg_sampler() and unreg_sampler(). This is an optional callback URL, should the user wish to subscribe to the sampler.):

>>> cl.reg_sampler("TemperatureMonitor", "StructureTemp", "")
'OK'
>>> cl.get_sampler_value("TemperatureMonitor", "StructureTemp,gbtts1_2001")
'5.29'
>>> cl.get_sampler_value("TemperatureMonitor", "StructureTemp,MJD")
'52914'
>>> cl.get_sampler_value("TemperatureMonitor", "StructureTemp,seconds")
'71808.809368'
>>> cl.unreg_sampler("TemperatureMonitor", "StructureTemp", "")
'OK'

Services

There are some common aspects of the Grail services that bear discussion

Service names

The names of most of the Grail SOAP methods are closely allied to the names of the corresponding Manager class methods. The exceptions are Grail methods that do not map to a Manager call. The biggest difference between Manager class method names and Grail method names is that Grail uses underscores to separate words, not capitalization. Also, one name in particular had to be changed (invoke() into invoke_param()) to avoid a name conflict with a SOAPpy proxy member function name. The descriptions and explanations of most of these functions were taken outright from Manager.h, or are closely based on those explanations.

Manager names

Ygor managers are named according to a 'major,minor' convention. Ygor managers can be part of a very shallow hierarchy of managers (coordinators and submanagers). Coordinators can have many submanagers. Submanagers cannot have their own submanagers. The 'major' portion of the name refers to the coordinator, and the 'minor' portion to the submanager. In single or top level managers, 'major' and 'minor' are the same. Grail reflects all this by requiring managers to be specified in a 'major,minor' format. For those managers where 'major' is the same as 'minor', Grail allows the shortcut of using only the 'major' portion. For example:

    cl.on("LO1")

is the same as

    cl.on("LO1,LO1")

A submanager must be named using the 'major,minor' format. 'LO1A' is not a shortcut for 'LO1,LO1A'.

NOTE: Grail allows either a period or a comma to delimit the manager names. For consistency with parameter paths, it is recommended that the comma be used.

Parameters and Samplers

Ygor parameters are dynamic objects that consist of many data elements. At this time, Grail cannot send or receive entire Ygor parameters in one function call. For this reason, and because it is often convenient to access single fields in a parameter, Grail supports the use of the Ygor TCL interface that allows individual fields in a parameter to be accessed using a field path. There are three simple rules in using field paths:

A path must extend from the top and end in a field that does not posess sub-fields.
Fields are represented by their names and separated from parent fields by commas, with no spaces between names and commas.
Array elements are represented by numbers (1 based), delimited from the owning field by a comma. Again, no spaces should be used between names or indexes and commas.

Here is an example that shows both of these rules in action:

    cl.get_value("Undulator", "dynamicParameter,1,a")

Here, we are accessing field 'a' of element '1' of a dynamic array 'dynamicParameter'.

Reading parameter values

Values are read simply by calling the get_value() function. This returns the latest value received from the manager by Grail. Grail does not support callbacks at this time, so parameter values must be polled.

Reading sampler values

Grail does not support callbacks at this time, so samplers must be polled to get values from them. A sampler must first be started before it is read. Once started, it can be read any number of times. When done, it is good practice to stop the sampler. Calls to start and stop samplers are reference counted. That is, if sampler 'a' is started by one client, and then started again by another client, Grail only starts it once. It will however take 2 calls to unreg_sampler() to close the sampler.

Writing parameter values

Changing parameter values must be done one field at a time, using set_value(). This changes a copy of the parameter on Grail without actually modifying anything on the manager. Once all changes are made to the parameter(s), Grail must be told to send all modified parameters to the manager. This is done with a call to send_values(). Finally, the manager itself must be told to incorporate these new parameters. This may be done with reg_change() or any of its related functions.

For examples illustrating all of these actions, see the Sample Python Client Examples section.

Alphabetical listing of services

xsd:string abort(xsd:string device)

This method places the Manager into a Aborting state which initiates any activity needed to stop the operation of a system. This is a transistion state and would be immediate for Managers that require no shutdown actions, i.e., have no need to for cleanup or resetting between scans. This is almost always relevant only for "synchronous" systems. The rule here is to halt the system as quickly as possible. Also in a Coordinator, unlike stop(), abort() is unconditional(), i.e., the command is acted on whenever possible, and passed on to child Managers.


NRAO Home > Green Bank \| Wiki Topic: GB > Data > GbtInterface

Changes \| Index \| Contents \| Search \| Statistics \| Jump to Topic

Revision r1.20 - 23 Feb 2005 - 18:27 GMT - RamonCreager	Content copyright © 1999-2007 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.