Introduction

All dynasonde data that has been received from EISCAT has now been catalogued and is stored on the central automatic tape store at RAL. The various stages of the analysis can be run individually or as part of a pipeline. Each component in the analysis procedure will be described in detail in this document, along with examples of the use of such routines in a pipeline.

Cataloguing Dynasonde Data using DYNACAT

This is a routine for retrieving a catalogue file of dynasonde data. It works in exactly the same way as TAPECAT, the RAL EISCAT cataloguing program. Inputs have to be in an exact order, but the line of input arguments can be truncated at any point. To run DYNACAT type;

DYNACAT {start-time} {end-time} {site title} {type}

Options

• {start-time} and {end-time} are in the form of a continuous number from year (Y) to second (S) YYYYMMDDHHMMSS. These numbers can be truncated at any point. For instance, the start and end times 19950213 19950213 would produce a catalogue of all the files after the start of the 13th February 1995 (i.e. 00:00:00 hours) until the end of the 13th February 1995 (i.e. 23:59:59). In this way, a catalogue of data covering a whole week or month is possible. It is important to remember however, that when the analysis program, dyAn, acts on files from a catalogue, it writes the results to a single file. If this is extremely large, it may prove too large to transfer, or to load into MATLAB. For this reason, it is recommended that the user analyses data from a single day at most.
• {site} is tro for Tromsø. Currently, there is only one EISCAT dynasonde (and this is at Tromsø). This option exists in order to expand the database to other sounders if required.
• {title} limits the output to files that contain the string specified in title only. When the data is originally recorded, the file written to PC disk has a specific file extension that explains what mode of sounding was in operation. These strings are preserved in the RAL archive filenames and can be used to select the type of soundings the user is interested in. The original extensions consist of three letters. The first indicates the sounding mode, the second letter is a modifier (usually left as an underscore, but can be set to x to represent an example sounding, or r for a receiver test) and the third is a quality designator (either g for good, b for bad or t for test). Several strings can be used to select data simultaneously using the unix command line syntax. Strings are separated by the logical OR symbol, |, and the entire command is surrounded by single quotes, for example; 'b_g|i_g' selects all good b mode and i mode soundings (between the specified dates).

Types of Sounding

There are four types of dynasonde sounding.

• I or Ionosonde mode. This is a monotonically increasing sweep through a range of frequencies. It is the most basic of ionosonde modes.
• The B mode sweeps through the frequency range in a series of ramps in such a way that the same frequency is broadcast several times during a sounding. It is similar to an I mode sounding, with smaller ramps superimposed on it. This enables echoes to be compared over a longer time interval and this reduces any ambiguity in Doppler velocity measurements.
• K mode soundings use a select few frequencies only (usually 4) that have been chosen to correspond to features of interest in the ionosphere (such as the F1 ledge).
• P mode soundings are similar to the K modes in the way the signal is broadcast, but the quality of the received echoes is not tested. As a result, P mode files can be enormous (several megabytes each). These require detailed analysis, not provided by dyAn and so these files are catalogued under a separate type category.

{type} specifies the type of data, either dyna (for data files) or text (for text files containing directory listings).

The output from DYNACAT is a list of where each file is located on the tape store or local disks. This is sent to the standard output, which can either be redirected into a catalogue file with the > symbol, or piped directly into get_dyna by using the | symbol.

Retrieving data from RAL tape store using get_dyna

get_dyna is a convenient, shorthand way of retrieving data from the tape store or local disk. It accepts as input the filenames and paths given by the output of DYNACAT and checks to see if the data already exists on the RAL EISCAT system. If the file is not to be found there, it retrieves the data from the automatic tape store and puts it on disk somewhere on the stager areas of the EISCAT system. get_dyna then sends a list of where the files are located to standard output that can either be redirected into a file or piped into the analysis program dyAn.

Stop Press: New Instructions about using get_dyna (April 2009)

For some reason, get_dyna does not appear to be working with all of our data at the moment - the problem seems to be that the tape numbers generated by the catalogue program are not recognised by the RAL tape store. However, for most Dynasonde data, it is easy to work around the problem by getting the data from the local disk. Use this link for instructions on how to do this.

Analysing Dynasonde Data using dyAn

dyAn is the program that actually analyses the dynasonde data.

dyAn [ -c Calibration-file ]

When running, the analysis program writes information to the screen about the type of sounding being processed; time, date, antenna configuration etc. On the faster machines, this scrolls too quickly to read. [Ctrl] S pauses the display, and [Ctrl] Q resumes. The output from the analysis is fed to standard output and can be redirected into a file.

Options

-c Calibration-file

The current version of dyAn automatically detects the most recent calibration file, and uses this in the analysis. If however, the user requires a particular calibration file, this option allows the analysis to accept a calibration file as a command line input. Calibration files can be found on the EISCAT system in /home/common/dyna_cal, with the name convention DYN[year,daynumber].CAL. A calibration file is automatically selected from these files, corresponding to a time earlier than the data you are analysing, as close in time to that data as possible. Calibration of the dynasonde is required in order to take account of any system losses. The system is tested periodically and this information is used to correct for any system effects in the analysis. The calibration data will only change dramatically if there have been any major changes to the hardware of the system. Calibrations are carried out after every major change, and the system is unlikely to vary much in between. As a result, it is not critical that the calibration file is close in time to the data being studied. A calibration is only really required if the spatial extent and Doppler shifts of the echoes are to be studied. For scaling parameters, such as critical frequencies, from an ionogram (which requires only range and height information) a calibration is only vital if there is considerable spread in the location of echoes, where overhead and oblique echoes need to be distinguished.

dyAn is a version of a program developed by BAS. It has been modified at RAL to accept inputs on the command line and on the standard input channel and has been compiled to run on the EISCAT system. The RAL EISCAT group would like to thank BAS for their support.

Converting output to MATLAB .mat and ASCII formats using dyAn2ascii or dyAn2mat The output format from dyAn is in ASCII, with header information for a single sounding followed by the information in that sounding. e.g.

TR 1995/331  0.04:00 - 1995/331  0.06:14  Type:B JWW & MVLP X-werpol 6-ant pulse set
TIME FREQ_MHZ RANGE_TF ECHO_AMP SIG:NSE RANGE_SP DOP_VEL NORTH_KM EAST_KM POL_DEG
29.28  1.645  230.00   4.2   29.8  229.77    -1.44   126.04   -34.20 81.84
28.96  1.645  297.00   1.8    0.0  295.53  -124.68   206.95   144.74 33.03
29.12  1.650  231.00   4.5   99.9  230.70    -0.32   124.20   -29.51 60.63
29.24  1.650  298.50   1.8   26.8  297.01    89.67   229.36   105.38 8.17
29.08  1.632  102.00   7.7  360.1  102.66   -45.29    -8.24   -31.34 71.29
29.08  1.632  229.50   3.9  182.1  230.11  -110.06   108.95   -45.58 72.37

This format has the advantage of being verbose and explicit but it cannot be loaded directly into MATLAB in this form. dyAn2ascii and dyAn2mat are routines that converts the above format into ones suitable for MATLAB. The output from dyAn can either be piped directly into these routines or redirected in from a file. After processing, with dyAn2ascii, the above file becomes;

1995 331  0  4  0  1.650 231.00    4.5   99.9 230.70   -0.32  124.20  -29.51 60.63
1995 331  0  4  0  1.650 298.50    1.8   26.8 297.01   89.67  229.36  105.38  8.17
1995 331  0  4  0  1.632 102.00    7.7  360.1 102.66  -45.29   -8.24  -31.34 71.29
1995 331  0  4  0  1.632 229.50    3.9  182.1 230.11 -110.06  108.95  -45.58 72.37

An ASCII file with this format can be read into the RAL EISCAT dynasonde visualisation package ui_dynasonde and the EISCAT calibration package ui_cal. It is not necessary to do so, but it is good practice to name ASCII files with a similar prefix, such as dyna.<date>. This ensures that MATLAB will not grab more memory than necessary if the user is analysing several large data files in one MATLAB session.

Analysis Examples

To produce a catalogue of good B and K mode files on the 15th January 1995 and save this in a file called jan1595.cat

DYNACAT 19950115 19950115 tro 'b_g|k_g' dyna > jan1995.cat

To retrieve the above data from the tape store without producing a catalogue file and create a file called files.lst that contains information about where the sounding files have been stored on the system.

DYNACAT 19950115 19950115 tro 'b_g|k_g' dyna | get_dyna > files.lst

To retrieve and analyse the above data using a (fictitious) calibration file and convert the resulting output into MATLAB readable ASCII format, saved in a file called dyna.jan1595.

DYNACAT 19950115 19950115 tro 'b_g|k_g' dyna | get_dyna | dyAn -c 1994342.CAL | dyAn2ascii > dyna.jan1595

To retrieve and analyse the above data using a (fictitious) calibration file and convert the resulting output into MATLAB format, saved in a file called jan1595.mat

DYNACAT 19950115 19950115 tro 'b_g|k_g' dyna | get_dyna | dyAn -c 1994342.CAL | dyAn2mat > jan1595.mat

Viewing Dynasonde Data from within MATLAB using ui_dynasonde

This suite of MATLAB routines, designed for easy visualisation of dynasonde data, is available to anyone wishing to view dynasonde data. It can be run on the RAL EISCAT system, or is available for ftp transfer onto the users local system (which must itself have version 4.2c of MATLAB or later). ui_dynasonde is able to fit true-height profiles to the data by supporting the polynomial analysis package POLAN, developed by J. E. Titheridge. This is a FORTRAN program, and will require compilation on the home environment. In order to access POLAN from MATLAB, it must be compiled using the subroutine polrun3.f. Details can be found with the POLAN source code. Currently, RAL has compiled versions of POLAN for DEC risc, DEC ALPHA and SGI only.

The dynasonde visualisation software is available from the EISCAT website at http://www.eiscat.rl.ac.uk/dynasonde/ui_dynasonde.html It may be freely copied, as long as the information concerning the origin of the source code is not removed from the routines. Once the file has been un-tarred (using the command tar xvf .tar) it can be run by entering MATLAB and typing ui_dynasonde. There is a readme file with this package that explains in detail about each of the user interface controls. Details of how to make the best use of POLAN are contained in the document How to use POLAN which is available at the above web address.