Some users may prefer not to use the pre-existing utilities to visualise analysed data, but to work with the data inside an interactive visualisation environment such as matlab or idl. For such users, RAL supports software to take analysed data in rslt format and read it directly into matlab or idl sessions.

Sanity Checks on rslt files

Before attempting to use rslt files in conjunction with any of our other software modules, it is probably advisable to make sure that the files have a regular size (n times * m gates, if that is appropriate - note that this may not be possible for scanning experiments whose analysis is based on altitude gating) and that the start and end dates in the rslt file are correct. To regularise the result matrix into an n*m array, if that is possible, use the routine ncar nrow_constant:

ncar nrow_constant {rslt filename}

Note that it is not always possible to do this, for instance in the case of scanning experiments where the data are gated on height rather than range. In this case, the nrow number in the rslt file will be set to 0, and a different nrow will be used in each data record.

The routine

ncar update {rslt filename}

can be used to check that a number of properties of the file are correct.

• The site code (Tromsø UHF=72, Kiruna=71, Tromsø VHF=74, Sodankyla=73, ESR=95) is an international standard number for each radar facility, as defined under the NCAR format.
• The experiment code is a specific code number which will allow the type of experiment to be recognised later, see /home/user2/ral/catalogue/ncar.exp for a list of the supported experiments.
• The end time of the file is updated. Note that the end time cannot be set during the analysis as it is contained in the header of the rslt file, which is written at the start of the data analysis. At the time that this is done, there is no way of knowing how long the analysis will be, so this end time has to be fixed up retrospectively.

It may be that neither of these checks (which currently can only be done on the older eiscat machines) is strictly necessary. However, they offer the user the security that the data is really of the length and dimensions expected.

Handling rslt files

The RAL group has a large amount of software to handle "RAL NCAR" (rslt) format data files. Much of this software is currently only available on the older eiscat machines (eiscatd etc). On these machines, giving the command

ncar

will produce a list of the available utilities, which are currently

• ncar update
• ncar nrow_constant
• ncar extract
• ncar rename
• ncar compress
• ncar remove_errors
• ncar identify
• ncar points
• ncar dir_split
• ncar time_expand
• ncar timesets
• ncar cycle
• ncar printout
• ncar cat
• ncar dump
• ncar convert1
• ncar plot
• ncar expand

as well as showing the syntax for using them.

Of these utilities, update and nrow_constant are described above, while identify is described in Section 11 and velcom in section 13.1. Of the remaining commands, the most useful are:

ncar extract

allows certain parameters (specified in a var file) to be extracted into another file. A special version of this command exists for the cp6 experiment, which has different analysis parameters to those of every other EISCAT experiment.

ncar compress

reduces the size of rslt files by removing certain fields (such as the fitted acf) that are not needed by most data analysts.

ncar remove_errors

reduces the size of rslt files by removing the error estimates on the fitted parameters if these are not needed by the data analyst

ncar points

produces a list of the radar pointing directions contained in the data

ncar dir_split

uses a file of pointing directions (produced by ncar points) to split a rslt file into several data files, one for each pointing direction.

ncar time_expand

can be used to "artificially expand" the integration time of a data file, i.e. it enables the user to fool other programs into thinking that the integration time is longer than it actually is. This can be a useful feature (e.g. in velocity combination).

ncar printout

writes the contents of a rslt file as ascii. See also ~ian/programs/ncar_2_ascii for a version with slightly cleaner ASCII output

Most of these utilities were developed by Steve Crothers, who can provide help in their use.

A note about var files

Despite being basically invisible to users, var files (also sometimes called pdf files) have an important role in all our software utilities for handling rslt format data. The var file is basically a list of all the permissible variables that an analysed data file might contain. These variables (ranging from altitude and electron density to various exotic covariance parameters produced by the fitting routine) are listed together with their code number (in most cases as defined by the international NCAR format), the instructions necessary to decode them and the units in which they are expressed. For instance, electron density is coded in rslt files as 1000*Log10(Ne) (in m-3) and it is the (de)coding information in the var file that allows it to be translated to and from this format.

On the newer machines, the directory /soft/eiscat/etc/ncar/var contains example var files. On the older machines the equivalent directory is /home/common/var.

While most users will happily use the default var files without ever realising that they are doing so, non-standard var files can be used, for example, to limit the contents of a rslt file to selected variables only, or to prepare a file containing only a subset of analysed parameters from a much larger set.

Reading rslt files into matlab.

The following syntax can be used to read data from a rslt-format data file called test.rslt into matlab:

[prolog,time,d1names,d1data,d2names,d2data,ntime,ngate,acfs]=read_ncar(test.rslt);

The d1names and d2names arrays contain the (ASCII) names of the one-dimensional and two-dimensional variables retrieved from the file. The d1data contains the one-dimensional variables (i.e. one value per integration period) such as azimuth, elevation and transmitter power. The d2data contains all of the two-dimensional parameters (those with gate-dependent values). The time array contains the times in julian day form, with a start date and time and an end date and time for each integration period. This means that the time array has dimensions of (ntimes,4).

The d2data array will initially come out as a 1-d array of length (gates*times). This has to be reshaped according to the number of times (which can be found from the dimension of the time array). For instance, if d2names(:,4) is "NE" then the electron density is the fourth of the d2names variables. In this case, the electron density values can be extracted from the file using:

ne=d2d(:,4)
ne=reshape(nheights,ntimes)

If this example seems too abstruse, a quick look at the inner workings of a matlab program based on the read_ncar utility, e.g. /home/eiscat/ian/matlab/plotting/cmp.m, will provide a practical example of how to use the utility.

Reading rslt files into idl.

A number of utilities exist to read rslt-format analysed data files into idl. These utilities exist on the clustered nodes in the directory /soft/eiscat/shlib/idl/ncar. The functions in this area provide some kind of basic toolkit for getting analysed data into IDL and to look at it.

For instance:

c=ncar_open1(`test.rslt')
d=ncar_read_profile(c)

will produce an idl data structure, d, containing all of the variables extracted from the rslt-format file, test.rslt.

Again, inspection of a simple program like /home/eiscat/ian/idl/prof_plot.pro will show a simple example of how these software utilities can be used.

Reading rslt files into FORTRAN programs

Because the format of rslt files is very simple, it is relatively easy to read data from rslt-format files into FORTRAN programs. The program /home/eiscat/ian/programs/ncar_read.f provides an example.