4: How to get at the data
Available Data Types
The first question which you need to ask yourself is what kind of EISCAT data you are interested in. Essentially, the EISCAT data are of two main kinds:
Analysed Data
These data consist of altitude profiles of plasma parameters such as electron density, ion and electron temperature and plasma velocity. In order to produce data in this form, the raw data (lag profiles or autocorrelation functions) have already been reduced with an analysis program such as GUISDAP.
If you simply want to take a first look at the data to see whether a given data set is of interest, then the analysed data are a good place to start.
In this case, you can skip ahead (for the moment at least) to Section 11 of these notes. Sections 3 to 10 aim to provide you with the information that you need if you are planning to analyse the radar data for yourself.
Note that, although EISCAT endeavours to ensure that the quality of its analysed data is as high as it can possibly be, no single analysed data set can ever be truly definitive, since incoherent scatter data analysis is an instrinsically under-determined problem, and any analysis is thus the result of the assumptions on which it was based. Hence, even if you start by looking at analysed data sets to identify periods of interest, you might eventually find yourself returning to the raw data in order to re-process it in some other way.
Raw Data
What we conventionally refer to as "raw data" from EISCAT are really auto-correlated data, made up of range-gated autocorrelation functions or lag profiles, pre-integrated to a given initial time resolution anywhere between 1 and 10 seconds.
If you are interested in this kind of data, it is probably because you want to run your own data analysis in order to obtain derived plasma parameters, using GUISDAP or (for mainland data older than 2000) the RAL analysis program. Alternatively, you may be interested in plotting either the correlation functions themselves or (more likely) the equivalent ion line or plasma line spectra.
Before you can begin to work with the raw data, you need to locate them in the data catalogue (Section 5) and extract them from the archive (Section 6). Because EISCAT data are semi-stochastic and hence noisy, you will probably need to perform some post-integration on the data before you can begin to use them (Section 8). Once you have some appropriately post-integrated raw data, you can either plot them (Section 9) or reduce them into plasma parameters (Section 10). Since either of these tasks will have involved several stages by the time that the data come to be plotted or analysed, you can create process pipelines using unix pipes (Section 7) to connect the various stages together.
Sample-Level Data
There is also a further type of EISCAT data, the rawest of all, comprising the storage of uncorrelated sample streams. Storing the data in this way allows the user the highest degree of processing flexibility, since one can control not just how the data are subsequently integrated, but also how they are initially autocorrelated.
However, the storage of sample-level data tends to be reserved to a few specialised applications, such as interferometry, on the basis that the data sets it produces can be huge (Gigabytes per day). Although some experiments do save their uncorrelated data (usually in a matlab array called d_raw), EISCAT does not guarantee to enter such data into its permanent archive, though currently it does so in practice. While EISCAT does support the collection of uncorrelated data to some extent, users working with sample-level data should in principle be prepared to collect such data using their own resources (e.g. by bringing removable disk-packs to the radar sites and returning data to their own institutions for subsequent post-processing).