ERS Raw Data Processing

This is a Quickstart guide on how to use the scripts ERS_pre_proc and ERS_proc_all to automate processing a stack of raw ERS data sets to generate SLC and MLI image products. The data stack should all cover the same image frame.

These scripts are run from the command line. Think of theses scripts as as assistants to help you automate the processing tasks. To become familiar practice with a stack of 1 or 2 images to become familiar with the processing flow. Then you will be ready to go and process a much larger stack. The maximum number of images is essentially limited by your available disk space :)

The script functions are summarized below:

ERS_pre_proc	Generate MSP processing parameter files, extract DELFT precision state vectors, determine Doppler parameters, and generate processing parameter file for ERS_proc_all
ERS_proc_all	Process the raw data sets and generate SLC and MLI images, generate ISP image parameter files

ERS_pre_proc is used to perform various pre-processing functions and generate a text format data file (proc_list) that contains various additional processing parameters that are used by ERS_proc_all. ERS_proc_all takes the proc list and its command line parameters to calculate a set of SLC images and MLI images from the raw data. The scene specific Doppler parameters and extent both in range and azimuth is specified in the proc_list.

1. Raw data pre-processing using ERS_pre_proc

To begin processing select or create a directory on the disk where the raw data and processed images will be stored. The data can be located in different directories, though this is not recommended. You should also have a directory where the Delft state vectors have all been downloaded. See the HTML documentation on DELFT_proc2 for information on how to download the state vectors.

Next run ERS_pre_proc without arguments to get the following screen output:

*** ERS_pre_proc

  *** Copyright 2014, Gamma Remote Sensing, v3.9 3-Feb-2014 clw
  ***

  *** ERS raw data pre-processing ***

  

  usage: ERS_pre_proc <CEOS_list> <path_to_DELFT>
  <out_dir> <log> <proc_list> <mode>
  [keyword] [value]

  

      CEOS_list    (input) parameter
  file with 1 column entry/record

            1. directory
  (including path) containing CEOS data set from ERS 

      DELFT_path  path to DELFT orbits
  (example: /usr/local/data/DELFT)

      out_dir     directory for
  output fixed raw data files and processing parameter files

     
  log         (output)
  ERS_pre_proc processing log file 

      proc_list   (output) processing list
  for use by ERS_proc_par and ERS_proc_all

      mode       
  processing mode:

         
            1: create
  processing parameter files from CEOS leaders

         
            2: fix raw
  data files (read and scan for missing lines) 

         
            3: extract
  and interpolate DELFT state vectors and update MSP processing
  parameter files

         
            4:
  estimate Doppler ambiguity and centroid

               
      5: set value in the processing parameter files for
  a keyword:value pair

                 
    6: generate processing list for use by ERS_proc_par and
  ERS_proc_all

         
            7: gzip
  fixed raw data files

      keyword     (mode 5) keyword in
  the MSP processing parameter file (example:
  doppler_polynomial)

      value    (mode 5) new value
  delimited by double quotes (example: "317.0 0. 0. 0.")

ERS_pre_proc generates a set of MSP processing parameter files, scans the raw data for missing lines, range aligns the echoes, extracts and interpolates the DELFT state vectors, estimates the Doppler ambiguity and centroid, and generates the process list (called a proc_list) used as input to the scripts ERS_proc_par and ERS_proc_all scripts. There are series of steps (modes) that are run through to accomplish the various functions This script is called several times, stepping through the individual modes.

To use ERS_pre_proc, you need to create a CEOS_list in a text editor which is a list of the paths to the raw data sets. These are images of the raw data sets. The CEOS data files should be copied to directories with the names derived from the numerical date of acquisition with the form YYYYMMDD (all numbers). By using this naming convention the directories are in time order when listed.

An example CEOS_list is shown below:



  19990915

  19991020

Only the dat_01.001 and lea_01.001 files are used.

ERS_pre_proc is called up several times stepping through the modes and performing each of the raw data pre-processing steps in sequence.

1. Read the CEOS leader files (LEA_01.001) and initialize and generate the MSP processing parameter files:

ERS_pre_proc CEOS_list /e/Delft raw
  ERS_pre_proc_1.log proc_list 1

The processing parameter files are all placed in the directory called raw. The text format log file is specified as ERS_pre_proc_1.log and contains the MSP commands and screen output from running those commands. The processing parameter files have names of the form pYYYYMMDD.slc.par

2. Scan the raw data for missing lines and range align the raw data echoes.

The script will call ERS_fix with appropriate parameters for each of the data files in the CEOS list:

ERS_pre_proc CEOS_list /e/Delft raw
  ERS_pre_proc_2.log proc_list 2

The fixed raw data files are written to the raw output directory.

3. Extract and interpolate precision DELFT state vectors

ERS_pre_proc CEOS_list /e/Delft raw
  ERS_pre_proc_3.log proc_list 3

The state vectors are written to the processing parameter files in the raw directory. The script assumes that you have downloaded the latest state vectors for ERS-1 and ERS-2 from the TU-DELFT web site:
http://www.deos.tudelft.nl/AS/pieter/Local/ERS/index.html. The ERS-1 state vectors should be in a directory ers1/eigen_gl04s and the ERS-2 state vectors in a directory ers2/eigen_gl04s.

4. Determine the Doppler centroid:

The next step runs the program azsp_IQ to estimate the Doppler centroid



   

  ERS_pre_proc CEOS_list /e/Delft raw ERS_pre_proc_4.log proc_list
  4

The Doppler centroid values are written into the MSP processing parameter files

5. Generate the proc_list file used by ERS_proc_par and ERS_proc_all

The proc_list has 9 columns, 1 line per scene and contains information on the scene root file name (YYYYMMDD), sensor parameter file and processing parameters:
1. scene_identifier (example: 19960816)
2. MSP sensor parameter file
3. offset in echoes to start of processed data (enter - for default)
4. number of echoes to process (enter - for default)
5. range offset in samples (enter - for default)
6. number of range samples to process (enter - for default)
7. Doppler centroid for scene (Hz)
8. Doppler slope for scene (Hz/m)
9. azimuth processing bandwidth fraction for scene (0 --> 1.0, default: .9)

This is done using mode 6 of ERS_pre_proc:

ERS_pre_proc CEOS_list /e/Delft raw ERS_pre_proc_6.log
  proc_list 6

An example proc_list file for the pair of frames is shown below:

19990915 ERS2_ESA.par  - - - -  26.8 0.0
  0.8

  19991020 ERS2_ESA.par - - - -   79.5 0.0
  0.8

The Doppler centroids are close to one another and the processing bandwidth fraction is set to 0.8. The processing bandwidth fraction is the fraction of the total signal azimuth bandwidth used to generate the SLC image and should not be greater than .9 to avoid excessive azimuth ambiguities in the final SAR image. The dashes in the other fields specify default processing extents in range and azimuth and results in processing of the full frame.



  

  ERS_proc_all proc_list1 raw e:/ slc mli 2 10 1

Generally you will only need to run step 5 if you want to change some default value in the processing parameter files and this is not generally necessary. Also, step 7 should only be done after processing of all the data to images is completed.

2. Using ERS_proc_all to process raw data to SLC and MLI images

ERS_proc_all uses the proc_list and fixed raw data to generate processing SLC and MLI images.



  

  *** ERS_proc_all

  *** Copyright 2014, Gamma Remote Sensing, v3.3 3-Feb-2014 clw
  ***

  *** ERS-1 and ERS-2 SAR image processing from RAW data ***

  

  usage: ERS_proc_all <proc_list> <raw_dir>
  <rc_dir> <SLC_dir> <MLI_dir> <rlks>
  <azlks> <SLC_format> [az_patch] [autof_min]

      proc_list  (input) processing list (9
  columns):

             
        1. scene_identifier (example:
  19960816)

         
            2. MSP sensor
  parameter file

                
     3. offset in echoes to start of processed data
  (enter - for default)

                
     4. number of echoes to process (enter - for
  default)

         
            5. range
  offset in samples (enter - for default)

         
            6. number
  of range samples to process (enter - for default)

                
     7. Doppler centroid for scene (Hz)

         
            8. Doppler
  slope for scene (Hz/m)

         
            9. azimuth
  processing bandwidth fraction for scene

      raw_dir     data directory
  containing fixed ERS raw data files

      rc_dir      directory to
  temporarily store intermediate range compressed data (example:
  /tmp)

      SLC_dir     directory to
  store output SLC data

      MLI_dir     directory to
  store multilook intensity (MLI) files derived from the slc
  data

      rlks       
  number of range looks to generate MLI images  
  (nominal: 1)

      azlks       number of
  azimuth looks to generate MLI images (nominal: 5)

      SLC_format  SLC image format
  (0:FCOMPLEX  1:SCOMPLEX)

      az_patch    azimuth patch size
  (default: 6144)

      autof_min   minimum SNR threshold
  for autofocus, 0.0 for no autofocus (nominal = 10)

     

      NOTE:  current directory is denoted by
  .

On the command line you must specify the directory where the parameter files and raw data reside, the directories for output SLC and MLI images, the number of range and azimuth looks for the MLI images, the format of the SLC images, the size of the azimuth patch and the minimum SNR threshold for autofocus. The autofocus step may not be able to evaluate the focus error if the image has no contrast, such as over the ocean. In this case it is usually safe to turn off autofocus because the use of the precision state vectors usually assures excellent image focus. In the case where autofocus is not able to evaluate the focus error it can be disabled by entering 0.0 for the autof_min command line parameter and the reprocessing.

The nominal number of range and azimuth looks for the multi-look intensity images is 1 look in range and 5 in azimuth. In addition to the MLI images, Sun raster images of the MLI data are produced and written into the MLI directory. To look at these images use xv or other image display tool. Be sure to examine all the output images to see if they are focused and otherwise acceptable. Log files from processing each of the scenes is generated and stored in the SLC directory under names YYYYMMDD_MSP.log