# Sentinel-5p CO data processing

This chapter describes the tasks performed for processing Sentinel-5p CO data.

## Product description

The product guides can be found at:

- SentiWiki / Sentinel-5P

  - `L2__CO___`, `PUM-CO` Product User Manual

Notes:

- From processor version v2.4.0 onwards, the units of the averaging kernel changed from `m` to a more standard unit-less repesentation. When processing older data versions, the conversion into CSO format (which was using unit-less representation already) is different.

## CSO processing

(See Tutorial chapter for introduction to CSO scripts and configuration)

An example configuration of the CSO processing of the S5p/CO data is available via the following settings:

- config/Copernicus/cso.rc

  Top-level settings that configure the job-tree with various sub-tasks. This is a generic file that could be used for multiple S5 products, edit it to select the CO processing.

- config/Copernicus/cso-user-settings.rc

  User-specific settings such as the work directory.

- config/Copernicus/cso-s5p-co.rc

  Specific settings for CO product.

Start the job-tree using:

    ./bin/cso  config/Copernicus/cso.rc

Selected sub-steps in the processing are described below.

## Inquire Sentinel-5p/CO archive

S5p/CO observations are available from the Copernicus DataSpace; see the cso_dataspace module module for a detailed description.

Data is available for different processing streams, each identified by a 4-character key:

- `NRTI` : Near real time, available with a day after observation;

- `OFFL` : Offline, available within weeks after observations;

- `RPRO` : re-processing of all previously made observations;

The portals provide data files created with the same retrieval algorithm, but with different processor versions. It is therefore necessary to first inquire both archives to see which data is available where, and what the version numbers are.

The `CSO_DataSpace_Inquire` class is available to inquire the Copernicus DataSpace. The settings used by this class allow selection on for example time range and intersection area. The result is a csv file which with columns for keywords such orbit number and processor version, as well as the filename of the data and the url that should be used to actually download the data:

    orbit;start_time;end_time;processing;collection;processor_version;filename;href
    23665;2022-05-08 09:44:59;2022-05-08 11:26:29;RPRO;03;020400;S5P_RPRO_L2__CO_____20220508T094459_20220508T112629_23665_03_020400_20230112T000125.nc;https://zipper.dataspace.copernicus.eu/odata/v1/Products('afbaef9f-5295-4d47-be43-f74270e07b4d')/$value
    23666;2022-05-08 11:26:29;2022-05-08 13:07:59;RPRO;03;020400;S5P_RPRO_L2__CO_____20220508T112629_20220508T130759_23666_03_020400_20230112T000126.nc;https://zipper.dataspace.copernicus.eu/odata/v1/Products('3beada57-7602-482e-bde4-7a10758327e3')/$value
    :

See the section on File name convention in the Product User Manual for the meaning of all parts of the filename.

To visualize what is available from the various portals, the `cso_inquire.CSO_Inquire_Plot` could be used to create an overview figure:

The jobtree configuration to inquire the portals and create the overview figure could look like:

    ! single step:
    cso.s5p.co.inquire.class                      :  utopya.UtopyaJobStep
    ! two tasks:
    cso.s5p.co.inquire.tasks                      :  table-dataspace plot

    !~ inquire files available on DataSpace:
    cso.s5p.co.inquire.table-dataspace.class      :  cso.CSO_DataSpace_Inquire
    cso.s5p.co.inquire.table-dataspace.args       :  '${PWD}/config/Copernicus/cso-s5p-co.rc', \
                                                         rcbase='cso.s5p.co.inquire-table-dataspace'

    !~ create plot of available versions:
    cso.s5p.co.inquire.plot.class                 :  cso.CSO_Inquire_Plot
    cso.s5p.co.inquire.plot.args                  :  '${PWD}/config/Copernicus/cso-s5p-co.rc', \
                                                          rcbase='cso.s5p.co.inquire-plot'