History and future

Versions

A summary of the versions and changes.

  • v1.0
    Initial release.
    (2020-06)
  • v1.1
    Support averaging of grid cells over footprints.
    Added output options for horizontal mapping info, model concentrations at native layers, and simulated profile.
    (2020-09)
  • v1.2
    Updates for retrieval products including apriori profiles.
    (2020-10)
  • v1.3
    Added operator for mapping vertical profiles.
    (2020-10)
  • v1.4
    Create listing files not as part of convert job but in a seperate job.
    Use weights expressed in area in horizontal mapping.
    Added tools to create gridded averages.
    (2020-11)
  • v1.5
    Added option to search for orbit files collected in tar files.
    Fixes for mapping information.
    (2020-11)
  • v1.6
    Put out user defined variables over pixels from a simulation model.
    (2021-01)
  • v1.7
    Introduced formula definitions for operations on pixel data.
    Support airmass factor corrections as used for NO2 products.
    (2021-05)
  • v1.8
    Replaced default download method for OpenSearch API.
    Added HCHO basic rc file.
    Extended support for DLR TROPOMI products.
    Add adjoint operations and adjoint test.
    (2021-06)
  • v2.0
    Added --single flag to run script to allow running just one job step.
    Added class to create listing of files downloaded from ESA/SciHub.
    Regridding operator using CSO Python routines rather than using mapping information saved from Fortran operator.
    For mapping between footprints and grid cells, triangles are now splitted into 2 new triangles based on the longest edge.
    (2021-10)
  • v2.1
    Read data on root and scatter to domains, instead of reading by all domains.
    Support S5p/NO2 level3 observations from KNMI.
    Store settings in subdirectory ‘config’.
    Postprocessing tools with agregation over regions.
    Added example for adjoint test to operator code.
    Added routines to swap operator data to a different domain decomposition.
    Support S5p/Glyoxal retrievals from GlyRetro.
    (2022-08)
  • v2.2
    Option to grid converted orbit files using listing file.
    (2023-08)
  • v2.3
    Added collection level to inquire overview plots.
    Convert selected orbit files based on selection query.
    (2023-08)
  • v2.4
    Reformatted using ‘black’.
    (2023-08)
  • v2.5
    Support new Copernicus DataSpace portal to download Sentinel data.
    (2023-11)
  • v2.6
    Support SO2 COBRA product from PAL website.
    (2024-01)
  • v2.7
    Fixed averaging over time values when creating super-observations.
    Use cartopy for map plots.
    Support orbits that overlap with dateline.
    (2024-01)
  • v2.8
    Re-introduced ‘Download’ class to download a series of S5P files without converting.
    Updated re-try settings for downloading.
    Extended support for creation of gridded averages.
    Extended support for region plots.
  • v2.9
    Initial support for S5P O3-profile product.
  • v2.10
    Support inquiry of ColHub mirror archive.
  • v2.11
    Support inquiry and download from NASA EarthData archive.

To be included

Features available in branches, that should be included in master version.

  • Create map plots as a list of pixels, in case track information is not available. (E.D.)

To be done

A wishlist of developments.

  • Regridding following Kuhlman et al. (2014).

  • Include time range in all filenames?

    • Proposal: use names similar to the original S5p files, up to the collection number:

      S5P_NRTI_L2__NO2____20140101T000000_20140102T000000_00099_01.nc
      
    • Also output by observation operator could use the same basename.

  • Add offline tool to re-create listing files.

    • Scan directories based on filename patterns.

    • Eventually specify order, thus RPRO if available, otherwise OFFL or NRTI.

  • If the original track is saved, add flags to log which selection criterium was used to exclude a pixel.

  • Store covariances as sparse matrix (diagonal?) if possible.

  • Storage of additional observation representation errors by assimilation system.

How to create a version tag

Example for tag “v2.11”.

  1. Update file formatting following conventions.

    1. For Python files:

      make black
      
    2. Eventually also cleanup Fortran sources.

  2. Ensure that all changes in source files are committed.

  3. Update change logs:

    1. Update the CHANGELOG.

    2. Update the source file doc/source/history.rst of this chapter.

    3. Change the release tag that will be shown in the documentation in the file doc/source/conf.py:

      # The full version, including alpha/beta/rc tags
      release = "v2.11"
      
    4. Commit and push these changes:

      git commit -m 'Defined v2.11' .
      git push
      
  4. Tag and push the code to the git repository:

    git tag -a 'v2.11' -m 'Tagged v2.11'
    git push origin v2.11