CHANGELOG

CSO - CAMS Satellite Operator
=============================


Version tags
------------

See "History" chapter in documentation for overview of versions.
Detailled changes per version are listed below.


v2.1
----

Updated documentation.
  doc/source/documentation.rst
  doc/source/s5p-so2.rst

Improved MPI data type selection.
  oper/src/cso_comm.F90
  oper/src/cso_ncfile.F90

Properly count user defined variables.
  oper/src/cso_pixels.F90

Fixed deallocation.
  oper/src/cso_sat.F90

Only setup exchange between domains if needed.
  oper/src/cso_sat.F90

Optional limit output variables to selection.
  oper/src/cso_sat.F90

Trap non-existing or empty listing files when creating catalogue of converted data.
  py/cso_catalogue.py

Updated documentation of special conversions.
  py/cso_s5p.py

Templates for S5p/CO product.
  doc/samples/S5p_OFFL_L2__CO____*.txt
  py/cso_s5p.py

Processing of NO2 superobservations provided by KNMI.
  doc/source/figs/NO2_superobs
  doc/source/index.rst
  doc/source/obsoper.rst
  doc/source/pymod-cso_s5p_superobs.rst
  doc/source/s5p-no2-superobs.rst
  py/cso.py
  py/cso_s5p.py
  py/cso_s5p_superobs.py

Support timeseries with different resolution.
  py/cso_catalogue.py
  py/cso_index.py

Updated documentation.
  doc/source/s5p-no2.rst
  doc/README.md

Read data on root, scatter to processes.
  oper/src/cso_comm.F90
  oper/src/cso_domains.F90
  oper/src/cso_ncfile.F90
  oper/src/cso_pixels.F90
  oper/src/cso_sat.F90

Create gridded averages using mapping functions similar as used in observation operator.
  py/cso_gridded.py
  py/cso_mapping.py
  py/cso_constants.py
  doc/source/gridding.rst
  doc/source/pymod-cso_mapping.rst
  doc/source/pymods.rst
  
Add time dimension to gridded averages to allow temporal averaging.
  py/cso_gridded.py

When mapping footprints to grid cells, split triangles into 2 new triangles 
based on the longest edge 
  py/cso_mapping.py

Retry reading of S5p data, sometimes fails for unknown reason.
  py/cso_s5p.py

Added class to create plot of S5p versions downloaded from SciHub.
  py/cso_scihub.py

Support processing of S5p/NO2 data from PAL.
  py/cso_pal.py
  
Changed S5p conversion class to download files while converting.
Eventually downloaded files are removed after conversion.
  py/cso_s5p.py
  py/cso_scihub.py

Aggregation over regions.
  py/cso_regions.py
  data/country-codes*.csv
  doc/source/pymod-cso_regions.rst
  doc/source/pymods.rst
  doc/source/gridding.rst

Support for S5p/Glyoxal product.
  config/ESA-S5p/cso-s5p-glyox.rc
  doc/source/s5p-glyoxal.rst
  doc/source/figs/GLYOX/
  doc/samples/S5P_OFFL_L2__CHOCHO___20200601T001519_20200601T015649_13643_01_010000_20210128.txt
  doc/source/index.rst


v2.2
----

Support batch job for Sun Grid Engine.
  py/utopya_jobscript.py
  py/utopya_jobtree.py

Improved inquire tasks for SciHub and PAL portals.
  py/cso_scihub.py
  py/cso_pal.py
  py/cso.py
  py/cso_inquire.py
  doc/source/pymod-cso_inquire.rst
  doc/source/pymod-cso_pal.rst

Updated documentations for inquire tasks.
  doc/source/tutorial.rst
  doc/source/s5p-so2.rst
  doc/source/s5p-no2.rst
  doc/source/s5p-hcho.rst
  config/tutorial/tutorial.rc

Support computation of grid cell areas for 2D arrays.
  py/cso_mapping.py

Added class to create super-observations.
  py/cso_superobs.py
  py/cso.py
  doc/source/pymod-cso_superobs.rst
  doc/source/pymods.rst

Specify output file template for catalogue figures rather than output directory.
Support steps over year when plotting series of gridded observations.
  py/cso_catalogue.py

Write files with packing and compression.
  py/cso_file.py
  py/cso_s5p.py
  oper/src/

Added class to create temporal means of gridded averages.
  py/cso_gridded.py
  doc/source/gridding.rst

Read Timestamp object from rcfile settings.
  py/rc.py

Support index timeseries over years.
  py/utopya_index.py

Updated product description of S5p/Glyoxal.
  doc/source/s5p-glyoxal.rst

Updated template settings.
  config/tutorial/tutorial.rc

Added tools to collect and plot time series of co-located pixel averages.
  py/cso_colocate.py
  py/cso.py
  py/cso_mapping.py
  doc/source/index.rst
  doc/source/colocate.rst
  doc/source/pymods.rst
  doc/source/pymod-cso_colocate.rst
  doc/source/documentation.rst

Create output directory if necessary.
  py/cso_inquire.py

Trap errors from plotting coastlines and country borders.
  py/cso_plot.py

Updated user guide.
  doc/source/figs/footprints_nc4_levels_and_area.png
  doc/source/obsoper.rst

Added option to plot specific layer of gridded averages.
Adhoc fix to trap nans in some files.
  py/cso_catalogue.py 

Support packed output for gridded averages.
  py/cso_gridded.py

Updated documentation.
  py/cso.py

Removed depricated DHUS downloads.
  py/cso_scihub.py

Define environment variable CSO_PREFIX.
  bin/cso
  
Use environment variable for installation prefix, use own filename for settings.
    config/tutorial/tutorial.rc


Changes for v2.2
----------------

Option to create gridded fields from orbit files in listing.
  py/cso_gridded.py
  py/cso_file.py
  config/tutorial/tutorial.rc


Changes for v2.3
----------------

Added collection level to inquire overview plots.
  py/cso_inquire.py

Convert selected orbit files based on selection query.
  py/cso_s5p.py

Synchronized template settings for Copernicus S5P products.
  config/Copernicus/

Updated User guide.
  doc/source/


v2.4
----

Added 'black' target to Makefile, including convention for maximum line length.
Reformatted files following new convention.
  Makefile
  py/*


v2.5
----

Fixed definition of CSO_PREFIX environment variable.  
  bin/cso

Updated tutorial.
  config/tutorial/tutorial.rc
  doc/source/tutorial.rst

Define lon/lat axes of gridded output as floats.
  py/cso_gridded.py

Use pandas concat to combine dataframes.
  py/cso_pal.py

Replaced `where` concstructs by loops after memory errors in output filling.
  oper/src/cso_ncfile.F90
  
Check on `pix_all` value to know if pixels are assigned to any of the sub-domains.
  src/tutorial_oper_S5p.F90

Migration from Copernicus SciHub to DataSpace.
  bin/dhusget.sh
  doc/source/*
  py/*
  config/*


Patch 2.5.1
~~~~~~~~~~~

Fixed bug in S5P orbit selection.
  py/cso_s5p.py


v2.6
----

Added 'black' target to Makefile, including convention for maximum line length.
  Makefile

Fixed averaging over time values when creating super-observations.
  py/cso_superobs.py

Improved output of packed variables.
Support files with "sample" coordinate instead of "pixel".
  oper/

Use cartopy for map plots.
  py/cso_plot.py
  py/cso_catalogue.py

Support orbits that overlap with dateline.
  py/cso_s5p.py
  py/cso_gridded.py
  py/cso_superobs.py


v2.7
----

Removed suppored for NO2-superobs, now create these using own classes.
  py/cso_s5p_superobs.py
  py/cso.py
  config/EMEP/emep-tropomi-superobs.rc
  doc/source/index.rst
  doc/source/pymod-cso_s5p_superobs.rst
  doc/source/s5p-no2-superobs.rst

Removed depricated references to S5p-Hub.
  doc/source
  py/

Removed whitespace from lines for easier comparison with other versions.
  py/

Synchronized layout with CSO public version for easier synchronization.
Synchronized change logs in source files between master and development versions.
  py/