Synchronized documenation. (7382e426) · Commits · CAMS / CSO

doc/source/s5p-chocho.rst

+25 −1

Original line number	Diff line number	Diff line
		@@ -56,7 +56,7 @@ Features:

		* :math:`\mathbf{y}_s` is the simulated retrieval (mol/m2) defined on :math:`n_r=1` layers;
		* :math:`\mathbf{A}` is the tropospheric averaging kernel matrix with shape :math:`(n_r,n_a)`;
		in this product, :math:`n_r=1` the number of a priori layers; see also the remarks below;
		with :math:`n_a` the number of a priori layers;
		* :math:`\mathbf{x}` is the atmospheric state, which consists of a 3D array of CHOCHO concentrations;
		* :math:`\mathbf{H}` extracts a simulated profile from the state using horizontal and vertical interpolation;
		the result should be defined on the :math:`n_a` a priori layers
		@@ -221,6 +221,30 @@ The example is based on the S5p CHOCHO file from which the header is available i
		* `doc/samples/S5P_OFFL_L2__CHOCHO___20200601T001519_20200601T015649_13643_01_010000_20210128.txt <../../samples/S5P_OFFL_L2__CHOCHO___20200601T001519_20200601T015649_13643_01_010000_20210128.txt>`_


		Orbit file selection
		--------------------

		Based on the inquiry the download and conversion could be limitted to files created with the most recent processor versions.

		For the S5P files a useful property is also the collection number, a 2-digit number that defines a collection of files
		(or actually processor versions) that together form a contineous series. The collection number is extracted from the filename,
		and stored as a column of the listing file.

		The following setting is used to select specific files from the archive based on the properities stored
		in the listing file::

		! Provide ';' seperated list of to decide if a particular orbit file should be processed.
		! If more than one file is available for a particular orbit (from "OFFL" and "RPRO" processing),
		! the file with the first match will be used.
		! The expressions should include templates '%{header}' for the column values.
		! Example to select files from collection '03', preferably from processing 'RPRO' but otherwise from 'OFFL':
		! (%{collection} == '03') and (%{processing} == 'RPRO') ; \
		! (%{collection} == '03') and (%{processing} == 'OFFL')
		!
		cso.s5p.chocho.convert.selection : (%{collection} == '03') and (%{processing} == 'RPRO') ; \
		(%{collection} == '03') and (%{processing} == 'OFFL')


		Pixel selection
		---------------

doc/source/s5p-co.rst

+6 −6

Original line number	Diff line number	Diff line
		@@ -44,7 +44,7 @@ Notes:

		* :math:`\mathbf{y}_s` is the simulated retrieval (mol/m2) defined on :math:`n_r=1` layers;
		* :math:`\mathbf{A}^{trop}` is the tropospheric averaging kernel matrix with shape :math:`(n_r,n_a)`;
		in this product, :math:`n_r=1` the number of a priori layers; see also the remarks below;
		with :math:`n_a` the number of a priori layers;
		* :math:`\mathbf{x}` is the atmospheric state, which consists of a 3D array of CO concentrations;
		* :math:`\mathbf{H}` extracts a simulated profile from the state using horizontal and vertical interpolation;
		the result should be defined on the :math:`n_a` a priori layers
		@@ -171,7 +171,7 @@ The jobtree configuration to inquire the portals and create the overview figure

		! task initialization:
		cso.s5p.co.convert.class : cso.CSO_S5p_Convert
		cso.s5p.co.convert.args : '${PWD}/config/ESA-S5p/cso-s5p-co.rc', rcbase='cso.s5p.co.convert'
		cso.s5p.co.convert.args : '${PWD}/config/Copernicus/cso-s5p-co.rc', rcbase='cso.s5p.co.convert'

		See the class documentation for the general configuration,
		below some specific choices are described.
		@@ -414,7 +414,7 @@ The jobtree configuration to inquire the portals and create the overview figure

		! catalogue creation task:
		cso.s5p.co.catalogue.task.figs.class : cso.CSO_Catalogue
		cso.s5p.co.catalogue.task.figs.args : '${PWD}/config/ESA-S5p/cso-s5p-co.rc', \
		cso.s5p.co.catalogue.task.figs.args : '${PWD}/config/Copernicus/cso-s5p-co.rc', \
		rcbase='cso.s5p.co.catalogue'

		The configuration describes where to find a listing file with orbits,
		@@ -454,7 +454,7 @@ The jobtree configuration to inquire the portals and create the overview figure

		! index creation task:
		cso.s5p.co.catalogue.task.index.class : utopya.Indexer
		cso.s5p.co.catalogue.task.index.args : '${PWD}/config/ESA-S5p/cso-s5p-co.rc', \
		cso.s5p.co.catalogue.task.index.args : '${PWD}/config/Copernicus/cso-s5p-co.rc', \
		rcbase='cso.s5p.co.catalogue-index'

		When succesful, the index creator displays an url that could be loaded in a browser::
		@@ -603,7 +603,7 @@ The jobtree configuration to inquire the portals and create the overview figure

		! catalogue creation task:
		cso.s5p.TRACER.sim-catalogue.task.class : cso.CSO_SimCatalogue
		cso.s5p.TRACER.sim-catalogue.task.args : '${PWD}/config/ESA-S5p/cso-s5p-TRACER.rc', \
		cso.s5p.TRACER.sim-catalogue.task.args : '${PWD}/config/Copernicus/cso-s5p-TRACER.rc', \
		rcbase='cso.s5p.TRACER.sim-catalogue'

		The configuration describes where to find a listing file with orbits,
		@@ -647,7 +647,7 @@ The jobtree configuration to inquire the portals and create the overview figure

		! index creation task:
		cso.s5p.co.catalogue.task.index.class : utopya.Indexer
		cso.s5p.co.catalogue.task.index.args : '${PWD}/config/ESA-S5p/cso-s5p-co.rc', \
		cso.s5p.co.catalogue.task.index.args : '${PWD}/config/Copernicus/cso-s5p-co.rc', \
		rcbase='cso.s5p.co.catalogue-index'

		When succesful, the index creator displays an url that could be loaded in a browser::

doc/source/s5p-hcho.rst

+144 −91

File changed.

Preview size limit exceeded, changes collapsed.

doc/source/s5p-no2.rst

+40 −18

Original line number	Diff line number	Diff line
		@@ -39,7 +39,7 @@ Features:
		* :math:`\mathbf{y}_s` is the simulated retrieval (mol/m2) defined on :math:`n_r=1` layers;
		* :math:`\mathbf{A}^{trop}` is the tropospheric averaging kernel matrix with shape :math:`(n_r,n_a)`,
		with :math:`n_a` the number of a priori layers;
		* :math:`\mathbf{x}` is the atmospheric state, which probably consists of a 3D array of NO2 concentrations;
		* :math:`\mathbf{x}` is the atmospheric state, which probably consists of a 3D array of NO\ :sub:`2` concentrations;
		* operators :math:`\mathbf{G}` and :math:`\mathbf{V}` together compute a simulated profile
		at the :math:`n_a` a priori layers from the state, using horizontal (:math:`\mathbf{G}`)
		and vertical (:math:`\mathbf{V}`) mappings;
		@@ -180,14 +180,14 @@ CSO processing

		(See :ref:`tutorial` chapter for introduction to CSO scripts and configuration)

		An example configuration of the CSO processing of the S5p/NO2 data is available via
		An example configuration of the CSO processing of the S5p/NO\ :sub:`2` data is available via
		the following settings:

		* `config/Copernicus/cso.rc <../../../config/Copernicus/cso.rc>`_

		Top-level settings that configure the job-tree with various sub-tasks.
		This is a generic file that could be used for multiple S5 products,
		edit it to select the NO2 processing.
		edit it to select the NO\ :sub:`2` processing.

		* `config/Copernicus/cso-user-settings.rc <../../../config/Copernicus/cso-user-settings.rc>`_

		@@ -195,7 +195,7 @@ the following settings:

		* `config/Copernicus/cso-s5p-no2.rc <../../../config/Copernicus/cso-s5p-no2.rc>`_

		Specific settings for NO2 product.
		Specific settings for NO\ :sub:`2` product.

		Start the job-tree using::

		@@ -259,7 +259,7 @@ To visualize what is available from the various portals, the
		.. figure:: figs/NO2/Copernicus_S5p_NO2.png
		:scale: 50 %
		:align: center
		:alt: Overview of available NO2 processings.
		:alt: Overview of available NO\ :sub:`2` processings.

		The jobtree configuration to inquire the portals and create the overview figure could look like::

		@@ -280,7 +280,6 @@ The jobtree configuration to inquire the portals and create the overview figure




		.. Label between '.. _' and ':' ; use :ref:`text <label>` for reference
		.. _s5p-no2-convert:

		@@ -302,7 +301,7 @@ which is initialized using the settings file::

		! task initialization:
		cso.s5p.no2.convert.class : cso.CSO_S5p_Convert
		cso.s5p.no2.convert.args : '${PWD}/config/ESA-S5p/cso-s5p-no2.rc', rcbase='cso.s5p.no2.convert'
		cso.s5p.no2.convert.args : '${PWD}/config/Copernicus/cso-s5p-no2.rc', rcbase='cso.s5p.no2.convert'

		See the class documentation for the general configuration,
		below some specific choices are described.
		@@ -311,6 +310,30 @@ The example is based on the S5p NO\ :sub:`2` file from which the header is avail
		* `doc/samples/S5P_OFFL_L2__NO2____20190601T011830_20190601T030000_08451_01_010301_20190607T025115.txt <../../samples/S5P_OFFL_L2__NO2____20190601T011830_20190601T030000_08451_01_010301_20190607T025115.txt>`_


		Orbit file selection
		--------------------

		Based on the inquiry the download and conversion could be limitted to files created with the most recent processor versions.

		For the S5P files a useful property is also the collection number, a 2-digit number that defines a collection of files
		(or actually processor versions) that together form a contineous series. The collection number is extracted from the filename,
		and stored as a column of the listing file.

		The following setting is used to select specific files from the archive based on the properities stored
		in the listing file::

		! Provide ';' seperated list of to decide if a particular orbit file should be processed.
		! If more than one file is available for a particular orbit (from "OFFL" and "RPRO" processing),
		! the file with the first match will be used.
		! The expressions should include templates '%{header}' for the column values.
		! Example to select files from collection '03', preferably from processing 'RPRO' but otherwise from 'OFFL':
		! (%{collection} == '03') and (%{processing} == 'RPRO') ; \
		! (%{collection} == '03') and (%{processing} == 'OFFL')
		!
		cso.s5p.no2.convert.selection : (%{collection} == '03') and (%{processing} == 'RPRO') ; \
		(%{collection} == '03') and (%{processing} == 'OFFL')


		Pixel selection
		---------------

		@@ -488,8 +511,7 @@ the later could include a template for the orbit number::
		! output directory and filename:
		! - times are taken from mid of selection, rounded to hours
		! - use '%{orbit}' for orbit number
		cso.s5p.no2.convert.output.dir : /Scratch/CSO/S5p/RPRO/NO2/Europe/%Y/%m
		cso.s5p.no2.convert.output.filename : S5p_RPRO_NO2_%{orbit}.nc
		cso.s5p.no2.convert.output.filename : /Scratch/CSO-data/Europe/S5p/NO2/C03/%Y/%m/S5p_NO2_%{orbit}.nc

		A flag is read to decide if existing files should be renewed or kept::

		@@ -557,8 +579,7 @@ file is found, the first match is used.
		This could be explored to create a listing that combines reprocessed data
		with near-real-time data::

		<rcbase>.patterns : RPRO/NO2/Europe/%Y/%m/S5p_*.nc \
		OFFL/NO2/Europe/%Y/%m/S5p_*.nc
		<rcbase>.patterns : CO3/%Y/%m/S5p_*.nc



		@@ -574,7 +595,7 @@ Configuration could look like::

		! catalogue creation task:
		cso.s5p.no2.catalogue.task.figs.class : cso.CSO_Catalogue
		cso.s5p.no2.catalogue.task.figs.args : '${PWD}/config/ESA-S5p/cso-s5p-no2.rc', \
		cso.s5p.no2.catalogue.task.figs.args : '${PWD}/config/Copernicus/cso-s5p-no2.rc', \
		rcbase='cso.s5p.no2.catalogue'

		The configuration describes where to find a listing file with orbits,
		@@ -606,7 +627,7 @@ Figures are saved to files with the basename of the original orbit file and the
		.. figure:: figs/NO2/S5p_RPRO_NO2_03278__vcd.png
		:scale: 50 %
		:align: center
		:alt: S5p NO2 columns
		:alt: S5p NO\ :sub:`2` columns

		To search for interesting features in the data,
		the :py:class:`Indexer <utopya_index.Indexer>` class could be used to create index pages.
		@@ -614,7 +635,7 @@ Configuration could look like::

		! index creation task:
		cso.s5p.no2.catalogue.task.index.class : utopya.Indexer
		cso.s5p.no2.catalogue.task.index.args : '${PWD}/config/ESA-S5p/cso-s5p-no2.rc', \
		cso.s5p.no2.catalogue.task.index.args : '${PWD}/config/Copernicus/cso-s5p-no2.rc', \
		rcbase='cso.s5p.no2.catalogue-index'

		When succesful, the index creator displays an url that could be loaded in a browser::
		@@ -696,7 +717,7 @@ Example settings::
		For the simulated values, also define a list of variable names that should be created::

		! state varaiables to be put out from model:
		tutorial.S5p.no2.vars : mod_conc mod_hp mod_tcc mod_cc hx ys shx M_m A_m yr_m ys_m
		tutorial.S5p.no2.vars : mod_conc mod_hp mod_tcc mod_cc xs ys Sx M_m A_m yr_m ys_m

		Example settings::

		@@ -822,7 +843,7 @@ Configuration could look like::

		! catalogue creation task:
		cso.s5p.no2.sim-catalogue.task.class : cso.CSO_SimCatalogue
		cso.s5p.no2.sim-catalogue.task.args : '${PWD}/config/ESA-S5p/cso-s5p-TRACER.rc', \
		cso.s5p.no2.sim-catalogue.task.args : '${PWD}/config/Copernicus/cso-s5p-TRACER.rc', \
		rcbase='cso.s5p.no2.sim-catalogue'

		The configuration describes where to find a listing file with orbits,
		@@ -871,7 +892,7 @@ Configuration could look like::

		! index creation task:
		cso.s5p.no2.catalogue.task.index.class : utopya.Indexer
		cso.s5p.no2.catalogue.task.index.args : '${PWD}/config/ESA-S5p/cso-s5p-no2.rc', \
		cso.s5p.no2.catalogue.task.index.args : '${PWD}/config/Copernicus/cso-s5p-no2.rc', \
		rcbase='cso.s5p.no2.catalogue-index'

		When succesful, the index creator displays an url that could be loaded in a browser::
		@@ -882,5 +903,6 @@ When succesful, the index creator displays an url that could be loaded in a brow
		.. figure:: figs/NO2/CSO_NO2_sim-catalogue.png
		:scale: 50 %
		:align: center
		:alt: Index for Simulated and S5p NO2 columns
		:alt: Index for Simulated and S5p NO\ :sub:`2` columns

doc/source/s5p-so2.rst

+161 −99

File changed.

Preview size limit exceeded, changes collapsed.