cso_regions module

The cso_regions module provides classes to create and apply region masks.

Class hierchy

The classes and are defined according to the following hierchy:

Classes

class cso_regions.CSO_CreateCountryMask(rcfile, rcbase='', env={}, indent='')

Bases: utopya_rc.UtopyaRc

Create country mask at grid.

Arguments:

  • rcfile, rcbase, env : settings file, prefix for keys, environment

Specify how the grid is defined:

! how is grid defined?
! - "sample"   : file with 'longitude' and 'latitude' coordinates
! - "regular"  : define regular grid
<rcbase>.grid.type            :   regular

For grid type regular, the definition should look like:

! define regular grid:
<rcbase>.grid.regular.west    :   -180
<rcbase>.grid.regular.east    :    180
<rcbase>.grid.regular.south   :    -90
<rcbase>.grid.regular.north   :     90
<rcbase>.grid.regular.dlon    :   0.1
<rcbase>.grid.regular.dlat    :   0.1

For grid type sample, define the sample file with:

! sample file that defines target grid:
<rcbase>.grid.sample.file   :  /data/FLEXPART/grid_time_20170103000000.nc

The country borders are taken from shape files shipped with the cartopy module. Specify the resolution to be used:

! required resolution of regions shape files:
!   10m  110m
<rcbase>.resolution          :  110m

The region map will be written to a netCDF file together with the region names and codes. In addition, a csv with just the region names/codes will be written. Specify the basename of both files:

! basename of region map files:
!  <basename>.nc    : netcdf file with map and region names etc
!  <basename>.csv   : text file with region names etc (same as in nc file)
<rcbase>.basename            :  /data/regions/regions-map_COSMO-7_countries

Existing files are only replaced if the following flag is set:

! renew files if already present?
<rcbase>.renew               :  True

The csv table with region records has for example as content:

code;name
OCE;ocean
ALB;Albania
AUT;Austria
BEL;Belgium
:

Based on this map it is posslible to create additional maps with aggregated countries. Provide a list with names of aggeragated maps to be created:

! create aggregated maps?
<rcbase>.aggregates          :  aggr total

For each aggregate, provide:

  • The name of csv table with codes and names for the aggregated regions. For example, if the aggreation should use “BNL” for combined Belgium/Netherlands/Luxemburg, then the table should contain:

    code;name
BNL;BeNeLux
:

  • The name of csv table with mapping from original to aggregated codes:

    code;newcode
BEL;BNL
:

  • The basename of the aggregated map to be created.

These settings could look like:

<rcbase>.aggregate.aggr.table      :  samples/regions_aggr.csv
<rcbase>.aggregate.aggr.mapping    :  samples/regions_aggr-from-countries.csv
<rcbase>.aggregate.aggr.basename   :  /data/regions/regions-map_COSMO-7_aggr
cso_regions.minmax(array)

Find both min and max value of arr in one pass

cso_regions.intersection_possible(points, c_minx, c_miny, c_maxx, c_maxy)

Determine if country could contain any of the points.

The maximum and minimum x and y values of the country and the points are compared, to determine if any of the points lie in the bounding box of country.

For rectangles oriented along the coordinate axes, Polygon.intersects() is faster than this function. However, we’re working with arbitrarily oriented (and possibly distorted) gridcells, for which Polygon.intersects() is considerably slower.

Arguments:

  • points : array of shape (4,3) containing the 4 corners of a cell, ordered clockwise

  • c_minx,c_miny,c_maxx,c_maxy : country bounds

Returns:

  • True if an intersection is possible, False otherwise.

class cso_regions.CreateCountryMap(lons, lats, lon_bounds, lat_bounds, resolution)

Bases: object

Create map with indices in country list.

Arguments:

  • lons, lats : 1D arrays with grid cell centers

  • lon_bounds, lat_bounds : 3D arrays (nlat,nlon,ncorner) with grid cell corners

  • resolution : shape file resolution to be used: ‘10m’, ‘110m’

Attributes created:

  • map : 2D map with indices

  • attrs : dict with lists for indices

  • used_country_index : list with country indices

GetCountryTable()

Return DataFrame with used countries; icountry is the original country number, iregion a new numbering:

icountry;iregion;code;name
 0;0;OCE;ocean
12;1;ALB;Albania
:
GetRegionMap(ctab)

Return 2D array with region numbers as in the iregion column of ctab, the DataFrame returned by :py:meth`GetCountryTable`.

class cso_regions.RegionMap(basename=None)

Bases: object

Storage for region map.

The map and auxilary data are stored in the attributes:

  • ds : xarray.Dataset with coordinate:

    • latitude, latitude : coordinates of regular grid

    • region : coordinate for region values

    and variables:

    • map(latitude,longitude) : integer region indices

    • region_index : region indices used in map

    • region_code : short character code

    • region_name : character strings with region names

  • gattrs : dict with global attributes

  • regions : pandas.DataFrame with columns:

    • iregion : int index, same as region_index variable

    • code : str with region_code

    • name : str with region_name

GetRegionInfo(iregion)

Return info on region:

  • code : region code

  • name : region name

GetRegionMask(iregion)

Return bool array of shape (latitude,longiutde) that is True for the specified region.

GetRegionDomain(iregion, edgefactor=0.0)

Return bounding box [west,easth,south,north] around region, with eventually edgefactor extra edge.

Create(rmap, regions, grid, gattrs={})

Create new region map.

Arguments:

  • rmap[:,:] : 2D int array with region indices

  • regions : DataFrame with columns:

    • iregion : int index as used in rmap

    • code : str region code

    • name : str region name

  • grid : 3-element tupple with info for grid variables:

    • ('rlat','rlon') : tupple with dimension names

    • { 'rlon' : .., 'rlat' : .. } : dict with xarray.DataArray objects

    • {'grid_mapping' : 'rotated_pole', ... } : dict with attributes to be added to variables

  • gattrs : dict with global attributes

Write(basename, savetable=False)

Write region map to netCDF file (extension “.nc”), and optionally the regions table to a “.csv” file.

Read(basename)

Read region map from specified netcdf filename.

Remap(tonew, new, gattrs={}, indent='')

Assign regions to new names.

Arguments:

  • tonew : csv file with old-to-new mapping; shoud have at least the columns:

    • code : original region codes

    • newcode : new codes

  • new : csv file with new regions; shoud have at least the columns:

    • code : region codes

    • name : region names

  • gattrs : dict with global attributes

Coarsen(domain, dres, distribute=None, gattrs={}, indent='')

Coarsen a 2D (lat,lon) region map to a 3D (region,lat,lon) array.

Arguments:

  • domain=[west,east,south,north] : target domain

  • dres=[dlon,dlat] : target resolution

Optional arguments:

  • distribute : list with region codes to be distributed over other regions, for example ['OCE'] to assign ocean area to surrounding regions

  • gattrs : dict with global attributes

class cso_regions.CSO_Catalogue_RegionsMaps(rcfile, rcbase='', env={}, indent='')

Bases: cso_catalogue.CSO_CatalogueBase

CSO task to create catalogue of gridded averages of retrievals and simulations zoomed over regions.

The catalogue is created in an output directory specified with:

<rcbase>.output.dir     :  /Scratch/CSO/catalogue

A time range for which images should be created is specified with:

! specifiy timerange:
<rcbase>.timerange.start  :  2012-01-01 00:00
<rcbase>.timerange.end    :  2012-12-31 23:59
! step is one of: hour | day | month
<rcbase>.timerange.step   :  hour

Define the basename of the region definition files (netcdf and csv), as created with the CSO_CreateCountryMask class:

! basename for region definition files:
<rcbase>.regions.basename       :  ${WORK}/projects/EMEP-S5p/regions/EMEP-002_country-mask_50m

If some problematic countries should be skipped, specify their codes:

! skip some ...
<rcbase>.regions.skip            :  AND

The time range is used to scan for output files from the CSO_GriddedAverage class:

<rcbase>.input.file        :  /Scratch/model/output/CSO_output_%Y%m%d_%H%M_gridded.nc

Specify a list of variables to be plotted, for example the retrieved and simulated column:

<rcbase>.vars                   :  yr ys

For each variable, define the name of the source variable:

<rcbase>.var.yr.source               :  yr
<rcbase>.var.yr.source               :  ys

Optionally specify target units that are different from the input:

! convert units:
<rcbase>.var.yr.units              :  1e15 mlc/cm2

The value range of the colorbar could be tunes using (default limits are based on data values):

<rcbase>.var.vcd.vmin              :   0.0
<rcbase>.var.vcd.vmax              :  10.0

The colors in the colorbar could be changed using:

<rcbase>.var.qa_flag.colors        :   ['red','yellow','green']

The label below the colorbar will by default show the variable name, unless a long_name is defined:

<rcbase>.var.vcd.long_name         :   retrieved vertical column density

The name of the created image files is read from:

! target files, time tempates are replaced:
<rcbase>.output.file            :  %Y/%m/%d/S5p_RPRO_NO2_%Y%m%d_%H%M_gridded_%{var}_%{region}.png

which will give:

2020/01/01/CSO_output_20200101_0100_gridded_yr.png
           CSO_output_20200101_0100_gridded_ys.png
                                            :

Enable the following flag to re-create existing files, by default only non-existing files are created:

<rcbase>.renew                  :  False

The figures size could be specified to fine tune the shape:

! figure size (inches), default is (8,6):
<rcbase>.figsize             :  (6,6)

! extra keyword arguments for map:
<rcbase>.map                 :  resolution='l'

A list of region codes that where actually used to create plots (regions within the domain) is written to a file regions.csv to facilitate index generation.

class cso_regions.CSO_Catalogue_RegionsTimeSeries(rcfile, rcbase='', env={}, indent='')

Bases: cso_catalogue.CSO_CatalogueBase

CSO task to create catalogue of timeseries of region aggregates.

class cso_regions.CSO_Statistics_RegionsTables(rcfile, rcbase='', env={}, indent='')

Bases: utopya_rc.UtopyaRc

CSO task to create tables with statistics over aggregations over regions.