`rc` module¶

Python module to read settings from a specially formatted text file. This text file will be denoted as a ‘resource’ file, or simpy ‘rcfile’.

Format of rc-files¶

An rc-file is a text file with key/value pairs seperated by a colon ‘:’. This is the format also used for X-Resources, from which the name resource-file or simply rc-file has been adapted. An example of the format is:

my.flag    :  T
my.answer  :  42

The following functionality is supported.

Line continuation¶

Long values could be continued at the next line after a ‘\’ as last character:

my.longlist : value1 value2 \
              value3 value4

Annotation¶

The following formatting rules are useful to make the settings readible and understandable.

Empty lines are ignored.
Comment lines starting with a ‘!’ as first character are ignored.
Comment starting with ‘!’ is stripped from the values. To have a value including exclamation marks, use an escaped version ‘\!’:
```
my.value      :   -999    ! just an integer value
my.message    :   This value has 64 characters \! Count if you don't believe it ...
```
Note that currently the remainder of the value is not scanned for comment.

Variable substitution¶

Both the keys and the values might contain variable expressions, that are substituted by actual values on initialization. In case a variable substitution could not be resolved an error message will be raised.

Variables are enclosed by ‘${’ and ‘}’, e.g. ‘${HOME}’.

The following variable subsitutions are supported, in the order listed here (the first possible substitution is performed):

User defined variables might be passed on initialization using the optional ‘env’ dictionairy; see the initializaton of the RcFile class.
Substitution of operating system environment variables:
```
input.dir :   ${HOME}/Documents/data
```
Substitution of values assigned to other keys:
```
grid                 :  glb300x200
input.${grid}.path   :  /data/input/${grid}
```
The substitutions are performed in a loop until nothing has to be substituted anymore, or some substitutions could not be applied at all; for the later an error is raised. Values to be substituted could therefore be set before and after they are used.

Special variables:

${__filename__}   # absolute path to settings file

${__cwd__}        # current work directory

${__pid__}        # evaluates to the current process id; 
                  # useful for names of log files etc

${__script__}     # evaluates to the base name of the calling script, 
                  # thus without .py etc

${__hostname__}   # evaluates to the hostname

Note that it also possible to define other enclosing marks than ‘${’ and ‘}’ using the optional ‘marks’ argument on initialization.

Expressions¶

Evaluation of expressions is applied to all values enclosed by ‘$((..))’ . The enclosed value should be a valid Python expression after all variable subsitutions have been applied:

ntask       :  4
nthread     :  2
ncore       :  $(( ${ntask} * ${nthread} ))

Including other settings¶

Include the key/value pairs from another file:

#include path/to/some-other.rc

Import settings¶

To import specfic settings from another file, use:

#from path/to/some-other.rc import key1 key2 ...
#from path/to/some-other.rc import key as newkey

Conditional expressions¶

Case specific settings could be defined using a conditional expression:

#if ${my.number} == 1
message    : Welcome
#elif ${my.number} == 2
message    : Welcome back
#else
message    : Whatever ...
#endif

The condition should be a valid python expressions that evaluates to a boolean; variable substitutions are performed before evaluation. Examples:

${my.number} == 4
"${my.name}" == "UTOPyA"

Keep it simple! Very complicated and nested if-statements might not be resolved correctly, and are in any case not easy to understand for other users.

Error messages¶

In case an undesired condition is found, it is possible to raise have an error message raised using the special ‘#error’ mark. Everything behind the ‘#error’ mark is displayed as an error message, eventually use ‘\n’ for newline and ‘\t’ for tab (4 spaces):

#if ${my.number} < 0
#error No settings provided for number : ${my.number}
#endif

Loop evaluation¶

A for-loop could be used to quickly set a number of similar settings:

#for XX in AA BB CC :
setting.XX   :  This is the value for XX.
#endfor

This will expand to:

setting.AA   :  This is the value for AA.
setting.BB   :  This is the value for BB.
setting.CC   :  This is the value for CC.

Usage as Python module¶

Initialization of rcfile object¶

The RcFile class is used to read, process, and store all key/value pairs from a rcfile. Initialize an object of this class by passing the name of the rcfile:

rcf = rc.RcFile( 'settings.rc' )

All substitutions described under formatting are applied on reading, unless the optional ‘raw’ flag is enabled:

rcf = rc.RcFile( 'settings.rc', raw=True )

See the initializaton of the RcFile class for details on this and other optional arguments.

Rcfile keys¶

The RcFile.has_key() method is provided to test if a key is defined:

if rcf.has_key('my.flag') :
  print 'value of my flag is : ', rcf['my.flag']

Extract a list with all keys using the RcFile.keys() method:

rcf.keys()

Get values¶

The RcFile.get() method is provided to extract values.

By default, the ‘get’ function returns the value as a str type:

s = rcf.get('my.value')

A second argument is the name of the python type to which the value is converted to:

i = rcf.get('my.flag','int')

If the return type should be a ‘bool’, the result is:

True for values : ‘True’ , ‘T’, ‘yes’, or ‘1’ ,
False for values : ‘False’, ‘F’, ‘no’ , or ‘0’ .

For other values that should be converted to a ‘bool’ an error is raised.

A default value could be returned if the key is not found, without raising an exception:

rcf.get( 'version', default='v1.2' )

Print a debug message to the logging system for each extracted key:

rcf.get( 'my.flag', verbose=True ) 

Add new key/value pairs¶

Use the RcFile.add() method to add a new value:

rcf.add( 'my.iter', 2 )

This function is useful to pass changed settings to other methods, or to write modified rcfiles using the RcFile.WriteFile() method. Eventually specify a comment line; if the content of the object is written to a file, this comment will be written before the key/value pair:

rcf.add( 'my.iter', 2, comment='iteration number for restart' )

Replace values¶

Assign a new value to an existing key with the RcFile.replace() method:

rcf.replace( 'my.flag', True )

Substitute values¶

The RcFile.substitute() method could be used to replace keys by rcfile values in a character line. For example, suppose the rcfile stored had content:

name       : Model
version    : v1.2

A character line with the right templates could now be evaluated into a version with values inserted at the right place:

line = rcf.substitute( 'This is version %{version} of %{name}.' )
print line
This is version v1.2 of Model.

Write content¶

The RcFile.WriteFile() method could be used to write the content of the object to a new file, with all variables expanded and included files included:

rcf.WriteFile( 'newfile.rc' )

Usage via script¶

Use the ‘rcget’ script to extract values from a rcfile in non-python application.

Example of usage:

rcget 'mytest.rc' 'version'

To see all options:

rcget --help

History¶

1998? Arjo Segers, TU Delft

Initial implementation of configuration options in ‘rc’ file style.

Read routines in Fortran.
2001? Arjo Segers, KNMI

Implementation of shell script to read settings from within a shell script.
2008? Andy Jacobson, NOAA

Translation to python of original shell script ‘go_readrc’ .
2009-06 Wouter Peters, WUR

Support substitution of previously defined variables.
2009-06 Arjo Segers, TNO

Support include files.
2009-09 Arjo Segers, TNO

Re-coded into class.

Implemented substitution loop.
2009-11 Arjo Segers, JRC

Added main program to run this file as a shell script.

Added replace and substitute routines.
2010-03 Arjo Segers, JRC

Support simple if-statements.

Support comment in values.
2010-07 Wouter Peters, WUR

Downgraded to work for python 2.4.3 too.

Added read/write routines for backwards compatibility.
2010-07-27 Arjo Segers, JRC

Maintain list with rcfile names and line numbers to be displayed with error messages to identify where problematic lines are found.
2010-07-28 Andy Jacobson, NOAA

Add second dictionary of key,linetrace values to help track the provenance of #included keys (to debug multiple key instances).

Identify duplicate keys by checking on different source lines instead of checking if the values are different.
2010-10 Arjo Segers, JRC

Restructured processing using classes for source lines and rcfile values, and resolve using recursive calls.

Added evaluation of expression enclosed by $((.)) .

Added for-loop.

Removed main program and stored this in the auxilary script ‘rcget’ .
2015-04 Arjo Segers, TNO

Formatted helptext for Sphinx.
2015-05 Arjo Segers, TNO

Load included files relative to including file if necessary.

Classes¶

class rc.RcFile(filename, raw=False, marks=('${', '}'), env={}, debug=False)¶

Bases: object

Class to store settings read from a rcfile. The filename of the rcfile to be read should be passed as first argument.

Variable substitutions are applied and special lines are evaluated, unless ‘raw’ is set to True.

The 2-item tupple (mark1,mark2) could be used to re-define the default substitution pattern ‘${..}’ into something else:

<mark1>...<mark2>

An extra environment dictionairy ‘env’ could be passed to substitute variables. For example, if the name of an output directory should depend on an iteration step number that is only available at run time, one could use a setting:

! step depended output:
output.dir     :  /scratch/me/step-${__STEP__}/output

and the rcfile should then be initialized using argument:

env = { '__STEP__' : 12 }

Enable the ‘debug’ flag to have messages printed about the key/value pairs found.

has_key(key)¶: Return bool to test if specified key is defined in rcfile.

keys()¶: Return list of keys defined in rcfile.

get(key, totype='', default=None, verbose=False)¶

Return element ‘key’ from the dictionary.

If the element is not present but a default is specified, than return this value.

If ‘verbose’ is set to True, a debug message is send to the logging system on which value is returned for the given key.

The optional argument ‘totype’ defines the conversion to a Python type.

If ‘totype’ is set to ‘bool’, the return value is:
- True for values ‘T’, ‘True’, ‘yes’, and ‘1’;
- False for ‘F’, ‘False’, ‘no’, or ‘0’.
If ‘totype’ is set to ‘datetime’, the content is read into a datetime.datetime object.

For other values, an exception will be raised.

replace(key, val)¶: Replace the value of a key by a new value.

add(key, val, comment='')¶: Add a new key/value pair.

replace_add(key, val)¶: Replace the value of a key by a new value, or add the key/value pair if not present yet.

substitute(line, marks=('${', '}'))¶

Return a line with all ‘${..}’ parts replaced by the corresponding rcfile values. The 2-item tupple (mark1,mark2) could be used to re-define the default key pattern ‘${..}’ into something else:

<mark1>...<mark2>

getlines()¶: Return list with processed rcfile lines.

WriteFile(filename)¶: write the dictionary to file

`rc` module¶

Format of rc-files¶

Line continuation¶

Annotation¶

Variable substitution¶

Expressions¶

Including other settings¶

Import settings¶

Conditional expressions¶

Error messages¶

Loop evaluation¶

Usage as Python module¶

Initialization of rcfile object¶

Rcfile keys¶

Get values¶

Add new key/value pairs¶

Replace values¶

Substitute values¶

Write content¶

Usage via script¶

History¶

Classes¶

Table of Contents

Previous topic

Next topic

This Page

rc module¶

Format of rc-files¶

Line continuation¶

Annotation¶

Variable substitution¶

Expressions¶

Including other settings¶

Import settings¶

Conditional expressions¶

Error messages¶

Loop evaluation¶

Usage as Python module¶

Initialization of rcfile object¶

Rcfile keys¶

Get values¶

Add new key/value pairs¶

Replace values¶

Substitute values¶

Write content¶

Usage via script¶

History¶

Classes¶

`rc` module¶