# `utopya_tools` module

The `utopya_tools` module provides access to a number of general purpose methods and classes.

## Overview

The following methods and classes are provided:

- A method is provided to import classes from a module using a path-like name:

  - `ImportClass()`

- Two methods are provided to create directories:

  - `CreateFilePath()`

  - `CreateDirs()`

- The following tools facilitate (re)writting of text files:

  - `WriteTextFile()`

  - `UpdateTextFile()`

- A class is provided to call a subprocess with more informative exception handling:

  - `Call`

## Methods and classes

utopya_tools.ImportClass(moduleclass)

  Import class from a module. Specify the moduleclass as a string:

      [path/]module.classname

  The path to the module is optional; if not specified, the module should be available on the search path. Example:

      mycls = ImportClass( '/path/to/utopya/v1.4/py/utopya.UtopyaBase' )

utopya_tools.CreateFilePath(filename)

  Create directory in provided ‘filename’ if not present yet.

utopya_tools.CreateDirs(dirname, renew=False)

  Create directory (including subdirs) if not present yet. If renew flag is set, remove existing version first.

utopya_tools.WriteTextFile(filename, text)

  Write a text file.

  Arguments:

  - filename : target file name

  - text : list of strings, a line should end with the newline character ‘\n’

utopya_tools.UpdateTextFile(filename, newtext)

  Replace a file by a new text if the later differs from the current content.

  Arguments:

  - filename : target file name

  - newtext : list of strings, a line should end with the newline character ‘\n’

utopya_tools.ReadNcVariable(filename, varname)

  Return data and attributes from netcdf variable.

  Arguments:

    filename : input file (netcdf) varname : (path to) variable

  Return values:

    data : array with data values attrs : dictionairy with the variable attributes

utopya_tools.GetValuesFunction(part)

  Evaluate function description, return list of values. Supported functions:

  - `CsvFile( 'filename.csv' [, **kwargs] )`

    The csv file is by default supposed to have a header line and semi-colon seperated values:

        code;long_name
        NL1234;Oeverloos
        NL1235;Laagwater
        :

    The first column is returned as list of values. Optional kwargs are passed to `pandas.read_csv()` to fine-tune reading of the file:

        CsvFile( 'filename.csv', sep=',', skiprows=2 )

  - `Range( 0, 23, 1, '%2.2i00' )`

    Return list of formatted integer values. The arguments included in the function description are evaluated to a start, end, step increment, and a formatting template.

  - `TimeSeries( '2017-01', '2017-12', '1 month', 'conc_timeseries_%Y-%m' )`

    Return list of formatted time values. The arguments included in the function description are evaluated to a start time, end time, time step increment, and a formatting template that can be handled by the `datetime.datetime.strftime()` method.

class utopya_tools.Call(command, input=None, verbose=True, indent='', err2out=False, **kwargs)

  Bases: `object`

  Class to call a subprocess and trap exceptions.

  This routine differs from those provided by the standard `subprocess` module in the following ways.

  - In case the command could not be executed a `CallingError` is raised. The original command as well as the error message are included in the exception.

  - If the command returns with a non-zero exit status a `StatusError` is raised.

  - Standard output and standard error are written using the `logging` module. The standard output is written with loglevel logging.INFO if verbose=True, and otherwise with loglevel ‘logging.DEBUG’. The standard error is written with level logging.ERROR. All lines written by the logging module are preceded by the optional indent. With the ‘err2out’ flag enable, std.error is written to std.output, which is sometimes useful to maintain the order in which messages are issued.

  The command and optional keyword arguments are passed to the `subprocess.Popen` class. The opional input (list of str objects) will be passed to its `subprocess.Popen.communicate()` method.

  Returns an object with attributes:

  - retcode (integer)

  - stdout (list of strings)

  - stderr (list of strings)

  Example of usage:

      # modules:
      import logging

      # list file, log result outside call:
      try :
          p = Call( ['/bin/ls','-l'] )
      except CallingError as err :
          logging.error( str(err) )
          raise Exception
      except StatusError as err :
          for line in err.stderr : logging.error(line)
          logging.error( str(err) )
          raise Exception
      except Exception as err :
          logging.error( 'unknown error : %s' % err )
          raise Exception
      #endtry
      # display output:
      for line in p.stdout : logging.info(line)
      for line in p.stderr : logging.error(line)

      # idem, but exception handling and logging is done in the call:
      p = Call( ['/bin/ls','-l'] )

exception utopya_tools.CallingError(command, strerror)

  Bases: `Exception`

  This exception is raised when an attempt to call a command fails. The following attributes are defined:

  - command : str object or list of str objects with the command that failed

  - strerror : error message raised by command

  Use the ‘str()’ method to obtain a nicely formatted message including the command and the error message.

  Example of usage:

      try :
          ...
      except CallingError as err :
          print( str(err) )

exception utopya_tools.StatusError(command, returncode, stdout, stderr)

  Bases: `Exception`

  This exception is raised when a called command returns a non-zero exit status. The following attributes are defined:

  - command : str object (list of) with original command

  - returncode : integer return status

  - stdout : str list

  - stderr : str list

  Use the ‘str()’ method to obtain a nicely formatted message including the command and the return code.

  Example of usage:

      try :
          ...
      except StatusError as err :
          print( str(err) )