Logging and tests utilities

Utilities to ease the setting of logging and tests

Logging

Logging Levels

Python logging has five standard levels, in increasing order of severity:

DEBUG (10): Detailed information, typically useful for diagnosing problems
INFO (20): Confirmation that things are working as expected
WARNING (30): An indication something unexpected happened, but the program still works
ERROR (40): Due to a more serious problem, the software couldn’t perform some function
CRITICAL (50): A very serious error, indicating the program may be unable to continue

The function does return the root Logger, though typically you would not use it directly.

set_logging

 set_logging (level:int=20, format_file:str='%(asctime)s - %(name)s -
              %(levelname)s - %(message)s',
              format_console:str='%(levelname)s - %(message)s',
              datefmt:str='%Y-%m-%d %H:%M:%S', log_dir:str=None,
              filemode:str='a', backupCount:int=5, maxBytes:int=5242880)

Set up the root Logger

	Type	Default	Details
level	int	20	The logging level
format_file	str	%(asctime)s - %(name)s - %(levelname)s - %(message)s	The logging format for the file
format_console	str	%(levelname)s - %(message)s	The logging format for the console
datefmt	str	%Y-%m-%d %H:%M:%S	The date format
log_dir	str	None	The logging directory, if None, logs to console
filemode	str	a	The logging file mode. ‘a’ for append, ‘w’ for overwrite
backupCount	int	5	The number of backup files to keep
maxBytes	int	5242880	The maximum size of the log file in bytes
Returns	Logger

Example usage

We must make sure the logging works when we are testing indivudual notebooks and when running the complete code. To accomplish this we can do the following.

First in run.py or other entry point of the project, add the following to get the logging when the complete code is run:

from hopsa import set_logging

if __name__ == "__main__":
   log_dir = "../logs"
   set_logging(log_dir=log_dir, level=10)

import logging
from hopsa import set_logging

if __name__ == "__main__":
   log_dir = "../logs"
   set_logging(log_dir=log_dir, level=logging.DEBUG)

Then in each module/notebook, you create module-specific loggers:

At the top of each notebook (00_core.ipynb, 02_features.ipynb, etc.)

#| export
import logging

#| eval: false
from hopsa import lgtst

#| eval: false
lgtst.set_logging(log_dir="../logs", level=logging.DEBUG)

#| export
logger = logging.getLogger(__name__)

The #| export nbdev directive makes sure this cell will be used in the python module. The #| eval: false nbdev directive makes sure this cell will run when we run the notebook, but it won’t be used in the Python module and it also won’t be tested when we run nbdev_prepare.

Then use the logger throughout the module

logger.debug("Debug message")
logger.info("Info message")

rt_logger = set_logging()

INFO - Log file: /home/jelle/code/hopsa/logs/hopsa.log
INFO - Log file: /home/jelle/code/hopsa/logs/hopsa.log
INFO - Log file mode: a
INFO - Log file mode: a
INFO - Log backup count: 5
INFO - Log backup count: 5
INFO - Log max bytes: 5242880
INFO - Log max bytes: 5242880

for h in rt_logger.handlers:
    h.close()
    rt_logger.removeHandler(h)
    print(f"Removed handler: {h}")

Removed handler: <StreamHandler stderr (INFO)>
Removed handler: <RotatingFileHandler /home/jelle/code/hopsa/logs/hopsa.log (INFO)>