Notebook utilities

Utilities for working with notebooks

Create markdown table from a Pydantic dataclass

Especially confenient for Documentation written automatically from the Notebooks by nbdev.

pydantic_to_markdown_table

 pydantic_to_markdown_table (model_class:Type[pydantic.main.BaseModel])

*Convert a Pydantic model class to a markdown table and display it in Jupyter notebook.

Args: model_class: A Pydantic model class (subclass of BaseModel)*

Example usage

class DummyChild(BaseModel):
    """A simple dataclass model"""
    model_name: str = Field(..., description="Name or path of the model to use") # Name
    provider: str = Field(default="huggingface", description="Model provider (huggingface, openai, etc)")
    api_key_env_var: Optional[str] = Field(default=None, description="Environment variable name for API key")
    api_base_url: Optional[str] = Field(default=None, description="Base URL for API reqeuest")
    temperature: float = Field(default=0.7, description="Temperature for generation")

class DummyParent(BaseModel):
    """Main configuration for a chat application"""
    app_name: str = Field(..., description="Name of the application")
    description: str = Field(default="", description="Description of the application")
    system_prompt: str = Field(..., description="System prompt for the LLM")
    model: DummyChild
    show_system_prompt: bool = Field(default=True, description="Whether to show system prompt in UI")
    show_context: bool = Field(default=True, description="Whether to show context in UI")
pydantic_to_markdown_table(DummyParent)

DummyParent

Main configuration for a chat application

Variable Type Default Details
app_name str PydanticUndefined Name of the application
description str ’’ Description of the application
system_prompt str PydanticUndefined System prompt for the LLM
model DummyChild PydanticUndefined (see DummyChild table)
show_system_prompt bool True Whether to show system prompt in UI
show_context bool True Whether to show context in UI

Create a tree structure from a Python dictionary

Function to clearly display the structure of a Python dictionary. The output only shows the keys.

Export notebook variables to toml

A function to create a TOML file from the contents of a Jupyter Notebook. This helps to create a simple interface to set the parameters of the application, without the need to build a complete GUI or demand of the user to edit TOML files directly without the ease and explanations possible in a Jupyter Notebook.

TOML can’t handle None values. So if you want to set a value to None, just don’t add that variable to the Jupyter Notebook or comment it out.

The function will replace any spaces in a heading with underscores.

/opt/hostedtoolcache/Python/3.11.12/x64/lib/python3.11/site-packages/fastcore/docscrape.py:230: UserWarning: Unknown section Parameters:
  else: warn(msg)
/opt/hostedtoolcache/Python/3.11.12/x64/lib/python3.11/site-packages/fastcore/docscrape.py:230: UserWarning: Unknown section Returns:
  else: warn(msg)

export_ipynb_toml

 export_ipynb_toml (nb_path:Optional[str]=None,
                    output_path:Optional[str]=None)

*Export the content of the current Jupyter notebook to a TOML file.

This function reads the content of the notebook where it’s being executed, extracts all level 2 (##) markdown cells as groups, and all parameter assignments in code cells as key-value pairs within those groups. Regular text markdown cells are ignored.*

Usage of globals() and ast

The ast (Abstract Syntax Tree) module in Python allows you to parse Python source code into its syntax tree representation. This lets you analyze and manipulate Python code programmatically.

ast.parse(source): Parses Python code (as a string) into an AST tree.

globals() is a built-in function that returns the current global symbol table as a dictionary.

Why use globals() first?

Dynamic or computed variables: In Jupyter notebooks, variables can be set by code that isn’t just a literal assignment. For example:

a = 1 + 2
b = some_function()
c = [i for i in range(5)]
  • globals()[‘a’] will give 3.
  • globals()[‘b’] will give the result of some_function().
  • globals()[‘c’] will give the actual list [0, 1, 2, 3, 4].

ast.literal_eval limitations:

  • ast.literal_eval can only evaluate simple literals (strings, numbers, lists, dicts, etc.), not arbitrary Python expressions or anything involving variables/functions.
  • For example, it fails on a = 1 + 2, b = some_function(), or c = [i for i in range(5)].

Notebook context:

  • In a notebook, users often assign variables dynamically, not just with literals.
  • Using globals() ensures you get the actual value as it exists in the current kernel session, reflecting any computation or function calls.

Why fall back to ast.literal_eval?

For simple assignments: - If the variable isn’t found in globals() (maybe the cell wasn’t run, or the variable was deleted), you can try to parse the value directly from the code if it’s a literal. - This works for things like x = 42 or y = “hello”, but not for expressions or function calls.

Summary

  • globals(): Gets the current, actual value in the notebook, including results of computations.
  • ast.literal_eval: Only works for simple literals, but is a safe fallback if the variable isn’t in globals().

Write TOML files using binary

We write using binary, because tomli-w is designed to work the same way as the built-in tomllib, which only reads from binary streams.

This ensures consistent encoding (UTF-8) and avoids issues with text encodings across platforms.

Example usage

somewhere = "/home/jared/lost"
export_ipynb_toml("nbuse.ipynb", "../tests/variables_user.toml")
TOML file saved to: ../tests/variables_user.toml

then read the TOML file using the tomllib module by doing:

with open("../tests/variables_user.toml", "rb") as tml:
    usr_toml = tomllib.load(tml)

print_dict_structure(usr_toml)
├── Create_markdown_table_from_a_Pydantic_dataclass
├── Create_a_tree_structure_from_a_Python_dictionary
  ├── movie_dict
    ├── The Big Lebowski
      ├── characters
        ├── Dude
        ├── Walter
        ├── Donny
      ├── setting
        ├── location
        ├── object
    ├── Office Space
      ├── characters
        ├── Peter
        ├── Milton
        ├── Lumbergh
      ├── setting
        ├── company
        ├── object
    ├── The Princess Bride
      ├── characters
        ├── Westley
        ├── Inigo
      ├── setting
        ├── location
    ├── Labyrinth
      ├── characters
        ├── Sarah
        ├── Jareth
├── Export_notebook_variables_to_toml
  ├── somewhere
├── Default_nbdev_code_cells

Default nbdev code cells