Utilities — rl_mind.nn, rl_mind.notebook

Neural-network helpers — rl_mind.nn

Neural network helpers.

• build_mlp builds a multi-layer perceptron from a list of layer sizes.

• soft_update performs the Polyak averaging update of a target network, \(\theta' \leftarrow \tau \theta + (1 - \tau) \theta'\).

rl_mind.nn.build_mlp(sizes: list[int], activation: Module = nn.ReLU(), output_activation: Module | None = None) → Sequential

Build a multi-layer perceptron

Parameters:

• sizes – The sizes of the layers, e.g. [obs_dim, 64, 64, action_dim]

• activation – The activation module between hidden layers

• output_activation – An optional activation after the last layer (e.g. nn.Tanh() for actions in \([-1, 1]\))

Returns:

A torch.nn.Sequential of len(sizes) - 1 linear layers, separated by activation

rl_mind.nn.soft_update(source: Module, target: Module, tau: float) → None

Update the target network parameters towards the source network

Performs \(\theta' \leftarrow \tau \theta + (1-\tau) \theta'\) for each parameter \(\theta'\) of target, in place and without tracking gradients. A small tau makes the target network lag behind, which is what stabilizes the bootstrapped targets of DDPG/TD3/SAC.

Parameters:

• source – The network being trained

• target – The target network, updated in place

• tau – The interpolation factor, in \([0, 1]\) (\(\tau = 1\) copies the source network)

Notebook helpers — rl_mind.notebook

Notebook utilities.

Small helpers that adapt to the execution context (Jupyter notebook, Colab or plain Python script): displaying videos, launching tensorboard, and creating per-run output directories.

rl_mind.notebook.is_notebook() → bool

True when running inside a Jupyter notebook or Google Colab

Used by the helpers that must degrade gracefully when a source is run as a plain script (as make check does).

rl_mind.notebook.outputs_directory() → Path

The base directory for all outputs (tensorboard logs, videos, …)

Reads NOTEBOOK_PROFILE directly (rather than importing rl_mind.profiles, which needs cs-lab) so this stays available with a bare pip install rl-mind; make check sets it to fast-test (see jupytext-notebook-helper’s CHECK_PROFILE).

Returns:

outputs-testing when NOTEBOOK_PROFILE is fast-test, outputs otherwise (both relative to the working directory)

rl_mind.notebook.run_directory(name: str) → Path

Create (and return) a fresh output directory for a training run

Parameters:

name – A name for the run, e.g. "ddpg-CartPoleContinuous-v1-S1"

Returns:

The created directory, <outputs>/ — suitable as a tensorboard log directory and as the run_dir of an Evaluator

rl_mind.notebook.setup_tensorboard()

Show tensorboard in the notebook, or print the command to launch it

In Jupyter, a chooser lets the student pick where the dashboard lives: external (the default: the command to run from a shell is shown) or in the notebook (the dashboard is displayed inline). Colab has no shell to run it from, so the dashboard is always shown inline there; from a plain script (as make check runs the sources), the command is printed.

The logs of every run are written under outputs/ relative to the working directory of the kernel — the absolute path is printed so that a tensorboard started from a shell watches the right directory.

rl_mind.notebook.video_display(path: Path | None)

Display a video in the notebook (or print its path in a script)

Parameters:

path – The video file, or None (nothing was recorded)

Returns:

An IPython.display.Video to be shown as the cell result in a notebook, None otherwise

rl_mind.notebook.silence_known_warnings() → None

Silence third-party warnings that clutter the notebook output

pygame (used for rendering) and tensorboard both still import the deprecated pkg_resources API: the resulting warning is noise we cannot fix on our side.