API reference

Every public name is documented in the pages below, one per area of the library (listed in the sidebar), in the order in which it appears in the source. The documentation is generated from the docstrings of src/rl_mind, so it always describes the code of the version it was built from — shown in the sidebar, and the one students get from PyPI.

Import conventions. Everything listed below (except the extra environments, rl_mind.envs / rl_mind.mazemdp_env, and rl_mind.profiles) is re-exported by the package, so both of these work:

from rl_mind import VecEnv, Transitions          # short
from rl_mind.env import VecEnv                   # explicit, used by the sources

The extra environments are deliberately not re-exported: importing them registers gymnasium environments, and importing rl_mind must have no side effect.

Shapes. B is the number of parallel environments, N a flat batch size, T a number of time steps. A “structure” is a TensorStruct: a frozen dataclass whose fields are tensors, indexed and stacked as a whole.

Reading the signatures. Constructor arguments are documented on __init__; the fields of a dataclass (Action, EnvStep, Transitions, Rollout, Episode, EvalResult) are documented as :param: entries of the class itself, since that is how they are passed.

At a glance

The names a training loop is made of, in the order they come into play:

Name	What it is for	Page
Actor	the policy: actor(obs) explores, actor.act(obs) evaluates	Actors & actions
Action, StochasticAction	what an actor returns	Actors & actions
TensorStruct	the base of every container: index, stack, concatenate	Actors & actions
VecEnv	B gymnasium copies, torch tensors in and out	Environments
EnvStep	what one step of a VecEnv returns	Environments
Transitions, ReplayBuffer	off-policy data, and where it is kept	Data
Episode, Rollout	episodic and on-policy data	Data
TransitionCollector	off-policy collection (DQN, DDPG, TD3, SAC)	Collectors
EpisodeCollector	whole episodes (REINFORCE)	Collectors
RolloutCollector	[T, B] rollouts (A2C, PPO)	Collectors
Evaluator	periodic evaluation, keeps the best actor	Evaluation
RenderWidget	watching an environment live in a notebook	Rendering
build_mlp, soft_update	network helpers	Utilities
run_directory	where a run writes its outputs	Utilities