Run logs

Every run leaves a notebook

You do not redirect output to a log file. The kernel writes each run as an nbformat 4 notebook beside the source file, while it runs.

The __runs__ folder

experiments/
├─ train.py
└─ __runs__/
   └─ train.py/
      ├─ 20261003-142233-a1f0.ipynb
      ├─ 20261003-150102-77c2.ipynb
      └─ index.json

• One run is one request: the whole file or a few cells.
• run_id is YYYYMMDD-HHMMSS-<4 hex> in UTC, so names sort by time.
• index.json lists the runs newest first with id, status, start, end and mode. It is rebuilt if it is damaged.
• Any Jupyter viewer opens the .ipynb. Notebook metadata metadata.darkpyonix holds the run id, kernel id, file hash, mode, params, status, times, interpreter and host.

How it is written

The kernel rewrites the log atomically (temporary file, then rename) at most every second while the run is going. If the kernel is killed, the file is still valid JSON and holds the output up to about a second before the kill. Its status stays running; the next kernel for that file marks it crashed.

Status values are running, ok, error, interrupted and crashed. When one cell's stream output passes 16 MiB (DARKPYONIX_RUN_OUTPUT_LIMIT), the rest goes to <run_id>.cell<index>.log beside the notebook, so the notebook stays readable. Live events are not limited.

Mapping logs to cells

Each cell in a log carries its index, type, title, status, times and source_sha256, the hash of the cell body. An editor maps the latest log onto the file by id, then by source hash, then by index. A cell whose source changed since the run can be shown as an output of the previous source.

The __runs__ variable

Inside the kernel the same logs are a variable, so code (yours or an agent's) can read past results without opening files:

run = __runs__.latest                 # the last finished run, or None
run.run_id, run.status, run.params    # '20261003-150102-77c2', 'ok', {'lr': 0.01}
run.cells[0].text                     # stdout of the first recorded cell
run.cell("Train").result             # repr of that cell's last expression
__runs__.list(limit=5)                 # newest first
__runs__.current.path                 # the .ipynb being written right now

A run and a cell are dict subclasses with attribute access. An attribute is looked up in the dict first, then in metadata.darkpyonix. cell.text joins the stdout stream, cell.stderr joins stderr, and run.cell(key) finds a cell by id, title or file index. json.dumps(run) gives the notebook back unchanged, and changing the returned value never changes the log.

Variables are not checkpointed. To get a value back, rerun the code that the log recorded.

Logs and Git

__runs__/ is tracked by default, because a run's record is part of the work. Add it to .gitignore if you do not want that.