CLI, Parallel Runs, and Quick-Start Documentation
Type: Feature / Developer Experience
Priority: High
Labels: cli, parallel, docs, good-first-project, intern
Prerequisite: Issue Separate Data Interface from Model Logic
must be merged before starting this issue.
Background
After the data-interface refactor the model is easy to instantiate and drive
programmatically, but a collaborator who just cloned the repo still has to:
- Read Python source to understand how to set parameters
- Edit
ABM_CE_PV_MultipleRun.py directly to change anything
- Run sequential loops to get multiple replicates (one core, slow)
ABM_CE_PV_BatchRun.py once handled bulk runs and sensitivity analysis, but it
hard-codes the old 70-argument constructor and is no longer compatible with the
new ABM_CE_PV(config, data) interface. It should be considered deprecated and
superseded by the work in this issue.
This issue adds three things:
- A clean command-line interface — users run sims by typing a command, not
editing Python files
- Parallel batch runs — launch N independent replicates across all available
CPU cores from a single command
- Quick-start documentation — a collaborator can clone, set up the
environment, and run their first simulation in under 15 minutes
Goals
Proposed File Layout
ABSiCE/
│
├── run.py ← NEW: single CLI entry-point (replaces MultipleRun.py)
│
├── config/ ← from data-interface issue
│ ├── default_simulation.yaml
│ └── data_paths.yaml
│
├── absice/ ← from data-interface issue
│ ├── __init__.py
│ ├── schemas/
│ │ ├── __init__.py
│ │ ├── simulation_config.py
│ │ └── data_paths_config.py
│ ├── data/
│ │ ├── __init__.py
│ │ └── data_loader.py
│ ├── model/
│ │ ├── __init__.py
│ │ └── ABM_CE_PV_Model.py
│ └── runner.py ← NEW: pure-Python batch runner (no CLI)
│
├── docs/
│ └── quickstart.md ← NEW: step-by-step guide for new collaborators
│
├── ABM_CE_PV_ConsumerAgents.py ← unchanged
│ ... (other agent files unchanged)
├── ABM_CE_PV_MultipleRun.py ← superseded by run.py; keep but do not edit
└── results/ ← batch runs write here; one sub-folder per scenario
Phase 1 — Set Up the CLI Entry Point
1a. Understand what click is
click is a Python library for building
command-line interfaces. Instead of printing sys.argv by hand, you decorate
ordinary functions and click handles argument parsing, --help text, and
error messages for you automatically.
# Without click — fragile, no --help, error-prone:
import sys
config = sys.argv[1] # breaks if user forgets the argument
# With click — validated, self-documenting:
import click
@click.command()
@click.option("--config", default="config/default.yaml", help="Config file path.")
def run(config: str) -> None:
"""Run one simulation."""
print(f"Using config: {config}")
# Now `python run.py --help` prints usage automatically,
# and `python run.py --config config/my_scenario.yaml` just works.
Install it with conda env update after adding it to
pv_abm_env_platform_independent.yaml (see Step 1 below).
1b. Add click to the environment
# pv_abm_env_platform_independent.yaml
name: pv_abm
channels:
- defaults
- conda-forge
dependencies:
- mesa=3.2.0
- salib=1.5.1
- networkx=3.4.2
- matplotlib=3.10.0
- geopy=2.4.1
- openpyxl=3.1.5
- click=8.1.8 # ← add this line
- pydantic>=2.0 # ← add this line (from the data-interface issue)
- pytest>=8.0 # ← add this line
Then recreate your environment:
conda env update -f pv_abm_env_platform_independent.yaml --prune
1c. Create run.py
# run.py
"""
ABSiCE simulation runner.
Usage examples
--------------
Single run with the default config:
python run.py run
Single run with a custom config:
python run.py run --config config/my_scenario.yaml
Ten parallel replicates on 4 workers:
python run.py batch --n-runs 10 --workers 4
Write a default config YAML to disk:
python run.py init-config --output config/my_scenario.yaml
"""
from pathlib import Path
import click
PROJECT_ROOT: Path = Path(__file__).parent
@click.group()
def cli() -> None:
"""ABSiCE — Agent-Based Simulation of Circular Economy for PV."""
@cli.command()
@click.option(
"--config", "-c",
default="config/default_simulation.yaml",
show_default=True,
type=click.Path(exists=True, dir_okay=False, path_type=Path),
help="Path to simulation config YAML.",
)
@click.option(
"--paths", "-p",
default=None,
type=click.Path(exists=True, dir_okay=False, path_type=Path),
help="Path to data_paths YAML. Defaults to built-in project paths.",
)
@click.option(
"--output-dir", "-o",
default="results",
show_default=True,
type=click.Path(file_okay=False, path_type=Path),
help="Base directory for results. Output goes into OUTPUT_DIR/LABEL/.",
)
@click.option(
"--label", "-l",
default=None,
help="Sub-folder name for this run's output. Defaults to the config file stem "
"(e.g. 'my_scenario' for config/my_scenario.yaml). "
"Use this to keep results from different scenarios separate.",
)
def run(config: Path, paths: Path | None, output_dir: Path, label: str | None) -> None:
"""Run a single simulation and write output to OUTPUT_DIR/LABEL/."""
from absice.runner import run_single
run_single(
config_path=config,
paths_yaml=paths,
output_dir=output_dir,
label=label or config.stem,
project_root=PROJECT_ROOT,
)
@cli.command()
@click.option(
"--config", "-c",
default="config/default_simulation.yaml",
show_default=True,
type=click.Path(exists=True, dir_okay=False, path_type=Path),
help="Path to simulation config YAML.",
)
@click.option(
"--paths", "-p",
default=None,
type=click.Path(exists=True, dir_okay=False, path_type=Path),
help="Path to data_paths YAML. Defaults to built-in project paths.",
)
@click.option(
"--n-runs", "-n",
default=10,
show_default=True,
type=click.IntRange(min=1),
help="Number of independent replicates to run.",
)
@click.option(
"--workers", "-w",
default=None,
type=click.IntRange(min=1),
help="Number of parallel worker processes. Defaults to number of CPU cores.",
)
@click.option(
"--output-dir", "-o",
default="results",
show_default=True,
type=click.Path(file_okay=False, path_type=Path),
help="Base directory for results. Output goes into OUTPUT_DIR/LABEL/.",
)
@click.option(
"--label", "-l",
default=None,
help="Sub-folder name for this batch's output. Defaults to the config file stem "
"(e.g. 'my_scenario' for config/my_scenario.yaml). "
"Use this to keep results from different scenarios separate.",
)
def batch(
config: Path,
paths: Path | None,
n_runs: int,
workers: int | None,
output_dir: Path,
label: str | None,
) -> None:
"""Run N independent replicates in parallel and write results to OUTPUT_DIR/LABEL/."""
from absice.runner import run_batch
run_batch(
config_path=config,
paths_yaml=paths,
n_runs=n_runs,
workers=workers,
output_dir=output_dir,
label=label or config.stem,
project_root=PROJECT_ROOT,
)
@cli.command("init-config")
@click.option(
"--output", "-o",
required=True,
type=click.Path(dir_okay=False, path_type=Path),
help="Path to write the new YAML config file.",
)
def init_config(output: Path) -> None:
"""Write a default, fully-commented simulation config YAML to OUTPUT."""
from absice.runner import write_default_config
write_default_config(output)
click.echo(f"Default config written to {output}")
if __name__ == "__main__":
cli()
Phase 2 — Implement the Runner Logic
All the actual work lives in absice/runner.py. run.py is just the CLI
wrapper — keeping them separate means the runner can be imported and called
from notebooks or tests without needing a terminal.
Why ProcessPoolExecutor, not mesa.batch_run()
What is ProcessPoolExecutor?
Python normally runs one thing at a time because of the Global Interpreter
Lock (GIL) — two threads cannot execute Python bytecode simultaneously.
For CPU-heavy work like running an ABM, threads therefore give no speedup.
ProcessPoolExecutor from the standard library sidesteps this by spawning
separate processes (each with its own memory and Python interpreter).
You submit tasks, the pool distributes them across processes, and you collect
the results when they finish.
from concurrent.futures import ProcessPoolExecutor
def square(n: int) -> int:
return n * n
with ProcessPoolExecutor(max_workers=4) as pool:
results = list(pool.map(square, [1, 2, 3, 4, 5]))
# results == [1, 4, 9, 16, 25] — computed across 4 processes in parallel
One important rule: the function you submit must be defined at module level
(not inside another function or as a lambda). The pool pickles the function to
send it to worker processes, and only module-level functions can be pickled.
That is why _worker() in runner.py is a top-level function.
Why not mesa.batch_run()?
mesa.batch_run() (Mesa 3.x) is designed for parameter sweeps: it receives a
model class and a flat dictionary of parameters it passes as keyword arguments to
the model constructor. After the data-interface refactor, ABM_CE_PV takes
(config, data) — not 70 flat kwargs — so mesa.batch_run() no longer fits.
ProcessPoolExecutor is a better fit here:
- Each worker process is independent (no shared memory → no GIL contention)
- Each worker loads its own data from disk, so no large DataFrames are pickled
across process boundaries
- Straightforward to add a progress bar with
tqdm later
# absice/runner.py
import os
import time
from concurrent.futures import ProcessPoolExecutor, as_completed
from pathlib import Path
import pandas as pd
from absice.schemas.simulation_config import SimulationConfig
from absice.schemas.data_paths_config import DataPathsConfig
from absice.data.data_loader import DataLoader
from ABM_CE_PV_Model import ABM_CE_PV
# ── public API ────────────────────────────────────────────────────────────────
def run_single(
config_path: Path,
paths_yaml: Path | None,
output_dir: Path,
label: str,
project_root: Path,
) -> Path:
"""
Load config, load data, run one simulation, save results to a CSV.
Results are written to ``output_dir / label / Results_model_run_<seed>.csv``
so that runs from different scenarios never share a directory.
Parameters
----------
config_path : Path
Path to the simulation YAML config.
paths_yaml : Path | None
Optional path to a data_paths YAML. If None, uses DataPathsConfig.make_default().
output_dir : Path
Base results directory.
label : str
Sub-folder name that identifies this scenario (e.g. the config file stem).
project_root : Path
Absolute path to the repo root (used to resolve default data paths).
Returns
-------
Path
Path to the CSV file that was written.
"""
config = SimulationConfig.from_yaml(str(config_path))
paths = _resolve_paths(paths_yaml, project_root)
data = DataLoader(paths).load_all(
resolution=config.consumer.resolution.value,
model_states=config.consumer.model_states,
)
output_path = _make_output_path(output_dir, label=label, run_id=config.run.seed or 0)
_execute_and_save(config, data, output_path)
print(f"Results written to {output_path}")
return output_path
def run_batch(
config_path: Path,
paths_yaml: Path | None,
n_runs: int,
workers: int | None,
output_dir: Path,
label: str,
project_root: Path,
) -> list[Path]:
"""
Run N independent replicates in parallel, each with a unique seed.
Each worker process loads data independently from disk — no DataFrames are
serialised across process boundaries. All CSVs are written under
``output_dir / label /`` so results from different scenarios never
overwrite each other.
Parameters
----------
config_path : Path
paths_yaml : Path | None
n_runs : int
Number of replicates to run.
workers : int | None
Number of worker processes. None = use all available CPU cores.
output_dir : Path
Base results directory.
label : str
Sub-folder name that identifies this scenario.
project_root : Path
Returns
-------
list[Path]
Paths to each output CSV (one per replicate), in run order.
"""
scenario_dir: Path = output_dir / label
scenario_dir.mkdir(parents=True, exist_ok=True)
t0 = time.perf_counter()
print(f"Launching {n_runs} runs on {workers or os.cpu_count()} workers …")
futures_to_id: dict = {}
with ProcessPoolExecutor(max_workers=workers) as pool:
for run_id in range(n_runs):
future = pool.submit(
_worker,
run_id=run_id,
config_path=config_path,
paths_yaml=paths_yaml,
output_dir=scenario_dir,
project_root=project_root,
)
futures_to_id[future] = run_id
output_paths: list[Path | None] = [None] * n_runs
for future in as_completed(futures_to_id):
run_id = futures_to_id[future]
try:
output_paths[run_id] = future.result()
print(f" run {run_id} done → {output_paths[run_id]}")
except Exception as exc:
print(f" run {run_id} FAILED: {exc}")
raise
elapsed = time.perf_counter() - t0
print(f"All {n_runs} runs completed in {elapsed:.1f}s")
return [p for p in output_paths if p is not None]
def write_default_config(output: Path) -> None:
"""
Write a default SimulationConfig to output as a commented YAML file.
The YAML is generated from Pydantic defaults so it can never be out of
sync with the model.
Parameters
----------
output : Path
Path to write the YAML file.
"""
# CostConfig has required fields — fill with clearly labelled placeholders.
# The intern running init-config will need to fill these in.
import yaml
# Build a config with sentinel values so the YAML makes it obvious
# what still needs to be filled in.
raw = SimulationConfig.model_fields # read field metadata without instantiating
output.parent.mkdir(parents=True, exist_ok=True)
# Easiest approach: load the existing default_simulation.yaml if it exists,
# otherwise dump the schema as a commented skeleton.
default_yaml = Path(__file__).parent.parent / "config" / "default_simulation.yaml"
if default_yaml.exists():
import shutil
shutil.copy(default_yaml, output)
else:
# Fall back: write an empty skeleton with a warning comment.
output.write_text(
"# Run `python run.py init-config` after creating config/default_simulation.yaml\n"
"# (see Phase 1 of the data-interface-separation issue for how to generate it)\n"
)
# ── private helpers ───────────────────────────────────────────────────────────
def _worker(
run_id: int,
config_path: Path,
paths_yaml: Path | None,
output_dir: Path,
project_root: Path,
) -> Path:
"""
Top-level function executed in each worker process.
Must be a module-level function (not a lambda or closure) so that
ProcessPoolExecutor can pickle it.
Parameters
----------
run_id : int
Used as the random seed and to name the output file.
config_path : Path
paths_yaml : Path | None
output_dir : Path
project_root : Path
Returns
-------
Path
Path to the CSV file written by this worker.
"""
config = SimulationConfig.from_yaml(str(config_path))
# Give each replicate a unique, reproducible seed derived from its run_id.
config = config.model_copy(
update={"run": config.run.model_copy(update={"seed": run_id})}
)
paths = _resolve_paths(paths_yaml, project_root)
data = DataLoader(paths).load_all(
resolution=config.consumer.resolution.value,
model_states=config.consumer.model_states,
)
output_path = _make_output_path(output_dir, run_id=run_id)
_execute_and_save(config, data, output_path)
return output_path
def _resolve_paths(paths_yaml: Path | None, project_root: Path) -> DataPathsConfig:
"""
Return a DataPathsConfig from a YAML file, or the built-in defaults.
Parameters
----------
paths_yaml : Path | None
project_root : Path
Returns
-------
DataPathsConfig
"""
if paths_yaml is not None:
return DataPathsConfig.from_yaml(str(paths_yaml))
return DataPathsConfig.make_default(project_root)
def _make_output_path(output_dir: Path, label: str, run_id: int) -> Path:
"""
Return the output CSV path for a given scenario label and run_id.
The directory structure is ``output_dir / label /`` so results from
different scenarios are always stored separately.
Parameters
----------
output_dir : Path
Base results directory.
label : str
Scenario identifier used as a sub-folder name.
run_id : int
Returns
-------
Path
"""
dest: Path = output_dir / label
dest.mkdir(parents=True, exist_ok=True)
return dest / f"Results_model_run_{run_id}.csv"
def _execute_and_save(
config: SimulationConfig,
data: "LoadedData",
output_path: Path,
) -> None:
"""
Instantiate the model, step it, and write the datacollector output to CSV.
Parameters
----------
config : SimulationConfig
data : LoadedData
output_path : Path
"""
from absice.data.data_loader import LoadedData # local import avoids circular import
model = ABM_CE_PV(config=config, data=data)
for _ in range(config.run.last_step):
model.step()
df: pd.DataFrame = model.datacollector.get_model_vars_dataframe()
df.to_csv(output_path)
Phase 3 — Write Tests
# test/test_runner_cli.py
from pathlib import Path
from unittest.mock import patch
from click.testing import CliRunner
from run import cli
def test_cli_registers_expected_commands() -> None:
"""cli group exposes exactly the three sub-commands we defined."""
assert set(cli.commands.keys()) == {"run", "batch", "init-config"}
def test_run_delegates_to_run_single(tmp_path: Path) -> None:
"""The run command calls runner.run_single with the correct arguments."""
config = tmp_path / "sim.yaml"
config.write_text("{}")
runner = CliRunner()
with patch("absice.runner.run_single") as mock_run_single:
runner.invoke(cli, ["run", "--config", str(config), "--output-dir", str(tmp_path)])
mock_run_single.assert_called_once()
call_kwargs = mock_run_single.call_args.kwargs
assert call_kwargs["config_path"] == config
assert call_kwargs["label"] == "sim" # defaults to config file stem
def test_init_config_delegates_to_write_default_config(tmp_path: Path) -> None:
"""init-config delegates to runner.write_default_config."""
out = tmp_path / "test_config.yaml"
runner = CliRunner()
with patch("absice.runner.write_default_config") as mock_write:
runner.invoke(cli, ["init-config", "--output", str(out)])
mock_write.assert_called_once_with(out)
# test/test_runner_unit.py
from pathlib import Path
from unittest.mock import MagicMock, patch
from absice.runner import _make_output_path, _resolve_paths
def test_make_output_path_creates_subfolder_and_returns_correct_path(tmp_path: Path) -> None:
p = _make_output_path(tmp_path, label="baseline", run_id=3)
assert p == tmp_path / "baseline" / "Results_model_run_3.csv"
assert (tmp_path / "baseline").exists()
def test_resolve_paths_calls_make_default_when_yaml_is_none(tmp_path: Path) -> None:
"""When paths_yaml is None, DataPathsConfig.make_default is called."""
with patch(
"absice.schemas.data_paths_config.DataPathsConfig.make_default"
) as mock_default:
mock_default.return_value = MagicMock()
_resolve_paths(None, tmp_path)
mock_default.assert_called_once_with(tmp_path)
Phase 4 — Quick-Start Documentation
Create docs/quickstart.md. The goal is that a new collaborator with no prior
knowledge of the project can be running simulations within 15 minutes of cloning
the repo.
# ABSiCE Quick-Start Guide
## 1. Clone and set up the environment
```bash
git clone <repo-url> ABSiCE
cd ABSiCE
conda env create -f pv_abm_env_platform_independent.yaml
conda activate pv_abm
```
## 2. Check that the required data files are in place
The simulation reads pre-processed data files that are not stored in the repo
(they are too large). Ask a team member for the `TEMP/` and `PV_ICE/` directories
and place them under `ABSiCE/`.
To verify all paths resolve correctly before running:
```bash
python -c "
from pathlib import Path
from absice.schemas.data_paths_config import DataPathsConfig
paths = DataPathsConfig.make_default(Path('.'))
print('All paths configured — check that the files exist before running.')
"
```
## 3. Configure your scenario
Copy the default configuration and edit it:
```bash
python run.py init-config --output config/my_scenario.yaml
# Open config/my_scenario.yaml in any text editor and change what you need.
```
Common things to change:
- `run.seed` — set an integer for reproducible results (e.g. `42`)
- `consumer.model_states` — restrict to a subset of US states (e.g. `[California, Texas]`)
- `scenario.hazardous_waste_regulation_enabled` — turn the regulation on/off
## 4. Run a single simulation
```bash
python run.py run --config config/my_scenario.yaml
```
Output is written to `results/my_scenario/Results_model_run_0.csv`.
The sub-folder name defaults to the config file stem so results from
different scenarios never overwrite each other.
## 5. Run multiple replicates in parallel
```bash
python run.py batch --config config/my_scenario.yaml --n-runs 10 --workers 4
```
This launches 10 independent runs (seeds 0–9) across 4 CPU cores.
Each run writes its own CSV under `results/my_scenario/`:
`Results_model_run_0.csv` through `Results_model_run_9.csv`.
To run a second scenario without touching the first:
```bash
python run.py batch --config config/epr_scenario.yaml --n-runs 10 --workers 4
# writes to results/epr_scenario/ — completely separate from results/my_scenario/
```
You can also set the label explicitly:
```bash
python run.py batch --config config/my_scenario.yaml --label baseline_v2 --n-runs 10
# writes to results/baseline_v2/
```
## 6. Compare runs
Use the existing `compare_results.py` script to diff outputs:
```bash
python compare_results.py results/Results_model_run_0.csv results/Results_model_run_1.csv
```
Also update README.md with a short "Running the simulation" section that
points to docs/quickstart.md and shows the single-line baseline command:
## Running the simulation
See [docs/quickstart.md](docs/quickstart.md) for full setup instructions.
**Baseline run (one command)**:
```bash
conda activate pv_abm
python run.py run
---
## Step-by-Step Checklist
Work through this in order. Each step can be a separate commit.
- [ ] **Step 1**: Add `click>=8.1`, `pydantic>=2.0`, and `pytest>=8.0` to
`pv_abm_env_platform_independent.yaml`; rebuild the conda environment
- [ ] **Step 2**: Create `run.py` with the `cli`, `run`, `batch`, and `init-config`
commands as shown above — verify `python run.py --help` works
- [ ] **Step 3**: Create `absice/runner.py` with `run_single`, `run_batch`,
`write_default_config`, and the private helpers
- [ ] **Step 4**: Verify `python run.py run` works end-to-end with a valid config
- [ ] **Step 5**: Verify `python run.py batch --n-runs 3 --workers 2` produces
three output CSVs with different seeds
- [ ] **Step 6**: Write CLI tests in `test/test_runner_cli.py` using
`click.testing.CliRunner` (no real model needed)
- [ ] **Step 7**: Write unit tests in `test/test_runner_unit.py` for helpers
- [ ] **Step 8**: Create `docs/quickstart.md` following the template above
- [ ] **Step 9**: Update `README.md` with the "Running the simulation" section
- [ ] **Step 10**: Ask a colleague to follow `docs/quickstart.md` from scratch and
report any step that was unclear
- [ ] **Step 11**: Open a PR and request a code review
---
## Coding Standards to Follow
Same rules as the data-interface issue — all new code must have:
1. Type annotations on every function parameter and return value
2. A docstring on every public function
3. No `typing` module imports — use `X | None`, `list[str]` etc. (Python ≥ 3.10)
Additional rules specific to CLI code:
4. **`run.py` is only glue** — it parses arguments and delegates to `absice/runner.py`.
No simulation logic lives in `run.py`.
5. **Worker functions must be module-level** — `ProcessPoolExecutor` pickles the
function it sends to workers. Lambdas and inner functions cannot be pickled.
Always define worker callables at module level.
6. **Print progress to stdout, errors to stderr**:
```python
import sys
print("Run 3 done", flush=True) # progress → stdout
print("Run 3 FAILED: ...", file=sys.stderr) # errors → stderr
Out of Scope (Do NOT Do in This PR)
- Sensitivity analysis / parameter sweeps (SALib) — that is a separate issue
- Upgrading Mesa beyond 3.2.0
- Visualisation or plotting of results
- Any changes to agent logic
Definition of Done
References
CLI, Parallel Runs, and Quick-Start Documentation
Type: Feature / Developer Experience
Priority: High
Labels:
cli,parallel,docs,good-first-project,internPrerequisite: Issue Separate Data Interface from Model Logic
must be merged before starting this issue.
Background
After the data-interface refactor the model is easy to instantiate and drive
programmatically, but a collaborator who just cloned the repo still has to:
ABM_CE_PV_MultipleRun.pydirectly to change anythingABM_CE_PV_BatchRun.pyonce handled bulk runs and sensitivity analysis, but ithard-codes the old 70-argument constructor and is no longer compatible with the
new
ABM_CE_PV(config, data)interface. It should be considered deprecated andsuperseded by the work in this issue.
This issue adds three things:
editing Python files
CPU cores from a single command
environment, and run their first simulation in under 15 minutes
Goals
python run.py run --config <yaml>executes a single simulation end-to-endpython run.py batch --config <yaml> --n-runs N --workers Wlaunches Nreplicates in parallel, each with a unique seed, writing one CSV per run to
results/python run.py init-config --output <path>writes a fully-commented defaultYAML that a user can immediately edit
python run.py --help(and--helpon every sub-command) prints clear,human-readable usage text
docs/quickstart.mddocuments installation, configuration, and a completeexample run from scratch
README.mdlinks to the quickstart and shows the one-liner to run thebaseline scenario
Proposed File Layout
Phase 1 — Set Up the CLI Entry Point
1a. Understand what
clickisclick is a Python library for building
command-line interfaces. Instead of printing
sys.argvby hand, you decorateordinary functions and
clickhandles argument parsing,--helptext, anderror messages for you automatically.
Install it with
conda env updateafter adding it topv_abm_env_platform_independent.yaml(see Step 1 below).1b. Add
clickto the environmentThen recreate your environment:
1c. Create
run.pyPhase 2 — Implement the Runner Logic
All the actual work lives in
absice/runner.py.run.pyis just the CLIwrapper — keeping them separate means the runner can be imported and called
from notebooks or tests without needing a terminal.
Why
ProcessPoolExecutor, notmesa.batch_run()What is
ProcessPoolExecutor?Python normally runs one thing at a time because of the Global Interpreter
Lock (GIL) — two threads cannot execute Python bytecode simultaneously.
For CPU-heavy work like running an ABM, threads therefore give no speedup.
ProcessPoolExecutorfrom the standard library sidesteps this by spawningseparate processes (each with its own memory and Python interpreter).
You submit tasks, the pool distributes them across processes, and you collect
the results when they finish.
One important rule: the function you submit must be defined at module level
(not inside another function or as a lambda). The pool pickles the function to
send it to worker processes, and only module-level functions can be pickled.
That is why
_worker()inrunner.pyis a top-level function.Why not
mesa.batch_run()?mesa.batch_run()(Mesa 3.x) is designed for parameter sweeps: it receives amodel class and a flat dictionary of parameters it passes as keyword arguments to
the model constructor. After the data-interface refactor,
ABM_CE_PVtakes(config, data)— not 70 flat kwargs — somesa.batch_run()no longer fits.ProcessPoolExecutoris a better fit here:across process boundaries
tqdmlaterPhase 3 — Write Tests
Phase 4 — Quick-Start Documentation
Create
docs/quickstart.md. The goal is that a new collaborator with no priorknowledge of the project can be running simulations within 15 minutes of cloning
the repo.
Also update
README.mdwith a short "Running the simulation" section thatpoints to
docs/quickstart.mdand shows the single-line baseline command:Out of Scope (Do NOT Do in This PR)
Definition of Done
flake8with no errorspython run.py --help,python run.py run --help, andpython run.py batch --helpall display clear, accurate usage textpython run.py batch --n-runs 5 --workers 2completes without errors andproduces 5 output CSVs
test/covering the runner and CLIdocs/quickstart.mdexists and has been read through by at least one otherperson (e.g. send it to a lab-mate and ask if anything is confusing)
README.mdhas a "Running the simulation" sectionrun.pyreplaces and how to migrate from theold
ABM_CE_PV_MultipleRun.pyworkflowReferences
concurrent.futures.ProcessPoolExecutor(note:
mesa.batch_runexpects flat kwargs to the model constructor — it isnot compatible with the new
ABM_CE_PV(config, data)interface; useProcessPoolExecutorinstead for simple parallel replicates)ABM_CE_PV_BatchRun.py— shows the oldapproach; it is now deprecated and superseded by
absice/runner.py