Data analytics · Public project
Monster Lab
A weekend analytics project that grew into a reproducible data pipeline, empirical difficulty study, converter, and interactive exploration tool.
The question
What actually makes a monster difficult?
The first question was descriptive: which printed combat statistics track Shadowdark monster level? The more useful question followed: after removing rules-derived variables, which measurable traits still explain difficulty, and do those traits agree with simulated outcomes?
The project combines 243 freely licensed Shadowdark core monsters with 322 D&D 5e SRD monsters. It normalizes both systems into SQLite, parses attack text, builds a cross-system match table, fits simple models, and presents the results through a multipage Streamlit application.
Pipeline
Keep ingestion, analysis, and presentation reproducible.
Source-specific ingestion
Independent scripts rebuild each table from licensed source data, making failures attributable and the pipeline safe to rerun.
Explicit text parsing
A documented grammar converts attack clauses into structured features and reports clauses it cannot admit instead of silently guessing.
Cross-system matching
Exact and fuzzy matches create 143 comparable monster pairs, with lower-confidence candidates separated for manual review.
Shared analytical functions
The reports, command-line tools, and dashboard use the same fit and metric functions so the displayed result cannot drift from the analysis.
Model criticism
The strongest first model was also the least interesting.
An initial level model reached R² = 0.997. That looked impressive until the data-generating process was considered: Shadowdark assigns one hit die per level, so HP is largely derived from the target variable. The model confirmed the dataset followed the rules, but it did not explain difficulty.
The revised no-HP model reached R² = 0.817 using AC, attack bonus, damage, number of attacks, and best stat modifier. Its residuals were more informative. Spellcasters and monsters built around petrification, regeneration, curses, or other rider effects appeared as expected failures because those mechanics are not represented by attack math alone.
A high score is not automatically a useful model. The feature lineage determines what the score means.
Empirical validation
Compare printed difficulty with simulated outcomes.
A seeded Monte Carlo simulator runs a fixed reference party against each core monster. The simulation is intentionally bounded: it models attacks, defenses, targeting, critical hits, and individually constructed characters, while acknowledging that prose-driven abilities remain outside the model.
A derived threat score correlates with simulated win rate better than printed level on both the level-matched subset and the complete monster set. Disagreements become useful findings rather than model embarrassment: they identify monsters whose danger comes from mechanics the structured features cannot see.
- Per-monster seeded trials reproduce exact result files
- Fixed and rerolled party variance answer different questions explicitly
- Difficulty findings are reported alongside limitations
- Tests cover parser behavior, metric formulas, simulation invariants, and data gating
Licensing boundary
Public data stays public. Owned-book data stays local.
The public pipeline uses the freely licensed Shadowdark core dataset and the D&D 5e SRD. Optional parsing for personally owned books writes only to a gitignored local database and is gated from the deployed application. This separates a useful local workflow from what the project is allowed to redistribute.
Monster Lab is an independent product published under the Shadowdark RPG Third-Party License and is not affiliated with The Arcane Library, LLC. Shadowdark RPG © 2023 The Arcane Library, LLC. The project also uses SRD 5.1 material under CC BY 4.0.