A lingua franca for epi modeling
Version: 4.0.0rc4 | Date: 2026-11-12 | Status: Prerelease
Introduction
This project describes the design of a lingua franca for epidemiological modeling (a.k.a. disease modeling or infectious disease dynamics). The vision for the lingua franca is to:
- Capture the best aspects of many existing epi modeling frameworks (in terms of flexibility, simplicity, and elegance), and improve upon them when possible. In other words, a model should always be easier to write with the lingua franca than without it. For example, compared to existing epi models, implementing a model in the lingua franca should require no more lines of code than the original, and be as fast or faster.
- Provide a unified representation across different modeling paradigms, including at least:
- Compartmental/ODE models
- Stochastic compartmental/SDE models
- Metapopulation models
- Agent-based models
- Allow low- to zero-effort interconversion between modeling frameworks (e.g. EpiModel to epydemix) and paradigms (e.g. compartmental to agent-based).
- Be AI-native, serving as a rigorous-yet-simple and deterministic representation of modeling concepts (as expressed via code with other modeling frameworks, or even as prose) into a single consistent code representation.
Motivation
Why do we need a lingua franca? What’s wrong with existing modeling frameworks? And why can’t LLMs just write disease models directly?
- Unlike other fields such as neuroscience or astrophysics, epi modeling frameworks haven’t captured a significant “market share”. This is most likely because of the diversity of the epi modeling landscape, and because existing frameworks tend to work well for some things and poorly or not at all at others. A lingua franca would remove these barriers. (A humbling caveat is that there have been many previous attempts to do this, without much success.)
- While AI certainly can write epi models, even quite complex ones, it does not necessarily do it quickly or rigorously. AI is most efficient and accurate as a thin wrapper for code that already provides most desired functionality; imagine an LLM trying to correctly do a dataframe merge without access to R or
pandas: it probably could, but would be slow, expensive, and would likely mangle edge cases. - A lingua franca serves as a rigorous source of truth for comparing other models to, and provides a robust platform for developing AI tools (e.g. skills, guardrails, evals, and harnesses). If a compartmental model written in R with EpiModel gives a different result than an agent-based model written in Python in Vivarium, currently it’s extremely difficult to understand why. A lingua franca allows the differences between models to be clarified and explored.
Standing on the shoulders of giants
We have been using AI tools to summarize and evaluate the existing disease modeling landscape. Because of this, the lingua franca owes its existence to the many years (and hundreds of thousands of hours) of human contributors; see Influences for where we have incorporated ideas from other models. We can be (somewhat) confident that a lingua franca based on existing models can do at least as well as them in terms of performance and brevity. While AI is not always very creative, it is the perfect tool for this task: it can impartially review all existing models, see who did it “best” (using human-specified criteria for what that means), and at minimum copy that implementation (and improve upon it where possible).
Because of this, we acknowledge our indebtedness to the broader disease modeling community, and inclusiveness is one of our core principles. (More bluntly: we do not want to be the AI jerks who steal other people’s work and pass it off as our own!) We are in the process of reaching out to authors of the disease modeling frameworks we borrowed from, but please get in touch if we haven’t written to you yet.
Relationship to Starsim
Although the lingua franca aims to reflect a “consensus” view of disease modeling approaches, we needed a concrete basis to start from. We used Starsim as the basis for the lingua franca for two reasons. First, because we’re biased (a downside of human involvement in these projects!). Second, because agent-based modeling is the most complex paradigm; all models can be expressed as agent-based models, and of course real diseases act on agents (humans and/or other animals), so in a sense agent-based models are already a disease modeling lingua franca, even though the majority of epi models are written as ODEs in practice.
The lingua franca will be released as Starsim v4 (so “lingua franca”, “Starsim”, and “v4” are all synonyms), planned for the end of 2026. You can see the current code and the draft PR #1437. The prerelease version is also available on PyPI; you can try it out with pip install starsim --pre. We are also currently working on a publication. We enthusiastically invite the disease modeling community to join us as both code contributors and coauthors.
Project structure
The lingua franca was built up in stages:
| Section | What it is | |
|---|---|---|
| 1 | Approaches | A review of the current approaches used in the epi modeling landscape, based on the disease modeling landscape |
| 2 | Best practices | A distillation of the modeling landscape into best practices for each aspect of building disease models |
| 3 | Design | The design of the language, written as an extension of Starsim |
| 4 | Models | Other frameworks’ own example models, and their translations into Starsim v4 |
| 5 | Compare | A tool for comparing example models between the original source and the lingua franca |
| 6 | Benchmark | One script that regenerates every line count, run time, and check of agreement between the originals and the lingua franca |
Development notes
To run the tests and benchmark, install this repo with pip install -e . --group starsim (or just pip install -e . to use a Starsim you already have), then:
lingua_franca test [conversion|switching|threading|all]runs the tests.lingua_franca benchmarkchecks the north star: it counts each model’s lines, times it against the original, checks that their results agree, and ends with a scorecard of what passes and fails.--stages tablesprints the scorecard again from the last run, and--outsaves it. See the benchmark.lingua_franca envs installbuilds the frameworks’ environments into_envs/from the recipes indocs/models/<framework>, andpython scripts/clean_envs.pydeletes them.
The site’s source is in docs/, and is built with quarto render docs. scripts/ holds what the pages call as they are rendered, skills/ the skills that review v4, and tests/reviews/ the targets they check and their reports.
The status page tracks how much has been implemented so far. This site explains how and why the design came about; to use Starsim v4, see its documentation.
The design record
This repo is the design record of Starsim v4, which moved here from the v4_design/ folder of the Starsim repo on 2026-10-31; older text that mentions v4_design/ means this record. What governs, and what is current:
- Constitution: the principles every choice is checked against.
- Targets: what v4 has to achieve, model by model.
- Design: the spec, kept in line with the code.
- The working plan, now docs/admin/oct31/plan.md.
- Deferred: what was set aside, and “Won’t do”, what was decided against. What is done is recorded in each plan.
Starsim’s code, tests, and user docs stay in the Starsim repo. The dated folders of older plans and reviews (e.g. v4_design/oct17/) are not kept in either repo: the ones in git are at this commit of Starsim, and links to them from deferred.md and elsewhere are left as a record.
AI disclaimer
AI tools (mostly Claude) were used to summarize information, compare and assess frameworks, extract code examples, and create translations. Assessments and design decisions are owned by the humans. Authorship for each page is listed in a little note like this:
Where the ideas come from
What the design owes to each framework reviewed in approaches, area by area. Each row adds to 100%. New is what no framework did first, and Other is frameworks without a review here. These are rough judgments, not measurements. Hover over a cell to see what was taken; influences has the overall shares and how they were estimated.
Other
|
New
|
||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Principles, defaults and errors | 20 | 20 | 1 | 3 | 4 | 3 | 1 | 3 | 6 | 39 | |||||||||||||||
| Model structure | 19 | 19 | 1 | 7 | 3 | 2 | 3 | 3 | 6 | 2 | 1 | 6 | 3 | 1 | 2 | 22 | |||||||||
| Time and units | 60 | 11 | 2 | 4 | 3 | 1 | 1 | 1 | 17 | ||||||||||||||||
| Parameters and distributions | 40 | 13 | 1 | 3 | 3 | 9 | 3 | 2 | 5 | 1 | 3 | 17 | |||||||||||||
| Stratification | 12 | 12 | 30 | 3 | 8 | 4 | 1 | 6 | 24 | ||||||||||||||||
| Strains and immunity | 17 | 12 | 3 | 3 | 38 | 27 | |||||||||||||||||||
| Geospatial and metapopulation | 8 | 16 | 21 | 3 | 3 | 7 | 17 | 8 | 17 | ||||||||||||||||
| Mixing and networks | 38 | 2 | 2 | 3 | 19 | 4 | 2 | 12 | 18 | ||||||||||||||||
| Interactions and effects | 35 | 2 | 1 | 25 | 6 | 3 | 3 | 1 | 24 | ||||||||||||||||
| Interventions | 33 | 2 | 8 | 3 | 3 | 2 | 2 | 17 | 5 | 2 | 3 | 6 | 14 | ||||||||||||
| Results and observation | 27 | 8 | 6 | 5 | 9 | 3 | 15 | 4 | 2 | 7 | 14 | ||||||||||||||
| Randomness and reproducibility | 70 | 8 | 2 | 20 | |||||||||||||||||||||
| Calibration and inference | 14 | 19 | 7 | 4 | 8 | 1 | 5 | 5 | 1 | 10 | 26 | ||||||||||||||
| Model as code and interop | 25 | 6 | 2 | 2 | 2 | 3 | 5 | 25 | 30 | ||||||||||||||||
| Deterministic compartmental (ODE) | 19 | 10 | 10 | 12 | 8 | 3 | 8 | 3 | 27 | ||||||||||||||||
| Stochastic compartmental | 16 | 16 | 16 | 11 | 3 | 3 | 12 | 23 | |||||||||||||||||
| Agent-based | 64 | 3 | 3 | 4 | 1 | 6 | 3 | 1 | 15 | ||||||||||||||||
| Solver choice and paradigm conversion | 12 | 10 | 6 | 1 | 10 | 10 | 12 | 7 | 32 | ||||||||||||||||
| Overall | 29 | 9 | 4 | 3 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | <1 | <1 | 8 | 23 |
Does it pass its own test?
Each model below was written by the authors of another framework, usually to show that framework at its best. We translate it into Starsim and hold the translation to two numbers from the original: no more lines of model code, and no more time to run it again once it is set up. Where Starsim falls short, that is a gap for us to close. The model names link to the side-by-side code, and examples has the details.
| Model | Language | Lines | Time | Results |
|---|---|---|---|---|
| atomica hiv_dyn |
Excel spreadsheets (run from Python) | Original: 217 Starsim: 65 |
Original: first 140 ms, rerun 4.1 ms Starsim: first 32 ms, rerun 0.41 ms (Rust) |
Identical, to rounding error (a largest difference of 1 in 1014), in every compartment and every level of the cascade of both sexes, in all four runs: without programs, and with the default, doubled, and zero budgets |
| atomica sir |
Excel spreadsheets (run from Python) | Original: 62 Starsim: 16 |
Original: first 110 ms, rerun 2.2 ms Starsim: first 10 ms, rerun 0.21 ms (Rust) |
Every compartment identical, to rounding error |
| camdl sir |
camdl | Original: 30 Starsim: 13 |
Original: first 10 ms, rerun 4.7 ms Starsim: first 9.9 ms, rerun 0.25 ms (Rust) |
Agree. Compartments and weekly cases within 0.02% (compare.py asks for 0.1%); Agree. Mean of 200 runs within 0.6% for I and 0.9% for weekly cases, chain-binomial in both, with whole-day steps (compare.py asks for 3% and 4%) |
| camdl stratified |
camdl | Original: 47 Starsim: 16 |
Original: first 20 ms, rerun 12 ms Starsim: first 14 ms, rerun 0.68 ms (Rust) |
Agree. The infectious in every cell within 0.4%, with and without vaccination (compare.py asks for 0.5%, since camdl writes whole numbers, which matters in the smallest cells), and the totals of S, E, and R within 0.1%, apart from the day of the vaccination (finding below); Agree in size, not quite in timing. The peak in every cell is within 2%, but with a step of a day the south’s epidemic rises about a day sooner, so the mean of 200 runs differs by up to 7% of the peak on the way up (compare.py asks for 2% and 8%). The gap is about the two chain-binomial schemes at a coarse step: on 14 October it shrank from 7% to 2% with a quarter-day step |
| emulsion quickstart |
YAML (the EMULSION modeling language) | Original: 280 Starsim: 53 |
Original: first 1.2 s, rerun 1.1 s Starsim: first 330 ms, rerun 220 ms (Numba) |
Do not agree. The mean of 50 runs ends with 55 animals, against 65, and every count of adults is low from early on (finding 1). Up to step 13 the timing is right apart from the quarantine, which starts two steps sooner in EMULSION (finding 5). compare.py asks for 13 animals, so it records this miss and fails if it grows; Do not agree. The quarantine now matches (Q within 1.3), and births come on time, but the herd is still too small: 70 animals against 76 at step 53, and 55 against 65 at step 105 (finding 1) |
| emulsion sir |
YAML (the EMULSION modeling language) | Original: 81 (compartments), 86 (individuals) Starsim: 10 |
Original: first 710 ms, rerun 710 ms Starsim: first 8.5 ms, rerun 0.2 ms (Rust) |
Agree. Mean of 200 runs: a peak of 38 infectious on day 18, against 37 on day 18; 86 recovered at the end, against 85 (compare.py asks for 1.5 and 2 animals). This needs whole-day steps, substeps=1; Agree, within chance. Mean of 200 runs: a peak of 35 on day 18, against 37 on day 19; 79 recovered at the end, against 81, with 21% of runs dying out early, against about 19% (compare.py asks for 3 and 6 animals) |
| epidemics seasonal_scenarios |
R (models written in odin) | Original: 76 Starsim: 24 |
Original: first 50 ms, rerun 9 ms Starsim: first 21 ms, rerun 1.5 ms (Rust) |
Agree to 9.2e-4 of each state’s largest value, in total and in each age group, on every day. I peaks at 2,377,511 on day 275 (original 2,378,561 on day 275). The difference is the original’s: it gives odin the seasonal factor at whole days, and odin interpolates linearly between them. With the same interpolation and steps of a quarter of a day, Starsim agrees to 1e-4 in total. compare.py allows 1e-3; Agree to 8.6e-3 of each state’s largest value. The difference is how overlapping cuts combine: epidemics adds them and Starsim multiplies them (finding 1). With the cuts added by hand, Starsim agrees to 1.6e-3, and its peak is 1,374,560. compare.py allows 1e-2; Agree to 1.4e-3 of each state’s largest value. compare.py allows 2e-3 |
| epidemics seirv |
R (models written in odin) | Original: 36 Starsim: 12 |
Original: first 20 ms, rerun 9 ms Starsim: first 590 ms, rerun 1.1 ms (JAX) |
Agree to 1.4e-7 of each state’s largest value, in total and in each age group, on every day. I peaks at 1,231,054 on day 369 in both, and 23,438,153 people are infected in both. compare.py allows 1e-6 |
| epimodel hiv |
R | Original: 405 Starsim: 56 |
Original: first 44 s, rerun 43 s Starsim: first 2.3 s, rerun 2 s (Rust) |
Agree within the noise of the runs in three scenarios; with no intervention, the epidemic is larger, as before (tolerance: 2.5 percentage points for prevalence, 2 for the care cascade and PrEP; below) |
| epimodel sis-network |
R | Original: 13 Starsim: 9 |
Original: first 5.5 s, rerun 4.7 s Starsim: first 570 ms, rerun 300 ms (Rust) |
Agree, once the order within a step is allowed for (within 5% at equilibrium): 11.7 new infections a step at equilibrium, against 12.0 (means of 100 runs each). Over the last 200 steps 240 of 500 are infected, against 227.5: EpiModel lets people recover in the step in which they are infected and counts them after that, which makes its count 5% lower for the same epidemic (240 × 0.95 = 228). With the same allowance, 118 are infected after step 50, against 113, where the runs vary most |
| epipack siarx_tracing |
Python | Original: 61 Starsim: 16 |
Original: first 870 ms, rerun 480 ms Starsim: first 540 ms, rerun 500 ms (Rust) |
Agree to 2e-6 of the population on every day to day 100; Agree to 1.2e-2 of the population on every day, with steps of 0.1 days against epipack’s continuous time. The largest difference is in S as it falls fastest, about three times the standard error of the difference of the two means; Agree to 1.2e-2 of the population on every day, as on the network |
| epipack sirs |
Python | Original: 37 Starsim: 8 |
Original: first 54 ms, rerun 52 ms Starsim: first 8.5 ms, rerun 0.25 ms (Rust) |
Agree to 6.5e-6 of the population on every day, with a daily RK4 step against epipack’s adaptive solver; Agree to 5.6e-3 of the population on every day, within the error of a mean of 1,000 runs; Agree to 2.1e-2 of the population on every day, with steps of 0.025 days against epipack’s continuous time. The largest difference is in the first days, when the epidemic doubles in about 10 hours |
| epiworld covid_flu |
R | Original: 29 Starsim: 21 |
Original: first 69 ms, rerun 67 ms Starsim: first 18 ms, rerun 0.47 ms (Rust) |
Agree while few are infected, and within 5% after (mean of 1,000 runs each): on days 1 to 5 every mean agrees within three standard errors (725 infected on day 5, against 721). Later the epidemic is larger, as expected from the original’s chance of infection (see below): it peaks on day 11 in both, at 9,296 against 8,899 (+4.5%), the flu peaks on day 9 at 162 against 155, and 26,855 have recovered on day 50, against 26,509 (+1.3%) |
| epiworld seirconn |
R | Original: 12 Starsim: 8 |
Original: first 3.4 ms, rerun 2.5 ms Starsim: first 9 ms, rerun 0.21 ms (Rust) |
Agree (within three standard errors; mean of 1,000 runs each): 398 ± 17 recovered on day 50, against 394, and the infection dies out by day 50 in 22.6% of runs, against 20.8%. Every state agrees on days 10 to 50, and the 95% quantile of the recovered on day 50 is 1,601 against 1,403, 2.1 standard errors apart |
| epydemix covid19 |
Python | Original: 161 Starsim: 43 |
Original: first 34 ms, rerun 34 ms Starsim: first 23 ms, rerun 1.2 ms (Rust) |
Agree (tolerance 3% of the peak). The mean of 200 runs: weekly deaths within 0.7% of the peak (833 against 837, in the same week), and 0.5% and 0.4% in the two reopening scenarios; deaths by age within 3% of the peak in every group from age 40; E, I, R, and D within 0.5%; 10,507 deaths by 2 August against 10,525; Agree (tolerance: each posterior mean inside the original’s 5% to 95% range, and projected peaks within 10%). Posterior means: R0 2.28 (original 2.25), Delta 18.8 days (19.0), I0 4,777 (4,770), and seasonality_min 0.79 (0.80), with similar ranges. Peak weekly deaths projected from the posterior are 891 (897), 1,366 (1,363), and 2,231 (2,217) in the three scenarios. This is a change from before: a point estimate gave much smaller second waves |
| epydemix sir_kenya |
Python | Original: 7 Starsim: 15 |
Original: first 7 ms, rerun 6.5 ms Starsim: first 11 ms, rerun 0.34 ms (Rust) |
Agree (peaks within 1%, a day apart at most, final size within 0.1 of a percentage point): the mean of 100 runs peaks 0.4% higher than the original, a day sooner, and the final size is the same (99.63%). On any one day the difference is up to 6% of the peak, since the rise is a day ahead (see the findings) |
| epymorph pei_humidity |
Python | Original: 22 Starsim: 19 |
Original: first 210 ms, rerun 130 ms Starsim: first 70 ms, rerun 2 ms (Rust) |
Agree to within 3.4% at the peak in each state, 3 days in its timing, and 0.9% in the number infected. Florida, where it starts, peaks on day 59 in both; the other states peak 1 to 3 days earlier in Starsim, for the reason given for the SIRS in Arizona |
| epymorph sirs_centroids |
Python | Original: 29 Starsim: 10 |
Original: first 410 ms, rerun 320 ms Starsim: first 16 ms, rerun 4 ms (Rust) |
Agree to within 8% at the peak in each county, and 5 days in its timing; 1.0% in the number recovered. Starsim is 1 to 5 days early in every county; see the findings |
| flepimop seir_2pop |
YAML (a flepiMoP configuration) | Original: 42, plus 10 lines of CSV Starsim: 15 |
Original: first 8 s, rerun 7.7 s Starsim: first 13 ms, rerun 0.38 ms (Rust) |
Agree exactly (tolerance 1e-6 people): every compartment in both provinces, on every day, within 3e-12 people |
| flepimop vaccine_scenarios |
YAML (a flepiMoP configuration) | Original: 141, plus 10 lines of CSV Starsim: 33 |
Original: first 21 s, rerun 21 s Starsim: first 770 ms, rerun 330 ms (Numba) |
Agree exactly, by province (tolerance 1e-9). Run with the values of Ro and of the share of cases detected that the original drew, the mean, 5%, and 95% of all 28 series in each province (5 compartments, 5 incidences, and the 4 outcomes, including the number in hospital) agree within 1e-9 of their largest value in all three scenarios; Agree in distribution (tolerance 30%, since the original has only ten runs, drawn afresh each time it is run). With 100 runs of each scenario, the mean peak of I in the large province is 1,099, 466, and 17 (original, from 10 runs: 1,103, 482, and 15), and the mean deaths are 89.0, 60.0, and 2.2 (88.8, 60.6, and 1.9) |
| LASER england_wales |
Python (Jupyter notebook) | Original: 126 Starsim: 46 |
Original: first 3.7 s, rerun 2.6 s Starsim: first 100 s, rerun 100 s (Rust) |
Agree. Over the 10 years, 107,761 infections against 106,639 (+1.1%; means of 3 and 50 runs, where compare.py now makes the last two of the three to keep the benchmark short; the three Starsim runs are 106,611, 106,913, and 109,760, against the original’s 5% to 95% range of 103,139 to 108,758); in each of seven places from Norwich to Southwold within 3%, except the smallest, Southwold, at +12%; 537,442 people at the end against 529,833 (+1.4%, finding 1). The share of weeks with no infections in each place, in the second half, is 0.631 against 0.628, and correlates at 0.985 across places. compare.py allows 5% in total, 20% in a place, 2% in people at the end, and 0.03 in the share of weeks |
| LASER hub_spoke_seir |
Python | Original: 50 Starsim: 17 |
Original: first 2.2 s, rerun 1 s Starsim: first 720 ms, rerun 610 ms (Numba) |
Agree. As agents, the means of 6 Starsim runs and 20 LASER runs: 1,529,575 recovered at the end, against 1,532,298, and within 0.5% in each place shown; peaks within 5% in size and 9 days in time. As an ODE: 1,532,507 recovered, peaks within 5% in size and 8 days in time. compare.py allows 2%, 15%, and 10 days. |
| measles metapopulation seir_districts |
Python | Original: 89 Starsim: 22 |
Original: first 820 ms, rerun 810 ms Starsim: first 440 ms, rerun 390 ms (Rust) |
Agree. Over 2010 to 2025, 1,734,028 infections against 1,740,643 and 17,325 reported cases against 17,396 (means of 50 runs each); monthly means correlate at 0.999 and 0.998; infections from 2020 in each of the 16 regions within 6% (compare.py asks for totals within 2%, each year within 20%, a correlation over 0.98, regions within 20%, and S and R at the end within 3%) |
| memilio secir |
Python (bindings to C++) | Original: 44 Starsim: 14 |
Original: first 0.41 ms, rerun 0.02 ms Starsim: first 13 ms, rerun 0.47 ms (Rust) |
Agree. Within 67 people of 83,000 at every time, which is the error of the reference itself (it is interpolated between steps of about two days); deaths within 0.5, of which 0.4 is the reference’s error (it remains with a step of 0.01 days) and 0.12 is Starsim’s daily step. compare.py allows 73 people and 0.6 deaths |
| memilio secir_ageres |
Python (bindings to C++) | Original: 45 Starsim: 22 |
Original: first 0.77 ms, rerun 0.06 ms Starsim: first 19 ms, rerun 0.67 ms (Rust) |
Agree. Within 1.5 people of 10,000 at every time (1.3 in Susceptible, 1.5 in Recovered), the same with a step of a day or a tenth of a day, which is the error of the reference itself (it is interpolated between solver steps); deaths within 0.02. compare.py allows 1.6 people and 0.02 deaths |
| odin seir_array |
R (the odin modeling language) | Original: 53 Starsim: 18 |
Original: first 710 ms, rerun 1.4 ms Starsim: first 19 ms, rerun 3.2 ms (Rust) |
Agree. S, E, I, and R within 0.23%, 0.36%, 0.25%, and 0.15% of their largest values on every day (a quarter of that with a step of half a day), and I peaks at 6.19 million on day 125, as in the original. It needs the workaround of finding 1: without it the epidemic peaks 15 days late. compare.py allows 0.5%; Agree, except in the youngest group, where E, I, and R differ by up to 15%, 5%, and 8%. The original’s delay does not let the latent age (finding 2); Starsim’s does. compare.py allows 1% in the other groups |
| odin stochastic_sir_arrays |
R (the odin modeling language) | Original: 27 Starsim: 11 |
Original: first 650 ms, rerun 0.95 ms Starsim: first 13 ms, rerun 0.91 ms (Rust) |
Agree. The mean of 20,000 replicates is within 6.1 people of 1,001 at every time, against the original’s 20,000 replicates; 44.8% die out. compare.py allows 12 people, three standard errors of the difference |
| pandemic_simulator ebola_legrand |
Python | Original: 306 Starsim: 30 |
Original: first 310 ms, rerun 5.8 ms Starsim: first 20 ms, rerun 1.1 ms (Rust) |
Agree to 2.9e-6 of the population on every day, with a daily RK4 step against an adaptive solver; Agree to 1.9e-4 of the population, or 3.1e-7 with the original in float64 (JAX_ENABLE_X64=1), where it leaves 179,407. See the findings for the difference |
| pandemic_simulator sir_reactive |
Python | Original: 99 Starsim: 8 |
Original: first 230 ms, rerun 4.6 ms Starsim: first 10 ms, rerun 0.38 ms (Rust) |
Agree to 1.2e-5 of the population on every day (6 people in I), with a daily RK4 step against an adaptive solver; Agree to 1.7e-3 of the population. The difference is when the threshold is noticed: Starsim reads it at each stage of its daily step, and the original within its adaptive step; Consistent: inside the original’s 95% band on every day. The original’s draws have no seed, so two of its runs differ by this much |
| siminf sir_herds |
R | Original: 12 Starsim: 21 |
Original: first 170 ms, rerun 75 ms Starsim: first 270 ms, rerun 140 ms (Rust) |
Agree. The mean of 200 runs (compare.py now makes 100, to keep the benchmark short) is within three standard errors (at most 2.7) of the original’s mean of 200 at every day checked, for the totals, the number of herds infected, and the first herd, including the total susceptible on day 100, where little else varies (199,901 ± 3 against 199,906). The infection dies out early in 4% of runs (7% in the original), so it takes about 200 runs to compare the means: 20 runs can easily have none of these runs, and then differ a lot |
| siminf sise3 |
R (and C, for the built-in model) | Original: 27 Starsim: 39 |
Original: first 280 ms, rerun 89 ms Starsim: first 760 ms, rerun 510 ms (Rust) |
Agree, within a herd, between herds, and in the demography. With 100 runs (compare.py now makes 50, to keep the benchmark short, and allows the same three standard errors), every series is within two standard errors of the difference from SimInf’s mean on every day checked (three is the tolerance), and 81% to 97% of the runs are inside SimInf’s 5% to 95% range for every series on every day. On day 1,457 the mean of 100 runs has 181 ± 13 herds infected (SimInf: 166, range 24 to 352) and 4,175 ± 309 animals (3,770; 519 to 8,138). The spread between herds runs a little high on every day (175 herds in 200 runs), within the noise, so a small difference there cannot be ruled out |
| summer2 seir-age-clinical |
Python | Original: 134 (55 for the model alone) Starsim: 16 |
Original: first 2.8 s, rerun 0.29 ms Starsim: first 14 ms, rerun 0.53 ms (Rust) |
Every compartment within 0.0006 people of 1,000; every derived output within 0.006% of its peak. From the same start, the stratified model and the same model written as four states (as_states() in compare.py) agree to 2e-14 people; At time 20 the mean of 100 stochastic runs has 542 ± 9 recovered and of 100 agent-based runs 547 ± 10, against 580 in the ODE, since the epidemic is still under way and runs take off at different times. No run died out. By time 100 they end with 848 recovered, as does the ODE (847) |
| summer2 sir-age-location |
Python | Original: 35 Starsim: 12 |
Original: first 780 ms, rerun 0.22 ms Starsim: first 12 ms, rerun 0.43 ms (Rust) |
Every compartment within 0.08 people of 1,000; The stochastic and agent-based runs give the same number recovered as the ODE, to within 1% (mean of 100 runs) |