Skip to main content

Data sources

🧠5% 🤖95%

The sources this package uses cover different sets of countries, are updated on different schedules, come under different licenses, and sometimes disagree with one another. This page compares them: what each is good for, its limitations, and how to choose between them. Much of the material on boundaries and population grids is adapted from laser-init; see Related projects.

The full attribution and license text is in NOTICE.md. You can also get each dataset’s source, license and citation in Python with ssd.provenance(which), or from df.attrs['provenance'] on any dataframe the package returns; see Provenance and reproducibility.

What comes from where

Data Source Access How you get it
Births, deaths, migration, population UN WPP 2024 bundled load()
Contact matrices Prem 2017 / Prem 2021 / Mistry 2021 bundled (prem), fetched (the rest) load()
Node table: centroids, areas, populations, p-codes UNOCHA COD-AB, geoBoundaries bundled load_nodes()
Annual series, life tables, other variants UN WPP 2024 bulk files downloaded download()
Administrative boundaries with geometry UNOCHA, geoBoundaries, GADM downloaded download_shapes()
Gridded population WorldPop downloaded download_raster(), download_nodes()

Bundled data is inside the package, works offline, and is the same for everyone using the same version. Fetched data is published with this package’s releases and downloaded the first time you use it. Downloaded data comes straight from its original publisher. Fetched and downloaded files are saved on your computer, so each is downloaded only once. What goes where depends mostly on size, which is why some UN WPP data is bundled and the rest is downloaded; see Bundled, fetched and downloaded data.


Demographics

UN World Population Prospects

Provider: UN Department of Economic and Social Affairs, Population Division · population.un.org/wpp · CC BY 3.0 IGO

The standard national demographic series, covering all countries from 1950 to 2100 and revised every two years.

Coverage 187 countries (regional totals are left out)
Levels National only
Variants Medium (bundled), plus High, Low, Constant fertility and others via download()
Update Every two years: 2022, 2024, …

Strengths. The estimates use the same methods for every country, so countries can be compared, and the time series is long and consistent. The methods are well documented, and the data is revised often enough to stay current.

Limitations. The data is national only, so you can’t get province-level rates from it. Projections are scenarios rather than predictions. Estimates can change a lot between revisions, since a new census can revise a country’s past figures too, which is why you can pick a revision with load(..., revision=2024).

Two ways to get it. The bundled data, which load() reads, covers every tenth year and is small enough to ship with the package. download() reads the UN’s own files instead, for annual data, complete life tables and other projection variants; each file is a 17–290 MB download, needed once.

If you read WPP files yourself: Namibia’s two-letter code is NA, which pandas reads as a missing value unless you pass keep_default_na=False. This package handles it for you.


Contact matrices

Three sources are available. prem2021 is bundled; the other two are fetched the first time you use them. They differ mainly in age groups and country coverage. The default contact matrices, prem, are prem2021, filled in with prem2017 for the ten locations prem2021 lacks; see Datasets.

Source Access Matrix Age bins Countries License
prem2021 bundled 16×16 5-year, 0–75+ 177 GPL-3.0
prem2017 fetched 16×16 5-year, 0–75+ 152 GPL-3.0
mistry2021 fetched 85×85 single-year, 0–84+ 36 Apache-2.0
mistry2021_us fetched 85×85 single-year, 0–84+ 51 US states (with DC) Apache-2.0

Prem et al. 2017 and 2021 estimate contact patterns for countries without their own contact survey, based on household composition and age structure. The 2021 paper is an update that covers more countries and compares the results with survey data; use it unless you are reproducing earlier work. Both provide the same five settings: all, home, work, school and community.

Mistry et al. 2021 has single-year age groups up to 84, but for only 36 countries. Use it when five-year age groups are too coarse, for example for school-entry effects, and your country is covered. The same study’s matrices for the US states are mistry2021_us; each reflects that state’s own households, schools and workplaces rather than the national average.

Not every country is in every source, so check with list_locations('prem2021') (or the source you want). The matrices were adapted from epydemix-data, which also has subnational matrices for other countries and US counties that aren’t included here; see Related projects.

US Census Bureau

Provider: US Census Bureau, Population Estimates Program · census.gov/popest · public domain

population_us is the Vintage 2023 estimate of each state’s population by single year of age, as compiled by epydemix-data. It covers the 50 states and the District of Columbia (not the territories), for 2023 only, both sexes together, with the last age group 84+ to match mistry2021_us. Population totals for other years, or by sex, are not included.


Administrative boundaries

The three boundary sources complement each other, which is why all three are available and why download_shapes() tries more than one.

UNOCHA COD-AB

Provider: UN Office for the Coordination of Humanitarian Affairs · downloaded from laser-base’s copy · CC BY 3.0 IGO

Coverage ~110 countries, weighted towards humanitarian operations
Levels 0–4, varying by country
Download size 0.2–5 MB per country
Key P-code

Strengths. The p-codes are the reason this source is tried first. A p-code is the standard code the UN uses for each subnational area, and most humanitarian and health data uses it too, including surveillance line lists, campaign coverage and datasets on HDX, so you can join that data to these boundaries by code. The boundaries are usually the official government ones, checked through UN operations, and are updated quickly during an emergency response.

Limitations. It covers about 110 countries, mostly those with humanitarian operations; many middle- and high-income countries are missing.

geoBoundaries

Provider: William & Mary geoLab · geoboundaries.org · CC BY 4.0

Coverage Near-universal (~200 countries)
Levels 0–5, varying by country
Download size 1–50 MB per country
Key shapeID (not a p-code)

Strengths. It covers the countries UNOCHA doesn’t. It is published in numbered releases, and this package uses a fixed one (v6.0.0), so the boundaries don’t change from one run to the next. The methods are peer-reviewed and documented.

Limitations. Quality varies more by country and level than GADM’s does. It uses its own IDs rather than p-codes, so joining p-coded data needs a lookup table between the two. Recent boundary changes may not be included yet.

GADM

Provider: Global Administrative Areas · gadm.org · non-commercial use only

Coverage Universal
Levels 0–5, the deepest hierarchies of the three
Download size 1–100 MB per country (all levels in one file)
Key GID_n

Strengths. The most complete and most detailed of the three, with names in several languages and frequent updates.

The license is the catch. GADM is free for non-commercial use only, which rules it out for much applied public-health work. So this package:

  • never uses GADM automatically: you have to ask for it with download_shapes(..., source='gadm'), which prints a reminder of the terms;
  • never redistributes anything derived from it: the bundled node table is not built from GADM.

If you download GADM data, following its terms is up to you.

Choosing between them

With no source argument, download_shapes() tries UNOCHA and then geoBoundaries. That suits most disease modeling: p-codes where they exist, and coverage everywhere else. Choose a source yourself when:

  • you need level 4–5 — geoBoundaries or GADM;
  • you are reproducing a published analysis — whichever it used;
  • you need a country’s official boundaries — check all three against the national statistical office, since they do disagree;
  • you are joining to p-coded data — UNOCHA; if UNOCHA lacks the country, you’ll need a lookup table between p-codes and the other source’s IDs.

One caution: the sources don’t always agree on how many units a country has, or where the boundaries are. Using different sources for different countries is usually fine; switching sources for the same country over time usually is not. The node table records the source of each row for this reason.


Gridded population

WorldPop

Provider: WorldPop, University of Southampton · worldpop.org · CC BY 4.0

Coverage Global, with gaps for very small territories
Resolution 1 km (this package); 100 m available upstream
Years 2000–2030 across two product series
Download size 1–40 MB per country and year
Method Census counts spread over a grid using a statistical model (satellite imagery, settlements, roads and so on)

WorldPop estimates how many people live in each 1 km square. Adding those up within each administrative unit gives a population for each node, which download_nodes() does for you.

Strengths. Global, open, detailed, adjusted to UN totals, and widely validated. Because there is a series of years, a model’s population distribution can change over time rather than staying fixed.

Limitations. These are modeled estimates, not census counts. They don’t add up exactly to the UN WPP national total for the same year; for Kenya in 2020 the WorldPop total is about 1.2% lower. Neither figure is wrong, as they estimate slightly different things. If your model needs the national total to match WPP, use rescale_to_national(). Estimates are also more accurate in cities than in rural areas, and come without uncertainty ranges.

WorldPop’s server is slow, so the first download for a country can take a while; after that it is saved on your computer and reused.


Reproducibility

The original publishers revise their data over time. To keep an analysis reproducible, pin the starsim_data version, pick the WPP revision explicitly, and record the boundary source you used. Provenance and reproducibility has the details.


Not included

These aren’t included; here is where else to look:

  • Subnational demographics. WPP provides none. National statistical offices, DHS and IPUMS are the usual alternatives.
  • Age-specific migration. WPP publishes the crude national rate only.
  • Other population grids — LandScan, GHSL, GRID3, Meta HRSL. WorldPop was chosen because it is open and globally consistent; the others are better for some questions.
  • OpenStreetMap and Natural Earth boundaries. OpenStreetMap’s quality varies a lot from place to place; Natural Earth is simplified for drawing maps rather than for analysis.
  • Health-system, mobility and climate data. Out of scope for this package.