# loaders

[loaders](../api/loaders.md#starsim_data.loaders)

Load bundled data.

The data ships inside the package under `starsim_data/data/`, keyed by location. There are two families of dataset, and [load](../api/loaders.md#starsim_data.loaders.load) returns the natural shape for each:

- Contact matrices - one file per source, each a `layout='matrix'` store. [load](../api/loaders.md#starsim_data.loaders.load) returns a square `(n, n)` frame whose rows and columns are indexed by the start age of each bin (e.g. `0, 5, ..., 75`); the `kind` argument selects the contact setting (`all`, `home`, …). `prem2021.h5` is bundled; the other sources (`prem2017`, `mistry2021`) are the same kind of store, fetched on demand by [fetch()](../api/fetching.md#starsim_data.fetching.fetch). The default, `prem`, is `prem2021`, or `prem2017` for the ten locations `prem2021` lacks (which are bundled, in `prem/prem2017.h5`).
- Demographic time series from UN WPP - grouped in `demographics-<revision>.h5`, one group per series (`births`, `deaths`, `migration`, `population`, `fertility`). [load](../api/loaders.md#starsim_data.loaders.load) returns a tidy frame with WPP-native column names, so e.g. `deaths` drops straight into `ss.Deaths(death_rate=...)`.

Demographic stores are versioned by UN WPP revision. [load](../api/loaders.md#starsim_data.loaders.load) uses the newest bundled revision unless `revision=` says otherwise, so a new WPP release is an additive rebuild rather than a silent change to data already in use; see [list_revisions()](../api/loaders.md#starsim_data.loaders.list_revisions).

[load](../api/loaders.md#starsim_data.loaders.load) never touches the network - everything it reads is inside the wheel. For the contact sources that are published but not bundled see [fetch()](../api/fetching.md#starsim_data.fetching.fetch), and for data from its original publisher (annual series, life tables, other projection variants) see [download()](../api/downloading.md#starsim_data.downloading.download); both cache what they download.

Locations are resolved by name, ISO-2/ISO-3 code, official name or alias - see [starsim_data.locations](../api/locations.md#starsim_data.locations). Every [load](../api/loaders.md#starsim_data.loaders.load) result is a `sciris.dataframe`, a drop-in replacement for a pandas `DataFrame`, carrying its source provenance in `.attrs`.

## Functions

| Name | Description |
|----|----|
| [list_datasets](#starsim_data.loaders.list_datasets) | Return the available datasets: contact sources plus WPP demographic series. |
| [list_locations](#starsim_data.loaders.list_locations) | Return the sorted locations available in dataset `which`. |
| [list_revisions](#starsim_data.loaders.list_revisions) | Return the bundled UN WPP revisions, oldest first; the last is the default. |
| [load](#starsim_data.loaders.load) | Load data for a location. |
| [provenance](#starsim_data.loaders.provenance) | Return where a dataset came from: source, license, citation and build details. |

### list_datasets

``` python
loaders.list_datasets(detailed=False, tier=None)
```

Return the available datasets: contact sources plus WPP demographic series.

#### Parameters

| Name | Type | Description | Default |
|----|----|----|----|
| detailed | bool / str | if True, return a dataframe with each dataset's name, description, access (bundled, fetched or downloaded) and location count, rather than just the names; if 'full', also its layout, admin levels, revisions and license. | `False` |
| tier | int | restrict to one tier (0 bundled, 1 fetched, 2 downloaded). | `None` |

#### Returns

| Name | Type | Description                                                  |
|------|------|--------------------------------------------------------------|
|      |      | list or sciris.dataframe: dataset names, or a table of them. |

#### Example

``` python
import starsim_data as ssd
'prem2021' in ssd.list_datasets() # Returns True
ssd.list_datasets(detailed=True) # A summary table
ssd.list_datasets(detailed='full') # Every column of the registry
```

### list_locations

``` python
loaders.list_locations(which, revision=None)
```

Return the sorted locations available in dataset `which`.

A fetched dataset lists its locations from the manifest, without downloading it.

#### Example

``` python
import starsim_data as ssd
'Kenya' in ssd.list_locations('prem2021') # Returns True
```

### list_revisions

``` python
loaders.list_revisions()
```

Return the bundled UN WPP revisions, oldest first; the last is the default.

#### Example

``` python
import starsim_data as ssd
ssd.list_revisions() # Returns ['2024']
```

### load

``` python
loaders.load(location, which='prem', kind='all', revision=None, level=None)
```

Load data for a location.

#### Parameters

| Name | Type | Description | Default |
|----|----|----|----|
| location | str | location name, ISO-2/ISO-3 code, official name or alias, e.g. `'Kenya'`, `'KE'`, `'KEN'`; case-, accent- and punctuation-insensitive. | *required* |
| which | str | dataset - the contact matrices (`'prem'`, the default, which is `'prem2021'` or, for the ten locations it lacks, `'prem2017'`, and can also be asked for as `'contacts'`; or `'prem2021'` itself), or a demographic series (`'births'`, `'deaths'`, `'migration'`, `'population'`, `'fertility'`). See [list_datasets()](../api/loaders.md#starsim_data.loaders.list_datasets); all of `'prem2017'` and `'mistry2021'` are fetched with [fetch()](../api/fetching.md#starsim_data.fetching.fetch), and annual series are downloaded with [download()](../api/downloading.md#starsim_data.downloading.download). | `'prem'` |
| kind | str | for contact-matrix datasets only, the contact setting - one of `'all'`, `'home'`, `'work'`, `'school'`, `'community'`. Ignored otherwise. | `'all'` |
| revision | str / int | for demographic series only, the UN WPP revision to read (default: the newest bundled). See [list_revisions()](../api/loaders.md#starsim_data.loaders.list_revisions). | `None` |
| level | int / str | for the node table only, the administrative level to return (None returns every level). See [load_nodes()](../api/nodes.md#starsim_data.nodes.load_nodes), the readable spelling of `load(..., which='nodes', level=...)`. | `None` |

#### Returns

| Name | Type | Description |
|----|----|----|
|  |  | sciris.dataframe: a drop-in replacement for a pandas `DataFrame`, with source |
|  |  | provenance in `.attrs['provenance']`. For contact matrices, a square `(n, n)` |
|  |  | matrix whose rows and columns are indexed by the start age of each bin (e.g. |
|  |  | `0, 5, ..., 75`). For demographic series, a tidy frame - `births`/`migration` |
|  |  | have columns `[Time, CBR]`/`[Time, CNMR]`; `deaths` has |
|  |  | `[Time, Sex, AgeGrpStart, mx]`; `population` has `[Time, Sex, AgeGrp, Value]`; |
|  |  | `fertility` has `[Time, AgeGrpStart, ASFR]`. |

#### Raises

| Type       | Description                                                 |
|------------|-------------------------------------------------------------|
| ValueError | if `which`, `kind` or `revision` is not recognized.         |
| KeyError   | if the location is unknown, or is absent from this dataset. |

#### Example

``` python
import starsim_data as ssd
M = ssd.load(location='kenya') # (16, 16) contact matrix
d = ssd.load(location='KEN', which='deaths') # tidy mortality
```

### provenance

``` python
loaders.provenance(which=None, revision=None)
```

Return where a dataset came from: source, license, citation and build details.

#### Parameters

| Name | Type | Description | Default |
|----|----|----|----|
| which | str | dataset name; if None, return a dict keyed by every dataset. | `None` |
| revision | str / int | for demographic series, the UN WPP revision (default newest). | `None` |

#### Returns

| Name | Type | Description |
|----|----|----|
|  |  | sciris.objdict: with keys including [source](../api/registry.md#starsim_data.registry.Dataset.source), `source_url`, `license`, |
|  |  | `citation`, `built` and `builder`. Contents vary by source. |

#### Example

``` python
import starsim_data as ssd
ssd.provenance('prem2021').license # Returns 'GPL-3.0'
```
