# fetching

[fetching](../api/fetching.md#starsim_data.fetching)

Fetch the data this package publishes but does not bundle.

Some stores are built exactly like the bundled ones but are too large, or needed by too few users, to go in the wheel: the full `prem2017` and `mistry2021` contact matrices, and the US states' (`mistry2021_us`, `population_us`). They are published as files with this package's releases, listed with their locations and provenance in `data/fetched.json`. [fetch()](../api/fetching.md#starsim_data.fetching.fetch) downloads one the first time it is used, checks it against the SHA-256 recorded there, caches it, and reads it as [load()](../api/loaders.md#starsim_data.loaders.load) does - so a given version of the package fixes their contents exactly, as it does for bundled data.

Anything else that is not bundled comes from the organization that publishes it, through [download()](../api/downloading.md#starsim_data.downloading.download). [fetch()](../api/fetching.md#starsim_data.fetching.fetch) passes such a request on to [download()](../api/downloading.md#starsim_data.downloading.download) with a warning, so that reaching a server this package does not control is never a surprise.

``` python
import starsim_data as ssd
ssd.fetch('japan', which='prem2017', kind='home')                      # a contact source not bundled
ssd.fetch('United States:Massachusetts', which='population_us')        # a US state's population
```

## Functions

| Name | Description |
|----|----|
| [fetch](#starsim_data.fetching.fetch) | Fetch a store this package publishes but does not bundle, caching it for next time. |
| [list_fetchable](#starsim_data.fetching.list_fetchable) | Return the datasets [fetch()](../api/fetching.md#starsim_data.fetching.fetch) can retrieve: the stores published with this package. |

### fetch

``` python
fetching.fetch(
    location,
    which,
    kind='all',
    refresh=False,
    verbose=None,
    **kwargs,
)
```

Fetch a store this package publishes but does not bundle, caching it for next time.

Unlike [load()](../api/loaders.md#starsim_data.loaders.load), this touches the network on a cache miss - deliberately, and under a different name, so that [load()](../api/loaders.md#starsim_data.loaders.load)'s "never needs the internet" guarantee stays intact. What it downloads is a file published with this package's releases and checked against its recorded SHA-256, so it is the same for everyone on a version.

Data that is not packaged at all, such as annual UN WPP series, is passed on to [download()](../api/downloading.md#starsim_data.downloading.download), with a warning that it comes from an external source.

#### Parameters

| Name | Type | Description | Default |
|----|----|----|----|
| location | str | location name, ISO-2/ISO-3 code, official name or alias, exactly as for [load()](../api/loaders.md#starsim_data.loaders.load). A US state is `'United States:Massachusetts'`; see [resolve()](../api/locations.md#starsim_data.locations.resolve). | *required* |
| which | str | a contact source that is not bundled (`'prem2017'`, `'mistry2021'`, and `'mistry2021_us'` for the US states), or `'population_us'` (the population of each US state by single year of age). See [list_fetchable()](../api/fetching.md#starsim_data.fetching.list_fetchable). | *required* |
| kind | str | for a contact source only, the contact setting, as for [load()](../api/loaders.md#starsim_data.loaders.load). | `'all'` |
| refresh | bool | re-download even if the file is already cached. | `False` |
| verbose | bool | report progress (default: `options.verbose`). | `None` |
| kwargs | dict | for data that is not packaged, passed to [download()](../api/downloading.md#starsim_data.downloading.download) (`years`, `variant`, `revision`). |  |

#### Returns

| Name | Type | Description |
|----|----|----|
|  |  | sciris.dataframe: exactly the schema [load()](../api/loaders.md#starsim_data.loaders.load) returns for the same layout; |
|  |  | `population_us` has that of `population`, for the one year 2023 and `Sex` |
|  |  | `'Both sexes'`. Provenance is in `.attrs['provenance']`. |

#### Raises

| Type | Description |
|----|----|
| ValueError | if `which` is not fetchable, or an argument does not apply to it. |
| KeyError | if the location is unknown, or is absent from this dataset. |

#### Example

``` python
import starsim_data as ssd
M = ssd.fetch('Japan', which='prem2017', kind='home')
```

### list_fetchable

``` python
fetching.list_fetchable()
```

Return the datasets [fetch()](../api/fetching.md#starsim_data.fetching.fetch) can retrieve: the stores published with this package.

#### Example

``` python
import starsim_data as ssd
'mistry2021' in ssd.list_fetchable() # Returns True
```
