Skip to main content

fetching

fetching

Fetch the data this package publishes but does not bundle.

Some stores are built exactly like the bundled ones but are too large, or needed by too few users, to go in the wheel: the full prem2017 and mistry2021 contact matrices, and the US states’ (mistry2021_us, population_us). They are published as files with this package’s releases, listed with their locations and provenance in data/fetched.json. fetch() downloads one the first time it is used, checks it against the SHA-256 recorded there, caches it, and reads it as load() does – so a given version of the package fixes their contents exactly, as it does for bundled data.

Anything else that is not bundled comes from the organization that publishes it, through download(). fetch() passes such a request on to download() with a warning, so that reaching a server this package does not control is never a surprise.

import starsim_data as ssd
ssd.fetch('japan', which='prem2017', kind='home')                      # a contact source not bundled
ssd.fetch('United States:Massachusetts', which='population_us')        # a US state's population

Functions

Name Description
fetch Fetch a store this package publishes but does not bundle, caching it for next time.
list_fetchable Return the datasets fetch() can retrieve: the stores published with this package.

fetch

fetching.fetch(
    location,
    which,
    kind='all',
    refresh=False,
    verbose=None,
    **kwargs,
)

Fetch a store this package publishes but does not bundle, caching it for next time.

Unlike load(), this touches the network on a cache miss – deliberately, and under a different name, so that load()’s “never needs the internet” guarantee stays intact. What it downloads is a file published with this package’s releases and checked against its recorded SHA-256, so it is the same for everyone on a version.

Data that is not packaged at all, such as annual UN WPP series, is passed on to download(), with a warning that it comes from an external source.

Parameters

Name Type Description Default
location str location name, ISO-2/ISO-3 code, official name or alias, exactly as for load(). A US state is 'United States:Massachusetts'; see resolve(). required
which str a contact source that is not bundled ('prem2017', 'mistry2021', and 'mistry2021_us' for the US states), or 'population_us' (the population of each US state by single year of age). See list_fetchable(). required
kind str for a contact source only, the contact setting, as for load(). 'all'
refresh bool re-download even if the file is already cached. False
verbose bool report progress (default: options.verbose). None
kwargs dict for data that is not packaged, passed to download() (years, variant, revision).

Returns

Name Type Description
sciris.dataframe: exactly the schema load() returns for the same layout;
population_us has that of population, for the one year 2023 and Sex
'Both sexes'. Provenance is in .attrs['provenance'].

Raises

Type Description
ValueError if which is not fetchable, or an argument does not apply to it.
KeyError if the location is unknown, or is absent from this dataset.

Example

import starsim_data as ssd
M = ssd.fetch('Japan', which='prem2017', kind='home')

list_fetchable

fetching.list_fetchable()

Return the datasets fetch() can retrieve: the stores published with this package.

Example

import starsim_data as ssd
'mistry2021' in ssd.list_fetchable() # Returns True