cache
Download cache for data that is too big to bundle.
Tier-0 data ships inside the wheel and load() never touches the network. Anything larger – the stores behind fetch(), and the UN WPP bulk files and spatial sources behind download() and the other download_* functions – is downloaded once into a local cache and reused from then on.
Every cached file gets a provenance sidecar (<file>.provenance.json) recording the URL it came from, when it was downloaded, its size and its SHA-256. That is laser-init’s idea, and it is the right one for cached files: unlike bundled data, where provenance is written into the HDF5 itself, a cache entry is an ordinary file on disk and a sidecar is the only place the lineage can live.
import starsim_data as ssd
ssd.cache_dir() # where things are cached
ssd.cache_info() # what is cached, from where, and when it was downloaded
ssd.clear_cache() # delete it allFunctions
| Name | Description |
|---|---|
| cache_dir | Return the directory where downloaded data is cached. |
| cache_info | Return a dataframe of what is currently cached: file, size, source URL and date. |
| cache_provenance | Return the recorded provenance for a cached file, or None if there is none. |
| cached_download | Download url into the cache, or return the existing path if already cached. |
| clear_cache | Delete cached downloads. Bundled data is never touched. |
cache_dir
cache.cache_dir(subdir=None, create=False)Return the directory where downloaded data is cached.
Resolved at call time from ssd.options.cache_dir, falling back to a platform default (~/.cache/starsim_data on Linux, ~/Library/Caches/starsim_data on macOS, %LOCALAPPDATA%\starsim_data\Cache on Windows). Nothing is created unless create=True, so merely asking where the cache is has no side effects.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| subdir | str | optional subdirectory within the cache, e.g. 'wpp'. |
None |
| create | bool | whether to create the directory if it does not exist. | False |
Returns
| Name | Type | Description |
|---|---|---|
| pathlib.Path: the cache directory. |
Example
import starsim_data as ssd
ssd.cache_dir().name # Returns 'starsim_data'cache_info
cache.cache_info()Return a dataframe of what is currently cached: file, size, source URL and date.
Returns
| Name | Type | Description |
|---|---|---|
sciris.dataframe: with columns [file, size, bytes, url, downloaded], empty if |
||
| nothing is cached. |
Example
import starsim_data as ssd
ssd.cache_info()cache_provenance
cache.cache_provenance(path)Return the recorded provenance for a cached file, or None if there is none.
cached_download
cache.cached_download(
url,
filename=None,
subdir=None,
refresh=False,
verbose=None,
)Download url into the cache, or return the existing path if already cached.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| url | str | the URL to fetch. | required |
| filename | str | name to save as (default: the last path component of the URL). | None |
| subdir | str | subdirectory of the cache to put it in, e.g. 'wpp'. |
None |
| refresh | bool | re-download even if a cached copy exists. | False |
| verbose | bool | report progress (default: ssd.options.verbose). |
None |
Returns
| Name | Type | Description |
|---|---|---|
| pathlib.Path: the path to the cached file. |
clear_cache
cache.clear_cache(pattern=None, verbose=None)Delete cached downloads. Bundled data is never touched.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| pattern | str | a glob matched against each cached file’s path relative to the cache root, e.g. 'wpp/*'. If None, the whole cache is removed. |
None |
| verbose | bool | report what was deleted (default: ssd.options.verbose). |
None |
Returns
| Name | Type | Description |
|---|---|---|
| list | the paths that were deleted. |
Example
import starsim_data as ssd
ssd.clear_cache('wpp/*')