Remote Data
Locations
| Form | Example |
|---|---|
| Local path | data/dem.tif |
| HTTP(S) | https://example.com/dem.tif |
| Amazon S3 | s3://bucket/key.tif |
| Google Cloud Storage | gs://bucket/key.tif (public buckets) |
| Azure Blob Storage | az://container/blob.tif (public, or with a SAS token) |
| Zip archive | roads.zip (its main dataset), roads.zip!dir/roads.shp |
| GDAL prefixes | /vsicurl/URL, /vsis3/bucket/key, /vsigs/..., /vsiaz/..., /vsizip/archive.zip/member |
Prefixes nest as in GDAL: /vsizip//vsicurl/https://example.com/data.zip/roads.shp.
Reading in place
GeoTIFFs and COGs are opened lazily: terrio info fetches the header only, and window or overview
reads fetch only the tiles they touch, merging adjacent ranges into one request. Other formats are
downloaded to a temporary directory first; a remote shapefile's .shx, .dbf, .prj and .cpg
come with it.
terrio info https://prd-tnm.s3.amazonaws.com/StagedProducts/Elevation/1/TIFF/current/n40w106/USGS_1_n40w106.tif
terrio convert https://.../B04.tif window.tif --srcwin 4096 4096 512 512
terrio convert https://.../B04.tif preview.tif --overview 2
From Python, terrio.open_raster(url, lazy=True).read(1, window=(x, y, w, h), overview=0).
Amazon S3
Requests are signed (AWS Signature Version 4) when credentials are found, and unsigned otherwise:
AWS_ACCESS_KEY_ID,AWS_SECRET_ACCESS_KEY,AWS_SESSION_TOKEN;- or a profile in
~/.aws/credentialsand~/.aws/config(AWS_PROFILE,AWS_SHARED_CREDENTIALS_FILE,AWS_CONFIG_FILE); AWS_NO_SIGN_REQUEST=YESforces unsigned requests for public buckets;AWS_REGIONsets the region; requests are retried in the bucket's region when S3 reports another;AWS_S3_ENDPOINT,AWS_HTTPSandAWS_VIRTUAL_HOSTINGtarget S3-compatible services.
Azure
AZURE_STORAGE_ACCOUNT names the account; AZURE_STORAGE_SAS_TOKEN authorizes private
containers.
HTTP settings
The GDAL variable is read when the terrio one is not set.
| Variable | GDAL equivalent | Default |
|---|---|---|
TERRIO_HTTP_TIMEOUT (seconds, per request) |
GDAL_HTTP_TIMEOUT |
none |
TERRIO_HTTP_CONNECTTIMEOUT (seconds) |
GDAL_HTTP_CONNECTTIMEOUT |
30 |
TERRIO_HTTP_MAX_RETRY |
GDAL_HTTP_MAX_RETRY |
3 |
TERRIO_HTTP_RETRY_DELAY (seconds, doubled each retry) |
GDAL_HTTP_RETRY_DELAY |
0.5 |
TERRIO_HTTP_BLOCK_SIZE (bytes) |
CPL_VSIL_CURL_CHUNK_SIZE |
65536 |
TERRIO_HTTP_CACHE_SIZE (bytes) |
CPL_VSIL_CURL_CACHE_SIZE |
67108864 |
TERRIO_HTTP_USERAGENT |
GDAL_HTTP_USERAGENT |
terrio/VERSION |
TERRIO_HTTP_LOG=YES (log each request) |
CPL_CURL_VERBOSE |
off |
Transient errors (connection failures, HTTP 429, 500, 502, 503, 504) are retried with backoff. Signatures and SAS tokens are hidden in logs and error messages.