This function sets GDAL configuration options that can be used to control the behavior of GDAL operations. These options can be used to optimize performance, control caching, and manage HTTP requests (among other things)
Usage
gdal_config_options(
VSI_CACHE = "TRUE",
VSI_CACHE_SIZE = "268435456",
GDAL_NUM_THREADS = 1,
GDAL_DISABLE_READDIR_ON_OPEN = "EMPTY_DIR",
GDAL_GEOREF_SOURCES = NULL,
GDAL_PAM_ENABLED = NULL,
CPL_VSIL_CURL_CACHE_SIZE = "268435456",
GDAL_HTTP_MAX_RETRY = "10",
GDAL_HTTP_RETRY_DELAY = "0.5",
GDAL_HTTP_TIMEOUT = "60",
GDAL_HTTP_CONNECTTIMEOUT = "10",
GDAL_HTTP_LOW_SPEED_TIME = "30",
GDAL_HTTP_LOW_SPEED_LIMIT = "1000",
GDAL_HTTP_MULTIPLEX = NULL,
GDAL_HTTP_VERSION = NULL,
GDAL_HTTP_MERGE_CONSECUTIVE_RANGES = "YES",
GDAL_HTTP_COOKIEFILE = NULL,
GDAL_HTTP_COOKIEJAR = GDAL_HTTP_COOKIEFILE,
GDAL_MAX_DATASET_POOL_SIZE = NULL,
GDAL_INGESTED_BYTES_AT_OPEN = NULL,
CPL_VSIL_CURL_ALLOWED_EXTENSIONS = NULL,
CPL_VSIL_CURL_USE_HEAD = "YES",
CPL_VSIL_CURL_CHUNK_SIZE = NULL,
...
)
gdal_creation_options(
output_format = NULL,
COMPRESS = "DEFLATE",
PREDICTOR = "2",
NUM_THREADS = "ALL_CPUS",
BIGTIFF = "IF_NEEDED",
TILED = if (identical(output_format, "COG")) NULL else "YES",
BLOCKXSIZE = NULL,
BLOCKYSIZE = NULL,
COPY_SRC_OVERVIEWS = if (identical(output_format, "COG")) NULL else "YES",
cli_format = FALSE,
...
)
gdalwarp_options(
multi = FALSE,
warp_memory = "50%",
num_threads = if (identical(.Platform$OS.type, "windows")) NULL else 1,
unified_src_nodata = c("NO", "YES", "PARTIAL")
)
set_gdal_config(x, scope = c("gdalraster", "sys"))
set_gdal_cache_max(mem_fraction = 0.1)
gdal_raster_drivers(shortname = FALSE)
check_muparser(gdal_version = "3.11.4")Arguments
- VSI_CACHE
Should the Virtual File System (VSI) cache be used?
- VSI_CACHE_SIZE
Size of the VSI cache in bytes.
- GDAL_NUM_THREADS
Number of threads GDAL is permitted to use for processing. Defaults to 1 because vrtility parallelises across mirai daemons and additional GDAL-internal threads contend for CPU rather than helping.
- GDAL_DISABLE_READDIR_ON_OPEN
Disable directory listing on open?
- GDAL_GEOREF_SOURCES
Comma-separated list of georeferencing sources GDAL is allowed to consult on open. Defaults to NULL, leaving GDAL's full search list "PAM,INTERNAL,TABFILE,WORLDFILE,XML" in effect. Setting "INTERNAL" restricts GDAL to the source raster's own georeferencing and can suppress sidecar / aux probes.
- GDAL_PAM_ENABLED
Controls the Persistent Auxiliary Metadata (PAM) mechanism. Defaults to NULL (GDAL's default, PAM enabled). Setting "NO" disables PAM wholesale and can eliminate residual probe requests for .aux.xml sidecars.
- CPL_VSIL_CURL_CACHE_SIZE
Global /vsicurl chunk cache size in bytes, applied per process. Defaults to 256 MiB. vrtility parallelises across mirai daemons, so the effective total is this value times the number of daemons; keep it modest rather than large to avoid memory pressure across daemons.
- GDAL_HTTP_MAX_RETRY
Maximum number of retries for HTTP requests.
- GDAL_HTTP_RETRY_DELAY
Delay between retries in seconds.
- GDAL_HTTP_TIMEOUT
Total per-request timeout in seconds. Defaults to 60. Setting this prevents a single stalled /vsicurl request from hanging the pipeline indefinitely; GDAL's underlying default is 0 (no timeout).
- GDAL_HTTP_CONNECTTIMEOUT
TCP connection establishment timeout in seconds. Defaults to 10.
- GDAL_HTTP_LOW_SPEED_TIME
Abort an in-flight request if the transfer rate stays below
GDAL_HTTP_LOW_SPEED_LIMITfor this many seconds. Defaults to 30.- GDAL_HTTP_LOW_SPEED_LIMIT
Transfer rate floor in bytes/sec used by
GDAL_HTTP_LOW_SPEED_TIME. Defaults to 1000 (1 KB/s).- GDAL_HTTP_MULTIPLEX
Use HTTP/2 multiplexing? Defaults to NULL (GDAL's default, off). Multiplexing funnels many range reads over a single HTTP/2 connection; under vrtility's per-daemon model that single connection becomes a per-process bottleneck, so plain HTTP/1.1 with multiple parallel connections (the default) saturates at least as well and is more resilient.
- GDAL_HTTP_VERSION
HTTP version to negotiate. Defaults to NULL, leaving GDAL's default (HTTP/1.1), which opens multiple parallel connections per process. This matches the configuration used by fast cloud readers such as odc-stac and, in vrtility's many-daemon model, saturates the network at least as well as forcing HTTP/2 while being more resilient to throttling. Set "2" to negotiate HTTP/2 via TLS-ALPN, or "1.1" to force HTTP/1.1.
- GDAL_HTTP_MERGE_CONSECUTIVE_RANGES
Merge consecutive ranges in HTTP requests?
Path to the cookie file for HTTP requests. Defaults to NULL (no cookie file). Most sources (e.g. Planetary Computer, public buckets) need no cookies. Set a path when accessing a source that authenticates via HTTP cookies, such as NASA Earthdata / URS. Under daemon parallelism prefer a per-daemon path to avoid concurrent writes to one file.
Path to the cookie jar for HTTP requests. Defaults to
GDAL_HTTP_COOKIEFILE(so NULL unless a cookie file is set).- GDAL_MAX_DATASET_POOL_SIZE
Maximum size of the dataset pool.
- GDAL_INGESTED_BYTES_AT_OPEN
Number of bytes to read at open.
- CPL_VSIL_CURL_ALLOWED_EXTENSIONS
Allowed file extensions for HTTP requests.
- CPL_VSIL_CURL_USE_HEAD
When "YES", GDAL issues an HTTP HEAD before the first GET on each cold /vsicurl open to retrieve content length. Defaults to "YES". Setting "NO" theoretically saves one round trip per cold open but in practice the HEAD warms the connection (DNS, TLS, redirect resolution) so skipping it produced no measurable gain in our testing.
- CPL_VSIL_CURL_CHUNK_SIZE
Chunk size for /vsicurl HTTP range reads, in bytes. Defaults to NULL so GDAL's underlying default of 16 KiB applies. Larger values inflate every cold metadata read by the chunk size and can hurt COG-open latency.
- ...
Additional -co options to set
- output_format
Output format equivalent to -of on the CLI. see details
- COMPRESS
Compression method
- PREDICTOR
Prediction method
- NUM_THREADS
Number of threads
- BIGTIFF
Use BigTIFF
- TILED
Use tiling
- BLOCKXSIZE
Block size in X
- BLOCKYSIZE
Block size in Y
- COPY_SRC_OVERVIEWS
Copy source overviews
- cli_format
Logical indicating whether to return the options in a format suitable for passing to system calls (i.e. with -co prefixes)
- multi
Logical indicating whether to use multi-threading, equivalent to -multi on the CLI
- warp_memory
Memory to use for warping equivalent to -wm on the CLI
- num_threads
Number of threads to use for warping equivalent to -wo NUM_THREADS on the CLI. "ALL_CPUS" (the default) will use all available CPUs, alternatively an integer can be supplied - or NULL to use a single threaded process.
- unified_src_nodata
Unified source nodata option equivalent to -wo UNIFIED_SRC_NODATA on the CLI. Can be "NO", "YES" or "PARTIAL". Default is "NO" (as was the default for earlier versions of GDAL).
- x
A named character vector of the configuration options
- scope
A character vector of the scope to set the options in. Either "gdalraster" or "sys".
- mem_fraction
Fraction of total RAM to use for GDAL cache, default is 0.1 (10% of total RAM)
- shortname
Logical indicating whether to return the short names of the drivers (default is FALSE)
- gdal_version
Minimum GDAL version to check alongside muparser support
Value
Character vector of options
set_gdal_cache_max returns (invisibly), a
memuse object - the value set for GDAL_CACHEMAX
A character vector of GDAL raster driver shortnames if
shortname = TRUE, otherwise a data frame with the full driver
information.
TRUE if muparser is available, FALSE otherwise
Details
Where a named argument is set to NULL, the default GDAL value will be used.
These arguments are currently included as NULL because they could in theory
improve performance but, from our limited testing they either have no or
a negative impact on performance.
output_format, equivalent to -of from the gdaltranslate or gdalwarp CLIs.
If NULL, then the output format will be inferred from the file extension.
set_gdal_cache_max is a very thin wrapper around
set_cache_max that allows you to conveniently
set the GDAL_CACHEMAX option as a fraction of the total RAM on your system.
check_muparser can be used to check if the installed gdal version was built
with muparser support; muparser is required for derived bands using
vrt_derived_block or if options("vrtility.use_muparser") is
set to TRUE when using vrt_set_maskfun with muparser-based
masks.
Examples
gdal_config_options(GDAL_HTTP_USERPWD = "user:password")
#> VSI_CACHE VSI_CACHE_SIZE
#> "TRUE" "268435456"
#> GDAL_NUM_THREADS GDAL_DISABLE_READDIR_ON_OPEN
#> "1" "EMPTY_DIR"
#> CPL_VSIL_CURL_CACHE_SIZE GDAL_HTTP_MAX_RETRY
#> "268435456" "10"
#> GDAL_HTTP_RETRY_DELAY GDAL_HTTP_TIMEOUT
#> "0.5" "60"
#> GDAL_HTTP_CONNECTTIMEOUT GDAL_HTTP_LOW_SPEED_TIME
#> "10" "30"
#> GDAL_HTTP_LOW_SPEED_LIMIT GDAL_HTTP_MERGE_CONSECUTIVE_RANGES
#> "1000" "YES"
#> CPL_VSIL_CURL_USE_HEAD GDAL_HTTP_USERPWD
#> "YES" "user:password"
gdal_creation_options(COMPRESS = "JPEG", JPEG_QUALITY = "90")
#> [1] "COMPRESS=JPEG" "PREDICTOR=2" "NUM_THREADS=ALL_CPUS"
#> [4] "BIGTIFF=IF_NEEDED" "TILED=YES" "COPY_SRC_OVERVIEWS=YES"
#> [7] "JPEG_QUALITY=90"
gdalwarp_options(multi = TRUE, warp_memory = "50%", num_threads = 4)
#> [1] "-multi" "-wm" "50%"
#> [4] "-wo" "NUM_THREADS=4" "-wo"
#> [7] "UNIFIED_SRC_NODATA=NO"
set_gdal_config(gdal_config_options())
gcm <- set_gdal_cache_max(0.05)
#> ℹ GDAL_CACHEMAX set to 799.485 MiB; to change this use
#> `vrtility::set_gdal_cache_max()`.
print(gcm)
#> 799.485 MiB