dicogis.listing.geodata_listing module

Look for geographic datasets.

dicogis.listing.geodata_listing.check_usable_pg_services(requested_pg_services)
Check if specified postgres services are actually referenced into the

pg_service.conf. Filters out services which are not present.

Parameters:

requested_pg_services (list[str]) – list of requested services names

Returns:

filtered list of services names

Return type:

list[str]

dicogis.listing.geodata_listing.find_geodata_files(start_folder, parallel_scan=False, max_workers=None, progress_reporter=None)

List compatible geo-files stored into a folder structure.

Parameters:

  • start_folder (Path) – folder to start.

  • parallel_scan (bool) – scan top-level subfolders in parallel worker threads (each recursing through its own branch independently) instead of a single sequential walk. Off by default: benchmarks show this is a net loss on local/fast storage (the per-file classification work is CPU-bound Python, so threads mostly fight over the GIL instead of overlapping I/O) and only pays off on high-latency storage such as network-mounted shares, with a tree deep enough to have many directories to overlap. Only enable it when scanning a known-slow/network location.

  • max_workers (int | None) – passed to ThreadPoolExecutor when parallel_scan is True; ignored otherwise. Defaults to ThreadPoolExecutor’s own default (based on CPU count).

  • progress_reporter (ProgressReporter | None) – reporter notified of folders as they’re scanned, and polled for cancellation. None (the default, and what dicogis-cli passes) disables both. Note that set_total() is never called: the number of folders can’t be known before walking the tree, so this stage reports a running count and the folder being scanned, not a percentage.

Return type:

tuple[int, list[str], list[str], list[str], list[str], list[str], list[str], list[str], list[str], list[str], list[str], list[str], list[str], list[str], list[str], list[str], list[str]]

Returns:

tuple[ int, list[str], list[str], list[str], list[str], list[str], list[str], list[str], list[str], list[str], list[str], list[str], list[str], list[str], list[str], ]: tuple with number of folders parsed and list of paths by formats

Raises:

OperationCanceled – if the progress reporter asked to stop. Nothing is returned in that case: a partial listing is indistinguishable from a complete one for callers downstream.