dicogis.georeaders.process_files module

class dicogis.georeaders.process_files.DatasetToProcess(file_path, file_format, georeader, processed=False, process_error=None, exported=False, export_error=None)

Bases: object

Model of a geofile to process.

__init__(file_path, file_format, georeader, processed=False, process_error=None, exported=False, export_error=None)

export_error: str | None = None

exported: bool = False

file_format: str

property file_name: str

Name of the dataset file, without its folders, to keep progress messages short.

file_path is typed as a Path but the listing actually provides plain strings (os.path.join), so both are accepted.

Returns:

file name, extension included

file_path: Path

georeader: object

process_error: str | None = None

processed: bool = False

class dicogis.georeaders.process_files.ProcessingFiles(serializer, localized_strings, li_cdao, li_dxf, li_flat_geodatabase_esri_filegdb, li_flat_geodatabase_geopackage, li_flat_geodatabase_spatialite, li_geojson, li_gxt, li_gml, li_kml, li_mapinfo_tab, li_shapefiles, li_vectors, li_rasters, li_file_databases, opt_analyze_esri_filegdb=True, opt_analyze_geojson=True, opt_analyze_geopackage=True, opt_analyze_gml=True, opt_analyze_gxt=True, opt_analyze_kml=True, opt_analyze_mapinfo_tab=True, opt_analyze_raster=True, opt_analyze_cdao=True, opt_analyze_shapefiles=True, opt_analyze_spatialite=True, progress_reporter=None, opt_quick_fail=False)

Bases: object

Geofiles processor.

MATRIX_DATASET_TYPE_METADATASET: dict[str, type[MetaDataset]] = {'flat_cad': <class 'dicogis.models.metadataset.MetaVectorDataset'>, 'flat_database': <class 'dicogis.models.metadataset.MetaDatabaseFlat'>, 'flat_raster': <class 'dicogis.models.metadataset.MetaRasterDataset'>, 'flat_vector': <class 'dicogis.models.metadataset.MetaVectorDataset'>}

MATRIX_FORMAT_GEOREADER = {'dxf': <class 'dicogis.georeaders.read_dxf.ReadCadDxf'>, 'esri_shapefile': <class 'dicogis.georeaders.read_vector_flat_dataset.ReadVectorFlatDataset'>, 'file_geodatabase_esri': <class 'dicogis.georeaders.read_vector_flat_geodatabase.ReadFlatDatabase'>, 'file_geodatabase_geopackage': <class 'dicogis.georeaders.read_vector_flat_geodatabase.ReadFlatDatabase'>, 'file_geodatabase_spatialite': <class 'dicogis.georeaders.read_vector_flat_geodatabase.ReadFlatDatabase'>, 'geojson': <class 'dicogis.georeaders.read_vector_flat_dataset.ReadVectorFlatDataset'>, 'geotiff': <class 'dicogis.georeaders.read_raster.ReadRasters'>, 'gml': <class 'dicogis.georeaders.read_vector_flat_dataset.ReadVectorFlatDataset'>, 'gxt': <class 'dicogis.georeaders.read_vector_flat_dataset.ReadVectorFlatDataset'>, 'kml': <class 'dicogis.georeaders.read_vector_flat_dataset.ReadVectorFlatDataset'>, 'mapinfo_tab': <class 'dicogis.georeaders.read_vector_flat_dataset.ReadVectorFlatDataset'>, 'raster': <class 'dicogis.georeaders.read_raster.ReadRasters'>}

__init__(serializer, localized_strings, li_cdao, li_dxf, li_flat_geodatabase_esri_filegdb, li_flat_geodatabase_geopackage, li_flat_geodatabase_spatialite, li_geojson, li_gxt, li_gml, li_kml, li_mapinfo_tab, li_shapefiles, li_vectors, li_rasters, li_file_databases, opt_analyze_esri_filegdb=True, opt_analyze_geojson=True, opt_analyze_geopackage=True, opt_analyze_gml=True, opt_analyze_gxt=True, opt_analyze_kml=True, opt_analyze_mapinfo_tab=True, opt_analyze_raster=True, opt_analyze_cdao=True, opt_analyze_shapefiles=True, opt_analyze_spatialite=True, progress_reporter=None, opt_quick_fail=False)

add_files_to_process_queue(list_of_datasets, dataset_format)

Add dataset to the processing queue.

Parameters:

• list_of_datasets (list) – list of datasets where to register the dataets

• dataset_format (str) – _description_

Return type:

list[DatasetToProcess]

Returns:

list of files to process

count_files_to_process()

Count number of files to process.

Return type:

int

Returns:

total of files to process

export_metadataset(dataset_to_process, metadataset_to_serialize)

Serialize metadataset in a generic way.

Parameters:

• dataset_to_process (DatasetToProcess) – dataset to process used to store status

• metadataset_to_serialize (MetaDataset) – metadataset to serialize

Return type:

tuple[DatasetToProcess, MetaDataset | None]

Returns:

dataset to process, metadataset or None if something went wrong

failed_metadataset(dataset_to_process, dataset_type, error)

Describe a dataset whose reading raised, so that it can be serialized as an error (e.g. a highlighted row in its Excel sheet).

Parameters:

• dataset_to_process (DatasetToProcess) – dataset which failed to be read

• dataset_type (str | None) – dataset_type of the georeader used, picking the model

• error (Exception) – exception raised while reading

Return type:

MetaDataset

Returns:

metadataset with name, path and error filled, processing_succeeded False

li_files_to_process: list[DatasetToProcess | None]

process_datasets_in_queue()

Process datasets in queue.

Raises:

OperationCanceled – if the progress reporter asked to stop. Raised only after post_serializing(), so whatever was processed before the cancellation is still written to the output file.

read_dataset(dataset_to_process)

Read dataset and store into metadataset.

Parameters:

dataset_to_process (DatasetToProcess) – dataset path or URI to read

Return type:

tuple[DatasetToProcess, MetaDataset | None]

Returns:

dataset and metadataset, flagged as failed (processing_succeeded is False) if reading raised

total_files: int | None

update_progress(message_to_display=None, increment_counter=False)

Helper method to update progress bar/status/counter.

Parameters:

• message_to_display (str | None) – message to display. Defaults to None.

• increment_counter (bool) – option to increment progress counter. Defaults to False.