1.2.5. tds2stac.tds2stac module#

class tds2stac.tds2stac.ExistenceValidator(stac_dir: str, logger_properties: dict | None = {})[source]#

Bases: object

A class for verifying the main STAC catalog’s existence. This class is implemented in STACCreator.

Parameters:

  • stac_dir (str) – Directory of the main STAC catalog (*)

  • logger_properties (dict, optional) – A dictionary of properties for logger. default is None.

logger_properties: dict | None#

A dictionary of properties for logger. default is None. You can look at keys in Logger class.

stac_dir: str#

Directory of the main STAC catalog. It can be a relative or absolute path.

class tds2stac.tds2stac.TDS2STACIntegrator(TDS_catalog: str, stac_dir: str, stac_id: str = 'TDS2STAC', stac_title: str | None = 'TDS2STAC', stac_description: str | None = None, auth: Tuple[str, str] | None = None, stac_existance: bool = False, stac_existance_collection: bool = False, collection_tuples: list[tuple] | None = None, datetime_filter: list | None = None, aggregated_dataset_url: str | None = None, depth_number: int | None = None, limited_number: int | None = None, spatial_information: list | None = None, temporal_format_by_dataname: str | None = None, item_geometry_linestring: bool = False, webservice_properties: dict | None = {}, asset_properties: dict | None = {}, extension_properties: dict | None = {}, logger_properties: dict = {})[source]#

Bases: object

This class is the central component of the TDS2STAC. It harvests the TDS catalog and then generates the STAC-Catalog, -Collections, and -Items through the TDS catalogs, based on the user’s input. This class mainly defines all configurations related to harvesting and STAC creation. In the first step, it recognizes the scenario of the TDS catalog using Recognizer. If it is recognized as a nested collection, NestedCollectionInspector is responsible for determining the nested collection’s ID, Title, and url of subdirectories. Other procedures follow in succession. For example, CollectionHarvester harvests the collection’s information and STACCreator creates the STAC-Catalog and -Collection. Then, ItemHarvester harvests the item’s information and STACCreator creates the STAC-Item and connect them to the related STAC-Collections. At the end each STAC-Collection will be connected to the main STAC-Catalog.

Parameters:

  • TDS_catalog (str) – The URL address of the TDS catalog that will be harvested.

  • stac_dir (str) – Directory of saving created STAC catalogs.

  • stac_id (str) – STAC catalog ID. default value is ‘TDS2STAC’.

  • stac_title (str, optional) – STAC catalog Title. default value is ‘TDS2STAC’.

  • stac_description (str, optional) – STAC catalog description.

  • auth (tuple, optional) – Authentication tuple for TDS catalog e.g.(‘user’, ‘password’).

  • stac_existance (bool, optional) – Verifying the presence of the STAC catalog in order to update an existing catalog; if not, a new catalog will be generated.

  • stac_existance_collection (bool, optional) – Verifying the presence of the STAC Collection in order to update an existing catalog; if not, a new collection will be generated.

  • collection_tuples (list, optional) – The elements of this tuple comprise the auto-TDS2STAC-generated ID , the user-defined ID, title, and description of the STAC-Collection respectively. (auto-ID, user-ID, user-title, user-description).

  • datetime_filter (list, optional) – Datetime-based filtering of harvesting. It works based on the modified tag in each dataset at TDS.

  • aggregated_dataset_url (str, optional) – Dataset’s URL of each data entry in the Aggregated datasets of TDS.

  • depth_number (int, optional) – depth number of nested datasets if it is a nested collection. default value is 0.

  • limited_number (int, optional) – The objective is to reduce the quantity of harvested items in each collection. It is beneficial for developing and testing purposes.

  • spatial_information (list, optional) – Spatial information of 2D datasets e.g. [minx, maxx, miny, maxy] or 1D dataset e.g. [x,y]. default value is None.

  • temporal_format_by_dataname (str, optional) – A preferred datetime format for datasets that include the time period in their names. e.g “e%y%m%d%H.%M%S%f”

  • item_geometry_linestring (bool, optional) – Set True to make a LineString geometry for STAC-Items from wms service. Otherwise it makes Polygon geometry for the given Item. default value is False.

  • extension_properties (dict, optional) – A dictionary of properties for extensions. default is None. For more information about the keys, please refer to the extension_properties.

  • webservice_properties (dict, optional) – A dictionary of properties for web_service. default is None (optional) For more information about the keys, please refer to the webservice_properties.

  • asset_properties (dict, optional) – A dictionary of properties for assets. default is None (optional) For more information about the keys, please refer to the asset_properties.

  • logger_properties (dict, optional) – A dictionary of properties for logger. default is None.

TDS_catalog: str#

TDS catalog URL address. Initial point of harvesting e.g. https://thredds.atmohub.kit.edu/thredds/catalog/caribic/IAGOS-CARIBIC_MS_files_collection_20231017/catalog.html

aggregated_dataset_url: str | None#

Dataset’s URL of each data entry in the Aggregated datasets of TDS.. default value None. The HTTPServer is not functional in the aggregated dataset. Therefore, in order to utilize this service as an asset in our STAC-Item, we should employ the aggregated_dataset_url, which links the individual datasets to the HTTPServer asset of the relevant Item.

asset_properties: dict | None#

A dictionary of properties for assets. default is None. When it’s None, keys look like the following example:

item_thumbnail (bool, optional):

A thumbnail asset for STAC-Items sourced from the Web Map Service (WMS) of the TDS. The default value is set to False.

item_overview (bool, optional):

A overview asset for STAC-Items sourced from the Web Map Service (WMS) of the TDS. The default value is set to False.

item_getminmax_thumbnail (bool, optional):

The TDS offers a function that allows users to obtain the minimum and maximum values of the colorbar associated with an image through the use of metadata. The aforementioned attribute is contingent upon both the item_thumbnail and item_overview. The default value is set to False.

collection_thumbnail (str, optional):

A thumbnail asset for STAC-collection sourced from the Web Map Service (WMS) of the TDS. It can be chosen from wms, link, or None. The default value is set to None.

collection_overview (str, optional):

A overview asset for STAC-collection sourced from the Web Map Service (WMS) of the TDS. It can be chosen from wms, link, or None. The default value is set to None.

collection_link (str, optional):

This property is reliant upon the values of collection_thumbnail and collection_overview. When the value of either of these attributes is set to link, it allows for the inclusion of a hyperlink to an image for collection_thumbnail or collection_overview.

item_assets_list_allowed (list, optional):

This is a list of permissible web services that will be incorporated as assets in the STAC-Item. The WebServiceScraper class provides access to the list of available web services. Default value is None.

item_assets_list_avoided (list, optional):

This is a list of web services that will be excluded from the STAC-Item asset list. The WebServiceScraper class provides access to the list of available webservices. Default value is None.

collection_assets_list_allowed (list, optional):

This is a list of permissible web services that will be incorporated as assets in the STAC-Collection. The WebServiceScraper class provides access to the list of available web services. Default value is None.

collection_assets_list_avoided (list, optional):

This is a list of web services that will be excluded from the STAC-Collection asset list. The WebServiceScraper class provides access to the list of available webservices. Default value is None.

explore_data (bool, optional):

By enabling the True setting, the inclusion of Godiva3 as an exploration asset will be implemented.

verify_explore_data (bool, optional):

This argument verifies the availability of the GetMetadata function. The provided function facilitates the retrieval of data necessary for generating maps using the Web Map Service (WMS) protocol. However, an error occurs when attempting to open Godiva3 when this function doesn’t work. In order to mitigate such errors, it would be advisable to establish this argument.

jupyter_notebook (bool, optional):

This argument posits the inclusion of the Jupyter Notebook as an asset.

auth: Tuple[str, str] | None#

Authentication tuple for TDS catalog e.g.(‘user’, ‘password’)

collection_tuples: list[tuple] | None#

STAC collection auto-generated ID, user-ID, user-Title and user-Description defined by user. It is worth mentioning that in order to obtain the list of automatically generated collection IDs, one can employ the NestedCollectionInspector for the given TDS Catalog and subsequently utilize this argument. Warning - Identifiers should consist of only lowercase characters, numbers, ‘_’, and ‘-‘. Default value None. e.g. (ID, Title, Description)

datetime_filter: list | None#

Datetime-based filtering. e.g. ['2010-02-18T00:00:00.000Z','2020-02-22T00:00:00.000Z'] Default value None. It should be noted it works based on the modified tag in each dataset at TDS.

depth_number: int | None#

The depth refers to the number of layered datasets. If the collection is nested, this argument is applicable; otherwise, employing this argument would be futile. default value None (optional)

extension_properties: dict | None#

A dictionary of properties for extensions. default is None.

item_extensions (list[str, tuple], optional):

The argument can consist of either a list of extension names (string) or a list of tuples containing three elements: the extension name, the function name or class name associated with the extension, and the Python script required for execution. For more explanation, refer to the Adding and Configuring Custom Extensions for STAC-Items and STAC-Collections.

collection_extensions (Union[list, tuple], optional):

It works as same as item_extensions argument. For more explanation, refer to the Adding and Configuring Custom Extensions for STAC-Items and STAC-Collections.

item_geometry_linestring: Literal[False]#

The default value for the LineString geometry in the STAC Items from the WMS service is set to False and the default geometry type for the STAC-Item is Polygon. However, in instances where the item has a POINT geometry, it can be automatically detected. However, in order to obtain the LineString geometry, it is necessary to set this argument to True.

limited_number: int | None#

The objective is to reduce the quantity of harvested items in each collection. It is beneficial for developing and testing purposes.. default value None (optional)

logger_properties: dict#

A dictionary of properties for logger. default is None. You can look at keys in Logger class.

spatial_information: list | None#

Spatial information of 2D datasets e.g. [minx, maxx, miny, maxy] or 1D dataset e.g. [x,y]. Default value `None`(optional)

stac_description: str | None#

STAC catalog description

stac_dir: str#

Directory of saving created STAC catalogs e.g. /path/to/stac/directory/

stac_existance: Literal[False]#

Verifying the existence of STAC catalog. If the catalog exists in the directory, it updates a existed catalog, otherwise it creates new catalog. default value False

stac_existance_collection: Literal[False]#

Verifying the existence of STAC Collections. If the collection exists in the directory, it updates a existed collection, otherwise it creates new collection. default value False

stac_id: str#

STAC catalog ID. default value TDS2STAC

stac_title: str | None#

STAC catalog Title. default value TDS2STAC

temporal_format_by_dataname: str | None#

A preferred datetime format for datasets that include the time period in their names e.g “e%y%m%d%H.%M%S%f”. Default value None (optional)

webservice_properties: dict | None#

A dictionary of properties for web_service. default is None.

It has the following keys.
web_service_config_file(str, opntional):

The primary tag_config.json file is situated in the primary directory of the installed TDS2STAC. However, the user has the ability to declare an alternative tag_config.json file, which allows for customization of the settings. The user can specify the location of their own JSON file in this section. To obtain further details on the creation of a tag_config.json file, refer: Creating the tag_config.json Configuration File: A Step-by-Step Guide. The default value is set to tag_config.json in the root directory of the installed app.