TableDataSource

laktory.models.datasources.TableDataSource

Bases: BaseDataSource

Data source using a data warehouse data table, generally used in the context of a data pipeline. Currently only supported by Spark dataframes.

ATTRIBUTE	DESCRIPTION
catalog_name	Name of the catalog of the source table TYPE: Union[str, None]
cdc	Change data capture specifications
schema_name	Name of the schema of the source table TYPE: Union[str, None]
table_name	Name of the source table TYPE: Union[str, None]

Examples:

from laktory import models

source = models.TableDataSource(
    catalog_name="dev",
    schema_name="finance",
    table_name="brz_stock_prices",
    selects=["symbol", "open", "close"],
    filter="symbol='AAPL'",
    as_stream=True,
)
# df = source.read(spark)

Attributes

full_name property

full_name

Table full name {catalog_name}.{schema_name}.{table_name}