Databricks DLT

laktory.models.pipeline.orchestrators.databricksdltorchestrator.DatabricksDLTOrchestrator

Bases: DLTPipeline, PipelineChild

Databricks DLT used as an orchestrator to execute a Laktory pipeline.

DLT orchestrator does not support pipeline nodes with views (as opposed to materialized tables). Also, it does not support writing to multiple schemas within the same pipeline.

Selecting this orchestrator requires to add the supporting notebook to the stack.

ATTRIBUTE	DESCRIPTION
config_file	Pipeline configuration (json) file deployed to the workspace and used by the job to read and execute the pipeline. TYPE: PipelineConfigWorkspaceFile
requirements_file	Pipeline requirements (json) file deployed to the workspace and used by the job to install the required python dependencies. TYPE: PipelineRequirementsWorkspaceFile

ATTRIBUTE	DESCRIPTION
additional_core_resources	configuration workspace file TYPE: list[PulumiResource]

Attributes

additional_core_resources property

additional_core_resources

• configuration workspace file
• configuration workspace file permissions
• requirements workspace file
• requirements workspace file permissions