Databricks Job

laktory.models.pipeline.orchestrators.databricksjoborchestrator.DatabricksJobOrchestrator ¤

Databricks job used as an orchestrator to execute a Laktory pipeline.

Job orchestrator supports incremental workloads with Spark Structured Streaming, but it does not support continuous processing.

Selecting this orchestrator requires to add the supporting notebook to the stack.

ATTRIBUTE	DESCRIPTION
`notebook_path`	Path for the notebook. If `None`, default path for laktory job notebooks is used. TYPE: `Union[str, None]`
`config_file`	Pipeline configuration (json) file deployed to the workspace and used by the job to read and execute the pipeline. TYPE: `PipelineConfigWorkspaceFile`
`requirements_file`	Pipeline requirements (json) file deployed to the workspace and used by the job to install the required python dependencies. TYPE: `PipelineRequirementsWorkspaceFile`

ATTRIBUTE	DESCRIPTION
`additional_core_resources`	configuration workspace file TYPE: `list[PulumiResource]`

additional_core_resources