Stack
laktory.models.stacks.Stack
¤
Bases: BaseModel
The Stack defines a collection of deployable resources, the deployment configuration, some variables and the environment-specific settings.
Examples:
from laktory import models
stack = models.Stack(
name="workspace",
resources={
"databricks_pipelines": {
"pl-stock-prices": {
"name": "pl-stock-prices",
"development": "${vars.is_dev}",
"libraries": [
{"notebook": {"path": "/pipelines/dlt_brz_template.py"}},
],
}
},
"databricks_jobs": {
"job-stock-prices": {
"name": "job-stock-prices",
"job_clusters": [
{
"job_cluster_key": "main",
"new_cluster": {
"spark_version": "16.3.x-scala2.12",
"node_type_id": "Standard_DS3_v2",
},
}
],
"tasks": [
{
"task_key": "ingest",
"job_cluster_key": "main",
"notebook_task": {
"notebook_path": "/.laktory/jobs/ingest_stock_prices.py",
},
},
{
"task_key": "pipeline",
"depends_on": [{"task_key": "ingest"}],
"pipeline_task": {
"pipeline_id": "${resources.dlt-pl-stock-prices.id}",
},
},
],
}
},
},
variables={
"org": "okube",
},
environments={
"dev": {
"variables": {
"is_dev": True,
}
},
"prod": {
"variables": {
"is_dev": False,
}
},
},
)
| PARAMETER | DESCRIPTION |
|---|---|
description
|
Description of the stack
TYPE:
|
environments
|
Environment-specific overwrite of config, resources or variables arguments.
TYPE:
|
iac_backend
|
IaC backend used for deployment.
TYPE:
|
name
|
Name of the stack.
TYPE:
|
organization
|
Organization
TYPE:
|
resources
|
Dictionary of resources to be deployed. Each key should be a resource type and each value should be a dictionary of resources who's keys are the resource names and the values the resources definitions.
TYPE:
|
settings
|
Laktory settings
TYPE:
|
terraform
|
Terraform-specific settings
TYPE:
|
| METHOD | DESCRIPTION |
|---|---|
apply_settings |
Required to apply settings before instantiating resources and setting default values |
build |
Build stack artifacts before preview or deploy. |
get_env |
Complete definition the stack for a given environment. It takes into |
inject_vars |
Same as |
to_terraform |
Create a terraform stack for a given environment |
apply_settings(data)
classmethod
¤
Required to apply settings before instantiating resources and setting default values
Source code in laktory/models/stacks/stack.py
552 553 554 555 556 557 558 559 560 561 562 | |
build(env_name, inject_vars=True, vars=None)
¤
Build stack artifacts before preview or deploy.
Pipeline config JSON files are written to the location determined by
settings.build_root (when set in stack.yaml under
settings:) or the default Laktory cache directory. For Databricks
Asset Bundles users, set settings.build_root to a
project-local path (e.g. .laktory/.resources/) so that DABs can
sync the files to the workspace.
| PARAMETER | DESCRIPTION |
|---|---|
env_name
|
Name of the environment
TYPE:
|
inject_vars
|
Inject stack variables
TYPE:
|
vars
|
Additional variables that override stack and environment variables.
TYPE:
|
Source code in laktory/models/stacks/stack.py
638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 | |
get_env(env_name)
¤
Complete definition the stack for a given environment. It takes into account both the default stack values and environment-specific overwrites.
| PARAMETER | DESCRIPTION |
|---|---|
env_name
|
Name of the environment
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
|
Environment definitions. |
Source code in laktory/models/stacks/stack.py
738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 | |
inject_vars(inplace=False, vars=None, objs=None)
¤
Same as BaseModel.inject_vars(), but resolves self.settings and
pushes it onto the global laktory._settings.settings singleton
first, deterministically, before resolving the rest of the stack.
BaseModel.inject_vars()'s field loop iterates model_fields_set (a
set, unordered) and may skip re-resolution entirely via its result
cache, so settings is not guaranteed to be resolved - and pushed
onto the singleton - before other fields (e.g. a ${settings.x}
reference elsewhere in the stack) are substituted in the same pass.
Source code in laktory/models/stacks/stack.py
568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 | |
to_terraform(env_name=None, vars=None)
¤
Create a terraform stack for a given environment env.
| PARAMETER | DESCRIPTION |
|---|---|
env_name
|
Target environment. If
TYPE:
|
vars
|
Additional variables that override stack and environment variables.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
TerraformStack
|
Terraform-specific stack definition |
Source code in laktory/models/stacks/stack.py
821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 | |
laktory.models.stacks.StackResources
¤
Bases: BaseModel
Resources definition for a given stack or stack environment.
| PARAMETER | DESCRIPTION |
|---|---|
databricks_accesscontrolrulesets
|
TYPE:
|
databricks_alerts
|
TYPE:
|
databricks_apps
|
TYPE:
|
databricks_budgetpolicies
|
TYPE:
|
databricks_budgets
|
TYPE:
|
databricks_catalogs
|
TYPE:
|
databricks_clusterpolicies
|
TYPE:
|
databricks_clusters
|
TYPE:
|
databricks_connections
|
TYPE:
|
databricks_currentusers
|
TYPE:
|
databricks_dashboards
|
TYPE:
|
databricks_dataqualitymonitors
|
TYPE:
|
databricks_dbfsfiles
|
TYPE:
|
databricks_directories
|
TYPE:
|
databricks_entitlements
|
TYPE:
|
databricks_externallocations
|
TYPE:
|
databricks_grant
|
TYPE:
|
databricks_grants
|
TYPE:
|
databricks_groups
|
TYPE:
|
databricks_instancepools
|
TYPE:
|
databricks_ipaccesslists
|
TYPE:
|
databricks_jobs
|
TYPE:
|
databricks_libraries
|
TYPE:
|
databricks_metastoreassignments
|
TYPE:
|
databricks_metastoredataaccesses
|
TYPE:
|
databricks_metastores
|
TYPE:
|
databricks_mlflowexperiments
|
TYPE:
|
databricks_mlflowmodels
|
TYPE:
|
databricks_mlflowwebhooks
|
TYPE:
|
databricks_modelservings
|
TYPE:
|
databricks_networkconnectivityconfig
|
TYPE:
|
databricks_notebooks
|
TYPE:
|
databricks_notificationdestinations
|
TYPE:
|
databricks_obotokens
|
TYPE:
|
databricks_permissions
|
TYPE:
|
databricks_pipelines
|
TYPE:
|
databricks_pythonpackages
|
TYPE:
|
databricks_queries
|
TYPE:
|
databricks_recipients
|
TYPE:
|
databricks_registeredmodels
|
TYPE:
|
databricks_repos
|
TYPE:
|
databricks_schemas
|
TYPE:
|
databricks_secrets
|
TYPE:
|
databricks_secretscopes
|
TYPE:
|
databricks_serviceprincipals
|
TYPE:
|
databricks_shares
|
TYPE:
|
databricks_storagecredentials
|
TYPE:
|
databricks_tables
|
TYPE:
|
databricks_tokens
|
TYPE:
|
databricks_users
|
TYPE:
|
databricks_vectorsearchendpoints
|
TYPE:
|
databricks_vectorsearchindexes
|
TYPE:
|
databricks_volumes
|
TYPE:
|
databricks_warehouses
|
TYPE:
|
databricks_workspacebindings
|
TYPE:
|
databricks_workspacefiles
|
TYPE:
|
databricks_workspacetrees
|
TYPE:
|
pipelines
|
TYPE:
|
providers
|
TYPE:
|
| METHOD | DESCRIPTION |
|---|---|
route_providers_by_key |
Use the provider key name (Terraform convention) as the discriminator. |
route_providers_by_key(v)
classmethod
¤
Use the provider key name (Terraform convention) as the discriminator.
AWSProvider and DatabricksProvider share fields like profile and token,
so Pydantic's union matching is ambiguous. The key name is the explicit
source of truth: databricks[...] → DatabricksProvider, aws[...] →
AWSProvider, azure[rm][...] → AzureProvider.
Source code in laktory/models/stacks/stack.py
368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 | |
laktory.models.stacks.stack.LaktorySettings
¤
Bases: BaseModel
Laktory Settings
| PARAMETER | DESCRIPTION |
|---|---|
build_root
|
Local directory where pipeline config JSON and resource files are written during build. Defaults to the Laktory cache directory. Use when deployment is delegated to third parties like Databricks Declarative Bundles.
TYPE:
|
dataframe_api
|
TYPE:
|
dataframe_backend
|
DataFrame backend
TYPE:
|
runtime_root
|
Laktory cache root directory. Used when a pipeline needs to write checkpoint files. The default assumes a DBFS FUSE mount, which is unavailable on newer/serverless Databricks workspaces - on those, set this explicitly to a Unity Catalog Volume path instead, e.g.
TYPE:
|
workspace_root
|
Root directory of a Databricks Workspace (excluding
TYPE:
|
laktory.models.stacks.stack.EnvironmentSettings
¤
Bases: BaseModel
Settings overwrite for a specific environments
| PARAMETER | DESCRIPTION |
|---|---|
resources
|
Dictionary of resources to be deployed. Each key should be a resource type and each value should be a dictionary of resources who's keys are the resource names and the values the resources definitions.
TYPE:
|
terraform
|
Terraform-specific settings
TYPE:
|
laktory.models.stacks.stack.Terraform
¤
Bases: BaseModel
| PARAMETER | DESCRIPTION |
|---|---|
backend
|
Terraform backend configuration. Accepts any standard Terraform backend block (e.g.
TYPE:
|