PipelinePipeline is the Schema for the Pipelines API.
TypeMR
Providerprovider-databricks
Groupcompute.databricks.m.upbound.io
Scopenamespaced
Versionv1beta1

1apiVersion: compute.databricks.m.upbound.io/v1beta1

2kind: Pipeline

kind
string
metadata
object
spec
object

PipelineSpec defines the desired state of Pipeline

forProvider
required
object

(No description available)

catalog
string
cause
string
channel
string
cluster
array

Clusters to run the pipeline. If none is specified, pipelines will automatically select a default cluster configuration for the pipeline. Please note that DLT pipeline clusters are supporting only subset of attributes as described in Also, note that autoscale block is extended with the mode parameter that controls the autoscaling algorithm (possible values are ENHANCED for new, enhanced autoscaling algorithm, or LEGACY for old algorithm).

autoscale
object

(No description available)

mode
string
object

(No description available)

object

(No description available)

object

(No description available)

dbfs
object

(No description available)

s3
object

(No description available)

object

(No description available)

array

(No description available)

abfss
object

(No description available)

dbfs
object

(No description available)

file
object

(No description available)

gcs
object

(No description available)

s3
object

(No description available)

volumes
object

(No description available)

workspace
object

(No description available)

label
string
policyId
string
sparkConf
object
array

(No description available)

clusterId
string
continuous
boolean
object

Deployment type of this pipeline. Supports following attributes:

edition
string
filters
object

Filters on which Pipeline packages to include in the deployed graph. This block consists of following attributes:

exclude
array

Paths to exclude.

include
array

Paths to include.

object

The definition of a gateway pipeline to support CDC. Consists of following attributes:

health
string
object

(No description available)

objects
array

Required. Settings specifying tables to replicate and the destination for the replicated tables.

report
object

(No description available)

sourceUrl
string
object

Configuration settings to control the ingestion of tables. These settings are applied to all tables in the pipeline.

array

(No description available)

scdType
string
array

(No description available)

schema
object

The default schema (database) where tables are read from or published to. The presence of this attribute implies that the pipeline is in direct publishing mode.

object

Configuration settings to control the ingestion of tables. These settings are applied to all tables in the pipeline.

array

(No description available)

scdType
string
array

(No description available)

table
object

(No description available)

object

Configuration settings to control the ingestion of tables. These settings are applied to all tables in the pipeline.

array

(No description available)

scdType
string
array

(No description available)

object

Configuration settings to control the ingestion of tables. These settings are applied to all tables in the pipeline.

array

(No description available)

scdType
string
array

(No description available)

array

(No description available)

state
string
updateId
string
library
array

Specifies pipeline code and required artifacts. Syntax resembles library configuration block with the addition of a special notebook & file library types that should have the path attribute. Right now only the

file
object

(No description available)

path
string
jar
string
maven
object

(No description available)

array

(No description available)

repo
string
notebook
object

& file types are supported.

path
string
pathRef
object

Reference to a Notebook in workspace to populate path.

name
required
string
namespace
string
policy
object

Policies for referencing.

resolve
string
object

Selector for a Notebook in workspace to populate path.

namespace
string
policy
object

Policies for selection.

resolve
string
whl
string
name
string
array

(No description available)

alerts
array

empty list of alert types. Right now following alert types are supported, consult documentation for actual list

array

empty list of emails to notify.

photon
boolean
object

(No description available)

array

(No description available)

startHour
number
schema
string
serverless
boolean
state
string
storage
string
target
string
trigger
object

(No description available)

cron
object

(No description available)

manual
object
url
string
object

THIS IS A BETA FIELD. It will be honored unless the Management Policies feature flag is disabled. InitProvider holds the same fields as ForProvider, with the exception of Identifier and other resource reference fields. The fields that are in InitProvider are merged into ForProvider when the resource is created. The same fields are also added to the terraform ignore_changes hook, to avoid updating them after creation. This is useful for fields that are required on creation, but we do not desire to update them after creation, for example because of an external controller is managing them, like an autoscaler.

catalog
string
cause
string
channel
string
cluster
array

Clusters to run the pipeline. If none is specified, pipelines will automatically select a default cluster configuration for the pipeline. Please note that DLT pipeline clusters are supporting only subset of attributes as described in Also, note that autoscale block is extended with the mode parameter that controls the autoscaling algorithm (possible values are ENHANCED for new, enhanced autoscaling algorithm, or LEGACY for old algorithm).

autoscale
object

(No description available)

mode
string
object

(No description available)

object

(No description available)

object

(No description available)

dbfs
object

(No description available)

s3
object

(No description available)

object

(No description available)

array

(No description available)

abfss
object

(No description available)

dbfs
object

(No description available)

file
object

(No description available)

gcs
object

(No description available)

s3
object

(No description available)

volumes
object

(No description available)

workspace
object

(No description available)

label
string
policyId
string
sparkConf
object
array

(No description available)

clusterId
string
continuous
boolean
object

Deployment type of this pipeline. Supports following attributes:

edition
string
filters
object

Filters on which Pipeline packages to include in the deployed graph. This block consists of following attributes:

exclude
array

Paths to exclude.

include
array

Paths to include.

object

The definition of a gateway pipeline to support CDC. Consists of following attributes:

health
string
object

(No description available)

objects
array

Required. Settings specifying tables to replicate and the destination for the replicated tables.

report
object

(No description available)

sourceUrl
string
object

Configuration settings to control the ingestion of tables. These settings are applied to all tables in the pipeline.

array

(No description available)

scdType
string
array

(No description available)

schema
object

The default schema (database) where tables are read from or published to. The presence of this attribute implies that the pipeline is in direct publishing mode.

object

Configuration settings to control the ingestion of tables. These settings are applied to all tables in the pipeline.

array

(No description available)

scdType
string
array

(No description available)

table
object

(No description available)

object

Configuration settings to control the ingestion of tables. These settings are applied to all tables in the pipeline.

array

(No description available)

scdType
string
array

(No description available)

object

Configuration settings to control the ingestion of tables. These settings are applied to all tables in the pipeline.

array

(No description available)

scdType
string
array

(No description available)

array

(No description available)

state
string
updateId
string
library
array

Specifies pipeline code and required artifacts. Syntax resembles library configuration block with the addition of a special notebook & file library types that should have the path attribute. Right now only the

file
object

(No description available)

path
string
jar
string
maven
object

(No description available)

array

(No description available)

repo
string
notebook
object

& file types are supported.

path
string
pathRef
object

Reference to a Notebook in workspace to populate path.

name
required
string
namespace
string
policy
object

Policies for referencing.

resolve
string
object

Selector for a Notebook in workspace to populate path.

namespace
string
policy
object

Policies for selection.

resolve
string
whl
string
name
string
array

(No description available)

alerts
array

empty list of alert types. Right now following alert types are supported, consult documentation for actual list

array

empty list of emails to notify.

photon
boolean
object

(No description available)

array

(No description available)

startHour
number
schema
string
serverless
boolean
state
string
storage
string
target
string
trigger
object

(No description available)

cron
object

(No description available)

manual
object
url
string
array

THIS IS A BETA FIELD. It is on by default but can be opted out through a Crossplane feature flag. ManagementPolicies specify the array of actions Crossplane is allowed to take on the managed and external resources. See the design doc for more information: https://github.com/crossplane/crossplane/blob/499895a25d1a1a0ba1604944ef98ac7a1a71f197/design/design-doc-observe-only-resources.md?plain=1#L223 and this one: https://github.com/crossplane/crossplane/blob/444267e84783136daa93568b364a5f01228cacbe/design/one-pager-ignore-changes.md

object

ProviderConfigReference specifies how the provider that will be used to create, observe, update, and delete this managed resource should be configured.

kind
required
string
name
required
string
object

WriteConnectionSecretToReference specifies the namespace and name of a Secret to which any connection details for this managed resource should be written. Connection details frequently include the endpoint, username, and password required to connect to the managed resource.

name
required
string
status
object

PipelineStatus defines the observed state of Pipeline.

object

(No description available)

catalog
string
cause
string
channel
string
cluster
array

Clusters to run the pipeline. If none is specified, pipelines will automatically select a default cluster configuration for the pipeline. Please note that DLT pipeline clusters are supporting only subset of attributes as described in Also, note that autoscale block is extended with the mode parameter that controls the autoscaling algorithm (possible values are ENHANCED for new, enhanced autoscaling algorithm, or LEGACY for old algorithm).

autoscale
object

(No description available)

mode
string
object

(No description available)

object

(No description available)

object

(No description available)

dbfs
object

(No description available)

s3
object

(No description available)

object

(No description available)

array

(No description available)

abfss
object

(No description available)

dbfs
object

(No description available)

file
object

(No description available)

gcs
object

(No description available)

s3
object

(No description available)

volumes
object

(No description available)

workspace
object

(No description available)

label
string
policyId
string
sparkConf
object
array

(No description available)

clusterId
string
continuous
boolean
object

Deployment type of this pipeline. Supports following attributes:

edition
string
filters
object

Filters on which Pipeline packages to include in the deployed graph. This block consists of following attributes:

exclude
array

Paths to exclude.

include
array

Paths to include.

object

The definition of a gateway pipeline to support CDC. Consists of following attributes:

health
string
id
string
object

(No description available)

objects
array

Required. Settings specifying tables to replicate and the destination for the replicated tables.

report
object

(No description available)

sourceUrl
string
object

Configuration settings to control the ingestion of tables. These settings are applied to all tables in the pipeline.

array

(No description available)

scdType
string
array

(No description available)

schema
object

The default schema (database) where tables are read from or published to. The presence of this attribute implies that the pipeline is in direct publishing mode.

object

Configuration settings to control the ingestion of tables. These settings are applied to all tables in the pipeline.

array

(No description available)

scdType
string
array

(No description available)

table
object

(No description available)

object

Configuration settings to control the ingestion of tables. These settings are applied to all tables in the pipeline.

array

(No description available)

scdType
string
array

(No description available)

object

Configuration settings to control the ingestion of tables. These settings are applied to all tables in the pipeline.

array

(No description available)

scdType
string
array

(No description available)

array

(No description available)

state
string
updateId
string
library
array

Specifies pipeline code and required artifacts. Syntax resembles library configuration block with the addition of a special notebook & file library types that should have the path attribute. Right now only the

file
object

(No description available)

path
string
jar
string
maven
object

(No description available)

array

(No description available)

repo
string
notebook
object

& file types are supported.

path
string
whl
string
name
string
array

(No description available)

alerts
array

empty list of alert types. Right now following alert types are supported, consult documentation for actual list

array

empty list of emails to notify.

photon
boolean
object

(No description available)

array

(No description available)

startHour
number
schema
string
serverless
boolean
state
string
storage
string
target
string
trigger
object

(No description available)

cron
object

(No description available)

manual
object
url
string
array

Conditions of the resource.

lastTransitionTime
required
string
message
string
reason
required
string
status
required
string
type
required
string