DataFlow is the Schema for the DataFlows API. Manages a Data Flow inside an Azure Data Factory.
Type
CRD
Group
datafactory.azure.upbound.io
Version
v1beta1
apiVersion: datafactory.azure.upbound.io/v1beta1
kind: DataFlow
DataFlowSpec defines the desired state of DataFlow
No description provided.
List of tags that can be used for describing the Data Factory Data Flow.
Reference to a Factory in datafactory to populate dataFactoryId.
Policies for referencing.
Selector for a Factory in datafactory to populate dataFactoryId.
Policies for selection.
The script lines for the Data Factory Data Flow.
One or more sink blocks as defined below.
A dataset block as defined below.
Reference to a DataSetJSON in datafactory to populate name.
Policies for referencing.
Selector for a DataSetJSON in datafactory to populate name.
Policies for selection.
A flowlet block as defined below.
A linked_service block as defined below.
A rejected_linked_service block as defined below.
A schema_linked_service block as defined below.
One or more source blocks as defined below.
A dataset block as defined below.
Reference to a DataSetJSON in datafactory to populate name.
Policies for referencing.
Selector for a DataSetJSON in datafactory to populate name.
Policies for selection.
A flowlet block as defined below.
A linked_service block as defined below.
A rejected_linked_service block as defined below.
A schema_linked_service block as defined below.
One or more transformation blocks as defined below.
A dataset block as defined below.
A flowlet block as defined below.
A linked_service block as defined below.
ProviderConfigReference specifies how the provider that will be used to create, observe, update, and delete this managed resource should be configured.
Policies for referencing.
ProviderReference specifies the provider that will be used to create, observe, update, and delete this managed resource. Deprecated: Please use ProviderConfigReference, i.e. providerConfigRef
Policies for referencing.
PublishConnectionDetailsTo specifies the connection secret config which contains a name, metadata and a reference to secret store config to which any connection details for this managed resource should be written. Connection details frequently include the endpoint, username, and password required to connect to the managed resource.
WriteConnectionSecretToReference specifies the namespace and name of a Secret to which any connection details for this managed resource should be written. Connection details frequently include the endpoint, username, and password required to connect to the managed resource. This field is planned to be replaced in a future release in favor of PublishConnectionDetailsTo. Currently, both could be set independently and connection details would be published to both without affecting each other.
DataFlowStatus defines the observed state of DataFlow.
No description provided.
Conditions of the resource.
example
apiVersion: datafactory.azure.upbound.io/v1beta1
kind: DataFlow
metadata:
annotations:
meta.upbound.io/example-id: datafactory/v1beta1/dataflow
labels:
testing.upbound.io/example-name: example
name: example
spec:
forProvider:
dataFactoryIdSelector:
matchLabels:
testing.upbound.io/example-name: factoryexample
script: |
source(
allowSchemaDrift: true,
validateSchema: false,
limit: 100,
ignoreNoFilesFound: false,
documentForm: 'documentPerLine') ~> source1
source1 sink(
allowSchemaDrift: true,
validateSchema: false,
skipDuplicateMapInputs: true,
skipDuplicateMapOutputs: true) ~> sink1
sink:
- dataset:
- nameSelector:
matchLabels:
testing.upbound.io/example-name: example2
name: sink1
source:
- dataset:
- nameSelector:
matchLabels:
testing.upbound.io/example-name: example1
name: source1
© 2022 Upbound, Inc.
Discover the building blocksfor your internal cloud platform.