DeliveryStream is the Schema for the DeliveryStreams API. Provides a AWS Kinesis Firehose Delivery Stream
Type
CRD
Group
firehose.aws.upbound.io
Version
v1beta1
apiVersion: firehose.aws.upbound.io/v1beta1
kind: DeliveryStream
DeliveryStreamSpec defines the desired state of DeliveryStream
No description provided.
Configuration options if elasticsearch is the destination. More details are given below.
The CloudWatch Logging Options for the delivery stream. More details are given below
The data processing configuration. More details are given below.
Array of data processors. More details are given below
Array of processor parameters. More details are given below
Reference to a Role in iam to populate roleArn.
Policies for referencing.
Selector for a Role in iam to populate roleArn.
Policies for selection.
The VPC configuration for the delivery stream to connect to Elastic Search associated with the VPC. More details are given below
Reference to a Role in iam to populate roleArn.
Policies for referencing.
Selector for a Role in iam to populate roleArn.
Policies for selection.
A list of security group IDs to associate with Kinesis Firehose.
A list of subnet IDs to associate with Kinesis Firehose.
Enhanced configuration options for the s3 destination. More details are given below.
Reference to a Bucket in s3 to populate bucketArn.
Policies for referencing.
Selector for a Bucket in s3 to populate bucketArn.
Policies for selection.
The CloudWatch Logging Options for the delivery stream. More details are given below
Nested argument for the serializer, deserializer, and schema for converting data from the JSON format to the Parquet or ORC format before writing it to Amazon S3. More details given below.
Nested argument that specifies the deserializer that you want Kinesis Data Firehose to use to convert the format of your data from JSON. More details below.
Nested argument that specifies which deserializer to use. You can choose either the Apache Hive JSON SerDe or the OpenX JSON SerDe. More details below.
Nested argument that specifies the native Hive / HCatalog JsonSerDe. More details below.
A list of how you want Kinesis Data Firehose to parse the date and time stamps that may be present in your input data JSON. To specify these format strings, follow the pattern syntax of JodaTime's DateTimeFormat format strings. For more information, see Class DateTimeFormat. You can also use the special value millis to parse time stamps in epoch milliseconds. If you don't specify a format, Kinesis Data Firehose uses java.sql.Timestamp::valueOf by default.
Nested argument that specifies the OpenX SerDe. More details below.
Nested argument that specifies the serializer that you want Kinesis Data Firehose to use to convert the format of your data to the Parquet or ORC format. More details below.
Nested argument that specifies which serializer to use. You can choose either the ORC SerDe or the Parquet SerDe. More details below.
Nested argument that specifies converting data to the ORC format before storing it in Amazon S3. For more information, see Apache ORC. More details below.
A list of column names for which you want Kinesis Data Firehose to create bloom filters.
Nested argument that specifies converting data to the Parquet format before storing it in Amazon S3. For more information, see Apache Parquet. More details below.
Nested argument that specifies the AWS Glue Data Catalog table that contains the column information. More details below.
Reference to a Role in iam to populate roleArn.
Policies for referencing.
Selector for a Role in iam to populate roleArn.
Policies for selection.
Reference to a CatalogTable in glue to populate tableName.
Policies for referencing.
Selector for a CatalogTable in glue to populate tableName.
Policies for selection.
The configuration for dynamic partitioning. See Dynamic Partitioning Configuration below for more details.
The data processing configuration. More details are given below.
Array of data processors. More details are given below
Array of processor parameters. More details are given below
Reference to a Role in iam to populate roleArn.
Policies for referencing.
Selector for a Role in iam to populate roleArn.
Policies for selection.
The configuration for backup in Amazon S3. Required if s3_backup_mode is Enabled. Supports the same fields as s3_configuration object.
The CloudWatch Logging Options for the delivery stream. More details are given below
Configuration options if http_endpoint is the destination. requires the user to also specify a s3_configuration block. More details are given below.
The CloudWatch Logging Options for the delivery stream. More details are given below
The data processing configuration. More details are given below.
Array of data processors. More details are given below
Array of processor parameters. More details are given below
The request configuration. More details are given below.
Reference to a Role in iam to populate roleArn.
Policies for referencing.
Selector for a Role in iam to populate roleArn.
Policies for selection.
Allows the ability to specify the kinesis stream that is used as the source of the firehose delivery stream.
Configuration options if redshift is the destination. Using redshift_configuration requires the user to also specify a s3_configuration block. More details are given below.
The CloudWatch Logging Options for the delivery stream. More details are given below
The data processing configuration. More details are given below.
Array of data processors. More details are given below
Array of processor parameters. More details are given below
Reference to a Role in iam to populate roleArn.
Policies for referencing.
Selector for a Role in iam to populate roleArn.
Policies for selection.
The configuration for backup in Amazon S3. Required if s3_backup_mode is Enabled. Supports the same fields as s3_configuration object.
Reference to a Bucket in s3 to populate bucketArn.
Policies for referencing.
Selector for a Bucket in s3 to populate bucketArn.
Policies for selection.
The CloudWatch Logging Options for the delivery stream. More details are given below
Reference to a Role in iam to populate roleArn.
Policies for referencing.
Selector for a Role in iam to populate roleArn.
Policies for selection.
Required for non-S3 destinations. For S3 destination, use extended_s3_configuration instead. Configuration options for the s3 destination (or the intermediate bucket if the destination is redshift). More details are given below.
Reference to a Bucket in s3 to populate bucketArn.
Policies for referencing.
Selector for a Bucket in s3 to populate bucketArn.
Policies for selection.
The CloudWatch Logging Options for the delivery stream. More details are given below
Reference to a Role in iam to populate roleArn.
Policies for referencing.
Selector for a Role in iam to populate roleArn.
Policies for selection.
Configuration options if splunk is the destination. More details are given below.
The CloudWatch Logging Options for the delivery stream. More details are given below
The data processing configuration. More details are given below.
Array of data processors. More details are given below
Array of processor parameters. More details are given below
ProviderConfigReference specifies how the provider that will be used to create, observe, update, and delete this managed resource should be configured.
Policies for referencing.
ProviderReference specifies the provider that will be used to create, observe, update, and delete this managed resource. Deprecated: Please use ProviderConfigReference, i.e. providerConfigRef
Policies for referencing.
PublishConnectionDetailsTo specifies the connection secret config which contains a name, metadata and a reference to secret store config to which any connection details for this managed resource should be written. Connection details frequently include the endpoint, username, and password required to connect to the managed resource.
WriteConnectionSecretToReference specifies the namespace and name of a Secret to which any connection details for this managed resource should be written. Connection details frequently include the endpoint, username, and password required to connect to the managed resource. This field is planned to be replaced in a future release in favor of PublishConnectionDetailsTo. Currently, both could be set independently and connection details would be published to both without affecting each other.
DeliveryStreamStatus defines the observed state of DeliveryStream.
Conditions of the resource.
example
apiVersion: firehose.aws.upbound.io/v1beta1
kind: DeliveryStream
metadata:
name: example
spec:
forProvider:
destination: extended_s3
extendedS3Configuration:
- bucketArnRef:
name: example
roleArnRef:
name: example
region: us-west-1
test-stream
apiVersion: firehose.aws.upbound.io/v1beta1
kind: DeliveryStream
metadata:
labels:
testing.upbound.io/example-name: test_stream
name: test-stream
spec:
forProvider:
destination: s3
region: us-west-1
s3Configuration:
- bucketArnSelector:
matchLabels:
testing.upbound.io/example-name: bucket
roleArnSelector:
matchLabels:
testing.upbound.io/example-name: firehose_role
tags:
LogDeliveryEnabled: placeholder
s3-stream
apiVersion: firehose.aws.upbound.io/v1beta1
kind: DeliveryStream
metadata:
annotations:
meta.upbound.io/example-id: cloudwatch/v1beta1/metricstream
labels:
testing.upbound.io/example-name: s3_stream
name: s3-stream
spec:
forProvider:
destination: s3
name: metric-stream-test-stream
region: us-west-1
s3Configuration:
- bucketArnRef:
name: bucket
roleArnRef:
name: firehose-to-s3
© 2022 Upbound, Inc.
Discover the building blocksfor your internal cloud platform.