CustomPipeline is the Schema for the CustomPipelines API. Provides a Datadog Logs Pipeline API https://docs.datadoghq.com/api/v1/logs-pipelines/ resource, which is used to create and manage Datadog logs custom pipelines. Each datadog_logs_custom_pipeline resource defines a complete pipeline. The order of the pipelines is maintained in a different resource: datadog_logs_pipeline_order. When creating a new pipeline, you need to explicitly add this pipeline to the datadog_logs_pipeline_order resource to track the pipeline. Similarly, when a pipeline needs to be destroyed, remove its references from the datadog_logs_pipeline_order resource.
Type
CRD
Group
logs.datadog.upbound.io
Version
v1alpha1
apiVersion: logs.datadog.upbound.io/v1alpha1
kind: CustomPipeline
CustomPipelineSpec defines the desired state of CustomPipeline
No description provided.
(Block List) (see below for nested schema)
(Block List, Max: 1) Arithmetic Processor. More information can be found in the official docs (see below for nested schema) Arithmetic Processor. More information can be found in the official docs
(Block List, Max: 1) Attribute Remapper Processor. More information can be found in the official docs (see below for nested schema) Attribute Remapper Processor. More information can be found in the official docs
(List of String) List of source attributes or tags. List of source attributes or tags.
(Block List, Max: 1) Category Processor. More information can be found in the official docs (see below for nested schema) Category Processor. More information can be found in the official docs
(Block List, Min: 1) List of filters to match or exclude a log with their corresponding name to assign a custom value to the log. (see below for nested schema) List of filters to match or exclude a log with their corresponding name to assign a custom value to the log.
(Block List, Max: 1) Date Remapper Processor. More information can be found in the official docs (see below for nested schema) Date Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) Date GeoIP Processor. More information can be found in the official docs (see below for nested schema) Date GeoIP Processor. More information can be found in the official docs
(Block List, Max: 1) Grok Processor. More information can be found in the official docs (see below for nested schema) Grok Processor. More information can be found in the official docs
(Block List, Min: 1, Max: 1) (see below for nested schema)
(List of String) List of sample logs for this parser. It can save up to 5 samples. Each sample takes up to 5000 characters. List of sample logs for this parser. It can save up to 5 samples. Each sample takes up to 5000 characters.
(Block List, Max: 1) Lookup Processor. More information can be found in the official docs (see below for nested schema) Lookup Processor. More information can be found in the official docs
(List of String) List of entries of the lookup table using key,value format. List of entries of the lookup table using key,value format.
(Block List, Max: 1) Message Remapper Processor. More information can be found in the official docs (see below for nested schema) Message Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) (see below for nested schema)
(Block List) (see below for nested schema)
(Block List, Max: 1) Arithmetic Processor. More information can be found in the official docs (see below for nested schema) Arithmetic Processor. More information can be found in the official docs
(Block List, Max: 1) Attribute Remapper Processor. More information can be found in the official docs (see below for nested schema) Attribute Remapper Processor. More information can be found in the official docs
(List of String) List of source attributes or tags. List of source attributes or tags.
(Block List, Max: 1) Category Processor. More information can be found in the official docs (see below for nested schema) Category Processor. More information can be found in the official docs
(Block List, Min: 1) List of filters to match or exclude a log with their corresponding name to assign a custom value to the log. (see below for nested schema) List of filters to match or exclude a log with their corresponding name to assign a custom value to the log.
(Block List, Max: 1) Date Remapper Processor. More information can be found in the official docs (see below for nested schema) Date Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) Date GeoIP Processor. More information can be found in the official docs (see below for nested schema) Date GeoIP Processor. More information can be found in the official docs
(Block List, Max: 1) Grok Processor. More information can be found in the official docs (see below for nested schema) Grok Processor. More information can be found in the official docs
(Block List, Min: 1, Max: 1) (see below for nested schema)
(List of String) List of sample logs for this parser. It can save up to 5 samples. Each sample takes up to 5000 characters. List of sample logs for this parser. It can save up to 5 samples. Each sample takes up to 5000 characters.
(Block List, Max: 1) Lookup Processor. More information can be found in the official docs (see below for nested schema) Lookup Processor. More information can be found in the official docs
(List of String) List of entries of the lookup table using key,value format. List of entries of the lookup table using key,value format.
(Block List, Max: 1) Message Remapper Processor. More information can be found in the official docs (see below for nested schema) Message Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) Reference Table Lookup Processor. Reference Tables are in public beta. More information can be found in the official docs (see below for nested schema) Reference Table Lookup Processor. Reference Tables are in public beta. More information can be found in the official docs
(Block List, Max: 1) Service Remapper Processor. More information can be found in the official docs (see below for nested schema) Service Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) Status Remapper Processor. More information can be found in the official docs (see below for nested schema) Status Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) String Builder Processor. More information can be found in the official docs (see below for nested schema) String Builder Processor. More information can be found in the official docs
(Block List, Max: 1) Trace ID Remapper Processor. More information can be found in the official docs (see below for nested schema) Trace ID Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) URL Parser Processor. More information can be found in the official docs (see below for nested schema) URL Parser Processor. More information can be found in the official docs
(List of String) List of source attributes or tags. List of source attributes.
Agent Parser Processor. More information can be found in the official docs (see below for nested schema) User-Agent Parser Processor. More information can be found in the official docs
(Block List, Max: 1) Reference Table Lookup Processor. Reference Tables are in public beta. More information can be found in the official docs (see below for nested schema) Reference Table Lookup Processor. Reference Tables are in public beta. More information can be found in the official docs
(Block List, Max: 1) Service Remapper Processor. More information can be found in the official docs (see below for nested schema) Service Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) Status Remapper Processor. More information can be found in the official docs (see below for nested schema) Status Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) String Builder Processor. More information can be found in the official docs (see below for nested schema) String Builder Processor. More information can be found in the official docs
(Block List, Max: 1) Trace ID Remapper Processor. More information can be found in the official docs (see below for nested schema) Trace ID Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) URL Parser Processor. More information can be found in the official docs (see below for nested schema) URL Parser Processor. More information can be found in the official docs
(List of String) List of source attributes or tags. List of source attributes.
Agent Parser Processor. More information can be found in the official docs (see below for nested schema) User-Agent Parser Processor. More information can be found in the official docs
THIS IS A BETA FIELD. It will be honored unless the Management Policies feature flag is disabled. InitProvider holds the same fields as ForProvider, with the exception of Identifier and other resource reference fields. The fields that are in InitProvider are merged into ForProvider when the resource is created. The same fields are also added to the terraform ignore_changes hook, to avoid updating them after creation. This is useful for fields that are required on creation, but we do not desire to update them after creation, for example because of an external controller is managing them, like an autoscaler.
(Block List) (see below for nested schema)
(Block List, Max: 1) Arithmetic Processor. More information can be found in the official docs (see below for nested schema) Arithmetic Processor. More information can be found in the official docs
(Block List, Max: 1) Attribute Remapper Processor. More information can be found in the official docs (see below for nested schema) Attribute Remapper Processor. More information can be found in the official docs
(List of String) List of source attributes or tags. List of source attributes or tags.
(Block List, Max: 1) Category Processor. More information can be found in the official docs (see below for nested schema) Category Processor. More information can be found in the official docs
(Block List, Min: 1) List of filters to match or exclude a log with their corresponding name to assign a custom value to the log. (see below for nested schema) List of filters to match or exclude a log with their corresponding name to assign a custom value to the log.
(Block List, Max: 1) Date Remapper Processor. More information can be found in the official docs (see below for nested schema) Date Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) Date GeoIP Processor. More information can be found in the official docs (see below for nested schema) Date GeoIP Processor. More information can be found in the official docs
(Block List, Max: 1) Grok Processor. More information can be found in the official docs (see below for nested schema) Grok Processor. More information can be found in the official docs
(Block List, Min: 1, Max: 1) (see below for nested schema)
(List of String) List of sample logs for this parser. It can save up to 5 samples. Each sample takes up to 5000 characters. List of sample logs for this parser. It can save up to 5 samples. Each sample takes up to 5000 characters.
(Block List, Max: 1) Lookup Processor. More information can be found in the official docs (see below for nested schema) Lookup Processor. More information can be found in the official docs
(List of String) List of entries of the lookup table using key,value format. List of entries of the lookup table using key,value format.
(Block List, Max: 1) Message Remapper Processor. More information can be found in the official docs (see below for nested schema) Message Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) (see below for nested schema)
(Block List) (see below for nested schema)
(Block List, Max: 1) Arithmetic Processor. More information can be found in the official docs (see below for nested schema) Arithmetic Processor. More information can be found in the official docs
(Block List, Max: 1) Attribute Remapper Processor. More information can be found in the official docs (see below for nested schema) Attribute Remapper Processor. More information can be found in the official docs
(List of String) List of source attributes or tags. List of source attributes or tags.
(Block List, Max: 1) Category Processor. More information can be found in the official docs (see below for nested schema) Category Processor. More information can be found in the official docs
(Block List, Min: 1) List of filters to match or exclude a log with their corresponding name to assign a custom value to the log. (see below for nested schema) List of filters to match or exclude a log with their corresponding name to assign a custom value to the log.
(Block List, Max: 1) Date Remapper Processor. More information can be found in the official docs (see below for nested schema) Date Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) Date GeoIP Processor. More information can be found in the official docs (see below for nested schema) Date GeoIP Processor. More information can be found in the official docs
(Block List, Max: 1) Grok Processor. More information can be found in the official docs (see below for nested schema) Grok Processor. More information can be found in the official docs
(Block List, Min: 1, Max: 1) (see below for nested schema)
(List of String) List of sample logs for this parser. It can save up to 5 samples. Each sample takes up to 5000 characters. List of sample logs for this parser. It can save up to 5 samples. Each sample takes up to 5000 characters.
(Block List, Max: 1) Lookup Processor. More information can be found in the official docs (see below for nested schema) Lookup Processor. More information can be found in the official docs
(List of String) List of entries of the lookup table using key,value format. List of entries of the lookup table using key,value format.
(Block List, Max: 1) Message Remapper Processor. More information can be found in the official docs (see below for nested schema) Message Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) Reference Table Lookup Processor. Reference Tables are in public beta. More information can be found in the official docs (see below for nested schema) Reference Table Lookup Processor. Reference Tables are in public beta. More information can be found in the official docs
(Block List, Max: 1) Service Remapper Processor. More information can be found in the official docs (see below for nested schema) Service Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) Status Remapper Processor. More information can be found in the official docs (see below for nested schema) Status Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) String Builder Processor. More information can be found in the official docs (see below for nested schema) String Builder Processor. More information can be found in the official docs
(Block List, Max: 1) Trace ID Remapper Processor. More information can be found in the official docs (see below for nested schema) Trace ID Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) URL Parser Processor. More information can be found in the official docs (see below for nested schema) URL Parser Processor. More information can be found in the official docs
(List of String) List of source attributes or tags. List of source attributes.
Agent Parser Processor. More information can be found in the official docs (see below for nested schema) User-Agent Parser Processor. More information can be found in the official docs
(Block List, Max: 1) Reference Table Lookup Processor. Reference Tables are in public beta. More information can be found in the official docs (see below for nested schema) Reference Table Lookup Processor. Reference Tables are in public beta. More information can be found in the official docs
(Block List, Max: 1) Service Remapper Processor. More information can be found in the official docs (see below for nested schema) Service Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) Status Remapper Processor. More information can be found in the official docs (see below for nested schema) Status Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) String Builder Processor. More information can be found in the official docs (see below for nested schema) String Builder Processor. More information can be found in the official docs
(Block List, Max: 1) Trace ID Remapper Processor. More information can be found in the official docs (see below for nested schema) Trace ID Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) URL Parser Processor. More information can be found in the official docs (see below for nested schema) URL Parser Processor. More information can be found in the official docs
(List of String) List of source attributes or tags. List of source attributes.
Agent Parser Processor. More information can be found in the official docs (see below for nested schema) User-Agent Parser Processor. More information can be found in the official docs
THIS IS A BETA FIELD. It is on by default but can be opted out through a Crossplane feature flag. ManagementPolicies specify the array of actions Crossplane is allowed to take on the managed and external resources. This field is planned to replace the DeletionPolicy field in a future release. Currently, both could be set independently and non-default values would be honored if the feature flag is enabled. If both are custom, the DeletionPolicy field will be ignored. See the design doc for more information: https://github.com/crossplane/crossplane/blob/499895a25d1a1a0ba1604944ef98ac7a1a71f197/design/design-doc-observe-only-resources.md?plain=1#L223 and this one: https://github.com/crossplane/crossplane/blob/444267e84783136daa93568b364a5f01228cacbe/design/one-pager-ignore-changes.md
ProviderConfigReference specifies how the provider that will be used to create, observe, update, and delete this managed resource should be configured.
Policies for referencing.
PublishConnectionDetailsTo specifies the connection secret config which contains a name, metadata and a reference to secret store config to which any connection details for this managed resource should be written. Connection details frequently include the endpoint, username, and password required to connect to the managed resource.
WriteConnectionSecretToReference specifies the namespace and name of a Secret to which any connection details for this managed resource should be written. Connection details frequently include the endpoint, username, and password required to connect to the managed resource. This field is planned to be replaced in a future release in favor of PublishConnectionDetailsTo. Currently, both could be set independently and connection details would be published to both without affecting each other.
CustomPipelineStatus defines the observed state of CustomPipeline.
No description provided.
(Block List) (see below for nested schema)
(Block List, Max: 1) Arithmetic Processor. More information can be found in the official docs (see below for nested schema) Arithmetic Processor. More information can be found in the official docs
(Block List, Max: 1) Attribute Remapper Processor. More information can be found in the official docs (see below for nested schema) Attribute Remapper Processor. More information can be found in the official docs
(List of String) List of source attributes or tags. List of source attributes or tags.
(Block List, Max: 1) Category Processor. More information can be found in the official docs (see below for nested schema) Category Processor. More information can be found in the official docs
(Block List, Min: 1) List of filters to match or exclude a log with their corresponding name to assign a custom value to the log. (see below for nested schema) List of filters to match or exclude a log with their corresponding name to assign a custom value to the log.
(Block List, Max: 1) Date Remapper Processor. More information can be found in the official docs (see below for nested schema) Date Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) Date GeoIP Processor. More information can be found in the official docs (see below for nested schema) Date GeoIP Processor. More information can be found in the official docs
(Block List, Max: 1) Grok Processor. More information can be found in the official docs (see below for nested schema) Grok Processor. More information can be found in the official docs
(Block List, Min: 1, Max: 1) (see below for nested schema)
(List of String) List of sample logs for this parser. It can save up to 5 samples. Each sample takes up to 5000 characters. List of sample logs for this parser. It can save up to 5 samples. Each sample takes up to 5000 characters.
(Block List, Max: 1) Lookup Processor. More information can be found in the official docs (see below for nested schema) Lookup Processor. More information can be found in the official docs
(List of String) List of entries of the lookup table using key,value format. List of entries of the lookup table using key,value format.
(Block List, Max: 1) Message Remapper Processor. More information can be found in the official docs (see below for nested schema) Message Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) (see below for nested schema)
(Block List) (see below for nested schema)
(Block List, Max: 1) Arithmetic Processor. More information can be found in the official docs (see below for nested schema) Arithmetic Processor. More information can be found in the official docs
(Block List, Max: 1) Attribute Remapper Processor. More information can be found in the official docs (see below for nested schema) Attribute Remapper Processor. More information can be found in the official docs
(List of String) List of source attributes or tags. List of source attributes or tags.
(Block List, Max: 1) Category Processor. More information can be found in the official docs (see below for nested schema) Category Processor. More information can be found in the official docs
(Block List, Min: 1) List of filters to match or exclude a log with their corresponding name to assign a custom value to the log. (see below for nested schema) List of filters to match or exclude a log with their corresponding name to assign a custom value to the log.
(Block List, Max: 1) Date Remapper Processor. More information can be found in the official docs (see below for nested schema) Date Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) Date GeoIP Processor. More information can be found in the official docs (see below for nested schema) Date GeoIP Processor. More information can be found in the official docs
(Block List, Max: 1) Grok Processor. More information can be found in the official docs (see below for nested schema) Grok Processor. More information can be found in the official docs
(Block List, Min: 1, Max: 1) (see below for nested schema)
(List of String) List of sample logs for this parser. It can save up to 5 samples. Each sample takes up to 5000 characters. List of sample logs for this parser. It can save up to 5 samples. Each sample takes up to 5000 characters.
(Block List, Max: 1) Lookup Processor. More information can be found in the official docs (see below for nested schema) Lookup Processor. More information can be found in the official docs
(List of String) List of entries of the lookup table using key,value format. List of entries of the lookup table using key,value format.
(Block List, Max: 1) Message Remapper Processor. More information can be found in the official docs (see below for nested schema) Message Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) Reference Table Lookup Processor. Reference Tables are in public beta. More information can be found in the official docs (see below for nested schema) Reference Table Lookup Processor. Reference Tables are in public beta. More information can be found in the official docs
(Block List, Max: 1) Service Remapper Processor. More information can be found in the official docs (see below for nested schema) Service Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) Status Remapper Processor. More information can be found in the official docs (see below for nested schema) Status Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) String Builder Processor. More information can be found in the official docs (see below for nested schema) String Builder Processor. More information can be found in the official docs
(Block List, Max: 1) Trace ID Remapper Processor. More information can be found in the official docs (see below for nested schema) Trace ID Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) URL Parser Processor. More information can be found in the official docs (see below for nested schema) URL Parser Processor. More information can be found in the official docs
(List of String) List of source attributes or tags. List of source attributes.
Agent Parser Processor. More information can be found in the official docs (see below for nested schema) User-Agent Parser Processor. More information can be found in the official docs
(Block List, Max: 1) Reference Table Lookup Processor. Reference Tables are in public beta. More information can be found in the official docs (see below for nested schema) Reference Table Lookup Processor. Reference Tables are in public beta. More information can be found in the official docs
(Block List, Max: 1) Service Remapper Processor. More information can be found in the official docs (see below for nested schema) Service Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) Status Remapper Processor. More information can be found in the official docs (see below for nested schema) Status Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) String Builder Processor. More information can be found in the official docs (see below for nested schema) String Builder Processor. More information can be found in the official docs
(Block List, Max: 1) Trace ID Remapper Processor. More information can be found in the official docs (see below for nested schema) Trace ID Remapper Processor. More information can be found in the official docs
(Block List, Max: 1) URL Parser Processor. More information can be found in the official docs (see below for nested schema) URL Parser Processor. More information can be found in the official docs
(List of String) List of source attributes or tags. List of source attributes.
Agent Parser Processor. More information can be found in the official docs (see below for nested schema) User-Agent Parser Processor. More information can be found in the official docs
Conditions of the resource.
sample-pipeline
apiVersion: logs.datadog.upbound.io/v1alpha1
kind: CustomPipeline
metadata:
annotations:
meta.upbound.io/example-id: logs/v1alpha1/custompipeline
labels:
testing.upbound.io/example-name: sample_pipeline
name: sample-pipeline
spec:
forProvider:
filter:
- query: source:foo
isEnabled: true
name: sample pipeline
processor:
- arithmeticProcessor:
- expression: (time1 - time2)*1000
isEnabled: true
isReplaceMissing: true
name: sample arithmetic processor
target: my_arithmetic
- attributeRemapper:
- isEnabled: true
name: sample attribute processor
overrideOnConflict: false
preserveSource: true
sourceType: tag
sources:
- db.instance
target: db
targetFormat: string
targetType: attribute
- categoryProcessor:
- category:
- filter:
- query: '@severity: "."'
name: debug
- filter:
- query: '@severity: "-"'
name: verbose
isEnabled: true
name: sample category processor
target: foo.severity
- dateRemapper:
- isEnabled: true
name: sample date remapper
sources:
- _timestamp
- published_date
- geoIpParser:
- isEnabled: true
name: sample geo ip parser
sources:
- network.client.ip
target: network.client.geoip
- grokParser:
- grok:
- matchRules: Rule %{word:my_word2} %{number:my_float2}
supportRules: ""
isEnabled: true
name: sample grok parser
samples:
- sample log 1
source: message
- lookupProcessor:
- defaultLookup: unknown service
isEnabled: true
lookupTable:
- 1,my service
name: sample lookup processor
source: service_id
target: service_name
- messageRemapper:
- isEnabled: true
name: sample message remapper
sources:
- msg
- pipeline:
- filter:
- query: source:foo
isEnabled: true
name: nested pipeline
processor:
- urlParser:
- name: sample url parser
normalizeEndingSlashes: true
sources:
- url
- extra
target: http_url
- serviceRemapper:
- isEnabled: true
name: sample service remapper
sources:
- service
- statusRemapper:
- isEnabled: true
name: sample status remapper
sources:
- info
- trace
- stringBuilderProcessor:
- isEnabled: true
isReplaceMissing: false
name: sample string builder processor
target: user_activity
template: "%{user.name} logged in at %{timestamp}"
- traceIdRemapper:
- isEnabled: true
name: sample trace id remapper
sources:
- dd.trace_id
- userAgentParser:
- isEnabled: true
isEncoded: false
name: sample user agent parser
sources:
- user
- agent
target: http_agent