CrawlerCrawler is the Schema for the Crawlers API. Manages a Glue Crawler
TypeMR
Providerprovider-aws-glue
Groupglue.aws.m.upbound.io
Scopenamespaced
Versionv1beta1

1apiVersion: glue.aws.m.upbound.io/v1beta1

2kind: Crawler

kind
string
metadata
object
spec
object

CrawlerSpec defines the desired state of Crawler

forProvider
required
object

(No description available)

array

List of nested AWS Glue Data Catalog target arguments. See Catalog Target below.

object

Reference to a CatalogDatabase in glue to populate databaseName.

name
required
string
namespace
string
policy
object

Policies for referencing.

resolve
string
object

Selector for a CatalogDatabase in glue to populate databaseName.

namespace
string
policy
object

Policies for selection.

resolve
string
tables
array

A list of catalog tables to be synchronized.

array

References to CatalogTable in glue to populate tables.

name
required
string
namespace
string
policy
object

Policies for referencing.

resolve
string
object

Selector for a list of CatalogTable in glue to populate tables.

namespace
string
policy
object

Policies for selection.

resolve
string
array

List of custom classifiers. By default, all AWS classifiers are included in a crawl, but these custom classifiers always override the default classifiers for a given classification.

object

Reference to a CatalogDatabase in glue to populate databaseName.

name
required
string
namespace
string
policy
object

Policies for referencing.

resolve
string
object

Selector for a CatalogDatabase in glue to populate databaseName.

namespace
string
policy
object

Policies for selection.

resolve
string
array

List of nested Delta Lake target arguments. See Delta Target below.

array

A list of the Amazon S3 paths to the Delta tables.

array

List of nested DynamoDB target arguments. See Dynamodb Target below.

path
string
scanAll
boolean
scanRate
number
array

List of nested Hudi target arguments. See Iceberg Target below.

array

A list of glob patterns used to exclude from the crawl.

paths
array

One or more Amazon S3 paths that contains Hudi metadata folders as s3://bucket/prefix.

array

List of nested Iceberg target arguments. See Iceberg Target below.

array

A list of glob patterns used to exclude from the crawl.

paths
array

One or more Amazon S3 paths that contains Hudi metadata folders as s3://bucket/prefix.

array

List of nested JDBC target arguments. See JDBC Target below.

object

Reference to a Connection in glue to populate connectionName.

name
required
string
namespace
string
policy
object

Policies for referencing.

resolve
string
object

Selector for a Connection in glue to populate connectionName.

namespace
string
policy
object

Policies for selection.

resolve
string
array

Specify a value of RAWTYPES or COMMENTS to enable additional metadata intable responses. RAWTYPES provides the native-level datatype. COMMENTS provides comments associated with a column or table in the database.

array

A list of glob patterns used to exclude from the crawl.

path
string
object

Specifies Lake Formation configuration settings for the crawler. See Lake Formation Configuration below.

object

Specifies data lineage configuration settings for the crawler. See Lineage Configuration below.

array

List of nested MongoDB target arguments. See MongoDB Target below.

object

Reference to a Connection in glue to populate connectionName.

name
required
string
namespace
string
policy
object

Policies for referencing.

resolve
string
object

Selector for a Connection in glue to populate connectionName.

namespace
string
policy
object

Policies for selection.

resolve
string
path
string
scanAll
boolean
object

A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run.. See Recrawl Policy below.

region
required
string
role
string
roleRef
object

Reference to a Role in iam to populate role.

name
required
string
namespace
string
policy
object

Policies for referencing.

resolve
string
object

Selector for a Role in iam to populate role.

namespace
string
policy
object

Policies for selection.

resolve
string
array

List of nested Amazon S3 target arguments. See S3 Target below.

array

A list of glob patterns used to exclude from the crawl.

path
string
schedule
string
object

Policy for the crawler's update and deletion behavior. See Schema Change Policy below.

tags
object
object

THIS IS A BETA FIELD. It will be honored unless the Management Policies feature flag is disabled. InitProvider holds the same fields as ForProvider, with the exception of Identifier and other resource reference fields. The fields that are in InitProvider are merged into ForProvider when the resource is created. The same fields are also added to the terraform ignore_changes hook, to avoid updating them after creation. This is useful for fields that are required on creation, but we do not desire to update them after creation, for example because of an external controller is managing them, like an autoscaler.

array

List of nested AWS Glue Data Catalog target arguments. See Catalog Target below.

object

Reference to a CatalogDatabase in glue to populate databaseName.

name
required
string
namespace
string
policy
object

Policies for referencing.

resolve
string
object

Selector for a CatalogDatabase in glue to populate databaseName.

namespace
string
policy
object

Policies for selection.

resolve
string
tables
array

A list of catalog tables to be synchronized.

array

References to CatalogTable in glue to populate tables.

name
required
string
namespace
string
policy
object

Policies for referencing.

resolve
string
object

Selector for a list of CatalogTable in glue to populate tables.

namespace
string
policy
object

Policies for selection.

resolve
string
array

List of custom classifiers. By default, all AWS classifiers are included in a crawl, but these custom classifiers always override the default classifiers for a given classification.

object

Reference to a CatalogDatabase in glue to populate databaseName.

name
required
string
namespace
string
policy
object

Policies for referencing.

resolve
string
object

Selector for a CatalogDatabase in glue to populate databaseName.

namespace
string
policy
object

Policies for selection.

resolve
string
array

List of nested Delta Lake target arguments. See Delta Target below.

array

A list of the Amazon S3 paths to the Delta tables.

array

List of nested DynamoDB target arguments. See Dynamodb Target below.

path
string
scanAll
boolean
scanRate
number
array

List of nested Hudi target arguments. See Iceberg Target below.

array

A list of glob patterns used to exclude from the crawl.

paths
array

One or more Amazon S3 paths that contains Hudi metadata folders as s3://bucket/prefix.

array

List of nested Iceberg target arguments. See Iceberg Target below.

array

A list of glob patterns used to exclude from the crawl.

paths
array

One or more Amazon S3 paths that contains Hudi metadata folders as s3://bucket/prefix.

array

List of nested JDBC target arguments. See JDBC Target below.

object

Reference to a Connection in glue to populate connectionName.

name
required
string
namespace
string
policy
object

Policies for referencing.

resolve
string
object

Selector for a Connection in glue to populate connectionName.

namespace
string
policy
object

Policies for selection.

resolve
string
array

Specify a value of RAWTYPES or COMMENTS to enable additional metadata intable responses. RAWTYPES provides the native-level datatype. COMMENTS provides comments associated with a column or table in the database.

array

A list of glob patterns used to exclude from the crawl.

path
string
object

Specifies Lake Formation configuration settings for the crawler. See Lake Formation Configuration below.

object

Specifies data lineage configuration settings for the crawler. See Lineage Configuration below.

array

List of nested MongoDB target arguments. See MongoDB Target below.

object

Reference to a Connection in glue to populate connectionName.

name
required
string
namespace
string
policy
object

Policies for referencing.

resolve
string
object

Selector for a Connection in glue to populate connectionName.

namespace
string
policy
object

Policies for selection.

resolve
string
path
string
scanAll
boolean
object

A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run.. See Recrawl Policy below.

role
string
roleRef
object

Reference to a Role in iam to populate role.

name
required
string
namespace
string
policy
object

Policies for referencing.

resolve
string
object

Selector for a Role in iam to populate role.

namespace
string
policy
object

Policies for selection.

resolve
string
array

List of nested Amazon S3 target arguments. See S3 Target below.

array

A list of glob patterns used to exclude from the crawl.

path
string
schedule
string
object

Policy for the crawler's update and deletion behavior. See Schema Change Policy below.

tags
object
array

THIS IS A BETA FIELD. It is on by default but can be opted out through a Crossplane feature flag. ManagementPolicies specify the array of actions Crossplane is allowed to take on the managed and external resources. See the design doc for more information: https://github.com/crossplane/crossplane/blob/499895a25d1a1a0ba1604944ef98ac7a1a71f197/design/design-doc-observe-only-resources.md?plain=1#L223 and this one: https://github.com/crossplane/crossplane/blob/444267e84783136daa93568b364a5f01228cacbe/design/one-pager-ignore-changes.md

object

ProviderConfigReference specifies how the provider that will be used to create, observe, update, and delete this managed resource should be configured.

kind
required
string
name
required
string
object

WriteConnectionSecretToReference specifies the namespace and name of a Secret to which any connection details for this managed resource should be written. Connection details frequently include the endpoint, username, and password required to connect to the managed resource.

name
required
string
status
object

CrawlerStatus defines the observed state of Crawler.

object

(No description available)

arn
string
array

List of nested AWS Glue Data Catalog target arguments. See Catalog Target below.

tables
array

A list of catalog tables to be synchronized.

array

List of custom classifiers. By default, all AWS classifiers are included in a crawl, but these custom classifiers always override the default classifiers for a given classification.

array

List of nested Delta Lake target arguments. See Delta Target below.

array

A list of the Amazon S3 paths to the Delta tables.

array

List of nested DynamoDB target arguments. See Dynamodb Target below.

path
string
scanAll
boolean
scanRate
number
array

List of nested Hudi target arguments. See Iceberg Target below.

array

A list of glob patterns used to exclude from the crawl.

paths
array

One or more Amazon S3 paths that contains Hudi metadata folders as s3://bucket/prefix.

array

List of nested Iceberg target arguments. See Iceberg Target below.

array

A list of glob patterns used to exclude from the crawl.

paths
array

One or more Amazon S3 paths that contains Hudi metadata folders as s3://bucket/prefix.

id
string
array

List of nested JDBC target arguments. See JDBC Target below.

array

Specify a value of RAWTYPES or COMMENTS to enable additional metadata intable responses. RAWTYPES provides the native-level datatype. COMMENTS provides comments associated with a column or table in the database.

array

A list of glob patterns used to exclude from the crawl.

path
string
object

Specifies Lake Formation configuration settings for the crawler. See Lake Formation Configuration below.

object

Specifies data lineage configuration settings for the crawler. See Lineage Configuration below.

array

List of nested MongoDB target arguments. See MongoDB Target below.

path
string
scanAll
boolean
object

A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run.. See Recrawl Policy below.

region
string
role
string
array

List of nested Amazon S3 target arguments. See S3 Target below.

array

A list of glob patterns used to exclude from the crawl.

path
string
schedule
string
object

Policy for the crawler's update and deletion behavior. See Schema Change Policy below.

tags
object
tagsAll
object
array

Conditions of the resource.

lastTransitionTime
required
string
message
string
reason
required
string
status
required
string
type
required
string