Marketplace
BrowsePublish
Marketplace
You are viewing an outdated version of provider-gcp.Go to Latest
upbound/provider-gcp@v0.41.2
Cluster
dataproc.gcp.upbound.io
Cluster
upbound/provider-gcp@v0.41.2dataproc.gcp.upbound.io

Cluster is the Schema for the Clusters API. Manages a Cloud Dataproc cluster resource.

Type

CRD

Group

dataproc.gcp.upbound.io

Version

v1beta1

apiVersion: dataproc.gcp.upbound.io/v1beta1

kind: Cluster

API Documentation
apiVersion
string
kind
string
metadata
object
spec
object
object

ClusterSpec defines the desired state of Cluster

forProvider
requiredobject
requiredobject

No description provided.

array

Allows you to configure various aspects of the cluster. Structure defined below.

array

The autoscaling policy config associated with the cluster. Note that once set, if autoscaling_config is the only field set in cluster_config, it can only be removed by setting policy_uri = "", rather than removing the whole block. Structure defined below.

policyUri
string
array

The Compute Engine accelerator (GPU) configuration for these instances. Can be specified multiple times. Structure defined below.

metrics
array
array

Metrics sources to enable.

array

One or more [available OSS metrics] (https://cloud.google.com/dataproc/docs/guides/monitoring#available_oss_metrics) to collect for the metric course.

array

The Customer managed encryption keys settings for the cluster. Structure defined below.

array

The config settings for port access on the cluster. Structure defined below.

array

Common config settings for resources of Google Compute Engine cluster instances, applicable to all instances in the cluster. Structure defined below.

metadata
object
network
string
array

Node Group Affinity for sole-tenant clusters.

array

Reservation Affinity for consuming zonal reservation.

key
string
values
array
array

Corresponds to the label values of reservation resource.

object

Reference to a ServiceAccount in cloudplatform to populate serviceAccount.

name
requiredstring
policy
object
object

Policies for referencing.

resolve
string
array

The set of Google API scopes to be made available on all of the node VMs under the service_account specified. Both OAuth2 URLs and gcloud short names are supported. To allow full access to all Cloud APIs, use the cloud-platform scope. See a complete list of scopes here.

object

Selector for a ServiceAccount in cloudplatform to populate serviceAccount.

policy
object
object

Policies for selection.

resolve
string
array

Shielded Instance Config for clusters using Compute Engine Shielded VMs.

tags
array
array

The list of instance tags applied to instances in the cluster. Tags are used to identify valid sources or targets for network firewalls.

zone
string
array

Commands to execute on each node after config is completed. You can specify multiple versions of these. Structure defined below.

script
string
array

The settings for auto deletion cluster schedule. Structure defined below.

array

The Google Compute Engine config settings for the master instances in a cluster. Structure defined below.

array

The Compute Engine accelerator (GPU) configuration for these instances. Can be specified multiple times.

array

Disk Config

imageUri
string
array

The config setting for metastore service with the cluster. Structure defined below.

array

The Google Compute Engine config settings for the additional instances in a cluster. Structure defined below.

array

Disk Config

array

Security related configuration. Structure defined below.

array

The config settings for software inside the cluster. Structure defined below.

array

The set of optional components to activate on the cluster. See Available Optional Components.

array

The Google Compute Engine config settings for the worker instances in a cluster. Structure defined below.

array

The Compute Engine accelerator configuration for these instances. Can be specified multiple times.

array

Disk Config

imageUri
string
labels
object
name
string
project
string
region
string
array

Allows you to configure a virtual Dataproc on GKE cluster. Structure defined below.

array

Configuration of auxiliary services used by this cluster. Structure defined below.

array

The config setting for metastore service with the cluster. Structure defined below.

array

The Spark History Server configuration for the workload.

array

The configuration for running the Dataproc cluster on Kubernetes. Structure defined below.

array

The configuration for running the Dataproc cluster on GKE.

array

GKE node pools where workloads will be scheduled. At least one node pool must be assigned the DEFAULT GkeNodePoolTarget.Role. If a GkeNodePoolTarget is not specified, Dataproc constructs a DEFAULT GkeNodePoolTarget. Each role can be given to only one GkeNodePoolTarget. All node pools must have the same location settings.

nodePool
string
array

(Input only) The configuration for the GKE node pool. If specified, Dataproc attempts to create a node pool with the specified shape. If one with the same name already exists, it is verified against all specified fields. If a field differs, the virtual cluster creation will fail.

array

The autoscaler configuration for this node pool. The autoscaler is enabled only when a valid configuration is present.

config
array
array

The node pool configuration.

array

The list of Compute Engine zones where node pool nodes associated with a Dataproc on GKE virtual cluster will be located.

roles
array
array

The roles associated with the GKE node pool. One of "DEFAULT", "CONTROLLER", "SPARK_DRIVER" or "SPARK_EXECUTOR".

array

The software configuration for this Dataproc cluster running on Kubernetes.

object

THIS IS A BETA FIELD. It will be honored unless the Management Policies feature flag is disabled. InitProvider holds the same fields as ForProvider, with the exception of Identifier and other resource reference fields. The fields that are in InitProvider are merged into ForProvider when the resource is created. The same fields are also added to the terraform ignore_changes hook, to avoid updating them after creation. This is useful for fields that are required on creation, but we do not desire to update them after creation, for example because of an external controller is managing them, like an autoscaler.

array

Allows you to configure various aspects of the cluster. Structure defined below.

array

The autoscaling policy config associated with the cluster. Note that once set, if autoscaling_config is the only field set in cluster_config, it can only be removed by setting policy_uri = "", rather than removing the whole block. Structure defined below.

policyUri
string
array

The Compute Engine accelerator (GPU) configuration for these instances. Can be specified multiple times. Structure defined below.

metrics
array
array

Metrics sources to enable.

array

One or more [available OSS metrics] (https://cloud.google.com/dataproc/docs/guides/monitoring#available_oss_metrics) to collect for the metric course.

array

The Customer managed encryption keys settings for the cluster. Structure defined below.

array

The config settings for port access on the cluster. Structure defined below.

array

Common config settings for resources of Google Compute Engine cluster instances, applicable to all instances in the cluster. Structure defined below.

metadata
object
network
string
array

Node Group Affinity for sole-tenant clusters.

array

Reservation Affinity for consuming zonal reservation.

key
string
values
array
array

Corresponds to the label values of reservation resource.

object

Reference to a ServiceAccount in cloudplatform to populate serviceAccount.

name
requiredstring
policy
object
object

Policies for referencing.

resolve
string
array

The set of Google API scopes to be made available on all of the node VMs under the service_account specified. Both OAuth2 URLs and gcloud short names are supported. To allow full access to all Cloud APIs, use the cloud-platform scope. See a complete list of scopes here.

object

Selector for a ServiceAccount in cloudplatform to populate serviceAccount.

policy
object
object

Policies for selection.

resolve
string
array

Shielded Instance Config for clusters using Compute Engine Shielded VMs.

tags
array
array

The list of instance tags applied to instances in the cluster. Tags are used to identify valid sources or targets for network firewalls.

zone
string
array

Commands to execute on each node after config is completed. You can specify multiple versions of these. Structure defined below.

script
string
array

The settings for auto deletion cluster schedule. Structure defined below.

array

The Google Compute Engine config settings for the master instances in a cluster. Structure defined below.

array

The Compute Engine accelerator (GPU) configuration for these instances. Can be specified multiple times.

array

Disk Config

imageUri
string
array

The config setting for metastore service with the cluster. Structure defined below.

array

The Google Compute Engine config settings for the additional instances in a cluster. Structure defined below.

array

Disk Config

array

Security related configuration. Structure defined below.

array

The config settings for software inside the cluster. Structure defined below.

array

The set of optional components to activate on the cluster. See Available Optional Components.

array

The Google Compute Engine config settings for the worker instances in a cluster. Structure defined below.

array

The Compute Engine accelerator configuration for these instances. Can be specified multiple times.

array

Disk Config

imageUri
string
labels
object
name
string
project
string
region
string
array

Allows you to configure a virtual Dataproc on GKE cluster. Structure defined below.

array

Configuration of auxiliary services used by this cluster. Structure defined below.

array

The config setting for metastore service with the cluster. Structure defined below.

array

The Spark History Server configuration for the workload.

array

The configuration for running the Dataproc cluster on Kubernetes. Structure defined below.

array

The configuration for running the Dataproc cluster on GKE.

array

GKE node pools where workloads will be scheduled. At least one node pool must be assigned the DEFAULT GkeNodePoolTarget.Role. If a GkeNodePoolTarget is not specified, Dataproc constructs a DEFAULT GkeNodePoolTarget. Each role can be given to only one GkeNodePoolTarget. All node pools must have the same location settings.

nodePool
string
array

(Input only) The configuration for the GKE node pool. If specified, Dataproc attempts to create a node pool with the specified shape. If one with the same name already exists, it is verified against all specified fields. If a field differs, the virtual cluster creation will fail.

array

The autoscaler configuration for this node pool. The autoscaler is enabled only when a valid configuration is present.

config
array
array

The node pool configuration.

array

The list of Compute Engine zones where node pool nodes associated with a Dataproc on GKE virtual cluster will be located.

roles
array
array

The roles associated with the GKE node pool. One of "DEFAULT", "CONTROLLER", "SPARK_DRIVER" or "SPARK_EXECUTOR".

array

The software configuration for this Dataproc cluster running on Kubernetes.

array

THIS IS A BETA FIELD. It is on by default but can be opted out through a Crossplane feature flag. ManagementPolicies specify the array of actions Crossplane is allowed to take on the managed and external resources. This field is planned to replace the DeletionPolicy field in a future release. Currently, both could be set independently and non-default values would be honored if the feature flag is enabled. If both are custom, the DeletionPolicy field will be ignored. See the design doc for more information: https://github.com/crossplane/crossplane/blob/499895a25d1a1a0ba1604944ef98ac7a1a71f197/design/design-doc-observe-only-resources.md?plain=1#L223 and this one: https://github.com/crossplane/crossplane/blob/444267e84783136daa93568b364a5f01228cacbe/design/one-pager-ignore-changes.md

object

ProviderConfigReference specifies how the provider that will be used to create, observe, update, and delete this managed resource should be configured.

name
requiredstring
policy
object
object

Policies for referencing.

resolve
string
object

PublishConnectionDetailsTo specifies the connection secret config which contains a name, metadata and a reference to secret store config to which any connection details for this managed resource should be written. Connection details frequently include the endpoint, username, and password required to connect to the managed resource.

configRef
object
object

SecretStoreConfigRef specifies which secret store config should be used for this ConnectionSecret.

name
requiredstring
policy
object
object

Policies for referencing.

resolve
string
metadata
object
object

Metadata is the metadata for connection secret.

labels
object
type
string
name
requiredstring
object

WriteConnectionSecretToReference specifies the namespace and name of a Secret to which any connection details for this managed resource should be written. Connection details frequently include the endpoint, username, and password required to connect to the managed resource. This field is planned to be replaced in a future release in favor of PublishConnectionDetailsTo. Currently, both could be set independently and connection details would be published to both without affecting each other.

name
requiredstring
namespace
requiredstring
status
object
object

ClusterStatus defines the observed state of Cluster.

object

No description provided.

array

Allows you to configure various aspects of the cluster. Structure defined below.

array

The autoscaling policy config associated with the cluster. Note that once set, if autoscaling_config is the only field set in cluster_config, it can only be removed by setting policy_uri = "", rather than removing the whole block. Structure defined below.

policyUri
string
bucket
string
array

The Compute Engine accelerator (GPU) configuration for these instances. Can be specified multiple times. Structure defined below.

metrics
array
array

Metrics sources to enable.

array

One or more [available OSS metrics] (https://cloud.google.com/dataproc/docs/guides/monitoring#available_oss_metrics) to collect for the metric course.

array

The Customer managed encryption keys settings for the cluster. Structure defined below.

array

The config settings for port access on the cluster. Structure defined below.

array

Common config settings for resources of Google Compute Engine cluster instances, applicable to all instances in the cluster. Structure defined below.

metadata
object
network
string
array

Node Group Affinity for sole-tenant clusters.

array

Reservation Affinity for consuming zonal reservation.

key
string
values
array
array

Corresponds to the label values of reservation resource.

array

The set of Google API scopes to be made available on all of the node VMs under the service_account specified. Both OAuth2 URLs and gcloud short names are supported. To allow full access to all Cloud APIs, use the cloud-platform scope. See a complete list of scopes here.

array

Shielded Instance Config for clusters using Compute Engine Shielded VMs.

tags
array
array

The list of instance tags applied to instances in the cluster. Tags are used to identify valid sources or targets for network firewalls.

zone
string
array

Commands to execute on each node after config is completed. You can specify multiple versions of these. Structure defined below.

script
string
array

The settings for auto deletion cluster schedule. Structure defined below.

array

The Google Compute Engine config settings for the master instances in a cluster. Structure defined below.

array

The Compute Engine accelerator (GPU) configuration for these instances. Can be specified multiple times.

array

Disk Config

imageUri
string
array

List of worker instance names which have been assigned to the cluster.

array

The config setting for metastore service with the cluster. Structure defined below.

array

The Google Compute Engine config settings for the additional instances in a cluster. Structure defined below.

array

Disk Config

array

List of worker instance names which have been assigned to the cluster.

array

Security related configuration. Structure defined below.

array

The config settings for software inside the cluster. Structure defined below.

array

The set of optional components to activate on the cluster. See Available Optional Components.

array

The Google Compute Engine config settings for the worker instances in a cluster. Structure defined below.

array

The Compute Engine accelerator configuration for these instances. Can be specified multiple times.

array

Disk Config

imageUri
string
array

List of worker instance names which have been assigned to the cluster.

id
string
labels
object
name
string
project
string
region
string
array

Allows you to configure a virtual Dataproc on GKE cluster. Structure defined below.

array

Configuration of auxiliary services used by this cluster. Structure defined below.

array

The config setting for metastore service with the cluster. Structure defined below.

array

The Spark History Server configuration for the workload.

array

The configuration for running the Dataproc cluster on Kubernetes. Structure defined below.

array

The configuration for running the Dataproc cluster on GKE.

array

GKE node pools where workloads will be scheduled. At least one node pool must be assigned the DEFAULT GkeNodePoolTarget.Role. If a GkeNodePoolTarget is not specified, Dataproc constructs a DEFAULT GkeNodePoolTarget. Each role can be given to only one GkeNodePoolTarget. All node pools must have the same location settings.

nodePool
string
array

(Input only) The configuration for the GKE node pool. If specified, Dataproc attempts to create a node pool with the specified shape. If one with the same name already exists, it is verified against all specified fields. If a field differs, the virtual cluster creation will fail.

array

The autoscaler configuration for this node pool. The autoscaler is enabled only when a valid configuration is present.

config
array
array

The node pool configuration.

array

The list of Compute Engine zones where node pool nodes associated with a Dataproc on GKE virtual cluster will be located.

roles
array
array

The roles associated with the GKE node pool. One of "DEFAULT", "CONTROLLER", "SPARK_DRIVER" or "SPARK_EXECUTOR".

array

The software configuration for this Dataproc cluster running on Kubernetes.

array

Conditions of the resource.

lastTransitionTime
requiredstring
message
string
reason
requiredstring
status
requiredstring
type
requiredstring
Marketplace

Discover the building blocks for your internal cloud platform.

© 2022 Upbound, Inc.

SolutionsProvidersConfigurations
LearnDocumentationTry for Free
MorePrivacy PolicyTerms & Conditions
Marketplace

© 2022 Upbound, Inc.

Marketplace

Discover the building blocksfor your internal cloud platform.