Crossplane in the Trenches: Deletion Pitfalls and How to Prevent Them

There's an infrastructure-as-code tool that offers dynamic infrastructure and continuous reconciliation - it's called Crossplane.

If you're using Crossplane right now, you should know about its deletion policies. Improper deletion handling can lead to orphaned cloud assets that continue accruing costs, and invisibly consume cloud quotas.

Leftover Kubernetes objects can cause the providers to continue making API calls against the cloud provider, adding to logging costs and just making your Kubernetes admins confused when debugging.

When writing Crossplane compositions, it's not a given that all resources will: a) delete cleanly, b) be easily discoverable, or c) be protected from accidental deletions.

To address these issues, we'll talk about:

Kubernetes Admission Protection
Crossplane Delete Policies
Crossplane Usages for Dependency Ordering
Crossplane Management Policies
Shift-left Validation via Pre-commit Hooks
Last Resort Tactics

Kubernetes Admission Protection

The first question you should ask is: "Should your users be able to delete this resource? How can I force an 'are you sure' confirmation to prevent accidental deletion?"

One way to answer that is with native Kubernetes validation. For example, you can have an "opt-in" annotation where if the resource has it, it's protected from deletion. To delete it, you first have to change or remove the annotation.

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
  name: "database-deletion-protection"
spec:
  matchConstraints:
    resourceRules:
      - apiGroups: ["contoso.com"]
        apiVersions: ["v1beta1"]
        operations: ["DELETE"]
        resources: ["databases"]
  validations:
    - expression: "oldObject.metadata.annotations['contoso.com/delete-protected'] == 'true'"
      messageExpression: "'This database is protected from deletion. Remove the contoso.com/delete-protected annotation or set it to false to allow deletion'"
      reason: Forbidden
  failurePolicy: Fail
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
  name: "database-deletion-protection-binding"
spec:
  policyName: "database-deletion-protection"
  validationActions: [Deny]
  matchResources:
    resourceRules:
      - apiGroups: ["contoso.com"]
        apiVersions: ["v1beta1"]
        resources: ["databases"]

Takeaway: Native Kubernetes validation policies provide the strongest deletion protection by preventing the delete operation from starting.

Crossplane Delete Policies

✅ Supported by: all Crossplane providers, it's a Crossplane setting, not a provider one.

In Crossplane v1, Claims create XRs which create the end Managed Resources - docs - You would think that if you delete the top level Claim, once it's gone, all resources associated with it are gone too.

That would be wrong.

By default, when you delete a Crossplane Claim, the Kubernetes garbage collector performs Background cascading deletion. This has two implications:

It's hard to find any leftover child resources that might be in a stuck state-or worse, in a healthy state but not deleting
If you have tests that apply a Claim and then delete it, the delete phase will be lying to you. It will make the Claim disappear, but the underlying resources will not be gone

If you want the Claim to wait for the clean deletion of resources in your Composition, add defaultCompositeDeletePolicy: Foreground to the CompositeResourceDefinition:

apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: xdatabases.platform.example.com
spec:
  defaultCompositeDeletePolicy: Foreground
  # ... rest of spec

Note this is just the “default”. It will only affect new resources. To change existing Claims’s delete policy you need to edit the claims themselves and add spec.compositeDeletePolicy: Foreground

apiVersion: platform.example.com/v1alpha1
kind: Database
metadata:
  name: my-database
spec:
  compositeDeletePolicy: Foreground
  # ... rest of spec

Deletion Policy docs.

Takeaway: Use Foreground deletion policy to ensure child resources are fully deleted before the parent Claim/XR disappears.

Crossplane Usages for Dependency Ordering

✅ Supported by: all Crossplane providers, it's a Crossplane setting.

Terraform and other CLI-driven infrastructure providers create internal trees of explicit and implicit dependencies, then order deletion to account for that.

Crossplane does not do that. It will delete both a Cluster and a Helm release sent to that cluster, or both a project and a Bucket in that project.

This can leave some Managed Resources hanging, forever erroring with "Unable to find GCP project" or "Delete failed: artifact registry does not have permission over CMEK key."

Crossplane's answer to that is a feature called Usages. With Usages, the original Claim will still go into deletion, but some of its resources will be blocked from deleting. You can also block on XRs, but not on itself.

This is a way to “wait” for deletion of resource A before deleting resource B.

flowchart TD

  XR["XR"] -- delete --> Usage
  XR -- delete --> Cluster
  Usage -. blocks .-> Cluster
  Usage -. waits for .-> Release
  XR -- delete --> Release

  style Release fill:#e06666,stroke:#333,stroke-width:2px

The XR sends a delete signal to all 3 Managed Resources. Only the Release is deleted. The Usage waits and the Cluster deletion is blocked.

flowchart TD
  XR["XR"] -- waiting to delete --> Usage
  XR -- blocked from deleting --> Cluster
  Usage -. blocks .-> Cluster

  style Usage fill:#e06666,stroke:#333,stroke-width:2px

The Helm Release is now gone so now the Usage will be deleted.

flowchart TD
  XR -- deleting --> Cluster
  style Cluster fill:#e06666,stroke:#333,stroke-width:2px

Finally the deletion signal gets to the Cluster and it starts getting deleted.

Usages with Deletion Policies

Usage relationships can be defined between Managed Resources and Composites.

However, a Composite as the using resource (spec.by) would be ineffective unless the compositeDeletePolicy: Foreground is used because it wouldn’t block deletion of its child resources before its own deletion with the default deletion policy Background.

And the yaml for a Usage:

apiVersion: apiextensions.crossplane.io/v1alpha1
kind: Usage
metadata:
  name: protect-database-from-cluster-deletion
spec:
  of:
    apiVersion: platform.example.com/v1alpha1
    kind: XDatabase
    resourceRef:
      name: my-production-database
  by:
    apiVersion: platform.example.com/v1alpha1
    kind: XCluster
    resourceRef:
      name: my-k8s-cluster

This ensures the database won't be deleted until the cluster that depends on it is deleted first.

Tip: If you find that deletion is taking much longer with Usages, you can set spec.replayDeletion: true to trigger an immediate retry of blocked deletions instead of waiting for exponential back-off.

Usages docs.

Takeaway: Usages ensure proper deletion ordering by blocking dependent resource deletion until dependencies are removed.

Crossplane Management Policies

Say you are managing a resource in GCP that can't be deleted, such as the default ProjectSink. You can change it, but GCP will not allow you to delete it.

To tell Crossplane to orphan it, add managementPolicies like so:

apiVersion: logging.gcp.upbound.io/v1beta2
kind: ProjectSink
metadata:
  name: default-project-sink
  annotations:
    crossplane.io/external-name: _Default
spec:
  managementPolicies: [Observe, Create, Update, LateInitialize]
  forProvider:
    project: my-project-id

And if you want to get the details of a resource-like a data resource in Terraform-you can do:

apiVersion: logging.gcp.upbound.io/v1beta2
kind: ProjectSink
metadata:
  name: default-project-sink
  annotations:
    crossplane.io/external-name: _Default
spec:
  managementPolicies: [Observe]
  forProvider:
    project: my-project-id

The above will not even attempt to edit it.

Management Policy Support

Management policy support is implemented per provider. All major providers (AWS, Azure, GCP, Helm, Kubernetes, Terraform) support it, but it needs to be enabled by --enable-management-policies. Check your specific provider documentation.

Management Policies docs.

Takeaway: Management policies give you fine-grained control over which CRUD operations the Crossplane provider performs on your resources.

Shift-left Validation via Pre-commit Hooks

If you have validation policies on deletion or other specifics for deletion, you can shift the validation left and add a pre-commit hook to your codebase that is done both on PRs and can be done by developers locally.

Example:

Create a .pre-commit-config.yaml file in your git repo:

repos:
  - repo: local
    hooks:
      - id: crossplane-deletion-policy-check
        name: Check Crossplane deletion policies
        entry: bash -c 'find . -name "*.yaml" -exec grep -l "kind.*CompositeResourceDefinition" {} \; | xargs grep -L "defaultCompositeDeletePolicy" && echo "ERROR: XRD missing defaultCompositeDeletePolicy" && exit 1 || exit 0'
        language: system
        files: '\.ya?ml$'

This ensures all CompositeResourceDefinitions have explicit deletion policies.

See the Pre-commit Hooks setup guide for the full setup.

Takeaway: Pre-commit hooks catch deletion policy misconfigurations before they reach production, saving debugging time later.

Last Resort Tactics

If all else fails and you're looking to delete a specific resource that is "stuck" in an Unhealthy state, you can remove the finalizer.

Understanding Finalizers

Finalizers are Kubernetes mechanisms that allow controllers to perform cleanup before a resource is deleted. They prevent the object from being deleted until specific conditions are met.

Only remove finalizers as a last resort when you're certain the cloud resources have been manually cleaned up.

Removing Finalizers

⚠️ Warning: This can leave hanging cloud resources behind that will continue to accrue costs.

# Check what finalizers exist first
kubectl get database my-stuck-database -o jsonpath='{.metadata.finalizers}'

# Remove finalizers from a stuck Crossplane resource
kubectl patch database my-stuck-database -p '{"metadata":{"finalizers":[]}}' --type=merge

Safer Finalizer Management

Before removing finalizers, try to understand why the resource is stuck:

# Install the crossplane CLI tool from https://docs.crossplane.io/latest/cli/
crossplane beta trace database.my-company.com -n my-namespace my-stuck-database

# Check the resource status
kubectl describe database.gcp.upbound.io my-stuck-database

# Check provider logs
kubectl get pods | grep "provider-gcp-container" # CHANGE FOR THE PROVIDER MANAGING YOUR RESOURCE
kubectl logs provider-gcp-container-xxx -f

# Look for related events
kubectl get events --field-selector involvedObject.name=my-stuck-database
kubectl get events | grep my-stuck-database

Takeaway: Finalizer removal should be your last resort-always investigate why resources are stuck first to avoid leaving orphaned cloud resources.

Conclusion

Crossplane’s deletion behaviors can be tricky, so you can use the following tools to keep your infrastructure safe and predictable:

Kubernetes Admission Policies – Block accidental deletions at the API level.
Foreground Deletion Policies – Ensure parent resources wait for children to be deleted.
Usages – Explicitly order deletions and prevent dependencies from being removed too soon.
Management Policies – Control which operations Crossplane performs on your resources.
Pre-commit Hooks – Catch misconfigurations before they reach production.
Finalizer Removal (Last Resort) – Unstick resources, but only after careful investigation.

By combining these techniques, you can avoid orphaned resources, reduce surprises, and keep your platform clean—even as your Crossplane usage grows.

This article is provided as a general guide for general information purposes only. It does not constitute advice. CECG disclaims liability for actions taken based on the materials.