Resource Customization And Bulk Changes

Resource customization lets an instance change restored Kubernetes resources before they land on the target cluster. It is configured under DisasterInstance.spec.restorePolicy and compiled by the server/operator into Velero ResourceModifier rules.

Use this when the target cluster has different StorageClasses, IngressClasses, registry prefixes, node labels, annotations, fixed service addresses, or environment-specific fields.

Where It Applies

Restore policy can apply to several paths:

Path	Apply target	Description
DataSync restore	`dataSync`	Used when DataSync restores Pod/PVC/PV data through Trafficless Restore
ResourceSync restore	`resourceSync`	Used when ResourceSync restores the standby resource skeleton
Drill restore	`drill`	Used by disaster drills or drill-level override policies
Failover / activation	depends on operation path	Uses the already prepared ResourceSync/DataSync state, then scales workloads

Do not apply the same rule blindly to every path. For example, rules that rewrite workload images usually belong to ResourceSync, while rules that change PVC StorageClass belong to data or restore paths that create PVCs.

Main Configuration Areas

Field	Purpose
`restorePolicy.resourceSelection`	Include/exclude namespaces, namespace-scoped resource kinds, and cluster-scoped resource kinds
`restorePolicy.storageClassMapping`	Map source PVC/PV StorageClasses to target classes
`restorePolicy.ingressClassMapping`	Map source IngressClasses to target classes
`restorePolicy.modifierRules`	Structured restore modifier rules
`restorePolicy.modifierRulesText`	JSON text input parsed by the server into `modifierRules`
`restorePolicy.bulkModifierActions`	Batch intent for replace/remove/image-rewrite operations
`restorePolicy.bulkModifierActionsText`	JSON text input parsed by the server into `bulkModifierActions`

If you only need application image registry rewrite, high-churn image tags, or rewriteImage examples, start with Application Image Rewrite.

When using modifierRules or bulkModifierActions, enable the unified resolver:

{
  "restorePolicy": {
    "useUnifiedDirectionResolver": true
  }
}

Without it, the operator may reject the new DSL during restore policy compilation.

Fine-Grained Resource Selection

The recommended scoped fields are:

restorePolicy:
  resourceSelection:
    includedNamespaceScopedResources:
      - deployments.apps
      - services
      - configmaps
      - secrets
    excludedNamespaceScopedResources:
      - events
    includedClusterScopedResources:
      - storageclasses.storage.k8s.io
    excludedClusterScopedResources: []

When includedClusterScopedResources is non-empty, ResourceSync runs an additional cluster restore phase before the namespace restore phase. This is expected. It avoids implicitly restoring broad cluster-scoped resources during namespace restore.

If only excludedClusterScopedResources is set and no cluster kinds are explicitly included, ResourceSync does not run the extra cluster phase.

Legacy fields also exist:

includedResources: []
excludedResources: []
includeClusterResources: true

For new instance configuration, prefer the scoped fields and avoid mixing them with includeClusterResources=true.

StorageClass Mapping

Use StorageClass mapping when the source and target clusters use different StorageClass names.

{
  "restorePolicy": {
    "storageClassMapping": {
      "mappings": [
        {
          "sourceClass": "nfs-client",
          "targetClass": "csi-rbd-sc",
          "namespaces": ["demo"]
        }
      ],
      "unmatchedPolicy": "Keep",
      "strictTargetValidation": true
    }
  }
}

Option	Meaning
`sourceClass`	StorageClass name in the source cluster
`targetClass`	StorageClass name to use in the target cluster
`namespaces`	Optional namespace filter for PVC mapping
`unmatchedPolicy=Keep`	Keep original values when no mapping matches
`unmatchedPolicy=Fail`	Fail policy compilation if required mappings are missing
`strictTargetValidation=true`	Validate that target classes exist before restore starts

If multiple StorageClass mappings produce different target classes for PV-level patches, the operator rejects the policy because PV is cluster-scoped and cannot be safely namespace-filtered in the current modifier schema.

IngressClass Mapping

Use IngressClass mapping when the target cluster uses a different ingress controller.

{
  "restorePolicy": {
    "ingressClassMapping": {
      "mappings": [
        {
          "sourceClass": "nginx",
          "targetClass": "traefik",
          "namespaces": ["demo"]
        }
      ],
      "unmatchedPolicy": "Keep",
      "strictTargetValidation": true
    }
  }
}

This generates patches for ingresses.networking.k8s.io and changes /spec/ingressClassName.

Modifier Rules

Modifier rules are used for precise JSON Patch operations.

Example: remove nodeSelector from restored Deployments:

{
  "restorePolicy": {
    "useUnifiedDirectionResolver": true,
    "modifierRules": [
      {
        "id": "remove-node-selector",
        "applyTo": ["resourceSync", "drill"],
        "mode": "veleroNative",
        "conditions": {
          "groupResource": "deployments.apps",
          "namespaces": ["demo"]
        },
        "veleroRule": {
          "patches": [
            {
              "operation": "remove",
              "path": "/spec/template/spec/nodeSelector"
            }
          ]
        },
        "directionPolicy": "ForwardOnly",
        "onConflict": "Fail"
      }
    ]
  }
}

Example: add an annotation to restored Deployments:

{
  "restorePolicy": {
    "useUnifiedDirectionResolver": true,
    "modifierRules": [
      {
        "id": "mark-restored-by-testudo",
        "mode": "veleroNative",
        "applyTo": ["resourceSync"],
        "conditions": {
          "groupResource": "deployments.apps",
          "namespaces": ["demo"]
        },
        "veleroRule": {
          "patches": [
            {
              "operation": "add",
              "path": "/metadata/annotations/testudo.softcdata.com~1restored-by",
              "value": "testudo"
            }
          ]
        }
      }
    ]
  }
}

JSON Pointer escaping applies:

Character	Escaped form
`/`	`~1`
`~`	`~0`

Reversible Rules

Reversible rules are useful when a field needs one value in the forward direction and another after failover/reprotect.

Example:

{
  "restorePolicy": {
    "useUnifiedDirectionResolver": true,
    "modifierRules": [
      {
        "id": "replace-image-registry",
        "applyTo": ["resourceSync", "drill"],
        "mode": "reversible",
        "directionPolicy": "Auto",
        "conditions": {
          "groupResource": "deployments.apps",
          "namespaces": ["demo"]
        },
        "pair": {
          "path": "/spec/template/spec/containers/0/image",
          "sourceValue": "harbor-a.example.com/demo/web:v1",
          "targetValue": "harbor-b.example.com/demo/web:v1"
        },
        "onConflict": "Fail"
      }
    ]
  }
}

directionPolicy=Auto chooses the proper value based on the current source and target roles. Use ForwardOnly or ReverseOnly when the rule must be limited to one direction. Reversible rules must use the canonical pair.path, pair.sourceValue, and pair.targetValue form; legacy transform-style examples are rejected by the server.

Example: Rewrite StorageClass For Specific PVCs

For a small set of PVCs, use reversible rules that match each PVC by name. This example rewrites two PVCs in the blueking namespace to the target cluster StorageClass nfs-client during DataSync restore:

{
  "restorePolicy": {
    "useUnifiedDirectionResolver": true,
    "modifierRules": [
      {
        "id": "rewrite-skywalking-pvc-sc",
        "mode": "reversible",
        "enabled": true,
        "applyTo": ["dataSync"],
        "priority": 100,
        "conditions": {
          "groupResource": "persistentvolumeclaims",
          "namespaces": ["blueking"],
          "resourceNameRegex": "^bk-skywalking-agent-nfs-pvc$"
        },
        "pair": {
          "path": "/spec/storageClassName",
          "sourceValue": "local-storage",
          "targetValue": "nfs-client"
        },
        "directionPolicy": "Auto",
        "onConflict": "Fail"
      },
      {
        "id": "rewrite-bkrepo-pvc-sc",
        "mode": "reversible",
        "enabled": true,
        "applyTo": ["dataSync"],
        "priority": 100,
        "conditions": {
          "groupResource": "persistentvolumeclaims",
          "namespaces": ["blueking"],
          "resourceNameRegex": "^bk-repo-bkrepo-nfs-pvc$"
        },
        "pair": {
          "path": "/spec/storageClassName",
          "sourceValue": "bkrepo-nfs",
          "targetValue": "nfs-client"
        },
        "directionPolicy": "Auto",
        "onConflict": "Fail"
      }
    ]
  }
}

When entering this in the console Resource customization text area, you usually enter only the modifierRules array, without the outer restorePolicy. The JSON must be a valid array or an object with an array field; do not paste two comma-separated object fragments by themselves.

Bulk Changes

Bulk actions let the UI/API express repeated field changes without manually writing every JSON Patch rule.

Supported actions:

Action	`action`	Behavior
Replace exact string value	`replaceExactValue`	Scans string leaf nodes and creates reversible pair rules for fields equal to `sourceValue`.
Remove map/object key	`removeKey`	Scans map/object keys equal to `key` and creates remove patches.
Rewrite image prefix	`rewriteImage`	Reads current workload images from the source cluster at ResourceSync/Drill runtime and generates image replacement rules dynamically.

The batch scan scope comes from:

DisasterInstance.spec.namespaces
DisasterInstance.spec.labelSelector
restorePolicy.resourceSelection

Static batch actions skip forbidden paths such as /status/**, /metadata/finalizers/**, and /metadata/ownerReferences/**. You do not need to exclude pods only to avoid matching status.containerStatuses[*].image.

Replace A Fixed Value

Example: many resources contain the source service address 10.10.0.12, and the target site must use 10.20.0.12:

{
  "restorePolicy": {
    "useUnifiedDirectionResolver": true,
    "bulkModifierActions": [
      {
        "id": "replace-service-ip",
        "action": "replaceExactValue",
        "enabled": true,
        "applyTo": ["resourceSync", "drill"],
        "sourceValue": "10.10.0.12",
        "targetValue": "10.20.0.12",
        "directionPolicy": "Auto"
      }
    ]
  }
}

The server scans source resources at submission time and persists the generated rules in:

DisasterInstance.spec.restorePolicy.modifierRuleSnapshot
DisasterInstance.spec.restorePolicy.modifierRuleSnapshotHash

Remove A Key

Example: source resources contain a production-only annotation:

metadata.annotations.site-role=primary

Remove the key with:

{
  "restorePolicy": {
    "useUnifiedDirectionResolver": true,
    "bulkModifierActions": [
      {
        "id": "drop-site-role",
        "action": "removeKey",
        "enabled": true,
        "applyTo": ["resourceSync"],
        "key": "site-role"
      }
    ]
  }
}

removeKey defaults to directionPolicy=ForwardOnly.

Rewrite Image Prefixes Dynamically

If image tags or digests change frequently, do not use replaceExactValue with a full image reference. Once the tag changes, the persisted static snapshot no longer represents the current source workload. Use rewriteImage and declare only stable source/target prefixes.

This rewriteImage is the dynamic action under restorePolicy.bulkModifierActions, not the legacy instance-level image registry mapping path. Use this DSL for new application image-prefix rewrite configurations:

{
  "restorePolicy": {
    "useUnifiedDirectionResolver": true,
    "bulkModifierActions": [
      {
        "id": "rewrite-primary-registry",
        "action": "rewriteImage",
        "enabled": true,
        "applyTo": ["resourceSync", "drill"],
        "directionPolicy": "Auto",
        "imageRewrite": {
          "sourcePrefix": "10.11.11.1:5000/",
          "targetPrefix": "registry-test.xxx.xxx.com:30088/dr_images/10_11_11_1_5000/",
          "unmatchedPolicy": "Keep",
          "digestPolicy": "Preserve"
        }
      }
    ]
  }
}

At restore build time, the operator reads the current source workload spec. For example:

10.11.11.1:5000/blueking/bcs-bkcmdb-synchronizer:v1.32.0

is rewritten to:

registry-test.xxx.xxx.com:30088/dr_images/10_11_11_1_5000/blueking/bcs-bkcmdb-synchronizer:v1.32.0

rewriteImage rules:

sourcePrefix and targetPrefix are required. Trailing / is normalized.
applyTo supports resourceSync and drill, not dataSync.
unmatchedPolicy supports Keep and Fail; empty defaults to Keep.
digestPolicy currently supports only Preserve.
When multiple prefixes match, the longest prefix wins. Equal-length conflicting matches fail compilation.
Runtime compilation covers workload spec image fields in Deployment, StatefulSet, DaemonSet, Job, CronJob, and Pod, including containers, initContainers, and ephemeralContainers.
Runtime compilation never modifies /status/containerStatuses/*/image.

Use replaceExactValue for fixed values such as service IPs or annotations. Use rewriteImage for registry-prefix changes where tags or digests change often.

Validation

After saving an instance, inspect the generated fields:

kubectl -n disaster-system get disasterinstance <name> -o yaml

For static batch actions, check:

spec:
  restorePolicy:
    bulkModifierActions:
    modifierRuleSnapshot:
    modifierRuleSnapshotHash:

If replaceExactValue or removeKey is configured but no snapshot/hash exists, static snapshot generation did not complete. If only rewriteImage is configured, an empty snapshot/hash is expected because image rules are compiled dynamically during ResourceSync/Drill restore construction.

After a sync, inspect the generated AppRestore:

kubectl -n disaster-system get apprestore \
  -l testudo.softcdata.com/app-resource-owner-kind=resourcesync \
  -o yaml

Look for:

spec.resourceModifierRules
testudo.softcdata.com/modifier-source
policy summary annotations
spec.template.includedResources
spec.template.excludedResources
spec.template.includeClusterResources

If ResourceSync is in scoped mode and cluster-scoped resources are explicitly included, expect two AppRestore objects: one ...-cluster restore and one ...-ns restore.

Production Guidance

Validate policies in a drill namespace before production failover.
Keep cluster-scoped resources out of per-instance restore unless they truly belong to the application.
Prefer StorageClass/IngressClass mapping over ad hoc patches when the intent is class mapping.
Use reversible rules for values that must change direction after failover or reprotect.
Use static batch actions for stable fixed values and rewriteImage for high-churn image tags.
Keep DataSync-specific rules conservative because that path exists to restore volume data safely.

Where It Applies​

Main Configuration Areas​

Fine-Grained Resource Selection​

StorageClass Mapping​

IngressClass Mapping​

Modifier Rules​

Reversible Rules​

Example: Rewrite StorageClass For Specific PVCs​

Bulk Changes​

Replace A Fixed Value​

Remove A Key​

Rewrite Image Prefixes Dynamically​

Validation​

Production Guidance​