Please use the form below to provide your feedback. Because your feedback is valuable to us, the information you submit in this form is recorded in our issue tracking system (JIRA), which is publicly available. You can track the status of your feedback using the ticket number displayed in the dialog once you submit the form.

How-to Guide: Data Topology Save and Restore

    +
    A how-to guide on using the "Write Once and Create Anywhere" capability of Autonomous Operator.

    Overview

    This guide will show you how to save the configuration of one Couchbase cluster, and how to restore it on a different Couchbase cluster using the "Write Once and Create Anywhere" capability of Couchbase Autonomous Operator (CAO).

    Prerequisites

    • If you are new to Couchbase Autonomous Operator (CAO), refer to the Autonomous Operator Introduction

    • Couchbase Scopes and Collections was added in Version 7.0. Refer to the Couchbase Scopes and Collections page to learn more about these

    • Some of the examples in this guide use the cao tool, see the installation page here for more details.

    Setup two Couchbase clusters. One with the sample bucket travel-sample configured, and a second cluster to restore the configuration of the first one. See Load the Sample Dataset to learn how to load a sample bucket.

    This requires the disableBucketManagement configuration in Couchbase to be set to false.

    In the images below, the travel-sample bucket is loaded along with its scopes and collections in the first cluster.

    data topology prereq buckets
    data topology prereq scopes collections

    In the next section we will save the configuration of the travel-sample bucket.

    Save Configuration

    To save the configuration of a Couchbase Cluster, use the cao tool provided by Couchbase.

    cao save --couchbase-cluster cluster-name --filename save.yaml

    The command above allows a specific cluster to be probed, and all data topology resources saved, directly from the Couchbase cluster. Saved data topology represents data as Kubernetes native resource types, and can later be used to restore data topology, allow it to be managed by the Operator, or even replicated to a completely new cluster.

    Here is an example YAML file generated by the above command:

    ---
apiVersion: couchbase.com/v2
kind: CouchbaseBucket
metadata:
  creationTimestamp: null
  name: bucket-27496aea-80f2-4a5a-aa24-a2329b036bbc
spec:
  compressionMode: passive
  conflictResolution: seqno
  evictionPolicy: valueOnly
  ioPriority: low
  maxTTL: 0s
  memoryQuota: 200Mi
  minimumDurability: none
  name: travel-sample
  replicas: 1
  scopes:
    managed: true
    resources:
    - kind: CouchbaseScope
      name: scope-702aed74-1426-49d1-b95d-b9b3e7707bfe
    - kind: CouchbaseScope
      name: scope-9a3bff9b-0729-4104-9a0c-990213693540
    - kind: CouchbaseScope
      name: scope-99429731-bdfd-42f2-8d22-7f66cbe04f8c
    - kind: CouchbaseScope
      name: scope-82867883-fa7b-4a1b-99e7-9895f0ff7b96
    - kind: CouchbaseScope
      name: scope-acd9b010-ba1a-44d4-a075-f1761ff9e413
    - kind: CouchbaseScope
      name: scope-68665dce-b85a-402e-a55d-fa13e3e7890a
    - kind: CouchbaseScope
      name: scope-9d41b1dd-1789-488f-8acf-d72719ac3647
---
apiVersion: couchbase.com/v2
kind: CouchbaseScope
metadata:
  creationTimestamp: null
  name: scope-702aed74-1426-49d1-b95d-b9b3e7707bfe
spec:
  collections:
    managed: true
    resources:
    - kind: CouchbaseCollection
      name: collection-7ea0d93a-3f1a-4f93-acdc-0ffa00400357
    - kind: CouchbaseCollection
      name: collection-058ec3ce-e9cb-4830-981f-5093989ea52d
    - kind: CouchbaseCollection
      name: collection-12ac4726-1df3-44e6-a930-1bac2dc43273
    - kind: CouchbaseCollection
      name: collection-d1059a0e-05ec-4788-89df-00efcc1d1a9e
    - kind: CouchbaseCollection
      name: collection-1c51f7fc-240d-45ca-9c4c-57eb8f40bd08
  name: inventory
---
apiVersion: couchbase.com/v2
kind: CouchbaseCollection
metadata:
  creationTimestamp: null
  name: collection-7ea0d93a-3f1a-4f93-acdc-0ffa00400357
spec:
  maxTTL: 0s
  name: route
---
apiVersion: couchbase.com/v2
kind: CouchbaseCollection
metadata:
  creationTimestamp: null
  name: collection-058ec3ce-e9cb-4830-981f-5093989ea52d
spec:
  maxTTL: 0s
  name: landmark
---
apiVersion: couchbase.com/v2
kind: CouchbaseCollection
metadata:
  creationTimestamp: null
  name: collection-12ac4726-1df3-44e6-a930-1bac2dc43273
spec:
  maxTTL: 0s
  name: hotel
---
apiVersion: couchbase.com/v2
kind: CouchbaseCollection
metadata:
  creationTimestamp: null
  name: collection-d1059a0e-05ec-4788-89df-00efcc1d1a9e
spec:
  maxTTL: 0s
  name: airport
---
apiVersion: couchbase.com/v2
kind: CouchbaseCollection
metadata:
  creationTimestamp: null
  name: collection-1c51f7fc-240d-45ca-9c4c-57eb8f40bd08
spec:
  maxTTL: 0s
  name: airline
---
apiVersion: couchbase.com/v2
kind: CouchbaseScope
metadata:
  creationTimestamp: null
  name: scope-9a3bff9b-0729-4104-9a0c-990213693540
spec:
  collections:
    managed: true
    resources:
    - kind: CouchbaseCollection
      name: collection-43653400-44c1-4917-96fd-a4a226fe48c3
    - kind: CouchbaseCollection
      name: collection-0cdf40ae-1021-4755-8ba9-70af7d6cd6a3
  name: tenant_agent_00
---
apiVersion: couchbase.com/v2
kind: CouchbaseCollection
metadata:
  creationTimestamp: null
  name: collection-43653400-44c1-4917-96fd-a4a226fe48c3
spec:
  maxTTL: 0s
  name: users
---
apiVersion: couchbase.com/v2
kind: CouchbaseCollection
metadata:
  creationTimestamp: null
  name: collection-0cdf40ae-1021-4755-8ba9-70af7d6cd6a3
spec:
  maxTTL: 0s
  name: bookings
---
apiVersion: couchbase.com/v2
kind: CouchbaseScope
metadata:
  creationTimestamp: null
  name: scope-99429731-bdfd-42f2-8d22-7f66cbe04f8c
spec:
  collections:
    managed: true
    resources:
    - kind: CouchbaseCollection
      name: collection-5ffe2f10-0938-4a6d-b10d-c32fdc46e7c3
    - kind: CouchbaseCollection
      name: collection-ef2d2b08-3f8e-42d9-b53d-dbe10c25a550
  name: tenant_agent_01
---
apiVersion: couchbase.com/v2
kind: CouchbaseCollection
metadata:
  creationTimestamp: null
  name: collection-5ffe2f10-0938-4a6d-b10d-c32fdc46e7c3
spec:
  maxTTL: 0s
  name: users
---
apiVersion: couchbase.com/v2
kind: CouchbaseCollection
metadata:
  creationTimestamp: null
  name: collection-ef2d2b08-3f8e-42d9-b53d-dbe10c25a550
spec:
  maxTTL: 0s
  name: bookings
---
apiVersion: couchbase.com/v2
kind: CouchbaseScope
metadata:
  creationTimestamp: null
  name: scope-82867883-fa7b-4a1b-99e7-9895f0ff7b96
spec:
  collections:
    managed: true
    resources:
    - kind: CouchbaseCollection
      name: collection-b630cfb5-b14a-432a-8a42-0630998b7172
    - kind: CouchbaseCollection
      name: collection-0c565252-14a8-450e-80e9-345b21317c93
  name: tenant_agent_02
---
apiVersion: couchbase.com/v2
kind: CouchbaseCollection
metadata:
  creationTimestamp: null
  name: collection-b630cfb5-b14a-432a-8a42-0630998b7172
spec:
  maxTTL: 0s
  name: users
---
apiVersion: couchbase.com/v2
kind: CouchbaseCollection
metadata:
  creationTimestamp: null
  name: collection-0c565252-14a8-450e-80e9-345b21317c93
spec:
  maxTTL: 0s
  name: bookings
---
apiVersion: couchbase.com/v2
kind: CouchbaseScope
metadata:
  creationTimestamp: null
  name: scope-acd9b010-ba1a-44d4-a075-f1761ff9e413
spec:
  collections:
    managed: true
    resources:
    - kind: CouchbaseCollection
      name: collection-19d05195-4020-44fa-bfb2-ea0ddedef885
    - kind: CouchbaseCollection
      name: collection-2fac624e-9acf-48ae-8cf5-fdf859fa8e63
  name: tenant_agent_03
---
apiVersion: couchbase.com/v2
kind: CouchbaseCollection
metadata:
  creationTimestamp: null
  name: collection-19d05195-4020-44fa-bfb2-ea0ddedef885
spec:
  maxTTL: 0s
  name: users
---
apiVersion: couchbase.com/v2
kind: CouchbaseCollection
metadata:
  creationTimestamp: null
  name: collection-2fac624e-9acf-48ae-8cf5-fdf859fa8e63
spec:
  maxTTL: 0s
  name: bookings
---
apiVersion: couchbase.com/v2
kind: CouchbaseScope
metadata:
  creationTimestamp: null
  name: scope-68665dce-b85a-402e-a55d-fa13e3e7890a
spec:
  collections:
    managed: true
    resources:
    - kind: CouchbaseCollection
      name: collection-e98d2b17-166e-4412-914e-72dff6b67fdb
    - kind: CouchbaseCollection
      name: collection-c3ac4637-2531-4ca8-bd8e-47e71017d5f6
  name: tenant_agent_04
---
apiVersion: couchbase.com/v2
kind: CouchbaseCollection
metadata:
  creationTimestamp: null
  name: collection-e98d2b17-166e-4412-914e-72dff6b67fdb
spec:
  maxTTL: 0s
  name: users
---
apiVersion: couchbase.com/v2
kind: CouchbaseCollection
metadata:
  creationTimestamp: null
  name: collection-c3ac4637-2531-4ca8-bd8e-47e71017d5f6
spec:
  maxTTL: 0s
  name: bookings
---
apiVersion: couchbase.com/v2
kind: CouchbaseScope
metadata:
  creationTimestamp: null
  name: scope-9d41b1dd-1789-488f-8acf-d72719ac3647
spec:
  collections:
    managed: true
    preserveDefaultCollection: true
  defaultScope: true

    Restore Configuration

    From the previous step, we now have a Couchbase cluster named cb-example with travel-sample bucket loaded. To create an exact mirror copy of this cluster, we will use the file created by the cao save command, and perform the restore on a different Couchbase cluster.

    In the image below, we can see that there are no buckets in the second cluster.

    data topology restore configuration

    To perform the restore, we’ll use the cao restore command. This command allows existing save data (as generated by cao save) to be applied to the selected cluster.

    As a precaution, the tool will only function if your cluster’s buckets are unmanaged, there is no label selector set and there are no existing resources, or a label selector is already in use.

    cao restore -f save.yaml
    Output
    Data topology solution:

  /
  └── travel-sample (bucket) create
      ├── _default (scope) create
      ├── inventory (scope) create
      │   ├── airline (collection) create
      │   ├── airport (collection) create
      │   ├── hotel (collection) create
      │   ├── landmark (collection) create
      │   └── route (collection) create
      ├── tenant_agent_00 (scope) create
      │   ├── bookings (collection) create
      │   └── users (collection) create
      ├── tenant_agent_01 (scope) create
      │   ├── bookings (collection) create
      │   └── users (collection) create
      ├── tenant_agent_02 (scope) create
      │   ├── bookings (collection) create
      │   └── users (collection) create
      ├── tenant_agent_03 (scope) create
      │   ├── bookings (collection) create
      │   └── users (collection) create
      └── tenant_agent_04 (scope) create
          ├── bookings (collection) create
          └── users (collection) create

WARNING! resources marked as delete may result in data loss.

OK to proceed? (y/N) y
couchbasebuckets/bucket-7ecdc1c1-66b6-4493-ba51-9dd57ccd2d14 created
couchbasescopes/scope-eec7b781-700b-4979-a915-2ef71858f60f created
couchbasescopes/scope-f9b3f83c-2d22-4a53-acd7-bb649dc5c4b0 created
couchbasescopes/scope-510ac404-95a0-4eed-8aa5-4a499ed40273 created
couchbasescopes/scope-333723a6-ce38-4b91-a18e-7b8dd57a82e3 created
couchbasescopes/scope-f4958b25-c579-4e31-b205-c29a47bdcc8d created
couchbasescopes/scope-9a78e23c-0c60-4cda-8757-d84183b75328 created
couchbasecollectiongroups/collectiongroup-ce4637f8-bd4f-427a-9b28-4d96a4f79510 created
couchbasecollectiongroups/collectiongroup-0f893d3e-8e1f-4b95-96cc-57690f70045c created
couchbasecollectiongroups/collectiongroup-20650eff-3499-43ad-9d73-905417f6b75e created
couchbasecollectiongroups/collectiongroup-6569eb8b-8900-46aa-8059-031f04865934 created
couchbasecollectiongroups/collectiongroup-cbc51cb7-b086-442d-a8a4-c0dd56763da3 created
couchbasecollectiongroups/collectiongroup-d25dc4d8-06d7-4664-8ae8-83ba08a9d1fd created
couchbasecluster/cb-example updated

    Once the configuration is restored, we can check the Couchbase UI of this cluster to verify.

    data topology restore buckets
    data topology restore scopes coll

    Further Reading