Skip to content

Commit

Permalink
Extract partial
Browse files Browse the repository at this point in the history
  • Loading branch information
henokgetachew committed Nov 27, 2024
1 parent c001af7 commit 61b9640
Show file tree
Hide file tree
Showing 3 changed files with 69 additions and 117 deletions.
60 changes: 60 additions & 0 deletions content/en/hosting/4.x/_partial_migration_3x_docker_to_4x_k3s.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
---
toc_hide: true
hide_summary: true
---

The hosting architecture differs entirely between CHT-Core 3.x and CHT-Core 4.x. When migrating from Docker Compose to K3s, specific steps are required using the [couchdb-migration](https://github.com/medic/couchdb-migration) tool. This tool interfaces with CouchDB to update shard maps and database metadata.

{{% alert title="Note" %}}
If after upgrading you get an error, `Cannot convert undefined or null to object` - please see [issue #8040](https://github.com/medic/cht-core/issues/8040) for a work around. This only affects CHT 4.0.0, 4.0.1, 4.1.0 and 4.1.1. It was fixed in CHT 4.2.0.
{{% /alert %}}


## Install Migration Tool
```shell
mkdir -p ~/couchdb-migration/
cd ~/couchdb-migration/
curl -s -o ./docker-compose.yml https://raw.githubusercontent.com/medic/couchdb-migration/main/docker-compose.yml
docker compose up
```

## Set Up Environment Variables
```shell
# Replace with your actual CouchDB URL from the Docker Compose setup
export COUCH_URL=http://<your-admin-user>:<your-admin-password>@<couchdb-host>:5984
```

## Run Pre-Migration Commands
```shell
cd ~/couchdb-migration/
docker compose run couch-migration pre-index-views 4.10.0
```

{{% alert title="Note" %}}
If pre-indexing is omitted, 4.x API will fail to respond to requests until all views are indexed. For large databases, this could take many hours or days.
{{% /alert %}}

## Save CouchDB Configuration
```shell
cd ~/couchdb-migration/
docker compose run couch-migration get-env
```

Save the output containing:
- CouchDB secret (used for encrypting passwords and session tokens)
- CouchDB server UUID (used for replication checkpointing)
- CouchDB admin credentials

The next part of the guide assumes your K3s cluster is already prepared. If not, please run the set of commands [here](https://docs.k3s.io/quick-start).

## Prepare Node Storage

```shell
# Create directory on the node
sudo mkdir -p /srv/couchdb1/data

# Copy data from Docker Compose installation to the k3s node
sudo rsync -avz --progress --partial --partial-dir=/tmp/rsync-partial \
/srv/storage/medic-core/couchdb/data/ \
<user>@<node1-hostname>:/srv/couchdb1/data/
```
63 changes: 5 additions & 58 deletions content/en/hosting/4.x/data-migration-3x-docker-to-4x-k3s-multi.md
Original file line number Diff line number Diff line change
@@ -1,71 +1,18 @@
---
title: "Migration from Docker Compose CHT 3.x to 3-Node Clustered CHT 4.x on K3s"
linkTitle: "Docker Compose to K3s Clustered Migration"
linkTitle: "Migration: 3.x Docker Compose to 4.x K3s (Multi-node)"
weight: 1
description: >
Guide to migrate existing data from CHT 3.x Docker Compose deployment to CHT 4.x clustered K3s deployment with 3 CouchDB nodes
relatedContent: >
---
{{ < read-content file="hosting/4.x/_partial_migration_3x_docker_to_4x_k3s.md" }}

The hosting architecture differs entirely between CHT-Core 3.x and CHT-Core 4.x. When migrating from Docker Compose to a clustered K3s deployment, specific steps are required using the [couchdb-migration](https://github.com/medic/couchdb-migration) tool. This tool interfaces with CouchDB to update shard maps and database metadata.

{{% alert title="Note" %}}
If after upgrading you get an error, `Cannot convert undefined or null to object` - please see [issue #8040](https://github.com/medic/cht-core/issues/8040) for a work around. This only affects CHT 4.0.0, 4.0.1, 4.1.0 and 4.1.1. It was fixed in CHT 4.2.0.
{{% /alert %}}


## Install Migration Tool
```shell
mkdir -p ~/couchdb-migration/
cd ~/couchdb-migration/
curl -s -o ./docker-compose.yml https://raw.githubusercontent.com/medic/couchdb-migration/main/docker-compose.yml
docker compose up
```

## Set Up Environment Variables
```shell
# Replace with your actual CouchDB URL from the Docker Compose setup
export COUCH_URL=http://<your-admin-user>:<your-admin-password>@<couchdb-host>:5984
```

## Run Pre-Migration Commands
```shell
cd ~/couchdb-migration/
docker compose run couch-migration pre-index-views 4.10.0
```

{{% alert title="Note" %}}
If pre-indexing is omitted, 4.x API will fail to respond to requests until all views are indexed. For large databases, this could take many hours or days.
{{% /alert %}}

## Save CouchDB Configuration
```shell
cd ~/couchdb-migration/
docker compose run couch-migration get-env
```

Save the output containing:
- CouchDB secret (used for encrypting passwords and session tokens)
- CouchDB server UUID (used for replication checkpointing)
- CouchDB admin credentials

The next part of the guide assumes your k3s cluster is already prepared. If not, please run the set of commands [here](https://docs.k3s.io/quick-start). Run k3s install command on the main node and run the k3s join commands on nodes 2 and 3.

## Prepare Node Storage
# Create directories on secondary nodes

```shell
# Create directory on the primary node.
# For production, make this a mounted disk instead of a directory on the root volume.
sudo mkdir -p /srv/couchdb1/data

# Copy data from Docker Compose installation
sudo rsync -avz --progress --partial --partial-dir=/tmp/rsync-partial \
/srv/storage/medic-core/couchdb/data/ \
/srv/couchdb1/data/

# Create directories on secondary nodes
ssh user@node2-hostname "sudo mkdir -p /srv/couchdb2/data/shards /srv/couchdb2/data/.shards"
ssh user@node3-hostname "sudo mkdir -p /srv/couchdb3/data/shards /srv/couchdb3/data/.shards"
ssh <user>@<node2-hostname> "sudo mkdir -p /srv/couchdb2/data/shards /srv/couchdb2/data/.shards"
ssh <user>@<node3-hostname> "sudo mkdir -p /srv/couchdb3/data/shards /srv/couchdb3/data/.shards"
```

## Create values.yaml for K3s Deployment
Expand Down
Original file line number Diff line number Diff line change
@@ -1,67 +1,12 @@
---
title: "Migration from Docker Compose CHT 3.x to Single-Node CHT 4.x on K3s"
linkTitle: "Docker Compose to K3s Single-Node Migration"
title: "Migration from 3.x Docker Compose to 4.x K3s (Single Node)"
linkTitle: "Migration: 3.x Docker Compose to 4.x K3s (Single-Node)"
weight: 1
description: >
Guide to migrate existing data from CHT 3.x Docker Compose deployment to CHT 4.x single-node K3s deployment
Guide on how to migrate existing data from CHT 3.x Docker Compose deployment to CHT 4.x single-node K3s deployment
relatedContent: >
---

The hosting architecture differs entirely between CHT-Core 3.x and CHT-Core 4.x. When migrating from Docker Compose to K3s, specific steps are required using the [couchdb-migration](https://github.com/medic/couchdb-migration) tool. This tool interfaces with CouchDB to update shard maps and database metadata.

{{% alert title="Note" %}}
If after upgrading you get an error, `Cannot convert undefined or null to object` - please see [issue #8040](https://github.com/medic/cht-core/issues/8040) for a work around. This only affects CHT 4.0.0, 4.0.1, 4.1.0 and 4.1.1. It was fixed in CHT 4.2.0.
{{% /alert %}}


## Install Migration Tool
```shell
mkdir -p ~/couchdb-migration/
cd ~/couchdb-migration/
curl -s -o ./docker-compose.yml https://raw.githubusercontent.com/medic/couchdb-migration/main/docker-compose.yml
docker compose up
```

## Set Up Environment Variables
```shell
# Replace with your actual CouchDB URL from the Docker Compose setup
export COUCH_URL=http://<your-admin-user>:<your-admin-password>@<couchdb-host>:5984
```

## Run Pre-Migration Commands
```shell
cd ~/couchdb-migration/
docker compose run couch-migration pre-index-views 4.10.0
```

{{% alert title="Note" %}}
If pre-indexing is omitted, 4.x API will fail to respond to requests until all views are indexed. For large databases, this could take many hours or days.
{{% /alert %}}

## Save CouchDB Configuration
```shell
cd ~/couchdb-migration/
docker compose run couch-migration get-env
```

Save the output containing:
- CouchDB secret (used for encrypting passwords and session tokens)
- CouchDB server UUID (used for replication checkpointing)
- CouchDB admin credentials

The next part of the guide assumes your K3s cluster is already prepared. If not, please run the set of commands [here](https://docs.k3s.io/quick-start).

## Prepare Node Storage

```shell
# Create directory on the node
sudo mkdir -p /srv/couchdb1/data

# Copy data from Docker Compose installation
sudo rsync -avz --progress --partial --partial-dir=/tmp/rsync-partial \
/srv/storage/medic-core/couchdb/data/ \
/srv/couchdb1/data/
```
{{ < read-content file="hosting/4.x/_partial_migration_3x_docker_to_4x_k3s.md" }}

## Create values.yaml for K3s Deployment
```yaml
Expand Down

0 comments on commit 61b9640

Please sign in to comment.