Skip to content

Latest commit

 

History

History
292 lines (223 loc) · 10.2 KB

couch-setup.md

File metadata and controls

292 lines (223 loc) · 10.2 KB

CouchDB Setup Tools

Basic Setup

First, create a DatabaseSetup object to describe the database:

const productsSetup: DatabaseSetup = {
  name: 'products',
}

Next, when your application starts up, use setupDatabase to ensure the database exists and has the right settings:

const connection = nano(couchUri)
await setupDatabase(connection, productsSetup)

This will ensure that the CouchDB instance contains a database named "products".

If the database needs special CouchDB settings, pass those as DatabaseSetup.options:

const productSetup: DatabaseSetup = {
  name: 'products',
  options: { partitioned: true },
}

Uploading Design Documents

Many databases use design documents to manage indices. You can use the DatabaseSetup.documents property to automatically upload these documents to the database when they are missing or their contents don't match:

const productSetup: DatabaseSetup = {
  name: 'products',
  documents: {
    '_design/mango-upc': makeMangoIndex('upc', ['upc']),
  },
}

This example will create a document named "_design/mango-upc". The contents will be a design document generated by the makeMangoIndex helper function.

makeMangoIndex

Use this function to create Mango index design documents. The first parameter is the name of the view (CouchDB doesn't really use this anywhere), and the second parameter is an array of properties to index over, using CouchDB's "sort syntax".

makeMangoIndex('createdByDate', ['created', 'date'])

You can pass an optional filter or partitioned flag using the final argument. Here is an example that only matches certain documents in a partition, and then sorts those by the "date" property:

makeMangoIndex('newestToys', ['date'], {
  filter: { type: 'toy' },
  partitioned: true,
})

makeJsDesign

This helper function creates JavaScript design documents. It takes the name of the view (which becomes part of the URL), a factory function to build the view methods, and an optional settings object:

makeJsDesign(
  'justId',
  ({ emit }) => ({
    map: function (doc) {
      emit(doc._id.replace(/^.*:/, ''), null)
    },
  }),
  { partitioned: false }
)

This example uses JavaScript to edit the document ID, removing the partition part. The exact syntax is very important here - by destructuring the emit function in the factory function's parameter list, TypeScript believes that emit exists (inside CouchDB it's a global). Then, using the old-school function syntax ensures that calling map.toString() will generate valid JavaScript that CouchDB can understand.

Since CouchDB is old, the JavaScript code may require extra hacks to be valid. In that case, you can pass a fixJs function in the settings object, which can further edit the JavaScript string:

makeJsDesign(
  'difference',
  ({ emit }) => ({
    map: function (doc) {
      const difference = doc.high - doc.low
      if (difference > 0) emit(doc._id, difference)
    },
    reduce: '_sum',
  }),
  {
    fixJs: (code) => code.replace(/let|const/g, 'var'),
  }
)

This example uses the fixJs option to convert "const" into "var" for the legacy JavaScript engine.

If you need to access to utility functions, you can use CommonJS to inject them into the view:

import { normalizeIp, parseIp } from './ip-utils'

makeJsDesign(
  'ip',
  ({ emit, require }) => ({
    map: function (doc) {
      const normalizeIp = require('views/lib/normalizeIp')
      const parseIp = require('views/lib/parseIp')
      const ip = normalizeIp(parseIp(doc.ip))
      emit([ip], null)
    },
  }),
  {
    fixJs: (code) => code.replace(/let|const/g, 'var'),
    lib: { normalizeIp, parseIp },
  }
)

This example inserts the normalizeIp and parseIp utility functions into the view using the lib option. Next, the map function imports them using CouchDB's global require function.

These library functions need to be completely standalone, and must use CouchDB's oldschool JavaScript syntax. They do pass through the fixJs function, if provided, which can help with syntax problems.

Watching Settings Documents

While CouchDB is good at storing application data, it is also good at storing settings. Instead of using environment variables or JSON files on disk, putting settings inside CouchDB provides a nice admin interface for editing them, as well as automatic replication across the various servers.

To watch a settings document for changes, use the syncedDocument helper function:

const appSettings = syncedDocument('appSettings', asAppSettings)

The syncedDocument helper function takes two parameters - the name of the document and a cleaner to validate its contents.

This cleaner should be capable of handling errors gracefully, replacing invalid properties with defaults. Consider using asHealingObject to accomplish this. That way, if the document is missing or broken, syncedDocument can re-build it during the sync process.

The synced document will contain default values to start (returned by the cleaner). To synchronize the actual values from the database, pass the synced document to setupDatabase:

const settingsSetup: DatabaseSetup = {
  name: 'settings',
  syncedDocuments: [appSettings],
}

// At app boot time:
const connection = nano(couchUri)
await setupDatabase(connection, settingsSetup)

The setupDatabase will perform an initial sync, ensuring the document exists, and will then watch the document for any future changes, keeping it up to date. Simply access the appSettings.doc property to see the latest value, or subscribe to changes using the onChange method:

appSettings.onChange((newSettings) => console.log(newSettings))

You can also access the database's raw change feed by including an onChange method in the database setup object itself:

const usersSetup: DatabaseSetup = {
  name: 'users',
  onChange(change: CouchChange) {
    console.log(`Document ${change.id} updated`)
  },
}

Replication Setup

The setupDatabase function can automatically create documents in the _replicator database. To do this, it needs a synced replicator setup document, which describes how the different CouchDB clusters should communicate. Here is an example replicator setup:

{
  "_id": "replicators",
  "_rev": "12-6ee74490125b924e323807d9363d64a2",
  "clusters": {
    "production": {
      "url": "https://production.example.com:6984/",
      "basicAuth": "ZXhhbXBsZTpleGFtcGxl",
      "pushTo": ["logs", "backup"]
    },
    "logs": {
      "url": "https://logs.example.com:6984/",
      "basicAuth": "ZXhhbXBsZTpleGFtcGxl",
      "include": ["logs-*"]
    },
    "backup": {
      "url": "https://backup.example.com:6984/",
      "basicAuth": "ZXhhbXBsZTpleGFtcGxl"
    }
  }
}

This example has three clusters, named "production", "logs", and "backup". Each cluster has a URL, a set of credentials, and options describing what to replicate. You can generate the credential strings by opening a browser console and running:

btoa('username:password')

You can also turn the base64 back to plain text by using atob.

In this example, the "production" cluster has a pushTo list, which tells it to push changes out to the "logs" and "backup" clusters. The setupDatabase function will not create any replication documents on the other clusters, since they don't have pushTo or pullFrom properties.

Since the "logs" cluster has an include filter, the setupDatabase routine will only create and replicate databases with names that start with "logs-" on this cluster. Clusters can also specify an exclude filter to avoid creating specific databases, and a localOnly filter to create databases but not replicate them. Databases can also have tags, such as "#archived" or "#secret", which also apply to the replicator filters.

Enabling Replication

To enable replication, you first need to create a settings database and put a replicator settings document inside. That way, editing the document will live-update the replicators database, without needing to restart the app. The following code snippet will configure this:

import {
  asReplicatorSetupDocument,
  setupDatabase,
  syncedDocument,
} from 'edge-server-tools'

export const replicatorSetup = syncedDocument(
  'replicators',
  asReplicatorSetupDocument
)

const settingsSetup: DatabaseSetup = {
  name: 'settings',
  syncedDocuments: [replicatorSetup],
}

// At app boot time:
const connection = nano(couchUri)
await setupDatabase(connection, settingsSetup)

Finally, pass replicatorSetup and currentCluster options to the various setupDatabase calls in your app:

const commonOptions = {
  replicatorSetup,

  // This could be read from a JSON config file instead:
  currentCluster: process.env.HOSTNAME,
}

// At app boot time:
const connection = nano(couchUri)
await setupDatabase(connection, settingsSetup, commonOptions)
await setupDatabase(connection, productsSetup, commonOptions)
await setupDatabase(connection, usersSetup, commonOptions)

Background Tasks

When setting up replication or synced documents, setupDatabase will create long-running background processes. These can interfere with CLI tools, which would like to exit quickly when they are done with their work. To avoid this problem, pass a disableWatching flag to setupDatabase:

setupDatabase(connection, someSetup, {
  disableWatching: true,
})

The setupDatabase function also returns a cleanup function, which will stop all background work (eventually - it can take a while).

const cleanup = setupDatabase(connection, someSetup)

// Later, at shutdown time:
cleanup()

The database setup process can generate errors and messages, which go to the console by default, but can be intercepted by passing log and onError options:

setupDatabase(connection, someSetup, {
  log(message: string) {
    console.log(message)
  },
  onError(error: unknown) {
    console.error(error)
  },
})