Skip to content

Commit

Permalink
Documenation updates for github.io documents (#28)
Browse files Browse the repository at this point in the history
* Minor cleanups, corrections, and clarifications of documentation.
  • Loading branch information
billkalter authored Sep 9, 2016
1 parent e356947 commit ce0a8cc
Show file tree
Hide file tree
Showing 15 changed files with 714 additions and 504 deletions.
78 changes: 58 additions & 20 deletions docs/_posts/2014-08-07-databus.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ Record provides two "intrinsic" properties on every object:
increases the version number by one. A Databus reader can compare version numbers to determine if events arrive
out-of-order. Version numbers should not be compared between data centers--due to weak consistency, objects in data
center A and B can both have the same version number but represent different content.

* `~signature` - A 128-bit hash of a sequence of updates (technically, a hash of the sequence of time UUIDs associated
with the updates). Every update results in a new object signature. There is no intrinsic ordering of signature
hash values--they can't be used to detect out-of-order events. They can be compared across data centers. If two
Expand Down Expand Up @@ -100,6 +99,13 @@ protected void initialize(Configuration configuration, Environment environment)
}
```

REST calls
----------

As with all parts of EmoDB the REST API requires [API keys]({{ site.baseurl }}/security) and the Java client includes these
in all requests automatically. For clarity the API key header is not included each REST example below, but in a
properly secured system you would need to add it to each request.

Subscription Management
-----------------------

Expand All @@ -115,16 +121,18 @@ HTTP:

Java:

void subscribe(String subscription, Condition tableFilter, Duration subscriptionTtl, Duration eventTtl);
```java
void subscribe(String subscription, Condition tableFilter, Duration subscriptionTtl, Duration eventTtl);
```

Request Body:

* `PUT` - The body of the request is optional. If specified, it must be a valid JSON condition string in the format
generated by `Condition.toString()`. See the [Conditions]({{ site.baseurl }}/deltas/#conditional)
section of the Delta documentation. From Java, use the [Conditions](https://github.com/bazaarvoice/emodb/blob/master/sor-api/src/main/java/com/bazaarvoice/emodb/sor/condition/Conditions.java)
class to create instances of `Condition`. The subscription will follow events on all tables for which the
condition evaluates to true. The condition is evaluated against the table template and with the `~table` intrinsic.
It is *not* evaluated against the specific content of the document being updated.
condition evaluates to true. The condition is evaluated against the table template and with the `~table` and
`~placement` intrinsics. It is *not* evaluated against the specific content of the document being updated.

Request HTTP Headers:

Expand All @@ -137,12 +145,11 @@ Request URL Parameters:
In general, applications should specify a TTL between 1 day and 1 week and should renew the subscription every few
hours. By specifying a TTL when a subscription is created, application developers and administrators are relieved
from the burden of cleaning up old subscriptions manually.

* `eventTtl` - optional - The number of seconds before any specific event followed by this subscription expires. The
default `ttl` is 86400 seconds, equal to 24 hours. In general, applications should specify a TTL between 1 day and
default `eventTtl` is 86400 seconds, equal to 24 hours. In general, applications should specify a TTL between 1 day and
1 week. The TTL should be long enough that the application will process events before they expire.

To subscribe to *all* tables in the System of Record, omit the condition from the body of the post, or pass the condition `alwaysTrue()':
To subscribe to *all* tables in the System of Record, omit the condition from the body of the post, or pass the condition `alwaysTrue()`:

$ curl -s -XPUT -H "Content-Type: application/x.json-condition" \
"http://localhost:8080/bus/1/demo-app" \
Expand Down Expand Up @@ -185,20 +192,44 @@ HTTP:

Java:

void unsubscribe(String subscription);
```java
void unsubscribe(String subscription);
```

### Count Events

Get an estimate of the number of unacknowledged events pending for a particular subscription. This is most useful for
debugging.
Get an estimate of the number of unacknowledged events pending for a particular subscription. This is most useful for debugging.

The estimated count may not take into account automatic de-duplication performed by the databus. For example, if a
single record was updated 10 times since the last poll the count may show 10 events but the next poll call may only
return a single document. Additionally, results are cached briefly after each call, so getting a subscription's size
seconds after getting the same subscription's size may not reflect events which have been added to the subscription
or acknowledged since the first call.

The time it takes to perform a full count of the subscription size scales linearly with the current size of the subscription.
For particularly large subscriptions this can result in long response times when getting a count. Frequently the caller
is only interested in an estimate up to a certain point. For example, an alert monitor may only take action if there are
more than 1,000 events and waiting for a complete total beyond 1,000 events provides little additional benefit. To support
this the caller can optionally pass in a limit parameter. This will perform a full count up to the limit then use a faster
heuristic approximation to count the remaining events. For example, with a limit of 1,000 a return value of 20,000 means
the count found 1,000 events and approximated there were an additional 19,000 events.

HTTP:

GET /bus/1/<subscription>/size
GET /bus/1/<subscription>/size?limit=1000

Java:

int getEventCount(String subscription);
```java
long getEventCount(String subscription);
long getEventCountUpTo(String subscription, long limit);

```

Request URL Parameters:

* `limit` - optional - If provided the count returned will heuristically approximate the number of events beyond the first limit events.

### Poll for Events

Expand All @@ -212,17 +243,17 @@ HTTP:

Java:

List<Event> poll(String subscription, Duration claimTtl, int limit);
```java
List<Event> poll(String subscription, Duration claimTtl, int limit);
```

Request URL Parameters:

* `subscription` - required - The name of the subscription to poll.

* `ttl` - optional - The number of seconds of the claim period. The default `ttl` is 30 seconds. Applications
should choose a time period long enough that they are confident they can process and acknowledge the returned
events before the claim expires. But if the claim period is too long, it may take a while for events to be
re-processed if an application dies while holding claims.

* `limit` - optional - The maximum number of events to return. The default `limit` is 10.

### Renew Claims
Expand All @@ -238,7 +269,9 @@ HTTP:

Java:

void renew(String subscription, Collection<String> eventKeys, Duration claimTtl);
```java
void renew(String subscription, Collection<String> eventKeys, Duration claimTtl);
```

Request HTTP Headers:

Expand All @@ -247,7 +280,8 @@ Request HTTP Headers:
### Acknowledge Claims

The application must acknowledge claims after processing them or else the Databus will assume the application died
and hand them out in future calls to poll.
and hand them out in future calls to poll. The exception is if the event is older that the subscription's `eventTtl`
then the event will no longer be available, event if it is never acknowledged.

HTTP:

Expand All @@ -257,7 +291,9 @@ HTTP:

Java:

void acknowledge(String subscription, Collection<String> eventKeys);
```java
void acknowledge(String subscription, Collection<String> eventKeys);
```

Request HTTP Headers:

Expand All @@ -277,9 +313,11 @@ HTTP:

Java:

String replayAsync(String subscription);
String replayAsyncSince(String subscription, Date since)
ReplaySubscriptionStatus getReplayStatus(String reference) // Check the status of Replay operation
```java
String replayAsync(String subscription);
String replayAsyncSince(String subscription, Date since)
ReplaySubscriptionStatus getReplayStatus(String reference) // Check the status of Replay operation
```

Request URL Parameters:

Expand Down
39 changes: 24 additions & 15 deletions docs/_posts/2014-08-07-quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ Quick Start

1. Download the [EmoDB binaries](https://github.com/bazaarvoice/emodb/releases)

2. Run the Emodb server locally. This will start zookeeper and cassandra locally.
2. Run the EmoDB server locally. This will start ZooKeeper and Cassandra locally.

$ bin/start-local.sh
...
INFO [2012-05-14 19:12:19,802] org.eclipse.jetty.server.AbstractConnector: Started [email protected]:8080
Expand All @@ -30,20 +30,29 @@ Quick Start
pong

$ curl -s "http://localhost:8081/healthcheck"
* deadlocks: OK
* emo-cassandra: OK
127.0.0.1(127.0.0.1):9160 879us
{"deadlocks":{"healthy":true},"ugc_global-cassandra":{"healthy":true,"message":"127.0.0.1(127.0.0.1):9160 124us"},...}

4. To erase the EmoDB data, simply delete the data folder:

$ rm -rf bin/data/
$ bin/start-local.sh
{:.workflow}


### API keys

EmoDB's REST API requires [API keys]({{ site.baseurl }}/security). For clarity the API key header is not included each
example below, but in a properly secured system you would need to add it to each request.

For the purposes of a quick tutorial you can use the default administrator password when using `start-local.sh`, "local_admin",
by adding the following parameters to each `curl` command in the tutorial:

-H "X-BV-API-Key: local_admin"

### Quick Tutorial

The following examples assume you have [jq](https://stedolan.github.io/jq/) or an equivalent (see Recommended
Software below). It is optional-- `jq .` just formats the JSON responses to make them easier to read.
The following examples assume you have [jq](https://stedolan.github.io/jq/) or an equivalent (see
[Recommended Software](#recommended-software) below). It is optional-- `jq .` just formats the JSON responses to make them easier to read.

1. Create a table in the System of Record. Specify a "table template" with properties that will be returned with
every object in the table:
Expand Down Expand Up @@ -157,8 +166,9 @@ Software below). It is optional-- `jq .` just formats the JSON responses to mak
"success": true
}

10. Look at the timeline showing all System of Record changes to the object (modulo compaction):


10. Look at the timeline showing all System of Record changes to the object (modulo compaction):

$ curl -s "http://localhost:8080/sor/1/review:testcustomer/demo1/timeline?audit=true" | jq .
[
{
Expand All @@ -182,24 +192,23 @@ Software below). It is optional-- `jq .` just formats the JSON responses to mak
}
}
]


{:.workflow}

Recommended Software
--------------------

For debugging, it's useful to have a JSON pretty printer. On a Mac with [Homebrew] (http://mxcl.github.com/homebrew/)
For debugging it's useful to have a JSON pretty printer. On a Mac with [Homebrew](http://brew.sh/)
installed:

brew install jq

Alternatively, you can use jsonpp
Alternatively, you can use `jsonpp`

brew install jsonpp

Alternatively, use Python's `json.tool`:

alias jsonpp='python -mjson.tool'

Many of the examples include `jq` or `jsonpp`. Running the examples without `jsonpp` will work just fine, but the results may
be more difficult to read.
Many of the examples include `jq` or `jsonpp`. Running the examples without pretty printing will work just fine, but
the results may be more difficult to read.
Loading

0 comments on commit ce0a8cc

Please sign in to comment.