Skip to content

Commit

Permalink
Add docs badge and doc.go file
Browse files Browse the repository at this point in the history
  • Loading branch information
Deliana Escobari committed May 31, 2018
1 parent 4396d9c commit dfc8501
Show file tree
Hide file tree
Showing 2 changed files with 131 additions and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Audit Star [![Build Status](https://travis-ci.org/enova/audit_star.svg?branch=master)](https://travis-ci.org/enova/audit_star)
# Audit Star [![Build Status](https://travis-ci.org/enova/audit_star.svg?branch=master)](https://travis-ci.org/enova/audit_star) [![Documentation](https://img.shields.io/badge/godoc-reference-5272B4.svg?style=flat-square)](https://godoc.org/github.com/enova/audit_star)

The purpose of audit_star is to maintain a history of all of the changes that happen to a database, and to store the history of those changes within the database itself. Additional documentation is provided by clicking the docs badge above.

Expand Down
130 changes: 130 additions & 0 deletions doc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
/*
Purpose
Audit_Star is a database building block that is intended to provide a standard, easy to use mechanism for capturing row-level data changes in your Postgres databases. It is to provide support for those scenarios where we do not already have another auditing mechanism in place (such as historian functionality or special application level events).
Benefits
The general benefits of auditing are well known. Aside from capturing a historical perspective of the data, it also provides a mechanism to assist in troubleshooting/debugging certain types of problems/issues (particularly those that can be tied to recent data changes). A secondary benefit includes providing a mechanism to communicate to ETL routines or downstream systems that a change (specifically a "delete") has taken place.
Some of the benefits that are specific to using this particular approach include the following:
- There is no maintenance required
- There is no custom code required
- The auditing mechanism never breaks, even when columns are added or removed from a table
- It is easy to apply/deploy
- It can be configured/customized (if needed) via a simple config file
How Does it Work?
This auditing mechanism involves (automatically) generating a number of database components for each table that is to be audited:
- Audit Table - Each "source" table will have a corresponding audit table (to hold the RECENT history of changes)
- Audit Trigger - Each source table will have a DML Trigger that propagates the data changes to its audit table
- Audit View - Each table will have a View that aids in querying/presenting the (JSON) data contained in the audit table
The schema of the audit tables do NOT match/mirror the source table they are auditing. Rather, the audit tables have a single column that holds a JSON representation of the entire "before" row, and a second column that holds a JSON representation of the diff between the "before" and "after" rows. The audit trigger handles creating the JSON representation of the changes, and inserting it into the audit table. The trigger also handle creating new audit table partitions as needed. The audit view converts the JSON back to a row-based representation of the data changes, so that the changes are easier to review. The view also stitches together certain data from the source table (as in the case of initial inserts).
Deployment
HStore Extension
The auditing solution is built on top of the HStore data type in Postgres. Hence, the HStore Extension must be installed in the database.
CREATE EXTENSION hstore with schema public;
Be sure that the extension is bubbled down to staging and development environments. Otherwise the migrations builds/tests will fail.
Run time parameters
The auditing solution uses PostgreSQL runtime parameters to pass metadata such as who made the change from the app to the audit system. The parameters currently used now are `audit_star.changed_by` and `audit_star.changed_reason`. These are defaults that must be set at the database level, usually to an empty string.
ALTER DATABASE <your-db-name> SET audit_star.changed_by TO '';
ALTER DATABASE <your-db-name> SET audit_star.change_reason TO '';
Be sure that the setting is bubbled down to staging and development environments. Otherwise the migrations builds/tests will fail.
Before using audit_star, database-specific configuration must be made to the ```audit.yml```
file. By default, audit_star will look in the current directory from which it is
being executed, but the optional parameter ```-cfg``` allows the user to provide
an alternative path to the ```audit.yml``` file.
The database-specific configuration along with accepted values are detailed in the
example file provided in `audit.yml` (copied below).
database config information
host: database host name
post: database port number; postgres default is 5432
db_name: database_name (name of the db to audit)
username: database username used to connect
password: databsase password used to connect
audit star config information
excluded_tables:
- exclude this_schema.this_table
excluded_schemas:
- exclude_this_schema
owner: app__owner (only audit tables owned by this user, if not specified will audit *every* table it can)
log_client_query: false (toggle logging of query that caused the change)
security: definer/invoker (security level of audit function - usually definer on release to avoid race conditions with defining permissions)
Installation
The preferred way to deploy Go binaries is using `go get` and `go install`.
Optionally, you can clone the repo manually into your $GOPATH and run `go build`.
Since audit_star is written in [Go](https://golang.org/), in order to build audit_star you must have Go
installed on your machine. Alternatively, you can build the binary on one machine
and then copy the binary to a machine, as needed, before invoking it, assuming the
machines share the same architecture. Go also supports [cross-compilation](https://dave.cheney.net/2012/09/08/an-introduction-to-cross-compilation-with-go)
which allows you to build the binary on a machine with one architecture and run
it on a machine with a completely different architecture.
Using `go get` (preferred)
You can download audit_star by running `go get github.com/enova/audit_star`.
This will download audit_star into your $GOPATH and install the binary into your $GOPATH/bin folder.
If your $GOPATH/bin folder is in your $PATH then you can run audit_star from anywhere by executing `audit_star`.
Building with ```go build```
After cloning the git repo, ```cd``` into the directory and run ```go build```. Assuming your Go is set up properly, this will create an ```audit_star``` binary in the current directly. You can then execute this binary by executing ```./audit_star```.
Testing
Audit Star tests use seprate configuration and db migration
Usage
Install pgmgr
go get github.com/rnubel/pgmgr
Resetting DB and Run migration
pgmgr db drop || true
pgmgr db create
pgmgr db migrate
Run tests
go test ./...
Note
Since the purpose of audit_star is to audit a database, in order to test this
functionality, the provided tests create a test database by running the migration
contained within the ```db/migrate``` folder. The migration was created and is
run using the [pgmgr Go migration tool](https://github.com/rnubel/pgmgr). So while
pgmgr is a dependency for the tests, it is not of dependency of audit_star itself.
*/
package main

/**/

0 comments on commit dfc8501

Please sign in to comment.