Skip to content

Commit

Permalink
2.0 (#23)
Browse files Browse the repository at this point in the history
* Move clients into their own repository

* Introduce INF/NAN string values normalizer (#19)

* Introduce INF/NAN string values normalizer

* Introduce INF/NAN string values normalizer

* Update CHANGELOG

* Fix CS

Co-authored-by: Andrew DalPino <[email protected]>

* Rename convert constants middleware

* Refactor convert sample constants middleware

* Upgrade ML to 2.0 (#20)

* Appease Stan

---------

Co-authored-by: Alex Torchenko <[email protected]>
  • Loading branch information
andrewdalpino and torchello authored Sep 19, 2023
1 parent 852dc1a commit 2a45e36
Show file tree
Hide file tree
Showing 50 changed files with 237 additions and 994 deletions.
6 changes: 5 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
- 2.0.0
- Move clients into their own repository
- INF and NAN string sample values are automatically converted to respective PHP constants

1.0.1
- Do not use deprecated ReactPHP class names

1.0.0
- 1.0.0
- Add pan and zoom to dashboard charts
- Rename anomaly scores HTTP resource

Expand Down
184 changes: 11 additions & 173 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Rubix Server
Rubix Server is a library for bringing your trained [Rubix ML](https://github.com/RubixML/ML) models into production. Inference servers are stand-alone services that run on your private or public network and wrap your trained estimator in an API that can be queried locally or over the network in real-time using standard protocols. In addition, the library provides async-compatible client implementations for making queries to the server from your PHP applications.
Rubix Server is a library for deploying your [Rubix ML](https://github.com/RubixML/ML) models to production. Our server wraps your trained estimator in an API that can be queried using standard protocols. Included is a real-time dashboard for monitoring the health and throughput of your models.

- **Optimized** for low latency predictions
- **Scalable** horizontally by adding more instances
- **Scale** by adding more instances
- **Monitoring** with real-time analytics dashboard
- **Robust** to common threats and failure modes

Expand All @@ -20,7 +20,7 @@ A [Docker Image](https://gitlab.com/torchello/rubix-ml-server-docker) is availab
- [PHP](https://php.net/manual/en/install.php) 7.4 or above

## Documentation
The latest documentation can be found below.
The latest documentation can be found in this README.

### Table of Contents
- [Servers](#servers)
Expand All @@ -30,13 +30,6 @@ The latest documentation can be found below.
- [Basic Authenticator](#basic-authenticator)
- [Shared Token Authenticator](#shared-token-authenticator)
- [Trusted Clients](#trusted-clients)
- [Clients](#clients)
- [REST Client](#rest-client)
- [Client Middleware](#client-middleware)
- [Backoff and Retry](#backoff-and-retry)
- [Basic Authenticator](#basic-authenticator-client-side)
- [Compress Request Body](#compress-request-body)
- [Shared Token Authenticator](#shared-token-authenticator-client-side)
- [Loggers](#loggers)
- [File](#file)
- [FAQs](#faqs)
Expand Down Expand Up @@ -124,9 +117,9 @@ Interfaces: [Server](#servers), [Verbose](#verbose-interface)

```php
use Rubix\Server\HTTPServer;
use Rubix\Server\HTTP\Middleware\Server\AccessLogGenerator;
use Rubix\Server\HTTP\Middleware\\AccessLogGenerator;
use Rubix\Server\Loggers\File;
use Rubix\Server\HTTP\Middleware\Server\BasicAuthenticator;
use Rubix\Server\HTTP\Middleware\\BasicAuthenticator;
use Rubix\Server\Services\Caches\InMemoryCache;

$server = new HTTPServer('127.0.0.1', 443, '/cert.pem', [
Expand Down Expand Up @@ -178,7 +171,7 @@ Generates an HTTP access log using a format similar to the Apache log format.
**Example**

```php
use Rubix\Server\HTTP\Middleware\Server\AccessLog;
use Rubix\Server\HTTP\Middleware\\AccessLog;
use Rubix\Server\Loggers\File;

$middleware = new AccessLog(new File('access.log'));
Expand All @@ -203,7 +196,7 @@ An implementation of HTTP Basic Auth as described in [RFC7617](https://tools.iet
**Example**

```php
use Rubix\Server\HTTP\Middleware\Server\BasicAuthenticator;
use Rubix\Server\HTTP\Middleware\\BasicAuthenticator;

$middleware = new BasicAuthenticator([
'morgan' => 'secret',
Expand All @@ -225,7 +218,7 @@ Authenticates incoming requests using a shared key that is kept secret between t
**Example**

```php
use Rubix\Server\HTTP\Middleware\Server\SharedTokenAuthenticator;
use Rubix\Server\HTTP\Middleware\\SharedTokenAuthenticator;

$middleware = new SharedTokenAuthenticator([
'secret', 'another-secret',
Expand All @@ -243,172 +236,17 @@ A whitelist of clients that can access the server - all other connections will b
**Example**

```php
use Rubix\Server\HTTP\Middleware\Server\TrustedClients;
use Rubix\Server\HTTP\Middleware\\TrustedClients;

$middleware = new TrustedClients([
'127.0.0.1', '192.168.4.1', '45.63.67.15',
]);
```

---
### Clients
Clients allow you to communicate directly with a model server using a friendly object-oriented interface inside your PHP applications. Under the hood, clients handle all the networking communication and content negotiation for you so you can write programs *as if* the model was directly accessible in your applications.

Return the predictions from the model:
```php
public predict(Dataset $dataset) : array
```

```php
use Rubix\Server\RESTClient;

$client = new RESTClient('127.0.0.1', 8080);

// Import a dataset

$predictions = $client->predict($dataset);
```

Calculate the joint probabilities of each sample in a dataset:
```php
public proba(Dataset $dataset) : array
```

Calculate the anomaly scores of each sample in a dataset:
```php
public score(Dataset $dataset) : array
```

### Async Clients
Clients that implement the Async Client interface have asynchronous versions of all the standard client methods. All asynchronous methods return a [Promises/A+](https://promisesaplus.com/) object that resolves to the return value of the response. Promises allow you to perform other work while the request is processing or to execute multiple requests in parallel. Calling the `wait()` method on the promise will block until the promise is resolved and return the value.

```php
public predictAsync(Dataset $dataset) : Promise
```

```php
$promise = $client->predictAsync($dataset);

// Do something else

$predictions = $promise->wait();
```

Return a promise for the probabilities predicted by the model:
```php
public probaAsync(Dataset $dataset) : Promise
```

Return a promise for the anomaly scores predicted by the model:
```php
public scoreAsync(Dataset $dataset) : Promise
```

### REST Client
The REST Client communicates with the [HTTP Server](#http-server) through the JSON REST API it exposes.

Interfaces: [Client](#clients), [AsyncClient](#async-clients)

#### Parameters
| # | Param | Default | Type | Description |
|---|---|---|---|---|
| 1 | host | '127.0.0.1' | string | The IP address or hostname of the server. |
| 2 | port | 8000 | int | The network port that the HTTP server is running on. |
| 3 | secure | false | bool | Should we use an encrypted HTTP channel (HTTPS)? |
| 4 | middlewares | | array | The stack of client middleware to run on each request/response. |
| 5 | timeout | | float | The number of seconds to wait before giving up on the request. |
| 6 | verify certificate | true | bool | Should we try to verify the server's TLS certificate? |

**Example**

```php
use Rubix\Server\RESTClient;
use Rubix\Server\HTTP\Middleware\Client\BasicAuthenticator;
use Rubix\Server\HTTP\Middleware\Client\CompressRequestBody;
use Rubix\Server\HTTP\Middleware\Client\BackoffAndRetry;
use Rubix\Server\HTTP\Encoders\Gzip;

$client = new RESTClient('127.0.0.1', 443, true, [
new BasicAuthenticator('user', 'password'),
new CompressRequestBody(new Gzip(1)),
new BackoffAndRetry(),
], 0.0, true);
```

### Client Middleware
Similarly to Server middleware, client middlewares are functions that hook into the request/response cycle but from the client end. Some of the server middlewares have accompanying client middleware such as [Basic Authenticator](#basic-authenticator) and [Shared Token Authenticator](#shared-token-authenticator).

### Backoff and Retry
The Backoff and Retry middleware handles Too Many Requests (429) and Service Unavailable (503) responses by retrying the request after waiting for a period of time to avoid overloading the server even further. An acceptable backoff period is gradually achieved by multiplicatively increasing the delay between retries.

#### Parameters
| # | Param | Default | Type | Description |
|---|---|---|---|---|
| 1 | max retries | 3 | int | The maximum number of times to retry the request before giving up. |
| 2 | initial delay | 0.5 | float | The number of seconds to delay between retries before exponential backoff is applied. |

**Example**

```php
use Rubix\Server\HTTP\Middleware\Client\BackoffAndRetry;

$middleware = new BackoffAndRetry(5, 0.5);
```

### Basic Authenticator (Client Side)
Adds the necessary authorization headers to the request using the Basic scheme.

#### Parameters
| # | Param | Default | Type | Description |
|---|---|---|---|---|
| 1 | username | | string | The user's name. |
| 2 | password | | string | The user's password. |

**Example**

```php
use Rubix\Server\HTTP\Middleware\Client\BasicAuthenticator;

$middleware = new BasicAuthenticator('morgan', 'secret');
```

### Compress Request Body
Apply the Gzip compression algorithm to the request body.

#### Parameters
| # | Param | Default | Type | Description |
|---|---|---|---|---|
| 1 | level | 1 | int | The compression level between 0 and 9 with 0 meaning no compression. |
| 2 | threshold | 65535 | int | The minimum size of the request body in bytes in order to be compressed. |

**Example**

```php
use Rubix\Server\HTTP\Middleware\Client\CompressRequestBody;

$middleware = new CompressRequestBody(5, 65535);
```

### Shared Token Authenticator (Client Side)
Adds the necessary authorization headers to the request using the Bearer scheme.

#### Parameters
| # | Param | Default | Type | Description |
|---|---|---|---|---|
| 1 | token | | string | The shared token to authenticate the request. |

**Example**

```php
use Rubix\Server\HTTP\Middleware\Client\SharedtokenAuthenticator;

$middleware = new SharedTokenAuthenticator('secret');
```

### Loggers
## Loggers
PSR-3 compatible loggers for capturing important server events.

#### File
### File
A simple append-only file logger.

#### Parameters
Expand Down
16 changes: 9 additions & 7 deletions composer.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "rubix/server",
"type": "library",
"description": "Serve your Rubix ML models in production with scalable stand-alone model servers.",
"description": "Deploy your Rubix ML models to production with scalable stand-alone inference servers.",
"homepage": "https://github.com/RubixML/Server",
"license": "MIT",
"readme": "README.md",
Expand All @@ -26,22 +26,21 @@
],
"require": {
"php": ">=7.4",
"guzzlehttp/guzzle": "^7.2",
"guzzlehttp/psr7": "^1.7",
"psr/container": "^1.1",
"psr/http-message": "^1.0",
"psr/log": "^1.1",
"react/http": "^1.1",
"rubix/ml": "^1.0",
"rubix/ml": "^2.0",
"symfony/polyfill-php80": "^1.17",
"webonyx/graphql-php": "^14.4"
},
"require-dev": {
"friendsofphp/php-cs-fixer": "^3.0",
"phpstan/phpstan": "0.12.*",
"phpstan/phpstan": "^1.0",
"phpstan/extension-installer": "^1.0",
"phpstan/phpstan-phpunit": "0.12.*",
"phpunit/phpunit": "8.5.*"
"phpstan/phpstan-phpunit": "^1.0",
"phpunit/phpunit": "^9.0"
},
"autoload": {
"psr-4": {
Expand Down Expand Up @@ -70,7 +69,10 @@
},
"config": {
"preferred-install": "dist",
"sort-packages": true
"sort-packages": true,
"allow-plugins": {
"phpstan/extension-installer": true
}
},
"funding": [
{
Expand Down
30 changes: 0 additions & 30 deletions examples/HTTP/client.php

This file was deleted.

4 changes: 2 additions & 2 deletions examples/HTTP/server.php
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@
use Rubix\ML\Datasets\Generators\Agglomerate;
use Rubix\ML\Classifiers\GaussianNB;
use Rubix\Server\HTTPServer;
use Rubix\Server\HTTP\Middleware\Server\AccessLogGenerator;
use Rubix\Server\HTTP\Middleware\Server\BasicAuthenticator;
use Rubix\Server\HTTP\Middleware\AccessLogGenerator;
use Rubix\Server\HTTP\Middleware\BasicAuthenticator;
use Rubix\Server\Services\Caches\InMemoryCache;
use Rubix\ML\Loggers\Screen;

Expand Down
1 change: 1 addition & 0 deletions phpstan.neon
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@ parameters:
paths:
- 'src'
- 'tests'
checkGenericClassInNonGenericObjectType: false
33 changes: 0 additions & 33 deletions src/AsyncClient.php

This file was deleted.

Loading

0 comments on commit 2a45e36

Please sign in to comment.