Skip to content

Commit

Permalink
Nuke protobuf and also JSON in favor of BSATN (#47)
Browse files Browse the repository at this point in the history
- Integrate protobufectomy, which replaces our protobuf and json WS message schemas
  with a SATS-ful schema for both BSATN and JSON.
- Notice that this SDK's implementation of JSON had diverged significantly
  from SpacetimeDB's use of `serde_json` leading to incompatible encodings.
  Rather than fixing this, remove JSON support in favor of only BSATN.
- Rewrite tests to use BSATN WS format.
- Also rewrite tests to be closer to legal in a few ways, but not entirely.
  E.g. don't put floats for `Point` fields with declared type `U16`.
- Hoist compression into a WS wrapper so that tests can bypass it.
- Update quickstart example for consistent filtering rules changes.
- Add `__identity_bytes`/`__address_bytes` getters to `Identity` and `Address`
  so serialization doesn't need to special-case them.
- Run prettier, which inflates this diff somewhat.
  If I had planned ahead I would have done this separately, but I didn't,
  and it's too late to separate the functional changes from the formatting now.

---------

Co-authored-by: Piotr Sarnacki <[email protected]>
  • Loading branch information
gefjon and drogus authored Jul 15, 2024
1 parent d95c0c3 commit 0e1ce88
Show file tree
Hide file tree
Showing 41 changed files with 5,058 additions and 6,338 deletions.
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
src/client_api/*.ts linguist-generated=true
examples/quickstart/client/src/module_bindings/*.ts linguist-generated=true
43 changes: 21 additions & 22 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,25 +9,24 @@ jobs:
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v2

- name: Set up Node.js
uses: actions/setup-node@v2
with:
node-version: 19

- name: Cache Yarn dependencies
uses: actions/cache@v2
with:
path: .yarn/cache
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-
- name: Install dependencies
run: yarn install --immutable

- name: Run tests
run: yarn test

- name: Checkout repository
uses: actions/checkout@v2

- name: Set up Node.js
uses: actions/setup-node@v2
with:
node-version: lts/*

- name: Cache Yarn dependencies
uses: actions/cache@v2
with:
path: .yarn/cache
key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-yarn-
- name: Install dependencies
run: yarn install --immutable

- name: Run tests
run: yarn test
33 changes: 7 additions & 26 deletions DEVELOP.md
Original file line number Diff line number Diff line change
@@ -1,31 +1,12 @@
# Notes for maintainers

The file `client_api.ts` is generated by [ts-proto](https://github.com/stephenh/ts-proto)
from [the SpacetimeDB client-api-messages proto definition](https://github.com/clockworklabs/SpacetimeDB/blob/master/crates/client-api-messages/protobuf/client_api.proto).
The directory `src/client_api` is generated from [the SpacetimeDB client-api-messages](https://github.com/clockworklabs/SpacetimeDB/tree/master/crates/client-api-messages).
This is not automated.
Whenever the `client_api.proto` changes, you'll have to manually re-run `protoc` to re-generate the definitions.
Whenever the `client-api-messages` crate changes, you'll have to manually re-generate the definitions.
See that crate's DEVELOP.md for how to do this.

```sh
cd spacetimedb-typescript-sdk
npm i # get the dev-dependencies
cd ~/path/to/SpacetimeDB/crates/client-api-messages/protobuf
protoc --plugin=/absolute/path/to/spacetimedb-typescript-sdk/node_modules/.bin/protoc-gen-ts_proto \
--ts_proto_out=/absolute/path/to/spacetimedb-typescript-sdk/src \
./client_api.proto
```
The generated files must be manually modified to fix their imports from the rest of the SDK.
Within each generated file:

Note that `protoc` cannot understand paths that start with `~`;
you must write the absolute path starting from `/`.

For reasons that escape me, `protoc-gen-ts` emits an incorrect import for `Long`.
After generating, you may have to manually edit `client_api.ts` by replacing:

```ts
import Long = require("long");
```

with:

```ts
import Long from "long";
```
- Change the import from `"@clockworklabs/spacetimedb-sdk"` to `"../index"`.
- If the type has generated a `class`, remove its `extends DatabaseTable`, remove the `public static db` member, and remove the call to `super()` within the constructor.
4 changes: 2 additions & 2 deletions examples/quickstart/client/src/App.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ function App() {
messages.sort((a, b) => (a.sent > b.sent ? 1 : a.sent < b.sent ? -1 : 0));

let messagesType: MessageType[] = messages.map((message) => {
let sender = User.filterByIdentity(message.sender);
let sender = User.findByIdentity(message.sender);
let name = sender ? userNameOrIdentity(sender) : "unknown";

return {
Expand All @@ -84,7 +84,7 @@ function App() {

client.current.on("initialStateSync", () => {
setAllMessagesInOrder();
var user = User.filterByIdentity(local_identity?.current!);
var user = User.findByIdentity(local_identity?.current!);
setName(userNameOrIdentity(user!));
});

Expand Down
22 changes: 8 additions & 14 deletions examples/quickstart/client/src/module_bindings/message.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 8 additions & 7 deletions examples/quickstart/client/src/module_bindings/user.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 0e1ce88

Please sign in to comment.