More fixes

.github/workflows/ci.yml

@@ -16,6 +16,13 @@ jobs:
       - name: Check out source code
         uses: actions/checkout@v2
 
+      - name: Check out FlowGo
+        uses: actions/checkout@v2
+        with:
+          repository: onflow/flow-go
+          ref: c0afa789365eb7a22713ed76b8de1e3efaf3a70a
+          path: flow-go
+
       - name: Install Go
         uses: actions/setup-go@v2
         with:
@@ -34,6 +41,41 @@ jobs:
       - name: Print Go version
         run: go version
 
+      # The protobuf steps uses the official instructions to install the
+      # pre-compiled binary, see:
+      # https://grpc.io/docs/protoc-installation/#install-pre-compiled-binaries-any-os
+      - name: Install Protobuf compiler
+        run: |
+          PB_REL="https://github.com/protocolbuffers/protobuf/releases"
+          curl -LO $PB_REL/download/v3.17.3/protoc-3.17.3-linux-x86_64.zip
+          unzip protoc-3.17.3-linux-x86_64.zip -d $HOME/.local
+          export PATH="$PATH:$HOME/.local/bin"
+          git clean -fd
+
+      # In order to be able to generate the protocol buffer and GRPC files, we
+      # need to install the related Go modules.
+      - name: Install Protobuf dependencies
+        run: |
+          go install google.golang.org/protobuf/cmd/[email protected]
+          go install google.golang.org/grpc/cmd/[email protected]
+          go install github.com/srikrsna/[email protected]
+
+      # Since building relic takes some time, we want to cache it.
+      - name: Cache Crypto package
+        uses: actions/cache@v2
+        with:
+          path: ./flow-go/crypto
+          key: ${{ runner.os }}-crypto
+          restore-keys: |
+            ${{ runner.os }}-crypto
+
+      # In order to be able to build with flow-go and the relic tag, we need to
+      # run its go generate target.
+      - name: Install Flow Go's crypto
+        run: |
+          cd ./flow-go/crypto
+          go generate .
+
       # This check makes sure that the `go.mod` and `go.sum` files for Go
       # modules are always up-to-date.
       - name: Verify Go modules

.github/workflows/release.yml

@@ -16,6 +16,13 @@ jobs:
         with:
           fetch-depth: 0
 
+      - name: Check out FlowGo
+        uses: actions/checkout@v2
+        with:
+          repository: onflow/flow-go
+          ref: c0afa789365eb7a22713ed76b8de1e3efaf3a70a
+          path: flow-go
+
       - name: Install Go
         uses: actions/setup-go@v2
         with:
@@ -34,6 +41,32 @@ jobs:
       - name: Print Go version
         run: go version
 
+      # The protobuf steps uses the official instructions to install the
+      # pre-compiled binary, see:
+      # https://grpc.io/docs/protoc-installation/#install-pre-compiled-binaries-any-os
+      - name: Install Protobuf compiler
+        run: |
+          PB_REL="https://github.com/protocolbuffers/protobuf/releases"
+          curl -LO $PB_REL/download/v3.17.3/protoc-3.17.3-linux-x86_64.zip
+          unzip protoc-3.17.3-linux-x86_64.zip -d $HOME/.local
+          export PATH="$PATH:$HOME/.local/bin"
+          git clean -fd
+
+      # In order to be able to generate the protocol buffer and GRPC files, we
+      # need to install the related Go modules.
+      - name: Install Protobuf dependencies
+        run: |
+          go install google.golang.org/protobuf/cmd/[email protected]
+          go install google.golang.org/grpc/cmd/[email protected]
+          go install github.com/srikrsna/[email protected]
+
+      # In order to be able to build with flow-go and the relic tag, we need to
+      # run its go generate target.
+      - name: Install Flow Go's crypto
+        run: |
+          cd ./flow-go/crypto
+          go generate .
+
       # This check makes sure that the `go.mod` and `go.sum` files for Go
       # modules are always up-to-date.
       - name: Verify Go modules

.gitignore

@@ -2,3 +2,4 @@
 *.cdc
 *.checkpoint
 *.log
+flow-go/

cmd/flow-rosetta-server/README.md

@@ -0,0 +1,51 @@
+# Flow Rosetta Server
+
+## Description
+
+The Flow Rosetta Server implements the Rosetta Data API specifications for the Flow network.
+It uses the Flow DPS Server's GRPC API as the backend to query the required data.
+Flow core contract addresses are derived from the chain ID with which the service is started.
+This allows the Rosetta API to access state remotely, or locally by running the Flow DPS Server on the same host.
+
+## Testing
+
+The Flow system model, where resources can be moved freely between accounts without generating events, makes it impossible to fully reconcile account balances on the Rosetta Data API.
+As a consequence, balance reconciliation has to be disabled when running the Rosetta CLI against the Flow Rosetta Server.
+
+Additionally, some of the events generated by Flow are not accurately reflecting the account address they relate to.
+This is also due to the same resource-based smart contract programming model.
+For instance, when an account receives vaults from different locations to execute a swap of tokens, the events related to this swap might indicate the swap
+contract's address, as it uses volatile vaults.
+
+Currently, the only way to work around this issue is to create exemptions for accounts which contain such smart contracts.
+As account balances using non-standard approaches to transfer Flow tokens can already not be reconciled, this is an acceptable limitation.
+In general, transactions by Rosetta should not be used to deduce account balances.
+Full historical account balance lookup is available and should thus be prefered to determine the account balance at any block height.
+
+The discussed configuration is available in the `flow.json` and `exemptions.json` files for the `mainnet-9` spork DPS.
+The following command can be executed to validate the Data API for that spork:
+
+```sh
+rosetta-cli check:data --configuration-file flow.json
+```
+
+## Usage
+
+```sh
+Usage of flow-rosetta-server:
+  -a, --api string              host URL for GRPC API endpoint (default "127.0.0.1:5005")
+  -e, --cache uint              maximum cache size for register reads in bytes (default 1073741824)
+  -l, --level string            log output level (default "info")
+  -p, --port uint16             port to host Rosetta API on (default 8080)
+  -t, --transaction-limit int   maximum amount of transactions to include in a block response (default 200)
+      --smart-status-codes      enable smart non-500 HTTP status codes for Rosetta API errors
+```
+
+## Example
+
+The following command line starts the Flow Rosetta server for a main network spork on port 8080.
+It uses a local instance of the Flow DPS Server for access to the execution state.
+
+```sh
+./flow-rosetta-server -a "127.0.0.1:5005" -p 8080
+```

cmd/flow-rosetta-server/exemptions.json

@@ -0,0 +1,20 @@
+[
+    {
+        "account_identifier": {
+            "address": "8624b52f9ddcd04a"
+        },
+        "currency": {
+            "symbol": "FLOW",
+            "decimals": 8
+        }
+    },
+    {
+        "account_identifier": {
+            "address": "c6c77b9f5c7a378f"
+        },
+	"currency": {
+            "symbol": "FLOW",
+            "decimals": 8
+        }
+    }
+]