generated from subsquid-quests/quest-template
-
Notifications
You must be signed in to change notification settings - Fork 6
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
0 parents
commit cfd693c
Showing
18 changed files
with
7,103 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
.DS_Store |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,193 @@ | ||
<p align="center"> | ||
<picture> | ||
<source srcset="https://uploads-ssl.webflow.com/63b5a9958fccedcf67d716ac/64662df3a5a568fd99e3600c_Squid_Pose_1_White-transparent-slim%201.png" media="(prefers-color-scheme: dark)"> | ||
<img src="https://uploads-ssl.webflow.com/63b5a9958fccedcf67d716ac/64662df3a5a568fd99e3600c_Squid_Pose_1_White-transparent-slim%201.png" alt="Subsquid Logo"> | ||
</picture> | ||
</p> | ||
|
||
[](https://docs.subsquid.io/) | ||
[](https://discord.gg/subsquid) | ||
|
||
[Website](https://subsquid.io) | [Docs](https://docs.subsquid.io/) | [Discord](https://discord.gg/subsquid) | ||
|
||
# XXXXX Subgraph migration | ||
|
||
This quest is to migrate the [XXXXX subgraph](https://thegraph.com/explorer) to Squid SDK. The resulting squid should match the GraphQL API of the subgraph as close as possible, by migrating `schema.graphql`. The judges reserve the right to request improvements afther the initial review of the submission. Reach out to the [Discord Channel]( https://discord.com/channels/857105545135390731/1155812879770058783) for any tech questions regarding this quest. Use ```template``` squid as a starter. | ||
|
||
# Quest Info | ||
|
||
| Category | Skill Level | Time required (hours) | Max Participants | Reward | Status | | ||
| ---------------- | ------------------------------------- | --------------------- | ---------------- | ------------------------------------- | ------ | | ||
| Squid Deployment | $\textcolor{orange}{\textsf{Medium}}$ | ~60 | 5 | $\textcolor{red}{\textsf{3000tSQD}}$ | open | | ||
|
||
# Acceptance critera | ||
|
||
Ultimately, the solutions are accepted at the discretion of judges following a manual review. This sections is a rough guide that is in no way binding on our side. | ||
|
||
Some of the reasons why the solution will not be accepted include: | ||
- squid does not start | ||
- squid fails to sync fully due to internal errors | ||
- [batch handler filters](https://docs.subsquid.io/evm-indexing/configuration/caveats/) are not set up correctly (leads to a late sync failure in [RPC-ingesting](https://docs.subsquid.io/evm-indexing/evm-processor/#rpc-ingestion) squids) | ||
- data returned for any query is not consistent with subgraph data | ||
|
||
You may find [this tool](https://github.com/abernatskiy/compareGraphQL) to be useful for squid to subgraph API comparisons. | ||
|
||
It is desirable that your solution: | ||
- includes a suite of test GraphQL queries that touches every [schema entity](https://docs.subsquid.io/store/postgres/schema-file/entities/) and, if used, every [custom resolver](https://docs.subsquid.io/graphql-api/custom-resolvers/) at least once, with corresponding subgraph queries (listing in README is enough) | ||
- has high code quality (readability, simplicity, comments where necessary) | ||
- uses [batch processing](https://docs.subsquid.io/basics/batch-processing/) consistently | ||
- avoids any "sleeping bugs": logic errors that accidentally happen to not break the data | ||
- follows the standard squid startup procedure: | ||
``` | ||
git clone <repo_url> | ||
cd <repo_url> | ||
npm ci | ||
sqd up | ||
sqd process & | ||
sqd serve | ||
``` | ||
If it does not, describe your startup procedure in the README. | ||
|
||
Please test your solutions before submitting. We do allow some corrections, but judges' time is not limitless. | ||
|
||
To submit, invite the following github accounts to your private repo : [@dariaag](https://github.com/dariaag), [@belopash](https://github.com/belopash), [@abernatskiy](https://github.com/abernatskiy) and [@dzhelezov](https://github.com/dzhelezov). | ||
|
||
# Rewards | ||
|
||
tSQD rewards will be delivered via the [quests page](https://app.subsquid.io/quests) of Subsquid Cloud. Make sure you use the same GitHub handle to make a submission and when linking to that page. | ||
|
||
Winners will be listed at the quest repository README. If you do not wish to be listed please tell us that in an issue in your submission repo. | ||
|
||
# Useful links | ||
|
||
- [Quickstart](https://docs.subsquid.io/deploy-squid/quickstart/) | ||
- [TheGraph Migration guide](https://docs.subsquid.io/migrate/migrate-subgraph/) | ||
- [Cryptopunks Subgraph source code](https://github.com/itsjerryokolo/CryptoPunks) | ||
|
||
# Setup and Common errors | ||
|
||
1. Install Node v16.x or newer [https://nodejs.org/en/download](https://nodejs.org/en/download) | ||
2. Install Docker [https://docs.docker.com/engine/install/](https://docs.docker.com/engine/install/) | ||
3. Install git [https://git-scm.com/book/en/v2/Getting-Started-Installing-Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) | ||
4. Install Squid CLI | ||
|
||
```bash | ||
npm i -g @subsquid/cli@latest | ||
``` | ||
|
||
## How to run a squid: | ||
|
||
Full startup procedure for newly developed squids: | ||
|
||
1. Install dependecies: | ||
|
||
```bash | ||
npm ci | ||
``` | ||
|
||
2. Generate model | ||
|
||
```bash | ||
sqd codegen | ||
``` | ||
|
||
3. Generate types | ||
|
||
```bash | ||
sqd typegen | ||
``` | ||
|
||
4. Build the squid | ||
|
||
```bash | ||
sqd build | ||
``` | ||
|
||
5. Open docker and run: | ||
|
||
```bash | ||
sqd up | ||
``` | ||
|
||
6. Generate migrations: | ||
|
||
```bash | ||
sqd migration:generate | ||
``` | ||
|
||
7. Start processing: | ||
|
||
```bash | ||
sqd process | ||
``` | ||
|
||
8. Start a local GraphQL server in a separate terminal: | ||
|
||
```bash | ||
sqd serve | ||
``` | ||
|
||
Types (`./src/abi`), models (`./src/model`) and migrations ('./db') are typically kept within squid repos after they become stable. Then the startup procedure simplifies to | ||
```bash | ||
npm ci | ||
sqd up | ||
sqd process & | ||
sqd serve | ||
``` | ||
|
||
## Possible Errors: | ||
|
||
1. Docker not installed | ||
|
||
```bash | ||
X db Error × query-gateway Error | ||
Error response from daemon: Get "https://registry-1.docker.jo/v2/": uri ting to 127.0.0.1:8888: dial cp 127.0.0.1:8888: connectex: No connection | ||
``` | ||
|
||
2. Git not installed | ||
|
||
```bash | ||
Error: Error: spawn git ENOENT | ||
at ChildProcess._handle.onexit (node: internal/child_process: 284:19) | ||
at onErrorNT (node: internal/child_process:477:16) | ||
at process.processTicksAndRejections (node: internal/process/task_queues:82:21) | ||
``` | ||
|
||
3. Dependencies not installed. Run `npm ci` | ||
|
||
```bash | ||
sqd typegen | ||
TYPEGEN | ||
Error: spawn squid-evm-typegen ENOENT | ||
Code: ENOENT | ||
``` | ||
|
||
4. Rate-limiting. Get a private RPC endpoint from [any node provider](https://ethereumnodes.com), then change the `rpcUrl` in `processor.ts` | ||
|
||
```bash | ||
will pause new requests for 20000ms {"rpcUrl":"https://rpc.ankr.com/eth", | ||
"reason" : "HttpError: got 429 from https://rpc.ankr.com/eth"} | ||
``` | ||
If necessary, [rate limit your RPC queries](https://docs.subsquid.io/evm-indexing/configuration/initialization/#set-data-source). | ||
|
||
## Best practices: | ||
|
||
1. Batch saving | ||
```bash | ||
let transfers: Transfers[] = []; | ||
... | ||
ctx.store.save(transfers); | ||
``` | ||
|
||
2. Using map instead of array to avoid duplicate values | ||
```bash | ||
let transfers: Map<string, Transfer> = new Map(); | ||
... | ||
ctx.store.upsert([...transfers.values()]); | ||
``` | ||
3. Verify log addresses, not only topics. | ||
```bash | ||
if (log.topics[0] === erc721.events.Transfer.topic && log.address === CONTRACT_ADDRESS) { | ||
... | ||
} | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
DB_NAME=squid | ||
DB_PASS=squid | ||
DB_PORT=23798 | ||
PROCESSOR_PROMETHEUS_PORT=3000 | ||
GQL_PORT=4350 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
/node_modules | ||
/lib | ||
|
||
# IDE files | ||
/.idea | ||
pnpm-lock.yaml | ||
**/*.log |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
# ABI folder | ||
|
||
This is a dedicated folder for ABI files. Place you contract ABI here and generate facade classes for type-safe decoding of the event, function data and contract state queries with | ||
|
||
```sh | ||
sqd typegen | ||
``` | ||
|
||
This `typegen` command is defined in `commands.json`. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,82 @@ | ||
{ | ||
"$schema": "https://cdn.subsquid.io/schemas/commands.json", | ||
"commands": { | ||
"clean": { | ||
"description": "delete all build artifacts", | ||
"cmd": ["npx", "--yes", "rimraf", "lib"] | ||
}, | ||
"build": { | ||
"description": "Build the squid project", | ||
"deps": ["clean"], | ||
"cmd": ["tsc"] | ||
}, | ||
"up": { | ||
"description": "Start a PG database", | ||
"cmd": ["docker", "compose", "up", "-d"] | ||
}, | ||
"down": { | ||
"description": "Drop a PG database", | ||
"cmd": ["docker", "compose", "down"] | ||
}, | ||
"migration:apply": { | ||
"description": "Apply the DB migrations", | ||
"cmd": ["squid-typeorm-migration", "apply"] | ||
}, | ||
"migration:generate": { | ||
"description": "Generate a DB migration matching the TypeORM entities", | ||
"deps": ["migration:clean"], | ||
"cmd": ["squid-typeorm-migration", "generate"], | ||
}, | ||
"migration:clean": { | ||
"description": "Clean the migrations folder", | ||
"cmd": ["npx", "--yes", "rimraf", "./db/migrations/*.js"], | ||
}, | ||
"migration": { | ||
"cmd": ["squid-typeorm-migration", "generate"], | ||
"hidden": true | ||
}, | ||
"codegen": { | ||
"description": "Generate TypeORM entities from the schema file", | ||
"cmd": ["squid-typeorm-codegen"] | ||
}, | ||
"typegen": { | ||
"description": "Generate data access classes for an ABI file(s) in the ./abi folder", | ||
"cmd": ["squid-evm-typegen", "./src/abi", {"glob": "./abi/*.json"}, "--multicall"] | ||
}, | ||
"process": { | ||
"description": "Load .env and start the squid processor", | ||
"deps": ["migration:apply"], | ||
"cmd": ["node", "--require=dotenv/config", "lib/main.js"] | ||
}, | ||
"process:prod": { | ||
"description": "Start the squid processor", | ||
"cmd": ["node", "lib/main.js"], | ||
"hidden": true | ||
}, | ||
"serve": { | ||
"description": "Start the GraphQL API server", | ||
"cmd": ["squid-graphql-server"] | ||
}, | ||
"serve:prod": { | ||
"description": "Start the GraphQL API server with caching and limits", | ||
"cmd": ["squid-graphql-server", | ||
"--dumb-cache", "in-memory", | ||
"--dumb-cache-ttl", "1000", | ||
"--dumb-cache-size", "100", | ||
"--dumb-cache-max-age", "1000" ] | ||
}, | ||
"check-updates": { | ||
"cmd": ["npx", "--yes", "npm-check-updates", "--filter=/subsquid/", "--upgrade"], | ||
"hidden": true | ||
}, | ||
"bump": { | ||
"description": "Bump @subsquid packages to the latest versions", | ||
"deps": ["check-updates"], | ||
"cmd": ["npm", "i", "-f"] | ||
}, | ||
"open": { | ||
"description": "Open a local browser window", | ||
"cmd": ["npx", "--yes", "opener"] | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
version: "3" | ||
|
||
services: | ||
db: | ||
image: postgres:15 | ||
environment: | ||
POSTGRES_DB: squid | ||
POSTGRES_PASSWORD: squid | ||
ports: | ||
- "${DB_PORT}:5432" | ||
# command: ["postgres", "-c", "log_statement=all"] |
Oops, something went wrong.