Skip to content

Commit 2a4b1cc

Browse files
committed
Merge branch 'main' into feat/aave-use-case
2 parents a86c93c + d0f7616 commit 2a4b1cc

25 files changed

+527
-127
lines changed

docs/Protocol/Specifications/Snapshotter/snapshot-build.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -117,7 +117,7 @@ Aggregate and higher-order snapshots that build on base snapshots are configured
117117

118118
- [Single Project Composition](/docs/protocol/data-composition#single-project-composition)
119119
- [Multi-Project Composition](/docs/protocol/data-composition#multiple-projects-composition)
120-
- [Walkthrough of the Snapshotter Implementation for Uniswap V2 Dashboard](docs/category/tour-of-the-existing-implementation)
120+
- [Walkthrough of the Snapshotter Implementation for Uniswap V2 Dashboard](/docs/build-with-powerloom/use-cases/existing-implementations/uniswapv2-dashboard/)
121121
:::
122122

123123
The order and dependencies of these compositions are specified according to the `aggregate_on` key.

docs/Protocol/Specifications/Snapshotter/state-machine.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,4 +7,4 @@ sidebar_position: 4
77
Refer to the following sections for detailed information on the state transitions that a snapshotter participates in while processing data sources per epoch.
88

99
* [Epoch Processing State Transitions](/docs/protocol/specifications/epoch#state-transitions)
10-
* [Snapshotter Internal APIs](/docs/build-with-powerloom/snapshotter-node/health-tracking)
10+
* [Snapshotter Internal APIs](/docs/build-with-powerloom/snapshotter-node/core-api)

docs/Protocol/Specifications/protocol-state.md

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -27,11 +27,7 @@ Presently, the protocol state is implemented as a smart contract on an EVM-compa
2727

2828
### Chain and Contract Address
2929

30-
Depending on the network being joined from the [`deploy`](https://github.com/Powerloom/deploy/) repository, you can find the details of the network RPC as well as the deployed contract address from the example [`env.example`](https://github.com/PowerLoom/deploy/blob/eth_india/env.example) file.
31-
32-
```bash reference
33-
https://github.com/PowerLoom/deploy/blob/7708906fc606df2eff84642fe54d2c47bd9137ab/env.example#L10-L14
34-
```
30+
Depending on the node type, the network RPC and the deployed contract address can be found in the `env.example` file in either the [`deploy`](https://github.com/Powerloom/deploy/) (for Full Nodes) and [`snapshotter`](https://github.com/Powerloom/snapshotter-lite) repositories (for Lite Nodes).
3531

3632
### ABI
3733

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
---
2+
sidebar_position: 0
3+
---
4+
5+
# Introduction to Devnet
6+
7+
8+
:::success info
9+
#### Our Devnet is live!
10+
🚀 Get access to the Devnet and build exciting data applications, you can mint a no-cost NFT slot on the Sepolia network 🌟. Devnet mint page: https://devnet-mint.powerloom.dev
11+
12+
💡 To mint the NFT slot, ETH on the Sepolia network is required. If your balance is insufficient, you can obtain Sepolia ETH through the following link: https://faucet.quicknode.com/drip
13+
:::
14+
15+
Devnet provides developers with a sandbox environment to build and test data-driven applications. Developers can create powerful tools such as dashboards, arbitrage bots, and wallet trackers. The Devnet allows for experimentation and iteration without the risks associated with using real assets on a live network.
16+
17+
Start building data application with Powerloom by setting up your own Snapshotter node on our devnet.
18+
Our Devnet environment is designed to facilitate developers in:
19+
- expanding upon [our pre-existing use cases](../../category/existing-implementations)
20+
- innovating [new applications](../../category/building-a-new-use-case)
21+
22+
## Chains Supported
23+
24+
We currently support the following chains (Mainnet and Testnet). If you want to request support for any other chains, please open an issue [here](https://github.com/PowerLoom/deploy/issues).
25+
26+
| Chain | Mainnet | Testnet |
27+
|----------|---------|---------|
28+
| Ethereum || |
29+
| Sepolia | ||
30+
31+
---
32+
33+
## Next Steps
34+
To begin building your applications, head over to the [getting started](./getting-started.md) section.
35+
36+
37+
## Quick Links
38+
- [Protocol Overview](../../category/protocol-overview)
39+
- [Use Cases](../../category/use-cases)
40+
- [Discord Support](https://discord.com/invite/powerloom)
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"label": "Devnet",
3+
"position": 1,
4+
"link": {
5+
"type": "generated-index",
6+
"description": "Build robust Web3 Data applications with powerloom."
7+
}
8+
}
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
sidebar_position: 4
3+
---
4+
5+
# Frequently Asked Questions
Lines changed: 220 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,220 @@
1+
---
2+
sidebar_position: 1
3+
---
4+
5+
import Admonition from '@theme-original/Admonition';
6+
7+
# Getting Started
8+
This section will guide you through the necessary steps to set up and get started with the Devnet environment. By following these instructions, you'll be able to quickly configure your Devnet and start building data applications on Powerloom.
9+
10+
## Minting the Devnet Slot
11+
To get started with Devnet, you'll need to mint a no-cost NFT slot on the Sepolia network.
12+
13+
<Admonition type="tip" icon="📢" title="Ethereum Sepolia Faucet">
14+
To mint the NFT slot, ETH on the Sepolia network is required. If your balance is insufficient, you can obtain Sepolia ETH through the following link: https://faucet.quicknode.com/drip
15+
</Admonition>
16+
17+
Follow these steps:
18+
19+
1. Visit https://devnet-mint.powerloom.dev to begin the minting process. You will be prompted to connect your wallet that contains Sepolia ETH.
20+
21+
2. Click mint to being the minting process after connecting your wallet.
22+
23+
![Mint Dashboard](/images/devnet_mint_dashboard.png)
24+
25+
3. Once the NFT slot is minted, use one of the following options to locate your minted Slot ID for later use in the node setup.
26+
- Navigate to the [Devnet Dashboard](https://devnet-mint.powerloom.dev/dashboard) to see a list of your minted slots.
27+
28+
![Devnet Dashboard](/images/devnet_slots_dashboard.png)
29+
30+
- Search for the transaction hash of your NFT mint on the [Sepolia block explorer](https://sepolia.etherscan.io/).
31+
- For Metamask: The transaction will be under the `Activity` section.
32+
- For other wallets: Search for your wallet address in the Sepolia block explorer and the mint will be the latest transaction.
33+
- Locate the minted Token ID under the `ERC-1155 Tokens Transferred` section.
34+
35+
![Block Explorer](/images/sepolia_blockexplorer_example.png)
36+
37+
After completing these steps, you'll be ready to start using Devnet. The next step is to configure and run our snapshotter node.
38+
39+
## Setting up the Snapshotter Node
40+
A snapshotter peer, as part of the Powerloom Protocol, does exactly what the name suggests: It synchronizes with other snapshotter peers over a smart contract running on the Powerloom Prost chain.
41+
42+
Whether you are developing your own application or extending our existing use cases, setting up your Snapshotter node is a crucial step.
43+
44+
45+
:::tip
46+
Here are some important topics you may want to check out:
47+
- [Understanding Epoch](../../Protocol/Specifications/Epoch.md)
48+
- [What are Data markets](../../Protocol/data-sources.md)
49+
- [Snapshotter Node Architecture](../snapshotter-node/architecture.md)
50+
- [Data Composition](../../Protocol/data-composition.md)
51+
:::
52+
53+
### System Requirements
54+
55+
- **RAM**: At least 4 GB.
56+
- **CPU Core**: Minimum of 2 Cores
57+
- **Disk Space**: A minimum of 40 GB.
58+
- **Python**: Ensure Python 3.11 is installed.
59+
- **Docker**: Latest version of `docker` (`>= 20.10.21`) and `docker-compose` (`>= v2.13.0`). If your system doesn't have docker installed, please follow the guide [here](https://www.baeldung.com/ops/docker-install-windows-linux-mac).
60+
- **RPC URL** URL for `Ethereum mainnet` or the chain you're working on.
61+
62+
:::tip
63+
Your RPC usage depends on your specific use case. If your application requires a high volume of RPC calls, we recommend running your own RPC node to avoid potentially expensive third-party service costs.
64+
65+
If you want to monitor your RPC usage, we recommend signing up with a provider like Alchemy, Infura or Quicknode.
66+
:::
67+
68+
### Running the Snapshotter Node
69+
Follow the process outlined below to ensure a smooth setup
70+
71+
#### Step 1: Clone the Snapshotter Deploy Repository
72+
73+
Clone the repository against the respective branch (main by default). Open the terminal and run the below command to clone the deploy repo in a directory named `powerloom-deploy`.
74+
75+
```bash
76+
git clone https://github.com/PowerLoom/deploy.git --single-branch powerloom_deploy --branch devnet && cd powerloom_deploy
77+
```
78+
79+
#### Step 2: Fork the Snapshotter Computes and Snapshotter Config Repositories
80+
81+
Our system leverages the Git submodule architecture to seamlessly manage these components. To dive deeper into how these elements integrate and function within our larger system, check out the [architecture documentation](https://docs.powerloom.io/docs/build-with-powerloom/snapshotter-node/architecture).
82+
83+
To optimize the development process, we recommend forking the templates for:
84+
85+
- Snapshotter Configs: https://github.com/PowerLoom/snapshotter-configs
86+
- Snapshotter Computes: https://github.com/PowerLoom/snapshotter-computes
87+
88+
:::info
89+
When forking the repository, ensure that the *"Copy the main branch only" * option is left unchecked. This will allow you to clone all the branches associated with the repository, not just the main branch.
90+
:::
91+
92+
Once you have forked the above repositories, next step is to clone them on your local system.
93+
94+
```bash
95+
git clone https://github.com/<your_github_username>/snapshotter-computes
96+
```
97+
```bash
98+
git clone https://github.com/<your_github_username>/snapshotter-configs
99+
```
100+
101+
#### Step 3: Configure the environment variables
102+
103+
1. In the deploy repo's directory, create a new file named `.env`.
104+
105+
2. Copy the contents of [`env.example`](https://github.com/PowerLoom/deploy/blob/devnet/env.example) to `.env`.
106+
107+
3. Fill in the required variables in the `.env` file:
108+
109+
- `SOURCE_RPC_URL`: The URL for Source RPC (Local node/Infura/Alchemy) service.
110+
111+
- `SIGNER_ACCOUNT_ADDRESS`: The address of the signer account. This should be an unused "burner" address that does not need to have any token balance. You can create a new burner wallet from [our tool](https://devnet-mint.powerloom.dev/burner). During later steps, this address will be registered against the `SLOT_ID` on the `PROTOCOL_STATE_CONTRACT` that is set in this `.env`.
112+
- `SIGNER_ACCOUNT_PRIVATE_KEY`: The private key corresponding to the burner wallet addresss
113+
- `SNAPSHOT_CONFIG_REPO`: https://github.com/your-username/snapshotter-configs [Change the username to your GitHub profile to point to your forked snapshotter-configs repository.]
114+
- `SNAPSHOT_CONFIG_REPO_BRANCH`: Change to your own branch or use the `devnet` branch if you are getting started.
115+
- `SNAPSHOTTER_COMPUTE_REPO`: https://github.com/your-username/snapshotter-computes [Change the username to your GitHub profile to point to your forked snapshotter-computes repository.]
116+
- `SNAPSHOTTER_COMPUTE_REPO_BRANCH`: Change to your own branch or use the `devnet` branch if you are getting started.
117+
- `PROST_RPC_URL`: The URL for the Powerloom Protocol Chain RPC service.
118+
- `PROTOCOL_STATE_CONTRACT`: The contract address for the protocol state.
119+
120+
:::note
121+
Currently, there are two variations of Protocol State Contracts available. One where [Epoch](../../Protocol/Specifications/Epoch.md) size is set to 1 and other where epoch size is set to 10 blocks.
122+
123+
You can find the latest Epoch Contracts in the [example.env](https://github.com/PowerLoom/deploy/blob/devnet/env.example) file.
124+
125+
This should allow developers to build and experiment with a variety of use cases. If you have any custom needs, reach out to the [team](https://discord.com/invite/powerloom)
126+
:::
127+
128+
- `RELAYER_HOST`: The relayer URL for the Powerloom Protocol Chain.
129+
- `NAMESPACE`: The unique key used to identify your project namespace.
130+
- `POWERLOOM_REPORTING_URL`: The URL for reporting to PowerLoom.
131+
- `PROST_CHAIN_ID`: Enter the Devnet chain ID
132+
- `SLOT_ID`: Enter your Devnet Slot ID. [You can find the slotID by visiting the [Devnet Dashboard](https://devnet-mint.powerloom.dev/dashboard), or on the [Sepolia block explorer](https://sepolia.etherscan.io/) by searching for the transaction hash of your node mint]
133+
134+
Optional variables:
135+
136+
- `IPFS_URL`: The URL for the IPFS (InterPlanetary File System) service in HTTP(s) (e.g., `https://ipfs.infura.io:5001`) or multiaddr format (e.g., `/dns4/ipfs.infura.io/tcp/5001/https`).
137+
- `IPFS_API_KEY`: The API key for the IPFS service (if required).
138+
- `IPFS_API_SECRET`: The API secret for the IPFS service (if required).
139+
- `SLACK_REPORTING_URL`: The URL for reporting to Slack.
140+
141+
#### Step 4: Set Up the Codebase
142+
143+
Set up the codebase by running the `bootstrap.sh` command in the terminal:
144+
145+
```bash
146+
bash bootstrap.sh
147+
```
148+
149+
:::info
150+
This is a one-time step that resets the codebase to the latest version of the branch.
151+
After completing this one-time setup, you'll be ready to dive into the codebase and start building amazing data applications!
152+
:::
153+
154+
#### Step 5:
155+
156+
Install the required dependencies using pip:
157+
158+
```bash
159+
pip install -r requirements.txt
160+
```
161+
162+
#### Step 6: Run the Snapshotter Node
163+
164+
Run the command
165+
166+
```bash
167+
bash build.sh
168+
```
169+
170+
#### Step 7: Assign your Burner Wallet
171+
172+
```bash
173+
Do you want to assign a burner wallet to a slot? (yes/no):
174+
```
175+
176+
Enter yes and then enter your burner wallet address when prompted.
177+
- This is the `SIGNER_ACCOUNT_ADDRESS` that was set during [Step 3](#step-3-configure-the-environment-variables).
178+
179+
180+
Next, you will be prompted to enter a private key:
181+
182+
```bash
183+
To assign a burner wallet to a slot, you need to sign a message with the private key of the Account holding the slot.
184+
Private Key:
185+
```
186+
187+
- Please note that this is *not* asking for the private key to your burner wallet.
188+
- Enter the private key of the wallet [**used to mint** the devnet slot](#minting-the-devnet-slot).
189+
190+
### Troubleshooting Errors
191+
192+
If the `.env` file is filled up correctly, all services will execute one by one.
193+
194+
If you see the following error:
195+
196+
```bash
197+
powerloom_depoy-pooler-1 | Snapshotter identity check failed on protocol smart contract
198+
powerloom_depoy-pooler-1 exited with code 1
199+
```
200+
201+
Make sure your snapshotter address is registered. Ensure that you have minted the Devnet Slot from the [devnet dashboard](https://mint-devnet.powerloom.network) and your burner wallet has been added correctly during Step 6.
202+
203+
Refer to our [troubleshooting section](../../build-with-powerloom/snapshotter-node/full-node/troubleshooting.md) if you encounter any other issues with your node.
204+
205+
### Stopping the Node
206+
207+
1. To shutdown services, press `Ctrl+C` (and again to force).
208+
209+
2. If you encounter issues with data, you can do a clean **reset** by running the following command before restarting from [Step 5](#step-5-run-the-snapshotter-node):
210+
211+
```bash
212+
docker-compose --profile ipfs down --volumes
213+
```
214+
215+
---
216+
217+
## Quick Links
218+
- [Protocol Overview](../../category/protocol-overview)
219+
- [Use Cases](../../category/use-cases)
220+
- [Discord Support](https://discord.com/invite/powerloom)
Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
---
2+
sidebar_position: 3
3+
---
4+
5+
# Troubleshooting and Monitoring
6+
7+
8+
## Debugging:
9+
10+
If you are getting an error that says:
11+
12+
```bash
13+
Container "<container_id>" is unhealthy.
14+
ERROR: Encountered errors while bringing up the project.
15+
```
16+
17+
Check if all the necessary docker containers are up and running. You should see an output against `docker ps` command with the following containers listed:
18+
19+
```bash
20+
#docker ps
21+
22+
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
23+
bfa1abe2b8aa powerloom-pooler-frontend "sh -c 'sh snapshott…" 2 hours ago Up 2 hours (healthy) 0.0.0.0:3000->3000/tcp, :::3000->3000/tcp deploy-pooler-frontend-1
24+
852f3445f11c powerloom-pooler "bash -c 'sh init_pr…" 2 hours ago Up 2 hours (healthy) 0.0.0.0:8002->8002/tcp, :::8002->8002/tcp, 0.0.0.0:8555->8555/tcp, :::8555->8555/tcp deploy-pooler-1
25+
ee652fda8513 powerloom-audit-protocol "bash -c 'sh init_pr…" 2 hours ago Up 2 hours (healthy) 0.0.0.0:9000->9000/tcp, :::9000->9000/tcp, 0.0.0.0:9002->9002/tcp, :::9002->9002/tcp deploy-audit-protocol-1
26+
5547fb5c1ab4 ipfs/kubo:release "/sbin/tini -- /usr/…" 2 hours ago Up 2 hours (healthy) 4001/tcp, 8080-8081/tcp, 4001/udp, 0.0.0.0:5001->5001/tcp, :::5001->5001/tcp deploy-ipfs-1
27+
999de5864a1b rabbitmq:3-management "docker-entrypoint.s…" 2 hours ago Up 2 hours (healthy) 4369/tcp, 5671/tcp, 0.0.0.0:5672->5672/tcp, :::5672->5672/tcp, 15671/tcp, 15691-15692/tcp, 25672/tcp, 0.0.0.0:15672->15672/tcp, :::15672->15672/tcp deploy-rabbitmq-1
28+
2c14926d7cfd redis "docker-entrypoint.s…" 2 hours ago Up 2 hours (healthy) 0.0.0.0:6379->6379/tcp, :::6379->6379/tcp deploy-redis-1
29+
```
30+
31+
### Accessing the logs:
32+
33+
You can access the logs folders if you need to debug or monitor the services running on your Powerloom Snapshotter node. Logs can be found in `/logs` folder in the root directory of your node and the cloned `deploy` repository.
34+
35+
36+
These logs can provide valuable information for debugging or troubleshooting purposes. You can use them to identify any service errors or issues and take appropriate actions to resolve them.
37+
38+
Feel free to explore the logs folders and review the logs of different services to gain insights into the functioning of your Powerloom Snapshotter node.
39+
40+
If everything is functioning properly and you are able to check the status on the dashboard, it indicates that your node is working correctly.
41+
42+
43+
---
44+
45+
## Node Status Monitoring
46+
47+
To Check if your node is running actively, navigate to the [Devnet Dashboard](https://devnet-mint.powerloom.dev/dashboard) and you can check the status of your node. To refresh the status, click on the "Check Node Status" button.
48+
49+
![Devnet Node status](/images/node-status-dashboard-devnet.png)
50+
51+
52+
:::info
53+
If the issues still persists, you can reach out to us on the `#Developer Channel` on our [Discord Support](https://discord.com/channels/777248105636560948/1146936525544759457)
54+
:::

docs/build-with-powerloom/snapshotter-node/architecture.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ Configuration files, located in the `/config` directory and linked to [snapshott
4646
The heart of the system resides in the `snapshotter/modules` directory, linked to [snapshotter-computes](https://github.com/Powerloom/snapshotter-computes/), where the actual computation logic for each project type is defined. These modules drive the snapshot generation process for specific project types.
4747

4848
## Building Your Own Use Case
49-
Working on a new use case is as simple as writing a new compute module and adding a new configuration file. Instructions to get started in `build-dev.sh` mode are available in the [Deploy repo](https://github.com/Powerloom/deploy/tree/eth_india).
49+
Working on a new use case is as simple as writing a new compute module and adding a new configuration file. Instructions to get started in `build-dev.sh` mode are available in the [Deploy repo](https://github.com/Powerloom/deploy/tree/devnet).
5050

5151
# Useful Links
5252

docs/build-with-powerloom/snapshotter-node/full-node/getting-started.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ sidebar_position: 2
66
Snapshotter (Full Node) has a complex architecture with multiple moving parts. The easiest way to get started is by using the Docker-based setup, recommended for most users.
77

88
:::info
9-
#### You can use powerloom to build extensive applications. Our Devnet is coming soon.Please signup [Here](https://airtable.com/appr3wKRsn4VoOvpi/pagxxT6QcL0OwGH0e/form), and we'll reach out to you once it goes live!
9+
#### You can use powerloom to build extensive data applications. Our Devnet is live. Read more about Devnet [here](../../devnet/getting-started.md)
1010
:::
1111

1212
## System Requirements

0 commit comments

Comments
 (0)