From 8410389fba30ed2e739c3b8d4709bcdf4471ea04 Mon Sep 17 00:00:00 2001 From: Johnpaul Date: Sun, 30 Jun 2024 00:28:18 +0100 Subject: [PATCH 1/4] Fetching Data from Indexers and Integrating it to a frontend interface --- .../smart-contracts/guides/dapps/indexers.mdx | 240 ++++++++++++++++++ 1 file changed, 240 insertions(+) create mode 100644 docs/smart-contracts/guides/dapps/indexers.mdx diff --git a/docs/smart-contracts/guides/dapps/indexers.mdx b/docs/smart-contracts/guides/dapps/indexers.mdx new file mode 100644 index 000000000..32cc1601e --- /dev/null +++ b/docs/smart-contracts/guides/dapps/indexers.mdx @@ -0,0 +1,240 @@ +--- +title: Fetching Data from Indexers and Integrating it to a frontend interface +--- + +### What are Indexers? + +Indexers in the Stellar ecosystem are specialized services that continuously process and organize blockchain data, making it easily accessible and queryable. They play a crucial role in data management and retrieval by providing a more efficient way to access information compared to directly querying the blockchain. + +### Role of Indexers + +- **Data Organization**: Indexers structure blockchain data into easily searchable formats. +- **Quick Retrieval**: They allow for fast and efficient querying of historical data. +- **Reduced Load**: By using indexers, applications can reduce the load on the Stellar network. + +### Types of Data Provided by Indexers + +Stellar indexers can provide various types of data, including: + +- Account balances and history +- Transaction details +- Operation logs +- Trade history +- Asset information +- Ledger entries +- Events from contracts + +## 2. Setting up Data Retrieval + +### Connecting with an Indexer + +1. **Choose an Indexer**: Select a reliable Stellar indexer. Some popular options include: + - **[Mercury](https://mercurydata.app/)** + - **[SubQuery](https://subquery.network/)** + +A bigger list of supported indexers can be found in the [tools](/docs/tools/developer-tools#data-indexers) section + +2. **API Documentation**: Familiarize yourself with the chosen indexer's API documentation. + +3. **Authentication**: Some indexers may require API keys. Obtain necessary credentials if required. + +4. **Setup HTTP/GraphQL Client**: Use a library like Axios, Fetch API, or Apollo Client to make HTTP requests to the indexer. + +### Step-by-Step Instructions for Querying Data + +We are going to use the [Mercury](https://mercurydata.app/) indexer for this example to index a smart contract ([Events](https://github.com/stellar/soroban-examples/tree/main/events)) deployed at the address [CC6MWZMG2JPQEENRL7XVICAY5RNMHJ2OORMUHXKRDID6MNGXSSOJZLLF](https://stellar.expert/explorer/testnet/contract/CC6MWZMG2JPQEENRL7XVICAY5RNMHJ2OORMUHXKRDID6MNGXSSOJZLLF) on the soroban testnet. + +The goal is to query the `'increment'` event emitted by the contract and display them in a frontend interface. + +1. **Setup Account**: + + - Sign up for an account on Mercury: https://test.mercurydata.app/signup + - Create a new project and add your contract address to the subscriptions: https://docs.mercurydata.app/video-tutorials/index-and-query-contract-events + - Collect API key from the project settings + +2. **GraphQL Playground**: + + - Open the GraphQL playground: https://api.mercurydata.app:2083/graphiql + - Use the following query to fetch the event data: + + ```graphql + query MyQuery { + eventByTopic(t2: "AAAADwAAAAlpbmNyZW1lbnQAAAA=") { + nodes { + contractId + topic1 + topic2 + data + txInfoByTx { + ledgerByLedger { + closeTime + sequence + } + memo + txHash + opCount + fee + } + } + } + } + ``` + + ```json + // fill the Headers with the API key + { + "Authorization": "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyIjoiY2FyZC1pZ..." + } + ``` + +## 3. Frontend Integration + +### Install dependencies + +1. **Install the GraphQL client and stellar client**: + ```bash + npm install @apollo/client graphql @stellar/stellar-sdk + ``` + +### Processing and Preparing Data + +1. **Encode Event Names**: The event names need to be encoded to base64 xdr before querying the indexer. You can use the following code snippet to encode the event names + + ```javascript + const { xdr, scValToNative } = require("@stellar/stellar-sdk"); + const val = xdr.ScVal.scvSymbol("increment").toXDR("base64"); + console.log(val); + + // Output: AAAADwAAAAlpbmNyZW1lbnQAAAA= + ``` + +2. **Create a Parse Function**: The data returned from the indexer is in XDR format. You can use the following function to parse the XDR data: + ```js + function parseXdr(eventXdr) { + const parseddr = scValToNative(xdr.ScVal.fromXDR(eventXdr, "base64")); + return parseddr; + } + ``` +3. **Create a Connection Function**: + + ```js + import React from "react"; + import { + ApolloClient, + InMemoryCache, + ApolloProvider, + useQuery, + gql, + } from "@apollo/client"; + + // Create the Apollo Client + const client = new ApolloClient({ + uri: "https://api.mercurydata.app:2083/graphql", + cache: new InMemoryCache(), + headers: { + Authorization: "Bearer eyJhbGciOiJIUzI1NiIsInR5cC...", + }, + }); + ``` + +4. **Fetch Data**: Here's an example of fetching event data: + + ```javascript + // Define the query + const GET_EVENT_BY_TOPIC = gql` + query MyQuery { + eventByTopic(t2: "AAAADwAAAAlpbmNyZW1lbnQAAAA=") { + nodes { + contractId + topic1 + topic2 + topic3 + topic4 + txInfoByTx { + ledgerByLedger { + closeTime + sequence + } + memo + data + txHash + opCount + fee + } + } + } + } + `; + + // Create a component to fetch the data + function EventDataFetcher({ setEventData }) { + const { loading, error, data } = useQuery(GET_EVENT_BY_TOPIC); + + useEffect(() => { + if (loading) console.log("Loading data..."); + if (error) console.error("Error fetching data:", error); + if (data) { + console.log("Fetched data:", data); + setEventData( + data.eventByTopic.nodes.map((node) => { + return { + currentCount: parseXdr(node.data).currentCount, + updateTime: new Date( + node.txInfoByTx.ledgerByLedger.closeTime * 1000, + ).toLocaleString(), + }; + }), + ); + } + }, [loading, error, data, setEventData]); + + return null; + } + ``` + +5. **State Management**: Use a state management solution (e.g., Redux, MobX, or React Context) to store and manage the fetched data. + +6. **Error Handling**: Implement proper error handling to manage API request failures or data inconsistencies. + +### Displaying Data Effectively + +1. **Component Structure**: Create reusable components for different data types. + + ```jsx + // Wrap your app with ApolloProvider + function App() { + const [eventData, setEventData] = useState([]); + + return ( + +
+

Stellar Events Data Fetcher

+ + {eventData ? ( +
+

Fetched Events:

+ {eventData.map((event, index) => ( +
+

Current Count: {event.currentCount}

+

Update Time: {event.updateTime}

+
+ ))} +
+ ) : ( +

No data fetched yet

+ )} +
+ + ); + } + + export default App; + ``` + +2. **Security Concerns**: Ensure that sensitive data like API keys are stored securely and not exposed in the frontend code. Using a backend or serverless function to fetch data from the indexer can help protect sensitive information. + +3. **Optimization**: Implement pagination or lazy loading to manage large datasets efficiently. + +### Demo + +You can find a working demo of this frontend integration [here](https://stellar-indexed.web.app). and the complete code [here](https://github.com/Myestery/stellar-indexed) From 69fb9678afc4c42b0ad57f23c5c237f0f6e2952d Mon Sep 17 00:00:00 2001 From: Johnpaul Date: Wed, 10 Jul 2024 14:17:18 +0100 Subject: [PATCH 2/4] format list and place indentation --- docs/build/guides/dapps/indexers.mdx | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/build/guides/dapps/indexers.mdx b/docs/build/guides/dapps/indexers.mdx index 32cc1601e..30b479966 100644 --- a/docs/build/guides/dapps/indexers.mdx +++ b/docs/build/guides/dapps/indexers.mdx @@ -2,7 +2,7 @@ title: Fetching Data from Indexers and Integrating it to a frontend interface --- -### What are Indexers? +## 1. What are Indexers? Indexers in the Stellar ecosystem are specialized services that continuously process and organize blockchain data, making it easily accessible and queryable. They play a crucial role in data management and retrieval by providing a more efficient way to access information compared to directly querying the blockchain. @@ -29,10 +29,11 @@ Stellar indexers can provide various types of data, including: ### Connecting with an Indexer 1. **Choose an Indexer**: Select a reliable Stellar indexer. Some popular options include: + - **[Mercury](https://mercurydata.app/)** - **[SubQuery](https://subquery.network/)** -A bigger list of supported indexers can be found in the [tools](/docs/tools/developer-tools#data-indexers) section + A more comprehensive list of supported indexers can be found in the [tools](/docs/tools/developer-tools#data-indexers) section 2. **API Documentation**: Familiarize yourself with the chosen indexer's API documentation. From a33c23d5657104c1a404a0b0f3d92fa9808d0f70 Mon Sep 17 00:00:00 2001 From: Johnpaul Date: Thu, 11 Jul 2024 13:44:39 +0100 Subject: [PATCH 3/4] Edit docs to index a different type of data --- docs/build/guides/dapps/indexers.mdx | 449 ++++++++++++++++----------- 1 file changed, 272 insertions(+), 177 deletions(-) diff --git a/docs/build/guides/dapps/indexers.mdx b/docs/build/guides/dapps/indexers.mdx index 30b479966..21ad050dd 100644 --- a/docs/build/guides/dapps/indexers.mdx +++ b/docs/build/guides/dapps/indexers.mdx @@ -32,6 +32,7 @@ Stellar indexers can provide various types of data, including: - **[Mercury](https://mercurydata.app/)** - **[SubQuery](https://subquery.network/)** + - **[BlockEden](https://www.blockeden.xyz/)** A more comprehensive list of supported indexers can be found in the [tools](/docs/tools/developer-tools#data-indexers) section @@ -43,198 +44,292 @@ Stellar indexers can provide various types of data, including: ### Step-by-Step Instructions for Querying Data -We are going to use the [Mercury](https://mercurydata.app/) indexer for this example to index a smart contract ([Events](https://github.com/stellar/soroban-examples/tree/main/events)) deployed at the address [CC6MWZMG2JPQEENRL7XVICAY5RNMHJ2OORMUHXKRDID6MNGXSSOJZLLF](https://stellar.expert/explorer/testnet/contract/CC6MWZMG2JPQEENRL7XVICAY5RNMHJ2OORMUHXKRDID6MNGXSSOJZLLF) on the soroban testnet. +We are going to use the [BlockEden](https://www.blockeden.xyz/) indexer for this example to query token metadata for all tokens indexed on the indexer sorted by recent on the soroban testnet. -The goal is to query the `'increment'` event emitted by the contract and display them in a frontend interface. +The goal is to query these data and display them in a frontend interface. 1. **Setup Account**: - - Sign up for an account on Mercury: https://test.mercurydata.app/signup - - Create a new project and add your contract address to the subscriptions: https://docs.mercurydata.app/video-tutorials/index-and-query-contract-events - - Collect API key from the project settings + - Sign up for an account on BlockEden: https://blockeden.xyz/dash/sign-up + - Create a new API key: https://blockeden.xyz/dash . This key will be used to authenticate requests to the indexer in step 3. 2. **GraphQL Playground**: - - Open the GraphQL playground: https://api.mercurydata.app:2083/graphiql - - Use the following query to fetch the event data: - - ```graphql - query MyQuery { - eventByTopic(t2: "AAAADwAAAAlpbmNyZW1lbnQAAAA=") { - nodes { - contractId - topic1 - topic2 - data - txInfoByTx { - ledgerByLedger { - closeTime - sequence - } - memo - txHash - opCount - fee - } - } - } - } - ``` - - ```json - // fill the Headers with the API key - { - "Authorization": "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyIjoiY2FyZC1pZ..." - } - ``` +The graphQL playground has a user-friendly interface that allows you to test queries and explore the available data. It also has schema documentation that helps in understanding the structure of the data. We are interested in fetching the token metadata from the indexer so we will query that section. + +- Open the GraphQL playground: https://blockeden.xyz/api-marketplace/stellar-soroban-indexer +- Use the following query to fetch the token metadata: + +```graphql +query { + token_metadata(limit: 50, order_by: { updated_at: desc }) { + admin_address + name + contract_id + symbol + updated_at + decimal + } +} +``` + + This query fetches the token metadata for the latest 50 tokens indexed on the indexer. + +The data displayed on this page can be safely integrated into our frontend application ## 3. Frontend Integration +This example will be using Vuejs, but the same principles can be applied to other frontend frameworks like React, Angular, or Svelte. + +### Create a Vue application + +```bash + npm create vue@latest +``` + ### Install dependencies -1. **Install the GraphQL client and stellar client**: - ```bash - npm install @apollo/client graphql @stellar/stellar-sdk - ``` - -### Processing and Preparing Data - -1. **Encode Event Names**: The event names need to be encoded to base64 xdr before querying the indexer. You can use the following code snippet to encode the event names - - ```javascript - const { xdr, scValToNative } = require("@stellar/stellar-sdk"); - const val = xdr.ScVal.scvSymbol("increment").toXDR("base64"); - console.log(val); - - // Output: AAAADwAAAAlpbmNyZW1lbnQAAAA= - ``` - -2. **Create a Parse Function**: The data returned from the indexer is in XDR format. You can use the following function to parse the XDR data: - ```js - function parseXdr(eventXdr) { - const parseddr = scValToNative(xdr.ScVal.fromXDR(eventXdr, "base64")); - return parseddr; - } - ``` -3. **Create a Connection Function**: - - ```js - import React from "react"; - import { - ApolloClient, - InMemoryCache, - ApolloProvider, - useQuery, - gql, - } from "@apollo/client"; - - // Create the Apollo Client - const client = new ApolloClient({ - uri: "https://api.mercurydata.app:2083/graphql", - cache: new InMemoryCache(), - headers: { - Authorization: "Bearer eyJhbGciOiJIUzI1NiIsInR5cC...", - }, - }); - ``` - -4. **Fetch Data**: Here's an example of fetching event data: - - ```javascript - // Define the query - const GET_EVENT_BY_TOPIC = gql` - query MyQuery { - eventByTopic(t2: "AAAADwAAAAlpbmNyZW1lbnQAAAA=") { - nodes { - contractId - topic1 - topic2 - topic3 - topic4 - txInfoByTx { - ledgerByLedger { - closeTime - sequence - } - memo - data - txHash - opCount - fee - } - } - } - } - `; - - // Create a component to fetch the data - function EventDataFetcher({ setEventData }) { - const { loading, error, data } = useQuery(GET_EVENT_BY_TOPIC); - - useEffect(() => { - if (loading) console.log("Loading data..."); - if (error) console.error("Error fetching data:", error); - if (data) { - console.log("Fetched data:", data); - setEventData( - data.eventByTopic.nodes.map((node) => { - return { - currentCount: parseXdr(node.data).currentCount, - updateTime: new Date( - node.txInfoByTx.ledgerByLedger.closeTime * 1000, - ).toLocaleString(), - }; - }), - ); - } - }, [loading, error, data, setEventData]); - - return null; - } - ``` - -5. **State Management**: Use a state management solution (e.g., Redux, MobX, or React Context) to store and manage the fetched data. - -6. **Error Handling**: Implement proper error handling to manage API request failures or data inconsistencies. +1. **Install the GraphQL client**: + +```bash + npm install @apollo/client graphql @vue/apollo-composable + # tailwind css will be used for styling + npm install -D tailwindcss postcss autoprefixer + npx tailwindcss init -p +``` + +The above libraries help in making GraphQL queries and managing the Apollo client in Vue applications. + +### Setup Tailwind CSS + +Add the following to the `tailwind.config.js` file: + +```js +// tailwind.config.js +/** @type {import('tailwindcss').Config} */ +export default { + content: ["./index.html", "./src/**/*.{js,ts,jsx,tsx}"], + theme: { + extend: {}, + }, + plugins: [], +}; +``` + +Add the following to the `assets/main.css` file: + +```css +@tailwind base; +@tailwind components; +@tailwind utilities; +``` + +### Setup Application + +Edit the `main.js` file to setup the Apollo client and provide it to the Vue app: + +Notice how we are using the `import.meta.env` to access the environment variables in the Vite application. + +```js +// main.js + +import "./assets/main.css"; +import { + ApolloClient, + InMemoryCache, + createHttpLink, +} from "@apollo/client/core"; + +import App from "./App.vue"; +import { DefaultApolloClient } from "@vue/apollo-composable"; +import { createApp } from "vue"; + +const apiToken = import.meta.env.VITE_API_TOKEN; + +const httpLink = createHttpLink({ + uri: `https://api.blockeden.xyz/stellar/mainnet/soroban/indexer/${apiToken}/v1/graphql`, +}); + +const apolloClient = new ApolloClient({ + link: httpLink, + cache: new InMemoryCache(), +}); + +const app = createApp(App); + +app.provide(DefaultApolloClient, apolloClient); +app.mount("#app"); +``` + +Set the API token in the `.env` file: + +```bash +# .env +VITE_API_TOKEN=token +``` ### Displaying Data Effectively -1. **Component Structure**: Create reusable components for different data types. - - ```jsx - // Wrap your app with ApolloProvider - function App() { - const [eventData, setEventData] = useState([]); - - return ( - -
-

Stellar Events Data Fetcher

- - {eventData ? ( -
-

Fetched Events:

- {eventData.map((event, index) => ( -
-

Current Count: {event.currentCount}

-

Update Time: {event.updateTime}

-
- ))} -
- ) : ( -

No data fetched yet

- )} -
- - ); - } - - export default App; - ``` - -2. **Security Concerns**: Ensure that sensitive data like API keys are stored securely and not exposed in the frontend code. Using a backend or serverless function to fetch data from the indexer can help protect sensitive information. - -3. **Optimization**: Implement pagination or lazy loading to manage large datasets efficiently. +We are going to create a component that fetches the token metadata from the indexer and displays it in the frontend. + +Firstly, we will setup a basic table view to display the token metadata. This table is styled with [Tailwind CSS](https://tailwindcss.com/docs/installation) classes for a clean and responsive design. + +We wil also implement pagination to manage the large dataset efficiently. + +1. **components/TokenList.vue**: + +```html + +``` + +Now we will add the script section to fetch the data from the indexer and display it in the frontend: + +```js + +``` + +The code above does fetching and paginating token metadata using GraphQL with Apollo Client and `@vue/apollo-composable`. + +It sets up a query (`GetTokenMetadata`) to fetch metadata like `admin_address`, `name`, `symbol`, `updated_at`, and `decimal`, handling pagination via `limit` and `offset` parameters. The `useQuery` hook manages data loading (`loading`), result handling (`result`), and errors (`error`), while `watchEffect` keeps `tokenMetadata` synced with fetched results. + +Pagination is achieved through `nextPage` and `prevPage` functions, adjusting `currentPage` and `offset` values and triggering a refresh (`refetch`) for navigating through token data pages seamlessly in Vue.js applications. + +Next we import the component into `App.vue` + +```html + + + +``` + +```bash +npm run dev +``` + +### App is ready + +We were able to fetch data from the indexer and display it in a frontend interface. The data is paginated to manage the large dataset efficiently. + +### Security Concerns: + +Ensure that sensitive data like API keys are stored securely and not exposed in the frontend code. Using a backend or serverless function to fetch data from the indexer can help protect sensitive information. + +### Conclusion + +By following the steps outlined above, you can effectively fetch data from Stellar indexers and integrate it into a frontend interface. This approach allows you to build powerful applications that leverage the rich data provided by indexers. ### Demo From 286a60d7de7f901525a8dd40778f5a4f7f1aa7ee Mon Sep 17 00:00:00 2001 From: Johnpaul Date: Thu, 11 Jul 2024 14:10:08 +0100 Subject: [PATCH 4/4] text correction, use mainnet --- docs/build/guides/dapps/indexers.mdx | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/build/guides/dapps/indexers.mdx b/docs/build/guides/dapps/indexers.mdx index 21ad050dd..f3359be03 100644 --- a/docs/build/guides/dapps/indexers.mdx +++ b/docs/build/guides/dapps/indexers.mdx @@ -44,7 +44,7 @@ Stellar indexers can provide various types of data, including: ### Step-by-Step Instructions for Querying Data -We are going to use the [BlockEden](https://www.blockeden.xyz/) indexer for this example to query token metadata for all tokens indexed on the indexer sorted by recent on the soroban testnet. +We are going to use the [BlockEden](https://www.blockeden.xyz/) indexer for this example to query token metadata for all tokens indexed on the indexer sorted by recent on the soroban mainnet. The goal is to query these data and display them in a frontend interface. @@ -73,7 +73,7 @@ query { } ``` - This query fetches the token metadata for the latest 50 tokens indexed on the indexer. +This query fetches the token metadata for the latest 50 tokens indexed on the indexer. The data displayed on this page can be safely integrated into our frontend application @@ -241,6 +241,10 @@ We wil also implement pagination to manage the large dataset efficiently. ``` +This Vue.js template file renders a Stellar token list fetched from BlockEden's GraphQL API using Apollo Client and `@vue/apollo-composable`. It begins with a title and handles loading and error states while data is fetched asynchronously. + +Once data is loaded, it populates a table with columns for token `name`, `symbol`, `contract_id`, `admin_address`, `decimal`, and `updated_at`. Pagination controls are provided with "Previous" and "Next" buttons that adjust the current page (`currentPage`) and trigger data refetching (`refetch`) based on the page size (`pageSize`). This setup ensures a responsive user interface for browsing through paginated token metadata effectively. + Now we will add the script section to fetch the data from the indexer and display it in the frontend: ```js