Skip to content

Commit

Permalink
Add dune upload support
Browse files Browse the repository at this point in the history
  • Loading branch information
shuva10v committed Jan 9, 2025
1 parent 1e1da68 commit 2df4cd2
Show file tree
Hide file tree
Showing 3 changed files with 59 additions and 1 deletion.
40 changes: 40 additions & 0 deletions .github/workflows/dune.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Complile and upload datalake on AWS

on:
push:
branches: [ "main" ]

env:
AWS_REGION: eu-central-1

permissions:
contents: read

jobs:
deploy:
name: Compile and update table on Dune
runs-on: ubuntu-latest
environment: production

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up Python 3.11
uses: actions/setup-python@v3
with:
python-version: 3.11
- name: Install Dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: Compile
run: |
python build_assets.py
- name: Upload to Dune
run: |
python upload_to_dune.py output/csv/assets.csv
env:
DUNE_API_KEY: ${{ secrets.DUNE_API_KEY }}


17 changes: 17 additions & 0 deletions dune_upload.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import sys
from dune_client.client import DuneClient

dune = DuneClient.from_env()

file_path = sys.argv[1]
with open(file_path) as file:
data = file.read()

table = dune.upload_csv(
data=str(data),
description="Labeled dataset for TON blockchain",
table_name="labels",
is_private=False
)

print(table)
3 changes: 2 additions & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
pandas
loguru
gitpython
pytoniq_core
pytoniq_core
dune-client

0 comments on commit 2df4cd2

Please sign in to comment.