[bounty] make realtime transcription works with screenpipe-cloud

[louis030195]

atm getting 403

how to reproduce:

in screenpipe-actions/ai-proxy/index.ts

1. comment out the auth checks
2. comment out langfuse part

# screenpipe-actions/ai-proxy/.dev.vars
DEEPGRAM_API_KEY=...  ask me ...

# terminal 1
cd screenpipe-actions/ai-proxy
# .dev.vars
bunx wrangler deploy 

# terminal 2
# find your user id 
less "$HOME/Library/Application Support/screenpipe/store.bin"
export CUSTOM_DEEPGRAM_API_TOKEN="<your user id>"
export DEEPGRAM_WEBSOCKET_URL="wss://..."
cargo build --release --features metal
./target/release/screenpipe   --enable-realtime-audio-transcription --data-dir /tmp/sp --port 3035

same works locally using bunx wrangler dev but get 403 when deployed

/bounty 100

my guesses:

• something is wrong in how we do the ws open message or something else, it seems the rust deepgram sdk disconnect instantly for some reason?
• something else

try bunx wrangler tail btw

[algora-pbc]

💎 $100 bounty • Screenpi.pe

Steps to solve:

1. Start working: Comment /attempt #1260 with your implementation plan
2. Submit work: Create a pull request including /claim #1260 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to mediar-ai/screenpipe!

Add a bounty • Share on socials

[kumarvivek1752]

@louis030195 i want to work on this. I'm trying to reproduce this error by adding config as you mentioned. but how to test on cloud?

edit :
I figure it out to test with cloud I've to run this command bunx wrangler deploy then
change these env variables with Cloudflare workers deployed URL:

➜  screenpipe git:(main) ✗ export HOST=http://localhost:8787 # replace this with deployed url
➜  screenpipe git:(main) ✗  export TOKEN=796e3f5b4xxxxxx25f3
➜  screenpipe git:(main) ✗ curl -X POST $HOST/v1/listen \
  -H "Content-Type: audio/wav" \
  -H "detect_language: en" \
  -H "Authorization: Bearer $TOKEN" \
  --data-binary "@./screenpipe-audio/test_data/poetic_kapil_gupta.wav"

but the issue it in dev mode it is working fine and I don't have an active Cloudflare active subscription to test on cloud.
bunx wrangler deploy output :

➜  ai-proxy git:(main) ✗ bunx wrangler deploy

 ⛅️ wrangler 3.80.4 (update available 3.107.2)
--------------------------------------------------------

Total Upload: 771.58 KiB / gzip: 146.36 KiB
Your worker has access to the following bindings:
- Durable Objects:
  - RATE_LIMITER: RateLimiter

✘ [ERROR] A request to the Cloudflare API (/accounts/9728a78xxxxxx0a232d/workers/scripts/ai-proxy) failed.

  In order to use Durable Objects, you must switch to a paid plan at
  https://dash.cloudflare.com/9728a7xxxxxxa0a232d/workers/plans. [code: 10084]

  If you think this is a bug, please open an issue at:
  https://github.com/cloudflare/workers-sdk/issues/new/choose