Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial support for Supabase Sync #554

Open
wants to merge 35 commits into
base: v1-dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
5bfc29f
wip on settings
joelmnz May 13, 2024
f719ace
settings saving
joelmnz May 13, 2024
3ec5482
wip on supabase connection
joelmnz May 13, 2024
9a0c165
wip on sync
joelmnz May 14, 2024
eaa2b4c
Merge remote-tracking branch 'origin/main' into main-supabase
joelmnz May 14, 2024
7aafb73
.
joelmnz May 14, 2024
0f623bb
sync wip
joelmnz May 14, 2024
a1c41b0
save to db working
joelmnz May 15, 2024
cf36e31
wip on import from server
joelmnz May 15, 2024
b6f78c3
added docs
joelmnz May 15, 2024
4c1b65f
better messages
joelmnz May 15, 2024
1ab20f0
Merge remote-tracking branch 'origin/main' into main-supabase
joelmnz May 15, 2024
da8c26e
publish fixes
joelmnz May 15, 2024
f4be5dc
PR tidy up
joelmnz May 15, 2024
9823cb8
Merge remote-tracking branch 'origin/main' into main-supabase
joelmnz May 17, 2024
460068b
improve sync + support sync from Module settings (e.g. first sync)
joelmnz May 17, 2024
319a33c
Merge pull request #1 from joelmnz/main-supabase
joelmnz May 18, 2024
1b3f193
Merge branch 'enricoros:main' into main
joelmnz May 21, 2024
002c650
wip not working, but might be close. need to put session info in app …
joelmnz May 25, 2024
67303e4
Merge branch 'enricoros:main' into main-supabaseUserAuth
joelmnz May 27, 2024
649ddc0
Merge branch 'enricoros:main' into main
joelmnz May 29, 2024
23afb25
doc: supabase db schema
joelmnz Jun 3, 2024
ade4299
wip: so close yet so far
joelmnz Jun 3, 2024
93ad2dd
Merge branch 'enricoros:main' into main
joelmnz Jun 6, 2024
cc69d20
Merge branch 'enricoros:main' into main-supabaseUserAuth
joelmnz Jun 6, 2024
72d61f5
Merge branch 'enricoros:main' into main
joelmnz Jun 11, 2024
ba1c7c1
Merge branch 'enricoros:main' into main
joelmnz Jun 26, 2024
161b104
enh: use Supabase user auth
joelmnz Jun 26, 2024
2121bcc
Merge branch 'main' into main-supabaseUserAuth
joelmnz Jun 26, 2024
bf97301
tidy: fix up code for PR, login consistency + db table check
joelmnz Jul 1, 2024
96dd36f
doc: update documentation
joelmnz Jul 1, 2024
dd7a344
fix: " vs ' for import
joelmnz Jul 1, 2024
58eff5e
fix: " vs ' in imports
joelmnz Jul 1, 2024
9457b2c
Merge pull request #2 from joelmnz/main-supabaseUserAuth
joelmnz Jul 1, 2024
7b30214
Merge branch 'main' into main
joelmnz Jul 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
135 changes: 132 additions & 3 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,9 @@
"@next/third-parties": "^14.2.3",
"@prisma/client": "^5.13.0",
"@sanity/diff-match-patch": "^3.1.1",
"@supabase/auth-ui-react": "^0.4.7",
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@enricoros I wasn't able to get the supabase auth UI working nicely, I think these might be able to be removed, thoughts?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll take a look at the default components provided by the package, I shall be able to integrate this. Worst case we drop this package and code a manual UI.

"@supabase/auth-ui-shared": "^0.1.8",
"@supabase/supabase-js": "^2.43.4",
"@t3-oss/env-nextjs": "^0.10.1",
"@tanstack/react-query": "~4.36.1",
"@trpc/client": "10.44.1",
Expand Down
4 changes: 4 additions & 0 deletions src/apps/settings-modal/SettingsModal.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { BrowseSettings } from '~/modules/browse/BrowseSettings';
import { DallESettings } from '~/modules/t2i/dalle/DallESettings';
import { ElevenlabsSettings } from '~/modules/elevenlabs/ElevenlabsSettings';
import { GoogleSearchSettings } from '~/modules/google/GoogleSearchSettings';
import { SupabaseSyncSettings } from '~/modules/supabasesync/SupabaseSyncSettings';
import { ProdiaSettings } from '~/modules/t2i/prodia/ProdiaSettings';
import { T2ISettings } from '~/modules/t2i/T2ISettings';

Expand Down Expand Up @@ -206,6 +207,9 @@ export function SettingsModal(props: {
<Topic icon={<SearchIcon />} title='Google Search API' startCollapsed>
<GoogleSearchSettings />
</Topic>
<Topic icon={<SearchIcon />} title='Supabase Sync' startCollapsed>
<SupabaseSyncSettings />
</Topic>
{/*<Topic icon='🛠' title='Other tools...' />*/}
</Topics>
</TabPanel>
Expand Down
2 changes: 2 additions & 0 deletions src/common/state/store-chats.ts
Original file line number Diff line number Diff line change
Expand Up @@ -386,12 +386,14 @@ export const useChatStore = create<ConversationsStore>()(devtools(
_get()._editConversation(conversationId,
{
autoTitle,
updated: Date.now(),
}),

setUserTitle: (conversationId: string, userTitle: string) =>
_get()._editConversation(conversationId,
{
userTitle,
updated: Date.now(),
}),

}),
Expand Down
1 change: 1 addition & 0 deletions src/modules/backend/backend.router.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ export const backendRouter = createTRPCRouter({
hasLlmPerplexity: !!env.PERPLEXITY_API_KEY,
hasLlmTogetherAI: !!env.TOGETHERAI_API_KEY,
hasVoiceElevenLabs: !!env.ELEVENLABS_API_KEY,
hasSupabaseSync: !!env.SUPABASE_SYNC_URL && !!env.SUPABASE_SYNC_KEY,
llmConfigHash: generateLlmEnvConfigHash(env),
};
}),
Expand Down
2 changes: 2 additions & 0 deletions src/modules/backend/store-backend-capabilities.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ export interface BackendCapabilities {
hasLlmPerplexity: boolean;
hasLlmTogetherAI: boolean;
hasVoiceElevenLabs: boolean;
hasSupabaseSync: boolean;
llmConfigHash: string;
}

Expand Down Expand Up @@ -53,6 +54,7 @@ const useBackendCapabilitiesStore = create<BackendStore>()(
hasLlmPerplexity: false,
hasLlmTogetherAI: false,
hasVoiceElevenLabs: false,
hasSupabaseSync: false,
llmConfigHash: '',

loadedCapabilities: false,
Expand Down
55 changes: 55 additions & 0 deletions src/modules/supabasesync/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Supabase Sync - Experimental

## Module Goal

> To sync all conversations from big-agi's localDb to a server and back allowing use on multiple devices while preserving big-agi's private and local approach.

Supabase supports multi user authentication so this module assumes you will have users setup and users can save their own data/chats to this database and they will not be accessable by other users (e.g. not used for team conversation sharing, if that is desired then get all team members to use same user account)

## Module Status

**Whats working:**

- Sync "Chat Conversations" to Supabase

**Planned:**

- Sync Conversation folders
- Sync other shared user settings like theme, what the "Enter" key does etc

## Supabase Setup

- Supabase project setup (free account is fine), you will need your url & anon-key
- Table called `user_conversation` with the following schema
- Row Level Security (RLS) turned on for this table and policies setup
- One or more supabase user accounts with access to the `user_conversation` table

```sql

create table user_conversation (
id uuid not null,
"systemPurposeId" character varying(255) null,
"folderId" uuid null,
created bigint not null,
updated bigint not null,
"userTitle" character varying(255) null,
"autoTitle" character varying(255) null,
messages json null,
user_id uuid null default auth.uid (),
constraint user_conversation_pkey primary key (id)
);

create policy "Users can mange their own data"
on "public"."user_conversation"
to public
using (
(auth.uid() = user_id)
);

```

## Big-Agi Setup

Navigate to your hosted instance and set your Supabase URL & KEY under the `Preferences -> Tools -> Supabase Sync` then login with your supabase user

NOTE: the `Last Synced` is a way of tracking what chnages you need to get. To do a full sync (possibly loosing any un-synced data) reset this value to 0.
Loading