Any plans to add Queues (like SQS etc.)

[ajonno]

The only thing I see missing from Supabase (which is AWESOME great work) that would at least in our case make it a 1:1 replacement to AWS, is queues. Same as SQS or Azure Service Bus etc.. , with features like FIFO, dead letter etc.

Is this on the radar at all ?

[steffenstolze]

I asked myself the same but at the same time I think it would be easy to I integrate AWS services like SQS using the AWS SDK within your application, e.g. in the server routes of a Nuxt or SvelteKit App.

Do you see a problem in the "coexistence" of Supabase and native AWS services in your project?

[ajonno]

Actually I agree with you. It's easy enough to drop an event into queues at aws or azure or gcp. Also those platforms have masses of options available for post processing.

[jcerjak]

I'm thinking something based on Postgres would be a good fit for Supabase. Quick search returns some interesting blogs/projects:

• https://www.crunchydata.com/blog/message-queuing-using-native-postgresql
• https://github.com/pgq/pgq
• https://github.com/tembo-io/pgmq

These approaches can probably be used even without support from Supabase, if these Postgres extensions can be installed on Supabase. I'll try with pgmq extension and create a Deno client for it, something similar as the existing Python client.

[jcerjak]

These approaches can probably be used even without support from Supabase, if these Postgres extensions can be installed on Supabase.

pgq and pgmq are not SQL-only extensions, so Supabase would have to provide them for installation...

[AndryHTC]

18 months have passed since the initial post, yet there's been no contribution to the discussion from anyone at Supabase. I'm curious, how have others tackled the issue of queue management in their projects?

[flogy]

Would love to have some queue functionality within Supabase too. I currently use a separate queue in AWS, which is not great but working.

[AndryHTC]

Strange... Supabase usually reply to the discussions/issues of his community. The silence here seems to be wanted other than accepted

[lukeocodes]

Maybe we're stumbled onto a feature they're building lol

[AndryHTC]

But didn't they already did that 3 years ago? https://github.com/supabase/workflows

[tmountain]

Lots of people are waiting for news regarding workflows, but it looks like the project is dead. The last commit was 3 years ago.

[tmountain]

Supabase is more than capable, but it takes some preliminary configuration.

https://blog.mansueli.com/building-a-queue-system-with-supabase-and-postgresql

[anil-zenollect]

@mansueli 👆

[tmountain]

After working through this for about a month now, I would recommend pgmq to anyone reading this. It's easy to install into Supabase via database.dev, and it works great.

[tazmaniax]

@tmountain is there a straightforward way to trigger an edge function from pgmq (e.g. push via a trigger or pull via a cron)?

[tmountain]

@tazmaniax -- here's a simple cron example showing how to trigger an edge function using pg_net. If you're using multiple hosts, you can use the supabase vault to store the hostname locally to the instance to make this work across prod/staging/etc.

select
  cron.schedule(
    'invoke-function-every-minute',
    '* * * * *', -- every minute
    $$
    select
      net.http_post(
          url:='https://project-ref.supabase.co/functions/v1/function-name',
          headers:='{"Content-Type": "application/json", "Authorization": "Bearer YOUR_ANON_KEY"}'::jsonb,
          body:=concat('{"time": "', now(), '"}')::jsonb
      ) as request_id;
    $$
  );

[joaopegoraro]

The problem with pg_net, is that it has a big problem when dealing with (not so) high loads, see this issue, which makes it completely unreliable for production, and seems the issue is not getting fixed anytime soon.

[rustygreen]

Seems like just adding pgmq as an extension is Supabase would appease most people's needs and be an easy win for Supabase. Supabase team, can we get pgmq extension included in the available extensions? It's a solid and well-maintained extension. What's the issue?

[tmountain]

Not sure what the issue is, but it's easily installed via https://database.dev/

[softmarshmallow]

Is anybody getting

{
  message: 'An invalid response was received from the upstream server'
}

502 / 500 on high demand?

It's very clear postgREST fails on high demand, randomly and strage nobody talks about it.

[Owez]

I'd love some sort of simple queue, even only having FIFO would be enough for a lot of typical tasks. It'd be nice to have "run a function everytime something is added to this queue" without any big setup :)

[SqlOwl]

This is a very wanted feature

[thisisgoldman]

We've just released open source project that brings a queue to Supabase. This thread helped us gain conviction to open source it, so I wanted to share it:

https://github.com/sequinstream/sequin

We wanted an HTTP interface, exactly-once processing guarantees, transactional enqueue, backfills, replays, and clean debugging / observability. It's a new project and we're actively improving it if you want to add an issue.

[AndryHTC]

I need to check the DX and vibe. If it's good you'll have an early adopter

[tazmaniax]

@thisisgoldman very interesting. Also need to check it out but if its good then it would be perfect if Supabase could bake this into their offering

[joaopegoraro]

This would be perfect for Supabase. Good job man

[IvanBernatovic]

They released the official queues feature yesterday as part of launch week 13: https://supabase.com/blog/supabase-queues
This is a great addition. Now we're just missing a feature that connects queues and edge functions, so new message automatically invoke a function. This is a common setup in AWS: SQS queues that trigger lambda functions.

[tmountain]

It's not ideal, but you can use a simple trigger to invoke the edge function. Here's an example:

CREATE OR REPLACE FUNCTION "public"."notify_edge_function"() RETURNS "trigger"
    LANGUAGE "plpgsql" SECURITY DEFINER
    AS $$
DECLARE
  anon_key TEXT;
  supabase_url TEXT;
  request_id BIGINT;
BEGIN
  -- Fetch the secrets from the vault.decrypted_secrets table
  SELECT decrypted_secret INTO anon_key FROM vault.decrypted_secrets WHERE name = 'anon_key';
  SELECT decrypted_secret INTO supabase_url FROM vault.decrypted_secrets WHERE name = 'supabase_url';

  -- Perform the HTTP POST request with authentication
  SELECT net.http_post(
    url := supabase_url || '/functions/v1/my-function',
    headers := jsonb_build_object(
      'Content-Type', 'application/json',
      'Authorization', 'Bearer ' || anon_key
    )::jsonb,
    body := jsonb_build_object('id', NEW.id)::jsonb
  ) AS request_id
  INTO request_id;

  RETURN NEW;

END;
$$;

[craft365]

It seems like queue functionality does not (perhaps temporarily) exist for local development. There is no such item in the 'integration' section.

[flogy]

I had the same issue and I think I solved it by updating to the latest prerelease version (2.0.5 or so) instead of @latest which points to 2.0.0. Maybe I also had to remove and recreate all docker containers.

[claezon]

For me, it worked to enable the Queues integration by updating my Supabase Cloud Postgres version, since I have a linked local development instance. Here’s what I did:

1. Updated the Cloud Postgres version (e.g., 15.8.1.020). You can do this from where you enable the Queues integration.
2. Ran: supabase link to synchronize the local and cloud instances.
3. Restarted the local Supabase instance.

This pulled the new Postgres Docker image, which includes the Queues integration.