OpenFGA: Performance Issues with Postgres [Consolidated Discussion] #98

rhamzeh · 2023-01-26T19:01:09Z

rhamzeh
Jan 26, 2023
Maintainer

This thread is meant to consolidate all the reports we have received regarding OpenFGA's performance especially when it comes to the PostgreSQL adapter to make them easier to track.

rhamzeh · 2023-01-26T19:01:23Z

rhamzeh
Jan 26, 2023
Maintainer Author

Too slow performance with postgres data store
- Reporter: @neeraj-ec
- Summary: @neeraj-ec has provided detailed performance metrics and the comparison in performance between the Postgres and the in-mem data store
- Storage Adapter: Postgres
- Affected Endpoint: Check
- Authorization Model Schema Version: 1.0
- Link: https://github.com/orgs/openfga/discussions/84

0 replies

rhamzeh · 2023-01-26T19:01:37Z

rhamzeh
Jan 26, 2023
Maintainer Author

[Bug] OpenFGA<>Postgres: too many clients already
- Reporter: @xXAvoraXx
- Summary: Too many clients error when issuing ListObjects
- Affected Endpoint: ListObjects (optimized, using and or but not in the mode)
- Storage Adapter: Postgres
- Authorization Model Schema Version: 1.1
- Link: [Bug] OpenFGA<>Postgres: too many clients already openfga#464
- Env Variables:

OPENFGA_RESOLVENODELIMIT 38
OPENFGA_LISTOBJECTSDEADLINE 100s
OPENFGA_HTTP_UPSTREAMTIMEOUT 110s
OPENFGA_EXPERIMENTALS list-objects-optimized

0 replies

rhamzeh · 2023-01-26T19:01:55Z

rhamzeh
Jan 26, 2023
Maintainer Author

Performance issues - how to scale ?
- Reporter: @psadovsky
- Summary: @psadovsky provides a case where they are getting a low check performance (6s for 16 sequential requests)
- Affected Endpoint: Check
- Storage Adapter: Postgres on Azure
- Authorization Model Schema Version: N/A
- Link: Performance issues - how to scale ? openfga#445
- Notes: On Azure AKS, DB not in same pod as OpenFGA. Not clear whether the time is measured from when the FE sends the request to receiving, or wether the time it is taking OpenFGA to evaluate

0 replies

rhamzeh · 2023-01-26T19:02:52Z

rhamzeh
Jan 26, 2023
Maintainer Author

Optimized ListObjects causes "too many clients" errors
- Reporter: Fede on Discord
- Summary: Fede provides a reproducible test case where OpenFGA errors when responding to ListObjects requests. When successful, duplicate results were being returned.
- Affected Endpoint: ListObjects (optimized)
- Storage Adapter: Postgres
- Authorization Model Schema Version: 1.1
- Link: https://discord.com/channels/759188666072825867/1065264152488181830/1065993274671763496

1 reply

rhamzeh Jan 26, 2023
Maintainer Author

Based on Fede's report, it is possible to reproduce this. In the reproducible case, here are the relevant settings:

Summary: A user is a member of a group. Group members are editors of some folders, each folder has a couple of documents in it.

We'd like to list the documents the user can view.

(Note: the user was reporting duplicate objects returned in some cases)

docker-compose.yaml

version: '3.8'

networks:
  openfga:

services:
  db:
    image: postgres:14-alpine
    networks:
      - openfga
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=password
      - POSTGRES_DB=openfga
    healthcheck:
      test: [ "CMD-SHELL", "pg_isready -U postgres" ]
      interval: 5s
      timeout: 5s
      retries: 5

  pause:
    image: alpine
    command: sleep 5
    depends_on:
      db:
        condition: service_started

  migrate:
    image: openfga/openfga:v0.3.2
    depends_on:
      pause:
        condition: service_completed_successfully
    command: |
      migrate
    environment: 
      - OPENFGA_DATASTORE_ENGINE=postgres
      - OPENFGA_DATASTORE_URI=postgres://postgres:password@db:5432/openfga
    networks:
      - openfga

  openfga:
    image: openfga/openfga:v0.3.2
    environment:
      - OPENFGA_DATASTORE_ENGINE=postgres
      - OPENFGA_DATASTORE_URI=postgres://postgres:password@db:5432/openfga
      - OPENFGA_LOG_FORMAT=json
      - OPENFGA_EXPERIMENTALS="list-objects-optimized"
    command: run --listObjects-max-results 50000 --listObjects-deadline 60s --http-upstream-timeout 65s
    depends_on:
      db:
        condition: service_started
      migrate:
        condition: service_completed_successfully
    networks:
      - openfga
    ports:
      - "8080:8080"

Authorization Model

model
  schema 1.1

type user

type group
  relations
    define admin: [user]
    define member: [user,group#member] or admin

type folder
  relations
    define editor: [user,group#member]
    define viewer: [user,group#member] or editor

type document
  relations
    define editor: [user,group#member] or editor from parent
    define viewer: [user,group#member] or editor or viewer from parent
    define parent: [folder]

Relationship Tuples

[{
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.0"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.1"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.2"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.3"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.4"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.5"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.6"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.7"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.8"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.9"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.10"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.11"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.12"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.13"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.14"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.15"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.16"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.17"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.18"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.19"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.20"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.21"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.22"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.23"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:0.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:0"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.0"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.1"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.2"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.3"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.4"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.5"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.6"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.7"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.8"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.9"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.10"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.11"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.12"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.13"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.14"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.15"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.16"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.17"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.18"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.19"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.20"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.21"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.22"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.23"
}, {
  "user": "folder:1",
  "relation": "parent",
  "object": "document:1.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:1"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.0"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.1"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.2"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.3"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.4"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.5"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.6"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.7"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.8"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.9"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.10"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.11"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.12"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.13"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.14"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.15"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.16"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.17"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.18"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.19"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.20"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.21"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.22"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.23"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:2.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:2"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.0"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.1"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.2"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.3"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.4"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.5"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.6"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.7"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.8"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.9"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.10"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.11"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.12"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.13"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.14"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.15"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.16"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.17"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.18"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.19"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.20"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.21"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.22"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.23"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:3.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:3"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.0"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.1"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.2"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.3"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.4"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.5"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.6"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.7"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.8"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.9"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.10"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.11"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.12"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.13"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.14"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.15"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.16"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.17"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.18"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.19"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.20"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.21"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.22"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.23"
}, {
  "user": "folder:4",
  "relation": "parent",
  "object": "document:4.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:4"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.0"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.1"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.2"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.3"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.4"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.5"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.6"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.7"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.8"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.9"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.10"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.11"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.12"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.13"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.14"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.15"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.16"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.17"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.18"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.19"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.20"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.21"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.22"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.23"
}, {
  "user": "folder:5",
  "relation": "parent",
  "object": "document:5.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:5"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.0"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.1"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.2"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.3"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.4"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.5"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.6"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.7"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.8"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.9"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.10"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.11"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.12"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.13"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.14"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.15"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.16"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.17"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.18"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.19"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.20"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.21"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.22"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.23"
}, {
  "user": "folder:6",
  "relation": "parent",
  "object": "document:6.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:6"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.0"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.1"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.2"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.3"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.4"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.5"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.6"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.7"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.8"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.9"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.10"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.11"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.12"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.13"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.14"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.15"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.16"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.17"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.18"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.19"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.20"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.21"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.22"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.23"
}, {
  "user": "folder:7",
  "relation": "parent",
  "object": "document:7.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:7"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.0"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.1"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.2"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.3"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.4"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.5"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.6"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.7"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.8"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.9"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.10"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.11"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.12"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.13"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.14"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.15"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.16"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.17"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.18"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.19"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.20"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.21"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.22"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.23"
}, {
  "user": "folder:8",
  "relation": "parent",
  "object": "document:8.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:8"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.0"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.1"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.2"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.3"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.4"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.5"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.6"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.7"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.8"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.9"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.10"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.11"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.12"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.13"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.14"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.15"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.16"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.17"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.18"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.19"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.20"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.21"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.22"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.23"
}, {
  "user": "folder:9",
  "relation": "parent",
  "object": "document:9.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:9"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.0"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.1"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.2"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.3"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.4"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.5"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.6"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.7"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.8"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.9"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.10"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.11"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.12"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.13"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.14"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.15"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.16"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.17"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.18"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.19"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.20"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.21"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.22"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.23"
}, {
  "user": "folder:10",
  "relation": "parent",
  "object": "document:10.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:10"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.0"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.1"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.2"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.3"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.4"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.5"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.6"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.7"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.8"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.9"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.10"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.11"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.12"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.13"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.14"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.15"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.16"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.17"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.18"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.19"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.20"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.21"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.22"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.23"
}, {
  "user": "folder:11",
  "relation": "parent",
  "object": "document:11.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:11"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.0"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.1"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.2"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.3"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.4"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.5"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.6"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.7"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.8"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.9"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.10"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.11"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.12"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.13"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.14"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.15"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.16"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.17"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.18"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.19"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.20"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.21"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.22"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.23"
}, {
  "user": "folder:12",
  "relation": "parent",
  "object": "document:12.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:12"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.0"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.1"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.2"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.3"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.4"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.5"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.6"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.7"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.8"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.9"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.10"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.11"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.12"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.13"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.14"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.15"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.16"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.17"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.18"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.19"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.20"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.21"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.22"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.23"
}, {
  "user": "folder:13",
  "relation": "parent",
  "object": "document:13.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:13"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.0"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.1"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.2"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.3"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.4"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.5"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.6"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.7"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.8"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.9"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.10"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.11"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.12"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.13"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.14"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.15"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.16"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.17"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.18"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.19"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.20"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.21"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.22"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.23"
}, {
  "user": "folder:14",
  "relation": "parent",
  "object": "document:14.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:14"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.0"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.1"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.2"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.3"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.4"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.5"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.6"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.7"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.8"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.9"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.10"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.11"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.12"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.13"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.14"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.15"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.16"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.17"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.18"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.19"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.20"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.21"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.22"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.23"
}, {
  "user": "folder:15",
  "relation": "parent",
  "object": "document:15.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:15"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.0"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.1"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.2"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.3"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.4"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.5"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.6"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.7"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.8"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.9"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.10"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.11"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.12"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.13"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.14"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.15"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.16"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.17"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.18"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.19"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.20"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.21"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.22"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.23"
}, {
  "user": "folder:16",
  "relation": "parent",
  "object": "document:16.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:16"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.0"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.1"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.2"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.3"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.4"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.5"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.6"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.7"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.8"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.9"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.10"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.11"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.12"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.13"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.14"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.15"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.16"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.17"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.18"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.19"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.20"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.21"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.22"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.23"
}, {
  "user": "folder:17",
  "relation": "parent",
  "object": "document:17.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:17"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.0"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.1"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.2"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.3"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.4"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.5"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.6"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.7"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.8"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.9"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.10"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.11"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.12"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.13"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.14"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.15"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.16"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.17"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.18"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.19"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.20"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.21"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.22"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.23"
}, {
  "user": "folder:18",
  "relation": "parent",
  "object": "document:18.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:18"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.0"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.1"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.2"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.3"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.4"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.5"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.6"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.7"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.8"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.9"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.10"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.11"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.12"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.13"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.14"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.15"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.16"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.17"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.18"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.19"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.20"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.21"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.22"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.23"
}, {
  "user": "folder:19",
  "relation": "parent",
  "object": "document:19.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:19"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.0"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.1"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.2"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.3"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.4"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.5"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.6"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.7"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.8"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.9"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.10"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.11"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.12"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.13"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.14"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.15"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.16"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.17"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.18"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.19"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.20"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.21"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.22"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.23"
}, {
  "user": "folder:20",
  "relation": "parent",
  "object": "document:20.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:20"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.0"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.1"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.2"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.3"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.4"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.5"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.6"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.7"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.8"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.9"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.10"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.11"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.12"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.13"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.14"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.15"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.16"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.17"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.18"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.19"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.20"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.21"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.22"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.23"
}, {
  "user": "folder:21",
  "relation": "parent",
  "object": "document:21.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:21"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.0"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.1"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.2"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.3"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.4"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.5"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.6"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.7"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.8"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.9"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.10"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.11"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.12"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.13"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.14"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.15"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.16"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.17"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.18"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.19"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.20"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.21"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.22"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.23"
}, {
  "user": "folder:22",
  "relation": "parent",
  "object": "document:22.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:22"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.0"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.1"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.2"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.3"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.4"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.5"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.6"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.7"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.8"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.9"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.10"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.11"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.12"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.13"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.14"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.15"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.16"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.17"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.18"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.19"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.20"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.21"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.22"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.23"
}, {
  "user": "folder:23",
  "relation": "parent",
  "object": "document:23.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:23"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.0"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.1"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.2"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.3"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.4"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.5"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.6"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.7"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.8"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.9"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.10"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.11"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.12"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.13"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.14"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.15"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.16"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.17"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.18"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.19"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.20"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.21"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.22"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.23"
}, {
  "user": "folder:24",
  "relation": "parent",
  "object": "document:24.24"
}, {
  "user": "group:marketing#member",
  "relation": "editor",
  "object": "folder:24"
}, {
  "user": "user:anne",
  "relation": "member",
  "object": "group:marketing"
}, {
  "user": "folder:0",
  "relation": "parent",
  "object": "document:1.1"
}, {
  "user": "folder:2",
  "relation": "parent",
  "object": "document:1.1"
}, {
  "user": "folder:3",
  "relation": "parent",
  "object": "document:1.1"
}]

The request being run is:

ListObjects({
  "user": "user:anne",
  "relation": "viewer",
  "type": "document"
})

Measurements

Case #	# of Folders	# of Documents in every folder	Postgres (v0.3.2)	In Memory (v0.3.2)	Postgres (`context-database-layer` branch)	More Options - 1 (v0.3.2)	More Options - 1 (`context-database-layer` branch)	Postgres (v0.3.3)
1	25	25	20s (but missed 11 documents)	196ms	34152ms	1606ms	1751ms	671ms
2	25	50	Timeout (>60s)	507ms	Timeout (>60s)	3326ms	2127ms	1191ms
3	500	20	Timeout (>60s)	31952ms	Timeout (>60s)	26385ms	27478ms	10117ms

[More Options - 1]: Added --datastore-conn-max-idle-time 10m --datastore-max-idle-conns 100 --datastore-max-open-conns 100 to docker run

aaguiarz · 2023-02-28T11:54:46Z

aaguiarz
Feb 28, 2023
Maintainer

These issues were addressed in https://github.com/openfga/openfga/releases/tag/v0.3.3

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OpenFGA

OpenFGA: Performance Issues with Postgres [Consolidated Discussion] #98

{{title}}

Replies: 5 comments 1 reply

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

OpenFGA

OpenFGA: Performance Issues with Postgres [Consolidated Discussion] #98

rhamzeh Jan 26, 2023 Maintainer

Replies: 5 comments · 1 reply

rhamzeh Jan 26, 2023 Maintainer Author

rhamzeh Jan 26, 2023 Maintainer Author

rhamzeh Jan 26, 2023 Maintainer Author

rhamzeh Jan 26, 2023 Maintainer Author

rhamzeh Jan 26, 2023 Maintainer Author

Measurements

aaguiarz Feb 28, 2023 Maintainer

rhamzeh
Jan 26, 2023
Maintainer

Replies: 5 comments 1 reply

rhamzeh
Jan 26, 2023
Maintainer Author

rhamzeh
Jan 26, 2023
Maintainer Author

rhamzeh
Jan 26, 2023
Maintainer Author

rhamzeh
Jan 26, 2023
Maintainer Author

rhamzeh Jan 26, 2023
Maintainer Author

aaguiarz
Feb 28, 2023
Maintainer