Merge pull request #486 from permitio/doc-upgardes-jul-23

tweaks to improve the docs

.github/pull_request_template.md

@@ -20,6 +20,7 @@
 
 ## Check List (Check all the applicable boxes) <!-- Follow the above conventions to check the box -->
 
+- [x] I sign off on contributing this submission to open-source
 - [ ] My code follows the code style of this project.
 - [ ] My change requires changes to the documentation.
 - [ ] I have updated the documentation accordingly.

documentation/docs/getting-started/intro.mdx

@@ -35,13 +35,12 @@ This is where [Cedar-Agent](https://github.com/permitio/cedar-agent) and OPAL co
 - OPAL uses policy-engines, but isn't one itself -
 - Check out <a href="https://github.com/permitio/cedar-agent" target="_blank">Cedar-Agent</a>, <a href="https://www.openpolicyagent.org/" target="_blank">Open-Policy-Agent</a>, and <a href="https://www.osohq.com/" target="_blank">OSO</a>
 
-#### OPAL is not a large scale global fine-grained authorization:
+#### OPAL is not a database for permission data
 
-- Currently OPAL is not meant for managing ridiculous (>100GB) amounts of data within one layer. Though it can complement a CDN to achieve a similar result - see [Key concepts and design](../overview/design#large-scale-fga).
 - Check out <a href="https://research.google/pubs/pub48190/" target="_blank">Google-Zanzibar</a>
 
-- Fullstack permissions:
+#### Fullstack permissions:
 
-  - OPAL + policy-agent essentially provide microservices for authorization
-  - Developers still need to add control interfaces on top (e.g. user-management, api-key-management, audit, impersonation, invites) both as APIs and UIs
-  - Check out <a href="https://permit.io" target="_blank">Permit.io</a>
+- OPAL + a policy-agent essentially provide microservices for authorization
+- Developers still need to add control interfaces on top (e.g. user-management, api-key-management, audit, impersonation, invites) both as APIs and UIs
+- Check out <a href="https://permit.io" target="_blank">Permit.io</a>

documentation/docs/getting-started/tldr.mdx

@@ -0,0 +1,31 @@
+# OPAL TL;DR
+
+OPAL is an advanced piece of software with many capabilities and configuration options, hence it has a lot of docs; but if you want just the gist of it - this is the article for you.
+
+## How OPAL works
+
+The OPAL server sends instructions to the OPAL-clients (via pub/sub subscriptions over websockets) to load policy and data into their managed policy-agents (e.g. OPA, Cedar-agent, AWS AVP)
+
+### Policy
+
+OPAL tracks [policies from Git](/tutorials/track_a_git_repo) or from [API bundle servers](/tutorials/track_an_api_bundle_server).
+
+With Git - directories with policy-code (e.g. `.rego` or `.cedar` files) are automatically mapped to topics - which a client can subscribe to with `OPAL_POLICY_SUBSCRIPTION_DIRS`
+Every time you push a change, the OPAL server will notify the subscribing OPAL-clients to load the new policy.
+
+### Data
+
+OPAL tracks data from various sources via webhooks and [Fetch-Providers](/tutorials/write_your_own_fetch_provider) (extensible python modules that teach it to load data from sources).
+
+[Initial data is indicated by the server](getting-started/running-opal/run-opal-server/data-sources) based on `OPAL_DATA_CONFIG_SOURCES`.
+Subsequent data updates are triggered via [the data update webhook](/tutorials/trigger_data_updates).
+Every time the policy agent (or it's managing OPAL-client) restarts, the data and policy are loaded from scratch.
+
+#### Data as part of policy bundle
+
+Data can also be loaded with the policy as part of `data.json` files, located in the folders next to the policy file.
+
+:::note
+The **folder path** is used as the **key path** in the policy engine cache.
+In order to avoid race conditions between policy data updates and regular data updates, make sure the key paths used by your policy-data and the ones used by your data-updates are different.
+:::

documentation/docs/tutorials/track_a_git_repo.mdx

@@ -36,9 +36,10 @@ At the moment, only GitHub-hosted repos support webhooks. We plan to expand to m
 
 ### 2) OPAL Client subscribes to policy update notifications
 
-OPAL client can subscribe to multiple policy topics, each policy topic is formatted as `policy:{path}` where path is a **directory** in the policy git repository (**relative path** to the root of the repository).
+OPAL client can subscribe to multiple policy topics, topics are based on paths where each path is a **directory** in the policy git repository (**relative path** to the root of the repository).
 
 The policy directories the client will subscribe to are specified by the environment variable `OPAL_POLICY_SUBSCRIPTION_DIRS` passed to the client. The default is `"."` meaning the root directory in the branch (i.e: essentially all `.rego` and `data.json` files in the branch).
+`:` is used by the environment variable parsing as a delimiter between directories.
 
 Let's look at at a more complex example. Let's say we have a multi-tenant application in the cloud with an on-prem component in each customer site. We want to apply the same base policy to all tenants, but also to allow special rules (different policy) per tenant.
 
@@ -70,6 +71,12 @@ We'll deploy a different OPAL client as part of our on-prem/edge container in ea
   - `edge2` will set `OPAL_POLICY_SUBSCRIPTION_DIRS=default:tenants/tenant2`.
   - `edge3` will set `OPAL_POLICY_SUBSCRIPTION_DIRS=default:tenants/tenant3`.
 
+:::note
+When translated from directories to pub/sub topics - each topic is formatted as `{scope}:{policy_topic_prefix}:{path}` where path is the **directory**.
+`policy_topic_prefix` would usually be just "policy" and `scope` "default".
+Relevant code: [packages/opal-client/opal_client/policy/updater.py](https://github.com/permitio/opal/blob/41cd3f33fb6c089fef3eb402c7457012a3526531/packages/opal-client/opal_client/policy/updater.py#L76C13-L76C13)
+:::
+
 #### 2.1) Loading Data.json files
 
 In general `data.json` files are mostly meant for backward as a backward compatibility to vanilla OPA bundles (in OPA the `data.json` in [bundles](https://www.openpolicyagent.org/docs/latest/management-bundles/) is the primary way to load data. For users migrating from plain OPA to OPAL being able to reuse their bundles can be useful).

documentation/docs/welcome.mdx

@@ -11,10 +11,14 @@ OPAL is designed to work with **live applications**, and bring policy to a real-
 stores - this could be your Git, your databases, SaaS services and so on - **OPAL will make sure the services are always in sync**
 with the authorization data and policy they need, and more importantly, **only** the services that they need.
 
-<!-- Check out OPAL's main site at **<a href="https://opal.ac">OPAL.ac</a>** -->
-
 ### How do I start with OPAL?
 
+We recommend that you:
+
+1. Get a quick understanding by reading the TL;DR article
+2. Explore the tutorials
+3. Play with OPAL's docker compose playground
+
 :::tip OPAL Playground - taking your first steps
 
 If you are new to OPAL and want to run step by step through setting up an OPAL playground - please follow

documentation/sidebars.js

@@ -24,6 +24,11 @@ const sidebars = {
           id: "getting-started/intro",
           label: "Introduction",
         },
+        {
+          type: "doc",
+          id: "getting-started/tldr",
+          label: "TL;DR",
+        },
         {
           type: "category",
           label: "Quickstart",