Connecting to smart contracts, & limitations of current architecture

[gigxz]

Up to this point, we've been imagining Metagov being used by a Driver to govern activities on social platforms. The main Driver we've considered during architecture is PolicyKit, with some thought to how Kybern and Dada could function as a drivers as well.

Current state of things

The current architecture requires:

1. For 1 instance of a Driver, there is 1 instance of Metagov.
2. The Driver and Metagov instances are running on the same server.

Requirement (1) lets us not have to deal with the situation where there are multiple drivers using a single instance of Metagov. In the code we assume that there is only one Driver, and that the Driver is governing 1 or more communities.

Requirement (2) lets us restrict (most of) the metagov endpoints to be accessibly to local traffic only. That's why https://prototype.metagov.org/api/internal/resource/sourcecred.cred?username=miriam is forbidden if you open it in a browser, but metagov instance of PolicyKit can access this URL because it's making the request from the same network.

Requirement (2) means that we don't have to deal with authenticating requests to those endpoints, because the Driver is fully trusted. The Driver is responsible for authentication and authorization–– ie making sure that people say who they say they are, and that people can only access Metagov data that they have access to (though the Driver) –– for example in PolicyKit, I as an author for community X should not be able to write a policy that performs an action in community Y.

Limitations

Here's a list of things that aren't currently supported, because of the requirements listed above.

A. Read access from non-Driver system: An external system wants to make a read-only request to Metagov to read a resource. For example, a separately hosted instance of Discourse[1] requests a revenue share config from Metagov. The community already uses PolicyKit as the driver.

→ we could either let certain read-only endpoints be configured as completely public to the internet, OR implement auth so that the caller can authenticate against metagov.

B. Write access from non-Driver system: An external system wants to make a write request to Metagov. For example, some code somewhere wants to perform an action. The community already uses policykit as the driver.

→ the caller needs to authenticate against metagov somehow

C. Smart contract as Driver: A smart contract wants to act as a single-community Driver, with the ability to perform metagov actions, governance processes, and be notified of events occurring on other platforms. This isn't possible right now because of requirement (2)–– that the Driver is collocated with Metagov.

D. Smart contract as governed platform, PolicyKit as Driver. I think this is basically the same as (C). In both cases, Metagov needs a way to push data to the chain.

📌 Feedback requested: Which of these unsupported behaviors do we actually need to support? Do we have a concrete use-case for (B)?
My current feeling: we can get by with the naive solution for (A) if needed since everything we're using is public anyway, skip (B) for now though we'll eventually want it(?), and need to figure out (C) ASAP because we'll need it to support the upcoming NEAR challenge and possible peerkat collaboration.

Smart contracts & Oracles

Smart contracts can't just make API requests directly to services on the internet. Oracles are a special kind of smart contracts that allow a smart contract to look like it's making an API call, but really the oracle is making the API calls off-chain and posting the result on-chain for smart contracts to use. This tutorial how/why. Existing tools like Chainlink make it easier to perform API requests, receive responses, & retrieve data from the outside world. See Chainlink docs: request and receive data.

TODO- figure out exactly how Metagov could act as an oracle. Once I have an actual proposal for that, I'll do a separate issue.

[1] I'm specifically thinking about a Discourse instance that has a Discourse plugin (that we created) that invokes Metagov. Obviously there aren't any existing non-driver platforms that want to invoke Metagov right now.

[shaunagm]

B. Write access from non-Driver system: An external system wants to make a write request to Metagov. For example, some code somewhere wants to perform an action. The community already uses policykit as the driver.

→ the caller needs to authenticate against metagov somehow

Can you give an example of an action that an external system would want to perform? Is this separate from Platform Actions?

[thelastjosh]

📌 Feedback requested: Which of these unsupported behaviors do we actually need to support? Do we have a concrete use-case for (B)?

So (B) is basically asking, what are the reasons for Metagov to have an independent authentication service? This question might deserve its own issue.

Here's one possible use-case for (B): DADA has a Telegram bot, e.g. something like Common Stack's Praise Bot, that wants to emit a notification through Metagov to three linked services: Discourse, Loomio, and Discord. The Telegram bot is essentially an independent service. Three ways to think about how the bot interacts with Metagov:

• Treat it as a Metagov governance service, i.e. someone has written a Metagov plugin for the bot and provided some sort of key or other private authentication method.
• Authenticate against Metagov (so Metagov has an independent authentication service), in order to get read/write access to resources.
• The current way, authenticate against PolicyKit, then script those three sets of notifications in PolicyKit (side question: would that be one policy, or three?). Assuming the community has installed and configured PolicyKit.

Question: are these options similar to how we might want to interact with a "minimal driver"?

[thelastjosh]

My current feeling: we can get by with the naive solution for (A) if needed since everything we're using is public anyway, skip (B) for now though we'll eventually want it(?), and need to figure out (C) ASAP because we'll need it to support the upcoming NEAR challenge and possible peerkat collaboration.

I agree, let's focus on (C), skip (B), and use the naive, everything-is-public option for (A).

[gigxz]

@thelastjosh I guess the Telegram bot use-case is almost an example of (B), but not quite because it could be implemented using webhooks. I would implement that use-case by making a metagov plugin for Telegram, like this:

• There is a Metagov bot for Telegram that sends webhook events to the prototype.metagov.org Metagov instance. The /hooks endpoint on metagov is public***
• There is a Telegram plugin for Metagov that receives webhook events and forwards them to the Driver (PolicyKit at policykit.metagov.org)
• In the PolicyKit UI, the "Dada admin" would activate that metagov plugin, configure it for their telegram channel, and write a single policy that says something like: "when this event comes from telegram, if it has this certain type/string/??, then perform action X on Discourse, action Y on Loomio, and action Z on Discord".

So yeah, to answer your question @shaunagm, I don't really have a use case for (B). @thelastjosh is right, the important question is "what are the reasons for Metagov to have an independent authentication service?" and that deserves its own issue. If we want a smart contract to BE a Driver, then we need authentication, because we can no longer rely on restricting metagov endpoints to local traffic only. Let me write up a spec for how a smart contract could act a driver (still learning all this Oracle stuff!)

***A note on webhooks: anyone on the internet can post requests to the metagov webhook receiver endpoints, so it's up to the plugin author to verify the incoming request. Metagov core does not protect the plugin author at all. The plugin author can take a few approaches:

1. Verify an event signature secret. This is ideal, because we are sure that the event is coming from the right place. However not all services support this. Discourse does support it, so we verify requests in the plugin like this.
2. Next-best thing is to use a hard-to-guess webhook url. Like "prototype.metagov.org/api/hooks/dada/telegram/a7db8bf6-5c36-48f0-a3f2-4155ba55b95b". That token is hard to guess, so we can be pretty sure that the event is coming from the right place. Metagov supports setting webhook URL slugs using a special config key called webhook_slug. We use this approach for OpenCollective, since it doesn't do event signatures.
3. Mitigate harm of potential spoof requests by fetching data from the service's API instead of relying on data that is in the webhook event body. We also do this for the open collective plugin here. That way even if someone spoofs and sends a message "new expense was created in OC," we hit OC to find that expense and realize it doesn't exist.

As an aside... It's also totally up to the plugin author how they want to implement the webhook receiver. The docs say "use this to send events to the Driver" but technically they can write whatever code they want. For example I could create my own version of the Discourse plugin with a receive_webhook function that implements the logic: "when a post is reported, immediately suspend the authors account." Obviously thats what we would consider a "policy" and the recommendation is to write such things in the DRIVER, not the PLUGIN code, so that (1) the plugins are reusable, and (2) the policy is potentially governable.

[thelastjosh]

As an aside... It's also totally up to the plugin author how they want to implement the webhook receiver. The docs say "use this to send events to the Driver" but technically they can write whatever code they want. For example I could create my own version of the Discourse plugin with a receive_webhook function that implements the logic: "when an post is reported, immediately suspend the authors account." Obviously thats what we would consider a "policy" and the recommendation is to write such things in the DRIVER, not the PLUGIN code, so that the policy is governable.

Hm, interesting... while we can't stop people from implementing policies in all sorts of places, we can build more support for logic that exists in the places that we want it to exist.

…

On Thu, Mar 25, 2021 at 2:06 PM Miriam Ashton ***@***.***> wrote: @thelastjosh <https://github.com/thelastjosh> I guess the Telegram bot use-case is almost an example of (B), but not quite because it could be implemented using webhooks. I would implement that use-case by making a metagov plugin for Telegram, like this: - There is a Metagov bot for Telegram that sends webhook events to the prototype.metagov.org Metagov instance. The /hooks endpoint on metagov is public*** - There is a Telegram plugin for Metagov that receives webhook events and forwards them to the Driver (PolicyKit at policykit.metagov.org) - In the PolicyKit UI, the "Dada admin" would activate that metagov plugin, configure it for their telegram channel, and write a single policy that says something like: "when this event comes from telegram, if it has this certain type/string/??, then perform action X on Discourse, action Y on Loomio, and action Z on Discord". So yeah, to answer your question @shaunagm <https://github.com/shaunagm>, I don't really have a use case for (B). @thelastjosh <https://github.com/thelastjosh> is right, the important question is "what are the reasons for Metagov to have an independent authentication service?" and that deserves its own issue. If we want a smart contract to BE a Driver, then we need authentication, because we can no longer rely on restricting metagov endpoints to local traffic only. Let me write up a spec for how a smart contract could act a driver (still learning all this Oracle stuff!) ****A note on webhooks:* anyone on the internet can post requests to the metagov webhook receiver endpoints, so it's *up to the plugin author* to verify the incoming request. Metagov core does not protect the plugin author at all. The plugin author can take a few approaches: 1. Verify an event signature secret. This is ideal, because we are sure that the event is coming from the right place. However not all services support this. Discourse does support it, so we verify requests in the plugin like this <https://github.com/metagov/metagov-prototype/blob/4c8f40804f2bf5d8cc4b29cb29f5f0d67d1f404a/metagov/metagov/plugins/discourse/models.py#L120-L133> . 2. Next-best thing is to use a hard-to-guess webhook url. Like " prototype.metagov.org/api/hooks/dada/telegram/a7db8bf6-5c36-48f0-a3f2-4155ba55b95b ". That token is hard to guess, so we can be pretty sure that the event is coming from the right place. Metagov supports <https://github.com/metagov/metagov-prototype/blob/4c8f40804f2bf5d8cc4b29cb29f5f0d67d1f404a/metagov/metagov/core/views.py#L111-L116> setting webhook URL slugs using a special config key called webhook_slug. We use this approach for OpenCollective, since it doesn't do event signatures. 3. Mitigate harm of potential spoof requests by fetching data from the service's API instead of relying on data that is in the webhook event body. We also do this for the open collective plugin here <https://github.com/metagov/metagov-prototype/blob/4c8f40804f2bf5d8cc4b29cb29f5f0d67d1f404a/metagov/metagov/plugins/opencollective/models.py#L121>. That way even if someone spoofs and sends a message "new expense was created in OC," we hit OC to find that expense and realize it doesn't exist. As an aside... It's also totally up to the plugin author how they want to implement the webhook receiver. The docs say "use this to send events to the Driver" but technically they can write whatever code they want. For example I could create my own version of the Discourse plugin with a receive_webhook function that implements the logic: "when an post is reported, immediately suspend the authors account." Obviously thats what we would consider a "policy" and the recommendation is to write such things in the DRIVER, not the PLUGIN code, so that the policy is governable. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACHA5PFGNWVCNPIUHH4E7LLTFN3R7ANCNFSM4ZX7RVBA> .

-- http://www.joshuatan.com/research/

[gigxz]

@thelastjosh plugin authors can now declare public action endpoints. I made three of the existing endpoints public, as you can see here under Actions (Public): https://prototype.metagov.org/redoc/#

So I can make a call like this from anywhere:
curl -H "X-Metagov-Community: some-real-community" -X POST "https://prototype.metagov.org/api/action/revshare.pick-pointer"