Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TT-5883] Python plugin example #464

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 15 additions & 15 deletions docs/api_definitions.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,22 +63,22 @@ An API Definition describes the configuration of an API. It instructs Tyk Gatewa

## Features

| Feature | Supported | Comments |
| ----------- | --------- | --------- |
| API Tagging | ✅ | - |
| [Config Data](./../config/samples/config_data_virtual_endpoint.yaml) | ✅ | - |
| Context Variables | ✅ | - |
| Feature | Supported | Comments |
| ----------- | --------- |------------------------------------------------------------------------|
| API Tagging | ✅ | - |
| [Config Data](./../config/samples/config_data_virtual_endpoint.yaml) | ✅ | - |
| Context Variables | ✅ | - |
| [Cross Origin Resource Sharing (CORS)](./../config/samples/httpbin_cors.yaml) | ⚠️ | [See ISSUE #3396 ](https://github.com/TykTechnologies/tyk/issues/3396) |
| Custom Plugins - Go | ⚠️ | Untested |
| [Custom Plugins - gRPC](./../bdd/features/api_http_grpc_plugin.feature) | ✅ | - |
| [Custom Plugins - Javascript](./api_definitions/custom_plugin.md) | ✅ | - |
| Custom Plugins - Lua | ⚠️ | Untested |
| Custom Plugins - Python | ️ | Untested |
| Global Rate Limit | ❌ | Not Implemented |
| [Segment Tags](./../config/samples/httpbin_tagged.yaml) | ✅ | - |
| Tag Headers | ❌ | Not Implemented |
| [Webhooks](./webhooks.md) | ❌ | [WIP #62](https://github.com/TykTechnologies/tyk-operator/issues/62) |
| [Looping](./api_definitions/looping.md) | ⚠️ | Untested |
| Custom Plugins - Go | ⚠️ | Untested |
| [Custom Plugins - gRPC](./../bdd/features/api_http_grpc_plugin.feature) | ✅ | - |
| [Custom Plugins - Javascript](./api_definitions/custom_plugin.md) | ✅ | - |
| Custom Plugins - Lua | ⚠️ | Untested |
| Custom Plugins - Python | ️ | - |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth linking to docs/api_definitions/custom_plugin_python/custom_plugin_python.md from here

| Global Rate Limit | ❌ | Not Implemented |
| [Segment Tags](./../config/samples/httpbin_tagged.yaml) | ✅ | - |
| Tag Headers | ❌ | Not Implemented |
| [Webhooks](./webhooks.md) | ❌ | [WIP #62](https://github.com/TykTechnologies/tyk-operator/issues/62) |
| [Looping](./api_definitions/looping.md) | ⚠️ | Untested |

## APIDefinition - Endpoint Middleware

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
## Custom Plugin example using python

We assume that you already have a developed python plugin with the associated python file and its manifest in JSON.
If not, you can start from the provided examples ("manifest.json" and "middleware.py").
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Provide links to the manifest.json and middleware.py example files.

Furthermore, in this example, we are going to serve those as a bundle using a small http python server.


Firstly to create the said bundle we need to run the following command:

```
IMAGETAG=v3.2.1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This command won't run if your directory has no manifest.json attached to this PR. So, users should be in docs/api_definitions/custom_plugin_python directory.


docker run \
--rm -w "/tmp" -v $(pwd):/tmp \
--entrypoint "/bin/sh" -it \
tykio/tyk-gateway:$IMAGETAG \
-c '/opt/tyk-gateway/tyk bundle build -y'

```

We are setting the shell variable IMAGETAG to be the version of the gateway we intend on loading the python bundle onto.
In this case, we are loading the plugin onto a Tyk gateway v3.2.1. When completed, you should see a bundle.zip in your current directory.

Now we must serve the bundle, and the easiest way to do so is through a local server using the following command:
```
python3 -m http.server
```

Your bundle should now be accessible locally through the URL http://localhost:8000/bundle.zip,
or through the url http://host.minikube.internal:8000/bundle.zip from within minikube.
Copy link
Member

@buraksekili buraksekili May 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may sound silly but why are we mentioning minikube here? The above part indicates that I should see bundle.zip in my current directory. And after serving it, shouldn't it be accessible only locally? It seems like it shouldn't be accessible via my cluster. Maybe i am missing a feature of minikube 😄

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mentioned it since the gateway needs to be able to access the bundle.zip being served by the python fileserver on the HOST; in minikube, host access is permitted through host.minikube.internal, which is also why the env variable TYK_GW_BUNDLEBASEURL refers to http://host.minikube.internal:8000/.

If the python fileserver was deployed in the minkube cluster, we would simply use the k8s namespacing to refer to the bundle's hosted location

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, i see. thanks @JRWu for explanation 👍 should we update doc based on your explanation? it may be helpful for users.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes absolutely, I can add some more information to the markdown file

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you 🙏

Copy link
Member

@asoorm asoorm May 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also a bit confused about this. Targeting bundle server http://host.minikube.internal:8000/bundle.zip might work fine locally / in Minikube, but how would it work in production? I need a bundle server right? So that bundle server needs to be deployable to K8s (dockerized) also with a Deployment and Service that the Gateway can target?


Next we need to ensure that the python custom plugin with bundle features are enabled
in the gateway. Before deploying the tyk stack we need to add these config
variables to the "values.yaml" file:


```yaml
extraEnvs: [
{
"name": "TYK_GW_ENABLEBUNDLEDOWNLOADER",
"value": "true"
},
{
"name": "TYK_GW_BUNDLEBASEURL",
"value": "http://host.minikube.internal:8000/"
},
{
"name": "TYK_GW_BUNDLEINSECURESKIPVERIFY",
"value": "true"
},
{
"name": "TYK_GW_COPROCESSOPTIONS_ENABLECOPROCESS",
"value": "true"
},
{
"name": "TYK_GW_COPROCESSOPTIONS_PYTHONPATHPREFIX",
"value": "/opt/tyk-gateway"
}
]
```

Next we should install the tyk stack and the operator in the usual way.

After everything is up and running we need to create an API that actually
makes use of the python plugin. For this we can use the provided yaml manifest
"custom_plugin_python.yaml" via the command (assuming the tyk stack has been deployed inside the tyk namespace):

```
kubectl apply -f custom_plugin_python.yaml -n tyk
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that we shouldn't set path and namespace explicitly in the doc. We might use

  • heredoc:
cat <<EOF | kubectl apply -n <namespace> -f -
apiVersion: tyk.tyk.io/v1alpha1
kind: ApiDefinition
metadata:
  name: httpbin
spec:
  name: httpbin
  use_keyless: true
  protocol: http
  active: true
  proxy:
    target_url: http://httpbin.org
    listen_path: /httpbin
    strip_listen_path: true
  custom_middleware_bundle: "bundle.zip"
EOF
  • or if users have access to repository,
kubectl apply -f ./docs/api_definitions/custom_plugin_python.yaml -n <namespace>

```

After this command is run, any request to the "/httpbin/get" dashboard endpoint
should have a "Testheader" header injected with value "testvalue".
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
apiVersion: tyk.tyk.io/v1alpha1
kind: ApiDefinition
metadata:
name: httpbin
spec:
name: httpbin
use_keyless: true
protocol: http
active: true
proxy:
target_url: http://httpbin.org
listen_path: /httpbin
strip_listen_path: true
custom_middleware_bundle: "bundle.zip"

17 changes: 17 additions & 0 deletions docs/api_definitions/custom_plugin_python/manifest.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"file_list": [
"middleware.py"
],
"custom_middleware": {
"driver": "python",
"pre": [
{
"name": "SetHeader",
"path": "/get",
"require_session": false,
"raw_body_only": false
}
]
}
}

8 changes: 8 additions & 0 deletions docs/api_definitions/custom_plugin_python/middleware.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
from tyk.decorators import *
from gateway import TykGateway as tyk

@Hook
def SetHeader(request, session, spec):
tyk.log("PreHook is called", "info")
request.add_header("testheader", "testvalue")
return request, session