Implement http.RoundTripper

[EtienneBruines]

To make the HTTP function more accessible to Go developers, it would be nice if go-pdk implemented http.RoundTripper to process standard HTTP requests. This way, a larger ecosystem of middleware can be used by plugins.

Ideally, this would allow a plugin like this:

package main

import (
    "net/http"

    "github.com/extism/go-pdk"
)

//export http_get_example1
func http_get_example1() int32 {
    req, _ := http.NewRequest("GET", "https://example.org", nil)
    resp, _ := pdk.HttpClient.Do(req)
    // Process the response

    return 0
}

//export http_get_example2
func http_get_example2() int32 {
    client := http.Client{
        RoundTripper: pdk.RoundTripper{},
    }

    req, _ := http.NewRequest("GET", "https://example.org", nil)
    resp, _ := client.Do(req)
    // Process the response

    return 0
}

There might be quite some things that the pdk.RoundTripper cannot support (streaming responses, for example). But since that's not supported anyways, we could just return an error explaining what isn't supported. For basic requests/responses this should be relatively doable.

The `http.RoundTripper interface is an easy one to implement:

https://github.com/golang/go/blob/a031f4ef83edc132d5f49382bfef491161de2476/src/net/http/client.go#L117-L143

Thoughts?

[bhelx]

Sorry, somehow we missed this issue. I'm definitely open to it! There is some overlap with our HTTP impl but we could probably re-use the objects.

[nilslice]

Out of curiosity, I decided to try and implement this. It works when compiling Wasm code from the go toolchain! But I can't figure out why tinygo compiled code blows up. I haven't spent much time looking around to see if tinygo has some inconsistencies with the stdlib net/http package though.

The problem with it, which I am not willing to really accept in order to make this part of the PDK is the size of the Wasm, which is over 5x the size when compiled with tinygo.

Here's the implementation, and a client that uses the RoundTripper:

package stdhttp_compat

import (
	"bytes"
	"fmt"
	"io"
	"net/http"

	"github.com/extism/go-pdk"
)

type roundTripper struct{}

// convert the *http.Request into an Extism HTTP Request, send it, and convert the
// Extism HTTP Response into the *http.Response.
func (r roundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
	var method pdk.HTTPMethod
	switch req.Method {
	case http.MethodGet:
		method = pdk.MethodGet
	case http.MethodHead:
		method = pdk.MethodHead
	case http.MethodPost:
		method = pdk.MethodPost
	case http.MethodPut:
		method = pdk.MethodPut
	case http.MethodPatch:
		method = pdk.MethodPatch
	case http.MethodDelete:
		method = pdk.MethodDelete
	case http.MethodConnect:
		method = pdk.MethodConnect
	case http.MethodOptions:
		method = pdk.MethodOptions
	case http.MethodTrace:
		method = pdk.MethodTrace
	default:
		method = pdk.MethodGet
	}

	extismReq := pdk.NewHTTPRequest(method, req.URL.String())

	var body []byte
	if req.Body != nil {
		var buf bytes.Buffer
		_, err := io.Copy(&buf, req.Body)
		if err != nil {
			return nil, err
		}
		body = buf.Bytes()
		req.Body.Close()
	}
	extismReq.SetBody(body)
	for k := range req.Header {
		extismReq.SetHeader(k, req.Header.Get(k))
	}

	extismResp := extismReq.Send()
	var buf bytes.Buffer
	conentLength, err := buf.Write(extismResp.Body())
	if err != nil {
		return nil, err
	}

	bufCloser := io.NopCloser(&buf)

	res := &http.Response{
		Status:           fmt.Sprintf("%d", extismResp.Status()),
		StatusCode:       int(extismResp.Status()),
		Proto:            req.Proto,
		ProtoMajor:       req.ProtoMajor,
		ProtoMinor:       req.ProtoMinor,
		Header:           nil,
		Body:             bufCloser,
		ContentLength:    int64(conentLength),
		TransferEncoding: nil,
		Close:            false,
		Uncompressed:     false,
		Trailer:          nil,
		Request:          req,
	}

	return res, nil
}

var Client = &http.Client{Transport: roundTripper{}}

[syke99]

hey @nilslice! the reason building it with tinygo doesn't end up working is because of exactly what you were curious about; tinygo's implementation of the net/http isn't implemented yet (not technically importable, you can find it in the list here).