NEOS-1545: Update TS API Docs (#2808)

docs/protos/go.md

@@ -8,40 +8,38 @@ slug: /go
 
 ## Introduction
 
-The Neosync Go SDK is publicly available and can be added to any Go-based project by running `go get github.com/nucleuscloud/neosync`.
+The Neosync Go SDK is publicly available and can be added to any Go project. With the Neosync Go SDK, you can:
 
-Neosync's CLI is the primary user of the Go SDK today, and can be referenced for examples of how to use the SDK.
+1. Anonymize structured data and generate synthetic data
+2. Anonymize free-form text data
+3. Create resources in Neosync such as Jobs, Connections, Transformers and more
 
-## Configuration
+## Installation
 
-There are a few inputs that the SDK needs in order to be properly configured.
+You can add the Neosync Go SDK using:
 
-1. API URL
-2. Account ID
-3. API Key (required for Neosync Cloud or self-hosted authenticated environments)
+`go get github.com/nucleuscloud/neosync`.
 
-### API Url
+## Prerequisites
 
-If using Neosync Cloud, the backend api url is: `https://neosync-api.svcs.neosync.dev`
+There are a few prerequisites that the SDK needs in order to be properly configured.
 
-The standard localhost url is: `http://localhost:8080`
+| **Properties** | **Details**                                                                                                                                                                 |
+| -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **API URL**    | Production: `https://neosync-api.svcs.neosync.dev`<br /> Local: `http://localhost:8080`                                                                                     |
+| **Account ID** | The account ID may be necessary for some requests and can be found by going into the `/:accountName/settings` page in the Neosync App                                       |
+| **API Key**    | An access token (API key, or user JWT) must be used to access authenticated Neosync environments. For an API Key, this can be created at `/:accountName/settings/api-keys`. |
 
-### Account ID
+## Authentication
 
-The account ID is necessary for some requests that do not have an obvious identifier like retrieving a list of jobs, or a list of connections.
-This can be found by going into the app on the `/:accountName/settings` page and found in the header.
+If you are using Neosync locally and in unauthenticated mode then there is no authentication required and you can move onto the [Getting Started](go#getting-started) section.
 
-### API Key
+If you are using Neosync locally in `auth mode` or using Neosync Cloud, you can authenticate with the Neosync server using an API URL and API Key. There are two ways to provide the authentication header.
 
-An access token (api key, or user jwt) must be used to access authenticated Neosync environments.
-For an API Key, this can be created at `/:accountName/settings/api-keys`.
-
-## Getting Started
-
-Neosync is made up of a number of different services that live inside of the same process.
-They are roughly split up in terms of their resource types, and correspond nicely with what resources that are found in the web application.
+1. Attaching to the HTTP client
+2. Providing an interceptor to the SDK Clients that patch in the header on every request.
 
-We can initialize the job service client to trigger a new job run like so:
+The example below shows the first option which attaches the API Key to the HTTP Client header:
 
 ```go
 package main
@@ -50,7 +48,6 @@ import (
 	"context"
 	"fmt"
 	"net/http"
-	"os"
 
 	"connectrpc.com/connect"
 	mgmtv1alpha1 "github.com/nucleuscloud/neosync/backend/gen/go/protos/mgmt/v1alpha1"
@@ -59,22 +56,49 @@ import (
 
 func main() {
 	jobclient := mgmtv1alpha1connect.NewJobServiceClient(
-		http.DefaultClient,
-		os.Getenv("NEOSYNC_API_URL"),
+		newHttpClient(map[string]string{ // create an instance of a newHttpClient
+			"Authorization": fmt.Sprintf("Bearer %s", os.GetEnv("API_KEY")), // pass in the API_KEY through an environment variable
+		}),
+		os.GetEnv("API_URL"), // pass in the API_URL through an environment variable
 	)
 
-	_, err := jobclient.CreateJobRun(context.Background(), connect.NewRequest(&mgmtv1alpha1.CreateJobRunRequest{
-		JobId: "<neosync-job-id>",
-	}))
-	if err != nil {
-		panic(err)
+	// rest of code to call an API in the JobServiceClient goes here
+	// ...
+}
+
+func newHttpClient(
+	headers map[string]string,
+) *http.Client {
+	return &http.Client{
+		Transport: &headerTransport{
+			Transport: http.DefaultTransport,
+			Headers:   headers,
+		},
+	}
+}
+
+type headerTransport struct {
+	Transport http.RoundTripper
+	Headers   map[string]string
+}
+
+func (t *headerTransport) RoundTrip(req *http.Request) (*http.Response, error) {
+	if req.Header == nil {
+		req.Header = http.Header{}
 	}
+	for key, value := range t.Headers {
+		req.Header.Add(key, value)
+	}
+	return t.Transport.RoundTrip(req)
 }
+
 ```
 
-## Go SDK Packages
+## Getting started
+
+In this section, we're going to walk through two examples that show you how to make an API call using Neosync's GO SDK. For a complete list of the APIs, check out the APIs in the `Services` section of our [protos](/api/mgmt/v1alpha1/job.proto#jobservice).
 
-There are two packages that are made available for connecting to a Neosync API.
+Neosync is made up of a number of different services that live inside of the same process. In order to connect to the Neosync API and use the services, we make two packages available:
 
 - `github.com/nucleuscloud/neosync/backend/gen/go/protos/mgmt/v1alpha1`
 - `github.com/nucleuscloud/neosync/backend/gen/go/protos/mgmt/v1alpha1/mgmtv1alpha1connect`
@@ -83,37 +107,192 @@ The first package is made up of the generated types. This includes all of the re
 
 The second package are where the client and server structs live for use with either creating a Neosync Client, or Neosync Server.
 
-## Adding Authentication to the SDK
-
-If developing Neosync locally, it's easy to run in the default mode that doesn't require authentication.
-However, if interacting with a production environment, it most likely will require authentication like an API Key.
+### Anonymizing Structured Data
+
+A straightforward use case is to anonymize sensitive data in an API request. Let's look at an example.
+
+```json
+// input
+{
+  "user": {
+    "name": "John Doe",
+    "email": "[email protected]"
+  },
+  "details": {
+    "address": "123 Main St",
+    "phone": "555-1234",
+    "favorites": ["dog", "cat", "bird"]
+  }
+}
+```
 
-Providing the authentication header can be a little wonky in Go and can be done in two different ways.
+Our input object is a simple user's object that we may get through a user sign up flow. In this object, we have a few sensitive fields that we want to anonymize: `name`, `email`, `address` and `phone`. We can leave the `favorites` as-is for now.
 
-1. Attaching to the HTTP client
-2. Providing an interceptor to the SDK Clients that patch in the header on every request.
+In order to anonymize this object, you can use Neosync's `AnonymizeSingle` API to send in a single object with sensitive data and get back an anonymized version of that object. You have full control over how you anonymize the data or generate new synthetic data.
 
-The example below shows how to augment the HTTP Client to include the header:
+Here's how you do it:
 
 ```go
 package main
 
 import (
+	"context"
+	"encoding/json"
 	"fmt"
 	"net/http"
-	"os"
 
+	"connectrpc.com/connect"
+	mgmtv1alpha1 "github.com/nucleuscloud/neosync/backend/gen/go/protos/mgmt/v1alpha1"
 	"github.com/nucleuscloud/neosync/backend/gen/go/protos/mgmt/v1alpha1/mgmtv1alpha1connect"
 )
 
+// define our User struct and use json tags to structure our json
+type User struct {
+	User    UserDefinition `json:"user"`
+	Details UserDetails    `json:"details"`
+}
+
+type UserDefinition struct {
+	Name  string `json:"name"`
+	Email string `json:"email"`
+}
+
+type UserDetails struct {
+	Address   string   `json:"address"`
+	Phone     string   `json:"phone"`
+	Favorites []string `json:"favorites"`
+}
+
+func main() {
+	anonymizeClient := mgmtv1alpha1connect.NewAnonymizationServiceClient(
+			newHttpClient(map[string]string{ // create an instance of a newHttpClient
+			"Authorization": fmt.Sprintf("Bearer %s", os.GetEnv("API_KEY")), // pass in the API_KEY through an environment variable
+		}),
+		os.GetEnv("API_URL"), // pass in the API_URL through an environment variable
+	)
+
+	inputData := User{
+		User: UserDefinition{
+			Name:  "Bob Smith",
+			Email: "[email protected]",
+		},
+		Details: UserDetails{
+			Address: "123 Main St",
+			Phone:   "555-1234",
+			Favorites: []string{
+				"cat", "dog", "cow",
+			},
+		},
+	}
+
+	transformerMappings := []*mgmtv1alpha1.TransformerMapping{
+		{
+			Expression: `(.. | objects | select(has("name")) | .name)`, // find and transform all name fields in objects
+			Transformer: &mgmtv1alpha1.TransformerConfig{
+				Config: &mgmtv1alpha1.TransformerConfig_TransformFullNameConfig{
+					TransformFullNameConfig: &mgmtv1alpha1.TransformFullName{
+						PreserveLength: true,
+					},
+				},
+			},
+		},
+		{
+			Expression: `.user.email`, // transform user.email field
+			Transformer: &mgmtv1alpha1.TransformerConfig{
+				Config: &mgmtv1alpha1.TransformerConfig_TransformEmailConfig{
+					TransformEmailConfig: &mgmtv1alpha1.TransformEmail{},
+				},
+			},
+		},
+		{
+			Expression: `.details.favorites[]`, // transform each element in details.favorite array
+			Transformer: &mgmtv1alpha1.TransformerConfig{
+				Config: &mgmtv1alpha1.TransformerConfig_TransformCharacterScrambleConfig{
+					TransformCharacterScrambleConfig: &mgmtv1alpha1.TransformCharacterScramble{},
+				},
+			},
+		},
+	}
+
+	// marshal our object into bytes
+	userBytes, err := json.Marshal(inputData)
+	if err != nil {
+		panic(err)
+	}
+
+	resp, err := anonymizeClient.AnonymizeSingle(context.Background(), connect.NewRequest(&mgmtv1alpha1.AnonymizeSingleRequest{
+		InputData:           string(userBytes), //stringify our bytes
+		TransformerMappings: transformerMappings,
+	}))
+	if err != nil {
+		fmt.Printf("Error in AnonymizeSingle: %v\n", err)
+		panic(err)
+	}
+
+	fmt.Printf("Anonymization response: %+v\n", resp.Msg.OutputData)
+}
+
+func newHttpClient(
+	headers map[string]string,
+) *http.Client {
+	return &http.Client{
+		Transport: &headerTransport{
+			Transport: http.DefaultTransport,
+			Headers:   headers,
+		},
+	}
+}
+
+type headerTransport struct {
+	Transport http.RoundTripper
+	Headers   map[string]string
+}
+
+func (t *headerTransport) RoundTrip(req *http.Request) (*http.Response, error) {
+	if req.Header == nil {
+		req.Header = http.Header{}
+	}
+	for key, value := range t.Headers {
+		req.Header.Add(key, value)
+	}
+	return t.Transport.RoundTrip(req)
+}
+
+```
+
+Let's take a closer look at what we're doing here. Neosync's AnonymizeSingle API uses [JQ](https://jqlang.github.io/jq/manual/) expressions to target field(s) in your object. This means that you don't have to parse your object before sending it to Neosync. You can pass it in as-is and just write JQ expressions to target the field(s) that you want to anonymize or generate.
+
+Our output will look something like this:
+
+```json
+// output
+"{\"details\":{\"address\":\"123 Main St\",\"favorites\":[\"idh\",\"tyj\",\"ean\"],\"phone\":\"555-1234\"},\"user\":{\"email\":\"[email protected]\",\"name\":\"Nim Racic\"}}"
+```
+
+That's it! The power of JQ is that you can use it to target any field of any type, even searching across multiple objects for similar named fields and more. It's truly the most flexible way to transform your data.
+
+### Triggering a Job Run
+
+Another common use case is to create resources in Neosync such as Jobs, Connections, Runs, Transformers and more. In this example, we'll trigger a Job which will create a Job Run. This can be used as part of a set-up script or custom workflow. Let's take a look at the code:
+
+Let's augment our code from above to call the `CreateJobRun` API.
+
+```go
 func main() {
 	jobclient := mgmtv1alpha1connect.NewJobServiceClient(
-		newHttpClient(map[string]string{
-			"Authorization": fmt.Sprintf("Bearer %s", os.Getenv("NEOSYNC_API_KEY")),
+		newHttpClient(map[string]string{ // create an instance of a newHttpClient
+			"Authorization": fmt.Sprintf("Bearer %s", os.GetEnv("API_KEY")), // pass in the API_KEY through an environment variable
 		}),
-		os.Getenv("NEOSYNC_API_URL"),
+		os.GetEnv("API_URL"), // pass in the API_URL through an environment variable
 	)
-	_ = jobclient
+
+	// Calling the CreateJobRun in the JobServiceClient
+	_, err := jobclient.CreateJobRun(context.Background(), connect.NewRequest(&mgmtv1alpha1.CreateJobRunRequest{
+		JobId: "<job-id>",
+	}))
+	if err != nil {
+		panic(err)
+	}
 }
 
 func newHttpClient(
@@ -141,4 +320,9 @@ func (t *headerTransport) RoundTrip(req *http.Request) (*http.Response, error) {
 	}
 	return t.Transport.RoundTrip(req)
 }
+
 ```
+
+## Moving forward
+
+Now that you've seen how to anonymize data, generate synthetic data and create resources in Neosync, you can use the Neosync Go SDK to do much more! And if you have any questions, we're always available in Discord to help.

docs/protos/home.md

@@ -8,10 +8,9 @@ slug: /
 
 ## Introduction
 
-Neosync, like many web services these days, is a full-fledged API. Check out the [architecture](/platform) page for an overview of how the Neosync API fits into the picture.
+Neosync has first-class APIs and SDKs that developers can use to integrate Neosync into their workflow. To learn more about how the Neosync API fits into the overall architecture, check out the Check out the [platform page](/platform).
 
-All of our APIs are generated from Protobuf files. For this reason, a client can easily be generated in any of the many supported languages.
-Internally, we make great use of the Go SDK as well as the TypeScript SDK.
+Neosync API serves up [Connect](https://github.com/connectrpc), which can listen using Connect, gRPC, or HTTP protocols. All of our APIs are generated from Protobuf files and our SDKs call Connect endpoints by default. Each SDK can be configured to use gRPC or REST in lieu of the default (Connect).
 
 ## Configuration