Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 60 additions & 62 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,23 +6,21 @@ This module implements several ways of sending authenticated requests
to the Kubernetes API from deno scripts.

Kubernetes is a complex architechure which likes using sophisticated networking concepts,
while Deno is a relatively young runtime, so there's some mismatch in capabilities.
Therefor one client implementation cannot work in every case,
and different Deno flags enable supporting different setups.
while Deno is a younger runtime, so there's some mismatch in capabilities.
Deno also has a granular permission system controlling what a program can access.
Therefore different Deno flags can be given to your program depending on where you are running it.

This library is intended as a building block.
If you are unsure how to issue a specific request from your own library/code,
or if your usage results in any `TODO: ...` error message from my code,
please feel free to file a Github Issue.
(NodeJS 20+ is also partially supported by this library)

## Usage

Here's a basic request, listing all Pods in the `default` namespace.
It uses the `autoDetectClient()` entrypoint which returns the first usable client.

Note: This example shows a manual HTTP request.
Note: The below example shows a manual HTTP request.
This library is intended as a building block for issuing raw HTTP requests.
To use the Kubernetes APIs more easily, consider also using
[/x/kubernetes_apis](https://deno.land/x/kubernetes_apis)
[@cloudydeno/kubernetes-apis](https://jsr.io/@cloudydeno/kubernetes-apis).

```ts
import { autoDetectClient } from 'https://deno.land/x/kubernetes_client/mod.ts';
Expand All @@ -38,18 +36,64 @@ console.log(podList);
// see demo.ts for more request examples (streaming responses, etc)
```

To get started on local development, `autoDetectClient` will most likely
decide to call out to your `kubectl`
installation to make each network call.
This only requires the `--allow-run=kubectl` Deno flag.
## Client Implementations

To use other clients, more flags are necesary.
See "Client Implementations" below for more information on flags and other HTTP clients.
The available clients are:

The `kubectl` client logs the issued commands if `--verbose` is passed to the Deno program.
1. `InCluster` which uses the current pod's service account to talk to the control plane.
With Deno, pass `--allow-read=/var/run/secrets/kubernetes.io --allow-net=kubernetes.default.svc.cluster.local`.
With NodeJS, set environment variable `NODE_EXTRA_CA_CERTS=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt`.

1. `KubectlRaw` which will run individual `kubectl` commands to send requests to Kubernetes.
This can be helpful when connecting to a cluster from a laptop.
Authentication won't be a problem since `kubectl` already handles all of that.
With Deno, pass `--allow-run=kubectl` to authorize this client.
Not yet supported with NodeJS.

1. `KubeConfig` which reads your `~/.kube/config` file and attempts to connect to the current default cluster.
Usually this works, but it might not always. Some of the issues would be fixable given a bug report.
With Deno, you'll probably want to pass at least `--allow-env --allow-net --allow-read=$HOME/.kube`.
Not yet supported with NodeJS.

1. `KubectlProxy` which simply sends HTTP requests to `http://localhost:8081`.
You'll need to first launch `kubectl proxy` in a separate terminal/shell and leave it running.
With Deno, pass `--allow-net=localhost:8001` to authorize this client.

An error message is shown when no client is usable, something like this:

```
Error: Failed to load any possible Kubernetes clients:
- InCluster PermissionDenied: Requires read access to "/var/run/secrets/kubernetes.io/serviceaccount/namespace", run again with the --allow-read flag
- KubeConfig PermissionDenied: Requires env access to "KUBECONFIG", run again with the --allow-env flag
- KubectlProxy PermissionDenied: Requires net access to "localhost:8001", run again with the --allow-net flag
- KubectlRaw PermissionDenied: Requires run access to "kubectl", run again with the --allow-run flag
```

## Programmatic API

You can also directly instantiate a particular client if you don't want to depend on autodetection.

* `KubectlRawRestClient` invokes `kubectl --raw` for every HTTP call.
Dependable for development, though a couple APIs are not possible to implement.

* `KubeConfigRestClient` uses `fetch()` to issue HTTP requests. There's a few different functions to configure it:

* `forInCluster()` uses a pod's ServiceAccount to automatically authenticate.

* `forKubectlProxy()` expects a `kubectl proxy` command to be runnin. This allows a full range-of-motion for development purposes regardless of the Kubernetes configuration.

* `readKubeConfig(path?, context?)` (or `forKubeConfig(config, context?)`) tries using the given config (or `$HOME/.kube/config` if none is given) as faithfully as possible. Passing a context name will override the `current-context` value from your config file.

## Development

Check out `lib/contract.ts` to see the type/API contract.

The `kubectl` client logs the issued commands if `--verbose` is passed to the Deno program.

If you are unsure how to issue a specific request from your own library/code,
or if your usage results in any `TODO: ...` error message from my code,
please feel free to file a Github Issue.

## Changelog

* `v0.7.0` on `2023-08-13`:
Expand Down Expand Up @@ -128,52 +172,6 @@ Check out `lib/contract.ts` to see the type/API contract.
* `v0.1.0` on `2020-11-16`: Initial publication, with `KubectlRaw` and `InCluster` clients.
Also includes `ReadableStream` transformers, useful for consuming watch streams.

# Client Implementations

An error message is shown when no client is usable, something like this:

```
Error: Failed to load any possible Kubernetes clients:
- InCluster PermissionDenied: Requires read access to "/var/run/secrets/kubernetes.io/serviceaccount/namespace", run again with the --allow-read flag
- KubeConfig PermissionDenied: Requires env access to "KUBECONFIG", run again with the --allow-env flag
- KubectlProxy PermissionDenied: Requires net access to "localhost:8001", run again with the --allow-net flag
- KubectlRaw PermissionDenied: Requires run access to "kubectl", run again with the --allow-run flag
```

Each client has different pros and cons:

* `KubectlRawRestClient` invokes `kubectl --raw` for every HTTP call.
Excellent for development, though a couple APIs are not possible to implement.

Flags: `--allow-run=kubectl`

* `KubeConfigRestClient` uses Deno's `fetch()` to issue HTTP requests.
There's a few different functions to configure it:

* `forInCluster()` uses a pod's ServiceAccount to automatically authenticate.
This is what is used when you deploy your script to a cluster.

Flags: `--allow-read=/var/run/secrets/kubernetes.io --allow-net=kubernetes.default.svc.cluster.local`

Lazy flags: `--allow-read --allow-net`

* `forKubectlProxy()` expects a `kubectl proxy` command to be running and talks directly to it without auth.

This allows a full range-of-motion for development purposes regardless of the Kubernetes configuration.

Flags: `--allow-net=localhost:8001` given that `kubectl proxy` is already running at that URL.

* `readKubeConfig(path?, context?)` (or `forKubeConfig(config, context?)`) tries using the given config (or `$HOME/.kube/config` if none is given) as faithfully as possible.

This requires a lot of flags depending on the config file,
and in some cases simply cannot work.
For example `https://<ip-address>` server values are not currently supported by Deno,
and invoking auth plugins such as `gcloud` aren't implemented yet,
so any short-lived tokens in the kubeconfig must already be fresh.
Trial & error works here :)

Entry-level flags: `--allow-env --allow-net --allow-read=$HOME/.kube`

## Related: API Typings

This module is only implementing the HTTP/transport part of talking to Kubernetes.
Expand Down
50 changes: 34 additions & 16 deletions lib/kubeconfig.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,24 @@ import { join as joinPath } from '@std/path/join';
import { resolve as resolvePath } from '@std/path/resolve';
import { parse as parseYaml } from '@std/yaml/parse';

// Remap several Deno APIs so that this library can possibly work under NodeJS
import { readFile } from 'node:fs/promises';
async function readTextFile(path: string) {
const buffer = await readFile(path);
return buffer.toString();
}
function getEnv(key: string) {
if (globalThis.Deno) return Deno.env.get(key);
return globalThis.process.env[key];
}

export class KubeConfig {
constructor(
public readonly data: RawKubeConfig,
) {}

static async readFromPath(path: string): Promise<KubeConfig> {
const data = parseYaml(await Deno.readTextFile(path));
const data = parseYaml(await readTextFile(path));
if (isRawKubeConfig(data)) {
resolveKubeConfigPaths(dirname(path), data);
return new KubeConfig(data);
Expand All @@ -25,13 +36,17 @@ export class KubeConfig {
}

static async getDefaultConfig(): Promise<KubeConfig> {
const delim = Deno.build.os === 'windows' ? ';' : ':';
const path = Deno.env.get("KUBECONFIG");
const isWindows = globalThis.Deno
? Deno.build.os === 'windows'
: globalThis.process.platform === 'win32';

const delim = isWindows ? ';' : ':';
const path = getEnv("KUBECONFIG");
const paths = path ? path.split(delim) : [];

if (!path) {
// default file is ignored if it't not found
const defaultPath = joinPath(Deno.env.get("HOME") || Deno.env.get("USERPROFILE") || "/root", ".kube", "config");
const defaultPath = joinPath(getEnv("HOME") || getEnv("USERPROFILE") || "/root", ".kube", "config");
try {
return await KubeConfig.readFromPath(defaultPath);
} catch (err: unknown) {
Expand All @@ -53,17 +68,20 @@ export class KubeConfig {
baseUrl = 'https://kubernetes.default.svc.cluster.local',
secretsPath = '/var/run/secrets/kubernetes.io/serviceaccount',
}={}): Promise<KubeConfig> {
// Avoid interactive prompting for in-cluster secrets.
// These are not commonly used from an interactive session.
const readPermission = await Deno.permissions.query({name: 'read', path: secretsPath});
if (readPermission.state !== 'granted') {
throw new Error(`Lacking --allow-read=${secretsPath}`);

if (globalThis.Deno) {
// Avoid interactive prompting for in-cluster secrets.
// These are not commonly used from an interactive session.
const readPermission = await Deno.permissions.query({name: 'read', path: secretsPath});
if (readPermission.state !== 'granted') {
throw new Error(`Lacking --allow-read=${secretsPath}`);
}
}

const [namespace, caData, tokenData] = await Promise.all([
Deno.readTextFile(joinPath(secretsPath, 'namespace')),
Deno.readTextFile(joinPath(secretsPath, 'ca.crt')),
Deno.readTextFile(joinPath(secretsPath, 'token')),
readTextFile(joinPath(secretsPath, 'namespace')),
readTextFile(joinPath(secretsPath, 'ca.crt')),
readTextFile(joinPath(secretsPath, 'token')),
]);

return new KubeConfig({
Expand Down Expand Up @@ -152,7 +170,7 @@ export class KubeConfigContext {
} | null> {
let serverCert = atob(this.cluster["certificate-authority-data"] ?? '') || null;
if (!serverCert && this.cluster["certificate-authority"]) {
serverCert = await Deno.readTextFile(this.cluster["certificate-authority"]);
serverCert = await readTextFile(this.cluster["certificate-authority"]);
}

if (serverCert) {
Expand All @@ -167,12 +185,12 @@ export class KubeConfigContext {
} | null> {
let userCert = atob(this.user["client-certificate-data"] ?? '') || null;
if (!userCert && this.user["client-certificate"]) {
userCert = await Deno.readTextFile(this.user["client-certificate"]);
userCert = await readTextFile(this.user["client-certificate"]);
}

let userKey = atob(this.user["client-key-data"] ?? '') || null;
if (!userKey && this.user["client-key"]) {
userKey = await Deno.readTextFile(this.user["client-key"]);
userKey = await readTextFile(this.user["client-key"]);
}

if (!userKey && !userCert && this.user.exec) {
Expand Down Expand Up @@ -202,7 +220,7 @@ export class KubeConfigContext {
return `Bearer ${this.user.token}`;

} else if (this.user.tokenFile) {
const token = await Deno.readTextFile(this.user.tokenFile);
const token = await readTextFile(this.user.tokenFile);
return `Bearer ${token.trim()}`;

} else if (this.user['auth-provider']) {
Expand Down
2 changes: 1 addition & 1 deletion transports/mod.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ export class ClientProviderChain {
const srcName = ` - ${label} `;
if (err instanceof Error) {
// if (err.message !== 'No credentials found') {
errors.push(srcName+(err.stack?.split('\n')[0] || err.message));
errors.push(srcName+(err.cause || err.stack?.split('\n')[0] || err.message));
// }
} else if (err) {
errors.push(srcName+err.toString());
Expand Down
9 changes: 3 additions & 6 deletions transports/via-kubeconfig.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import type { RestClient, RequestOptions, JSONValue, KubernetesTunnel } from '..
import { JsonParsingTransformer } from '../lib/stream-transformers.ts';
import { KubeConfig, type KubeConfigContext } from '../lib/kubeconfig.ts';

const isVerbose = Deno.args.includes('--verbose');
const isVerbose = globalThis.Deno?.args.includes('--verbose');

/**
* A RestClient which uses a KubeConfig to talk directly to a Kubernetes endpoint.
Expand All @@ -17,12 +17,9 @@ const isVerbose = Deno.args.includes('--verbose');
*
* Deno flags to use this client:
* Basic KubeConfig: --allow-read=$HOME/.kube --allow-net --allow-env
* CA cert fix: --unstable-http --allow-read=$HOME/.kube --allow-net --allow-env
* In-cluster 1: --allow-read=/var/run/secrets/kubernetes.io --allow-net --unstable-http
* In-cluster 2: --allow-read=/var/run/secrets/kubernetes.io --allow-net --cert=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
* In-cluster: --allow-read=/var/run/secrets/kubernetes.io --allow-net
*
* Unstable features:
* - using the cluster's CA when fetching (otherwise pass --cert to Deno)
* - using client auth authentication, if configured
* - inspecting permissions and prompting for further permissions (TODO)
*
Expand Down Expand Up @@ -77,7 +74,7 @@ export class KubeConfigRestClient implements RestClient {

let httpClient: Deno.HttpClient | null = null;
if (serverTls || tlsAuth) {
if (Deno.createHttpClient) {
if (globalThis.Deno && Deno.createHttpClient) {
httpClient = Deno.createHttpClient({
caCerts: serverTls ? [serverTls.serverCert] : [],
//@ts-ignore-error deno unstable API. Not typed?
Expand Down
2 changes: 1 addition & 1 deletion transports/via-kubectl-raw.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { TextLineStream } from '@std/streams/text-line-stream';
import type { RestClient, RequestOptions, JSONValue, KubernetesTunnel } from '../lib/contract.ts';
import { JsonParsingTransformer } from '../lib/stream-transformers.ts';

const isVerbose = Deno.args.includes('--verbose');
const isVerbose = globalThis.Deno?.args.includes('--verbose');

/**
* A RestClient for easily running on a developer's local machine.
Expand Down
Loading