Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate env vars with Zod #2362

Open
wants to merge 42 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
ddd10d1
WIP: Enables strict null checks in SDK
infomiho Oct 28, 2024
90baa94
Update handleApiError type
infomiho Oct 28, 2024
b5c3c63
Throw invalid credentials error explictly
infomiho Oct 28, 2024
7afc4f0
Formatting
infomiho Oct 28, 2024
b4e2362
Update comment and type
infomiho Oct 28, 2024
465856c
Update email env vars
infomiho Oct 28, 2024
5c329a4
Update isHttpErrorWithExtraMessage
infomiho Oct 28, 2024
e04aac3
Update comment
infomiho Oct 28, 2024
1928b6f
Remove TODO
infomiho Oct 28, 2024
e231dda
Fixes jobs types
infomiho Oct 28, 2024
f431153
Update todoApp tests. Comment update.
infomiho Oct 28, 2024
c8b2120
Update e2e tests
infomiho Oct 28, 2024
08304fe
Fixes headless tests
infomiho Oct 28, 2024
a509762
Fixes CORS error
infomiho Oct 28, 2024
9d618ea
Zod env validation WIP
infomiho Oct 25, 2024
7333b25
Define env vars validation. Use validate env vars.
infomiho Oct 29, 2024
80baf9c
Update e2e tests
infomiho Oct 29, 2024
fb899a0
Clean up
infomiho Oct 29, 2024
7593ac2
Update SKIP_EMAIL_VERIFICATION_IN_DEV validation
infomiho Oct 29, 2024
42d121c
Update headless tests
infomiho Oct 29, 2024
f877543
Update headless tests
infomiho Oct 30, 2024
025d185
Simplify the server config
infomiho Oct 30, 2024
1da4542
Fixes keycloak env usage
infomiho Oct 30, 2024
67de093
Merge branch 'main' into miho-zod-env
infomiho Nov 26, 2024
ee7c6de
Cleanup
infomiho Nov 26, 2024
7b0fedd
Cleanup
infomiho Nov 26, 2024
598eeab
Cleanup
infomiho Nov 26, 2024
4732a27
Cleanup
infomiho Nov 26, 2024
fcdc0d6
Cleanup
infomiho Nov 26, 2024
68e99ba
Fixes headless tests
infomiho Nov 26, 2024
c740e4e
Fixes e2e tests
infomiho Nov 26, 2024
88d7ea9
Cleanup
infomiho Nov 29, 2024
107fa51
Update e2e tests
infomiho Nov 29, 2024
42e9c78
Update API comment
infomiho Nov 29, 2024
9dc30c1
Update waspc/data/Generator/templates/sdk/wasp/server/env.ts
infomiho Dec 13, 2024
4766a94
Merge branch 'main' into miho-zod-env
infomiho Dec 13, 2024
ba46aa8
PR comments
infomiho Dec 16, 2024
920cc90
Merge branch 'main' into miho-zod-env
infomiho Jan 2, 2025
835e17a
Cleanup
infomiho Jan 2, 2025
d275338
Use single prettier.config.js for all templates
infomiho Jan 2, 2025
73e5e11
Update required error messages. jwtTokenSchema conditionally added.
infomiho Jan 2, 2025
41e24be
e2e tests
infomiho Jan 2, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions waspc/data/Generator/templates/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -58,12 +58,12 @@ COPY --from=server-builder /app/node_modules ./node_modules
# Copying the SDK because 'validate-env.mjs' executes independent of the bundle
# and references the 'wasp' package.
COPY --from=server-builder /app/.wasp/out/sdk .wasp/out/sdk
# Copying 'server/node_modules' because 'validate-env.mjs' executes independent
# of the bundle and references the dotenv package.
# Copying 'server/node_modules' because we require dotenv package to
# load environment variables
# TODO: replace dotenv with native Node.js environment variable loading
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is that, what is wrong with dotenv? Node.js now has native support for it that is equally good?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's always nicer to use natively supported methods than a package to do the same thing. It's available since Node.js 20 and we should probably go for it when Node.js 20 becomes our minimum version.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probabl make nodejs 20 our minimum version any moment hm. Ok, so it does do the same thing? If so, great.

COPY --from=server-builder /app/.wasp/build/server/node_modules .wasp/build/server/node_modules
COPY --from=server-builder /app/.wasp/build/server/bundle .wasp/build/server/bundle
COPY --from=server-builder /app/.wasp/build/server/package*.json .wasp/build/server/
COPY --from=server-builder /app/.wasp/build/server/scripts .wasp/build/server/scripts
infomiho marked this conversation as resolved.
Show resolved Hide resolved
COPY db/ .wasp/build/db/
EXPOSE ${PORT}
WORKDIR /app/.wasp/build/server
Expand Down
4 changes: 2 additions & 2 deletions waspc/data/Generator/templates/sdk/wasp/client/config.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{{={= =}=}}
import { stripTrailingSlash } from '../universal/url.js'
import { env } from './env.js'

const apiUrl = stripTrailingSlash(import.meta.env.REACT_APP_API_URL) || '{= defaultServerUrl =}';
const apiUrl = stripTrailingSlash(env.REACT_APP_API_URL)
infomiho marked this conversation as resolved.
Show resolved Hide resolved

// PUBLIC API
export type ClientConfig = {
Expand Down
14 changes: 14 additions & 0 deletions waspc/data/Generator/templates/sdk/wasp/client/env.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{{={= =}=}}
import * as z from 'zod'

import { ensureEnvSchema } from '../env/index.js'

const clientEnvSchema = z.object({
REACT_APP_API_URL: z
.string({
required_error: 'REACT_APP_API_URL is required',
})
.default('{= defaultServerUrl =}')
})

export const env = ensureEnvSchema(import.meta.env, clientEnvSchema)
3 changes: 3 additions & 0 deletions waspc/data/Generator/templates/sdk/wasp/client/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,6 @@ export type Route = { method: HttpMethod; path: string }

// PUBLIC API
export { config, ClientConfig } from './config'

// PUBLIC API
export { env } from './env.js'
sodic marked this conversation as resolved.
Show resolved Hide resolved
29 changes: 29 additions & 0 deletions waspc/data/Generator/templates/sdk/wasp/env/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
import * as z from 'zod'

const redColor = '\x1b[31m'

export function ensureEnvSchema<Schema extends z.ZodTypeAny>(
data: unknown,
schema: Schema
): z.infer<Schema> {
try {
return schema.parse(data)
} catch (e) {
// TODO: figure out how to output the error message in a better way
infomiho marked this conversation as resolved.
Show resolved Hide resolved
if (e instanceof z.ZodError) {
console.error()
console.error(redColor, '╔═════════════════════════════╗');
console.error(redColor, '║ Env vars validation failed ║');
console.error(redColor, '╚═════════════════════════════╝');
console.error()
for (const error of e.errors) {
console.error(redColor, `- ${error.message}`)
}
console.error()
console.error(redColor, '═══════════════════════════════');
throw new Error('Error parsing environment variables')
} else {
throw e
}
}
}
7 changes: 7 additions & 0 deletions waspc/data/Generator/templates/sdk/wasp/prettier.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
// Used for internal Wasp development only, not copied to generated app.
module.exports = {
sodic marked this conversation as resolved.
Show resolved Hide resolved
trailingComma: 'es5',
tabWidth: 2,
semi: false,
singleQuote: true,
}
15 changes: 0 additions & 15 deletions waspc/data/Generator/templates/sdk/wasp/server/auth/oauth/env.ts

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,23 +1,19 @@
import { OAuth2Provider, OAuth2ProviderWithPKCE } from "arctic";

export function defineProvider<
OAuthClient extends OAuth2Provider | OAuth2ProviderWithPKCE,
Env extends Record<string, string>
OAuthClient extends OAuth2Provider | OAuth2ProviderWithPKCE
>({
id,
displayName,
env,
oAuthClient,
}: {
id: string;
displayName: string;
env: Env;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of OAuth env vars having their special validation and living in a special place, they are now used directly from the env object.

oAuthClient: OAuthClient;
}) {
return {
id,
displayName,
env,
oAuthClient,
};
}
Original file line number Diff line number Diff line change
@@ -1,17 +1,12 @@
{{={= =}=}}
import { Discord } from "arctic";
import { Discord } from 'arctic';

import { defineProvider } from "../provider.js";
import { ensureEnvVarsForProvider } from "../env.js";
import { getRedirectUriForCallback } from "../redirect.js";
import { defineProvider } from '../provider.js';
import { getRedirectUriForCallback } from '../redirect.js';
import { env } from '../../../env.js';

const id = "{= providerId =}";
const displayName = "{= displayName =}";

const env = ensureEnvVarsForProvider(
["DISCORD_CLIENT_ID", "DISCORD_CLIENT_SECRET"],
displayName
);
const id = '{= providerId =}';
const displayName = '{= displayName =}';

const oAuthClient = new Discord(
env.DISCORD_CLIENT_ID,
Expand All @@ -23,6 +18,5 @@ const oAuthClient = new Discord(
export const discord = defineProvider({
id,
displayName,
env,
oAuthClient,
});
Original file line number Diff line number Diff line change
@@ -1,16 +1,11 @@
{{={= =}=}}
import { GitHub } from "arctic";
import { GitHub } from 'arctic';

import { ensureEnvVarsForProvider } from "../env.js";
import { defineProvider } from "../provider.js";
import { defineProvider } from '../provider.js';
import { env } from '../../../env.js';

const id = "{= providerId =}";
const displayName = "{= displayName =}";

const env = ensureEnvVarsForProvider(
["GITHUB_CLIENT_ID", "GITHUB_CLIENT_SECRET"],
displayName
);
const id = '{= providerId =}';
const displayName = '{= displayName =}';

const oAuthClient = new GitHub(
env.GITHUB_CLIENT_ID,
Expand All @@ -21,6 +16,5 @@ const oAuthClient = new GitHub(
export const github = defineProvider({
id,
displayName,
env,
oAuthClient,
});
Original file line number Diff line number Diff line change
@@ -1,17 +1,12 @@
{{={= =}=}}
import { Google } from "arctic";
import { Google } from 'arctic';

import { ensureEnvVarsForProvider } from "../env.js";
import { getRedirectUriForCallback } from "../redirect.js";
import { defineProvider } from "../provider.js";
import { getRedirectUriForCallback } from '../redirect.js';
import { defineProvider } from '../provider.js';
import { env } from '../../../env.js';

const id = "{= providerId =}";
const displayName = "{= displayName =}";

const env = ensureEnvVarsForProvider(
["GOOGLE_CLIENT_ID", "GOOGLE_CLIENT_SECRET"],
displayName,
);
const id = '{= providerId =}';
const displayName = '{= displayName =}';

const oAuthClient = new Google(
env.GOOGLE_CLIENT_ID,
Expand All @@ -23,6 +18,5 @@ const oAuthClient = new Google(
export const google = defineProvider({
id,
displayName,
env,
oAuthClient,
});
Original file line number Diff line number Diff line change
@@ -1,17 +1,12 @@
{{={= =}=}}
import { Keycloak } from "arctic";
import { Keycloak } from 'arctic';

import { ensureEnvVarsForProvider } from "../env.js";
import { getRedirectUriForCallback } from "../redirect.js";
import { defineProvider } from "../provider.js";
import { getRedirectUriForCallback } from '../redirect.js';
import { defineProvider } from '../provider.js';
import { env } from '../../../env.js';

const id = "{= providerId =}";
const displayName = "{= displayName =}";

const env = ensureEnvVarsForProvider(
["KEYCLOAK_REALM_URL", "KEYCLOAK_CLIENT_ID", "KEYCLOAK_CLIENT_SECRET"],
displayName,
);
const id = '{= providerId =}';
const displayName = '{= displayName =}';

const oAuthClient = new Keycloak(
env.KEYCLOAK_REALM_URL,
Expand All @@ -24,6 +19,5 @@ const oAuthClient = new Keycloak(
export const keycloak = defineProvider({
id,
displayName,
env,
oAuthClient,
});
111 changes: 32 additions & 79 deletions waspc/data/Generator/templates/sdk/wasp/server/config.ts
Original file line number Diff line number Diff line change
@@ -1,94 +1,47 @@
{{={= =}=}}
import merge from 'lodash.merge'
import { env } from './env.js'
import { stripTrailingSlash } from '../universal/url.js'

import { stripTrailingSlash } from "../universal/url.js";
type NodeEnv = typeof env.NODE_ENV

const nodeEnv = process.env.NODE_ENV ?? 'development'

// TODO:
// - Use dotenv library to consume env vars from a file.
// - Use convict library to define schema and validate env vars.
// https://codingsans.com/blog/node-config-best-practices

type BaseConfig = {
type Config = {
env: NodeEnv;
isDevelopment: boolean;
port: number;
databaseUrl: string;
frontendUrl: string;
serverUrl: string;
allowedCORSOrigins: string | string[];
{=# isAuthEnabled =}
auth: {
jwtSecret: string | undefined;
jwtSecret: string;
}
{=/ isAuthEnabled =}
}

type CommonConfig = BaseConfig & {
env: string;
isDevelopment: boolean;
port: number;
databaseUrl: string | undefined;
}
const frontendUrl = stripTrailingSlash(env.WASP_WEB_CLIENT_URL)
const serverUrl = stripTrailingSlash(env.WASP_SERVER_URL)

type EnvConfig = BaseConfig & {
frontendUrl: string;
serverUrl: string;
const allowedCORSOriginsPerEnv: Record<NodeEnv, string | string[]> = {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default CORS values per NODE_ENV, same as before.

development: '*',
production: [frontendUrl]
}

type Config = CommonConfig & EnvConfig

const config: {
all: CommonConfig,
development: EnvConfig,
production: EnvConfig,
} = {
all: {
env: nodeEnv,
isDevelopment: nodeEnv === 'development',
port: process.env.PORT ? parseInt(process.env.PORT) : {= defaultServerPort =},
databaseUrl: process.env.{= databaseUrlEnvVarName =},
allowedCORSOrigins: [],
{=# isAuthEnabled =}
auth: {
jwtSecret: undefined
}
{=/ isAuthEnabled =}
},
development: getDevelopmentConfig(),
production: getProductionConfig(),
}

const resolvedConfig: Config = merge(config.all, config[nodeEnv])
// PUBLIC API
export default resolvedConfig

function getDevelopmentConfig(): EnvConfig {
const frontendUrl = stripTrailingSlash(process.env.WASP_WEB_CLIENT_URL ?? '{= defaultClientUrl =}');
const serverUrl = stripTrailingSlash(process.env.WASP_SERVER_URL ?? '{= defaultServerUrl =}');
return {
// @ts-ignore
frontendUrl,
// @ts-ignore
serverUrl,
allowedCORSOrigins: '*',
{=# isAuthEnabled =}
auth: {
jwtSecret: 'DEVJWTSECRET'
}
{=/ isAuthEnabled =}
const allowedCORSOrigins = allowedCORSOriginsPerEnv[env.NODE_ENV]

const config: Config = {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simplified the creation of the config object, instead of having three separate things all, development, production, now we construct a single object and vary some of the values (e.g. allowed CORS origins) based on NODE_ENV.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we loose anyting this way? Anythihg int he old appraoch that was beneficial from having that split? Why was that split there otherwise, I do think it had some benefits possibly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the previous approach we clever since it envisioned many different options for dev and for production, but it in the end, we only vary the CORS origins between dev and production. So, it seemed like an overkill and I know I had these complex types to capture the different objects. But when I looked closer, it turned out this simple Config type is enough.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, makes sense, we can always modify it if needed!

frontendUrl,
serverUrl,
allowedCORSOrigins,
env: env.NODE_ENV,
isDevelopment: env.NODE_ENV === 'development',
port: env.PORT,
databaseUrl: env.{= databaseUrlEnvVarName =},
{=# isAuthEnabled =}
auth: {
jwtSecret: env.JWT_SECRET
}
{=/ isAuthEnabled =}
}

function getProductionConfig(): EnvConfig {
const frontendUrl = stripTrailingSlash(process.env.WASP_WEB_CLIENT_URL);
const serverUrl = stripTrailingSlash(process.env.WASP_SERVER_URL);
return {
// @ts-ignore
frontendUrl,
// @ts-ignore
serverUrl,
// @ts-ignore
allowedCORSOrigins: [frontendUrl],
{=# isAuthEnabled =}
auth: {
jwtSecret: process.env.JWT_SECRET
}
{=/ isAuthEnabled =}
}
}
// PUBLIC API
export default config
Loading
Loading