Add configuration for deployment on two separate domains [server-islands]

[Jacob-Roberts]

Summary

Allow deployment on two separate domains using server islands to separate “public” (static) and “sensitive”(dynamic) aspects of a website.

Background & Motivation

Every website is a mix of public and private information. Traditionally, JS and CSS have been the only truly static content. However Astro Server islands can truly change this paradigm.

In the past, self-hosting a website that deals with sensitive information requires making a performance/trust tradeoff. You can either:

1. Host the entire website on your own trusted servers, but lose the ability to take advantage of CDNs for solid user experiences.
2. Use a cloud provider to host/cache data to have fast performance, but trust your users’ data with a 3rd party.

Astro’s new server islands feature presents a unique opportunity to provide a middle ground that eliminates many of the downsides and satisfies both the privacy and the security requirements.

This is of particular interest to the “homelabber” community who want to host websites on their own trusted infrastructure, but want to cache as many static assets as possible on a CDN.

I know a full JAM-stack website is possible where the HTML is fully cached, but thin. However, this requires too much of the app to be client rendered. Astro is a great tool that allows flexibility and selectivity on what is hydrated and in what method. I want to server render components, AND I want the rest of the app to be statically cached.
Anything that is static should come in the initial HTML response.

It is currently possible to almost accomplish this goal with the build.assetsPrefix directive, but this only supports accessing JS/CSS from a separate domain. If my origin server is still far from users, they still get a slow experience.

The truly best state would be if the initial page request (HTML document) was also cached in a CDN, and all dynamic (server islands) were accessed from the origin server.

Note

An important distinction here is that for the static assets, the CDN holds the TLS certificate, whereas for the connection to the origin server, I hold the TLS certificate

Goals

• Astro build supports hosting all static assets including all HTML from a separate domain from the dynamic server island domain
  • https://mysite.com (static HTML, CSS, JS [cache control headers])
  • https://api.mysite.com (server islands, API endpoints)
• These two URLs are configurable in astro.config.js
• Example deployment using two Dockerfiles showing a static file webserver and a dynamic Astro node container

Details

For this example, I’ll refer to mysite.com as the static domain and api.mysite.com as the dynamic domain.

For a simplified use case, assume that both mysite.com and api.mysite.com are both being hosted on a single machine behind a reverse proxy.

However, mysite.com is behind a cloudflare DNS proxy that has the CDN setup correctly, meaning that any response returned will be properly cached and propagated worldwide. Cloudflare holds the TLS certificates for encryption with the users browser.

api.mysite.com is not behind a proxy. Any requests to that domain will go directly to the origin server. I hold the TLS certificates for encryption with the users browser.

Implementation - runtime

To implement this at runtime, I’ve been able to accomplish this through tinkering with the dist folder of a v5 built application. It requires changing the /_server-islands/… URL to simply have a fully qualified URL like https://api.mysite.com/_server-islands/…

Implementation - build time

I have attempted to build this as a separate adapter so as to not clutter the @astro/node integration with extra configuration that is most likely not applicable to everyone.

However. After looking through the code it appears that the server island code is not adapter specific and resides in the core package, so this seems like it will require adding more extensibility to the core package to allow this to be integration specific.

This is particularly where I am looking for help.

CORS

Since we will be making cross origin requests, api.mysite.com needs to provide access control headers to allowlist mysite.com as a valid origin.

It seems that currently all requests are GET so they are simple CORS requests, but this may require more advanced CORS headers if server islands needs it.

Header	Value
Access-Control-Allow-Origin	https://mysite.com
Access-Control-Allow-Credentials	true
Vary	Origin
Access-Control-Expose-Headers	MyCustomHeaders

Encryption secret

Since we will have two separate servers, the encryption secret will need to be kept in sync. This can be accomplished with the ASTRO_KEY environment variable, but it’s important to keep in mind as this complicates the deployment.

Static file server vs node server

For my initial exploration, I used the same node server for both the "static file server" and the "dynamic server". This was merely for convenience, but it should be possible to either:

1. Directly upload the static assets to something like S3 and then properly handle the index.html files
2. Use something like cloudflare CDN that uses the cache headers on the default node server to determine how to cache things.

The specifics aren't important here, and are not part of this RFC.

Why not [insert here]?

Use a single domain and put it all behind Cloudflare Proxy?

If the entire app was hosted and proxied through cloudflare, we could rely on cache headers to control which parts of the HTML is cached.

This is true, however the key difference is that if it's all on a single domain, then Cloudflare holds the TLS certificate. For this use case, imagine that I am serving highly sensitive data that cannot be served through Cloudflare. The TLS certificate for sensitive (dynamic) data must be between the browser and the origin directly.

Just use assetsPrefix?

As stated above, I want the initial index.html to be as fast as possible. I know I can cache that and optimize for the second visit, but there's nothing intrinsically uncacheable about the static bits of the HTML. In theory, it should be cacheable because it's the same for all users. So even for the first user it should come from their closest CDN server rather than from the origin.

Thanks

Thank you for all your hard work! This project is amazing, and I'm happy to contribute any way I can here.

Note

This is written in relation to a homelab project I wrote about here, but this is not required reading, and is somewhat separate from this discussion.

[Jacob-Roberts]

Draft PR created withastro/astro#12320

More work needed