Content Layer

[ascorbic]

This is the work-in-progress branch for the Content Layer
The Content Layer API is a new way to handle content and data in Astro. It builds upon content collections, taking them beyond local files in src/content and allowing you to fetch content from anywhere, including remote APIs, or files anywhere in your project. As well as being more powerful, the Content Layer is designed to be more performant, helping sites scale to thousands of pages. Data is cached between builds and updated incrementally. Markdown parsing is also 5-10x faster, with similar scale reductions in memory. While the feature is experimental and subject to breaking changes, we invite you to try it today and let us know how it works for you.

Enabling content layer

To enable, add the contentLayer flag to the experimental object in your Astro config:

{
	experimental: {
		contentLayer: true,
	}
}

Using the content layer

Tip

The content layer APIs are based on content collections, so you can learn more about them in the content collection docs. Any differences are highlighted below.

To use content layer, create a collection in src/content/config.ts with a loader property. For local files where there is one entry per file, use the glob() loader. You can put your content files anywhere, but not in src/content because these would be handled by the current content collections instead. In this example the files are in src/data.

import { defineCollection, z } from 'astro:content';
import { glob } from 'astro/loaders';

const blog = defineCollection({
  // By default the ID is a slug, generated from the path of the file relative to `base`
  loader: glob({ pattern: "**/*.md", base: "./src/data/blog" }),
  schema: z.object({
    title: z.string(),
    description: z.string(),
    pubDate: z.coerce.date(),
    updatedDate: z.coerce.date().optional(),
  }),
});

export const collections = { blog };

You can load multiple entries from a single JSON file using the file() loader. In this case the data must either be an array of objects, which each contain an id property, or an object where each key is the ID.

Array syntax:

[
  {
    "id": "labrador-retriever",
    "breed": "Labrador Retriever",
    "size": "Large",
    "origin": "Canada",
    "lifespan": "10-12 years",
    "temperament": [
      "Friendly",
      "Active",
      "Outgoing"
    ]
  },
  {
    "id": "german-shepherd",
    "breed": "German Shepherd",
    "size": "Large",
    "origin": "Germany",
    "lifespan": "9-13 years",
    "temperament": [
      "Loyal",
      "Intelligent",
      "Confident"
    ]
  }
]

Object syntax:

{
  "labrador-retriever": {
    "breed": "Labrador Retriever",
    "size": "Large",
    "origin": "Canada",
    "lifespan": "10-12 years",
    "temperament": [
      "Friendly",
      "Active",
      "Outgoing"
    ]
  },
  "german-shepherd": {
    "breed": "German Shepherd",
    "size": "Large",
    "origin": "Germany",
    "lifespan": "9-13 years",
    "temperament": [
      "Loyal",
      "Intelligent",
      "Confident"
    ]
  }
}

The collection is then defined using the file() loader:

import { defineCollection, z } from 'astro:content';
import { file } from 'astro/loaders';

const dogs = defineCollection({
	loader: file('src/data/dogs.json'),
	schema: z.object({
		id: z.string(),
    breed: z.string(),
		size: z.string(),
		origin: z.string(),
		lifespan: z.string(),
		temperament: z.array(z.string()),
	}),
});

export const collections = { dogs };

The collection can be queried in the same way as content collections:

import { getCollection, getEntry } from 'astro:content';

// Get all entries from a collection.
// Requires the name of the collection as an argument.
const allBlogPosts = await getCollection('blog');

// Get a single entry from a collection.
// Requires the name of the collection and ID
const labradorData = await getEntry('dogs', 'labrador-retriever');

Rendering content

Entries generated from markdown or MDX can be rendered directly to a page using the render() function.

Warning

The syntax for rendering content layer collections is different from existing content collections syntax.

---
import { getEntry, render } from 'astro:content';

const post = await getEntry('blog', Astro.params.slug);

const { Content, headings } = await render(entry);
---

<Content />

Creating a content layer loader

Content layer loaders aren't restricted to just loading local files. You can also use loaders to load or generate content from anywhere. The simplest type of loader is an async function that returns an array of objects, each of which has an id:

const countries = defineCollection({
  loader: async () => {
    const response = await fetch("https://restcountries.com/v3.1/all");
    const data = await response.json();
    // Must return an array of entries with an id property, or an object with IDs as keys and entries as values
    return data.map((country) => ({
      id: country.cca3,
      ...country,
    }));
  },
  // optionally add a schema
  // schema: z.object...
});

export const collections = { countries };

For more advanced loading logic, you can define an object loader. This allows incremental updates and conditional loading, and gives full access to the data store. See the API in the draft RFC.

Migrating a markdown or MDX collection to content layer

If you have used content collections, content layer will be very familiar as most of the APIs are the same. You can try converting an existing content collection to content layer if it uses markdown, MDX or JSON, with these steps:

1. Move the collection folder out of src/content. This is so it won't be handled as a current content collection. This example assumes the content has been moved to src/data. The config.ts file must remain in src/content.
2. Edit the collection definition. The collection should not have type set, and needs a loader defined.

import { defineCollection, z } from 'astro:content';
+ import { glob } from 'astro/loaders';

const blog = defineCollection({
  // For content layer you do not define a `type`
- type: 'content',
+ loader: glob({ pattern: "**/*.md", base: "./src/data/blog" }),	
  schema: z.object({
		title: z.string(),
		description: z.string(),
		pubDate: z.coerce.date(),
		updatedDate: z.coerce.date().optional(),
	}),
});

3. Change references from slug to id. Content layer collections do not have a slug field. You should use id instead, which has the same syntax.

---
export async function getStaticPaths() {
	const posts = await getCollection('blog');
	return posts.map((post) => ({
-   params: { slug: post.slug },
+   params: { slug: post.id },
    props: post,
	}));
}
---

4. Switch to the new render() function. Entries no longer have a render() method, as they are now serializable plain objects. Instead, import the render() function from astro:content.

---
- import { getEntry } from 'astro:content';
+ import { getEntry, render } from 'astro:content';

  const post = await getEntry('blog', params.slug);

- const { Content, headings } = await post.render();
+ const { Content, headings } = await render(entry);
---

<Content />

The getEntryBySlug and getDataEntryByID functions are deprecated and cannot be used with content layer collections. Instead, use getEntry, which is a drop-in replacement for both.

Learn more

To see the full API look at the draft RFC and share your feedback on the feature and API.

[changeset-bot]

🦋 Changeset detected

Latest commit: fdd7b83

The changes in this PR will be included in the next version bump.

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[github-actions]

This PR is blocked because it contains a minor changeset. A reviewer will merge this at the next release if approved.

[ascorbic]

@Princesseuh Will this surface the actual FS error message to the user?

[Princesseuh]

In the cause, yes! It's shown in both our CLI and our overlay, though the styling isn't quite amazing

Minor release