Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

get latest redirects based on nav-data not version metadata #2563

Merged
merged 5 commits into from
Sep 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions build-libs/__tests__/get-latest-content-sha-for-product.test.ts
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may be overlooking something here, but why is this a .ts file while the other two are .js?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we'd prefer to use TS in both places... but we import the redirects stuff into next.config.js, so it needs to be JS rather than TS. Tests can be TS since they don't need to run directly in that context πŸ‘

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah! Yeah that makes sense.

Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
/**
* Copyright (c) HashiCorp, Inc.
* SPDX-License-Identifier: MPL-2.0
*/

import getLatestContentShaForProduct from '../get-latest-content-sha-for-product'
import fetchGithubFile from '@build-libs/fetch-github-file'
import { PRODUCT_REDIRECT_ENTRIES } from '@build-libs/redirects'

describe('getLatestContentShaForProduct', () => {
PRODUCT_REDIRECT_ENTRIES.forEach(({ repo, path }) => {
it(`fetches the latest SHA for the "${repo}" repo`, async () => {
const latestSha = await getLatestContentShaForProduct(repo)
expect(typeof latestSha).toBe('string')
})

if (['hcp-docs', 'sentinel', 'ptfe-releases'].includes(repo)) {
console.log(`Skipping test for private repo "${repo}"`)
} else {
it(`fetches the latest SHA for the "${repo}" repo, then validates the SHA by fetching redirects`, async () => {
const latestSha = await getLatestContentShaForProduct(repo)
expect(typeof latestSha).toBe('string')
const redirectsFileString = await fetchGithubFile({
owner: 'hashicorp',
repo: repo,
path: path,
ref: latestSha,
})
expect(typeof redirectsFileString).toBe('string')
})
}
})
})
53 changes: 53 additions & 0 deletions build-libs/get-latest-content-sha-for-product.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
/**
* Copyright (c) HashiCorp, Inc.
* SPDX-License-Identifier: MPL-2.0
*/

/**
* We're using the docs endpoints to fetch the latest SHA, so we use
* the env var for the docs API.
*/
const MKTG_CONTENT_DOCS_API = process.env.MKTG_CONTENT_DOCS_API

/**
* A map of all possible `product` slugs to known content API endpoints that
* will return an object with a `ref` property that accurately reflects the
* ref from which the latest content was uploaded.
*/
const KNOWN_LATEST_REF_ENDPOINTS = {
boundary: '/api/content/boundary/nav-data/latest/docs',
nomad: '/api/content/nomad/nav-data/latest/docs',
vault: '/api/content/vault/nav-data/latest/docs',
vagrant: '/api/content/vagrant/nav-data/latest/docs',
packer: '/api/content/packer/nav-data/latest/docs',
consul: '/api/content/consul/nav-data/latest/docs',
'terraform-docs-common':
'/api/content/terraform-docs-common/nav-data/latest/docs',
'hcp-docs': '/api/content/hcp-docs/nav-data/latest/docs',
'ptfe-releases': '/api/content/ptfe-releases/nav-data/latest/enterprise',
sentinel: '/api/content/sentinel/nav-data/latest/sentinel',
heatlikeheatwave marked this conversation as resolved.
Show resolved Hide resolved
}

/**
* Fetch the latest sha from the content API for a given product.
* This relies on known `nav-data` endpoints for each product.
*
* @param {string} product
* @returns {Promise<string>}
*/
async function getLatestContentShaForProduct(product) {
const contentUrl = new URL(MKTG_CONTENT_DOCS_API)
const knownEndpoint = KNOWN_LATEST_REF_ENDPOINTS[product]
if (!knownEndpoint) {
throw new Error(
`getLatestContentShaForProduct failed, with unknown product: ${product}. Please add a known endpoint for this product to KNOWN_LATEST_REF_ENDPOINTS.`
)
}
contentUrl.pathname = knownEndpoint
const latestSha = await fetch(contentUrl.toString())
.then((resp) => resp.json())
.then((json) => json.result.sha)
return latestSha
}

module.exports = getLatestContentShaForProduct
63 changes: 29 additions & 34 deletions build-libs/redirects.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ const path = require('path')

const { isDeployPreview } = require('../src/lib/env-checks')
const fetchGithubFile = require('./fetch-github-file')
const getLatestContentShaForProduct = require('./get-latest-content-sha-for-product')
const { getTutorialRedirects } = require('./tutorial-redirects')
const {
getDocsDotHashiCorpRedirects,
Expand All @@ -26,21 +27,6 @@ const HOSTNAME_MAP = {
'test-st.hashi-mktg.com': 'sentinel',
}

/**
* Fetch the latest ref from the content API to ensure the redirects are accurate.
*
* @TODO save the redirects to the content database and expose them directly via the API
*/
async function getLatestContentRefForProduct(product) {
const contentUrl = new URL('https://content.hashicorp.com')
contentUrl.pathname = `/api/content/${product}/version-metadata/latest`
const latestRef = await fetch(contentUrl.toString())
.then((resp) => resp.json())
.then((json) => json.result.ref)

return latestRef
}

/**
* Load redirects from a content repository.
*
Expand All @@ -63,13 +49,10 @@ async function getLatestContentRefForProduct(product) {
* redirects and return early with an empty array.
*
* @param {string} repoName The name of the repo, owner is always `hashicorp`.
* @param {string?} redirectsPath Optional, custom path to the redirects file.
* @param {string} redirectsPath Path within the repo to the redirects file.
* @returns {Promise<Redirect[]>}
*/
async function getRedirectsFromContentRepo(
repoName,
redirectsPath = 'website/redirects.js'
) {
async function getRedirectsFromContentRepo(repoName, redirectsPath) {
/**
* Note: These constants are declared for clarity in build context intent.
*/
Expand All @@ -82,12 +65,12 @@ async function getRedirectsFromContentRepo(
let redirectsFileString
if (isDeveloperBuild) {
// For `hashicorp/dev-portal` builds, load redirects remotely
const latestContentRef = await getLatestContentRefForProduct(repoName)
const latestContentSha = await getLatestContentShaForProduct(repoName)
redirectsFileString = await fetchGithubFile({
owner: 'hashicorp',
repo: repoName,
path: redirectsPath,
ref: latestContentRef,
ref: latestContentSha,
})
} else if (isLocalContentBuild) {
// Load redirects from the filesystem, so that authors can see their changes
Expand All @@ -106,6 +89,24 @@ async function getRedirectsFromContentRepo(
return validRedirects
}

/**
* @type {{ repo: string, path: string}[]} An array of redirect
* entries. Each entry specifies a repo and the path within that repo to the
* redirects file.
*/
const PRODUCT_REDIRECT_ENTRIES = [
{ repo: 'boundary', path: 'website/redirects.js' },
{ repo: 'nomad', path: 'website/redirects.js' },
{ repo: 'vault', path: 'website/redirects.js' },
{ repo: 'vagrant', path: 'website/redirects.js' },
{ repo: 'packer', path: 'website/redirects.js' },
{ repo: 'consul', path: 'website/redirects.js' },
{ repo: 'terraform-docs-common', path: 'website/redirects.js' },
{ repo: 'hcp-docs', path: '/redirects.js' },
{ repo: 'ptfe-releases', path: 'website/redirects.js' },
{ repo: 'sentinel', path: 'website/redirects.js' },
]

async function buildProductRedirects() {
// Fetch author-oriented redirects from product repos,
// and merge those with dev-oriented redirects from
Expand All @@ -115,18 +116,11 @@ async function buildProductRedirects() {
}

const productRedirects = (
await Promise.all([
getRedirectsFromContentRepo('boundary'),
getRedirectsFromContentRepo('nomad'),
getRedirectsFromContentRepo('vault'),
getRedirectsFromContentRepo('vagrant'),
getRedirectsFromContentRepo('packer'),
getRedirectsFromContentRepo('consul'),
getRedirectsFromContentRepo('terraform-docs-common'),
getRedirectsFromContentRepo('hcp-docs', '/redirects.js'),
getRedirectsFromContentRepo('ptfe-releases'),
getRedirectsFromContentRepo('sentinel'),
])
await Promise.all(
PRODUCT_REDIRECT_ENTRIES.map((entry) =>
getRedirectsFromContentRepo(entry.repo, entry.path)
)
)
).flat()

return productRedirects
Expand Down Expand Up @@ -407,6 +401,7 @@ async function redirectsConfig() {
}

module.exports = {
PRODUCT_REDIRECT_ENTRIES,
redirectsConfig,
splitRedirectsByType,
groupSimpleRedirects,
Expand Down
Loading