Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nuxt 2.13.0 compatibility (nuxt static export) #143

Open
rutgerbakker95 opened this issue Jun 22, 2020 · 29 comments · May be fixed by #170
Open

Nuxt 2.13.0 compatibility (nuxt static export) #143

rutgerbakker95 opened this issue Jun 22, 2020 · 29 comments · May be fixed by #170
Assignees
Milestone

Comments

@rutgerbakker95
Copy link

What problem does this feature solve?

It would be nice if the sitemap automatically generate routes in combination with the Nuxt static option

This feature request is available on Nuxt community (#c106)
@mornir
Copy link

mornir commented Jun 22, 2020

haha I was just going to request the same feature 😄
To be more precise, it would be nice if the sitemap module could work with the new crawler option in full static mode.

At first I didn't want to use the crawler because I assumed it would increase the build time. But actually there is no difference, at least in my case. So I removed my routes and let the crawler do the job, but then I realised that my sitemap was now empty. Of course, I could still query the routes for the sitemap, but it would be awesome if the sitemap module could simply take the routes crawled by the Nuxt crawler.

@lluishi93
Copy link

Yes please! This would be an awesome feature actually...

@mornir
Copy link

mornir commented Jun 23, 2020

For fully static Nuxt websites hosted on Netlify, this build plugin could also be a viable solution: https://github.com/netlify-labs/netlify-plugin-sitemap

@lluishi93
Copy link

For fully static Nuxt websites hosted on Netlify, this build plugin could also be a viable solution: https://github.com/netlify-labs/netlify-plugin-sitemap

Woah! Didn't know about that one! Thank you!

@NicoPennec NicoPennec self-assigned this Jun 30, 2020
@gabrielsze
Copy link

This will be super great!!!

@lomashbhattarai
Copy link

It would be great. Would save us from the pain of generating the routes manually

@lluishi93
Copy link

How is this feature going?

@willin
Copy link

willin commented Aug 24, 2020

  i18n: {
    locales: [
      { name: '简体中文', code: 'zh', iso: 'zh-CN', file: 'zh.js' },
      { name: 'English', code: 'en', iso: 'en-US', file: 'en.js' }
    ],
    strategy: 'prefix',
    rootRedirect: 'zh',
    defaultLocale: 'zh',
    lazy: true,
    langDir: 'i18n/',
    seo: true,
    detectBrowserLanguage: {
      useCookie: true,
      cookieKey: 'i18n_redirected'
    },
    vueI18n: {
      fallbackLocale: 'zh'
    }
  },
  sitemap: {
    hostname: 'https://hanzi.press',
    i18n: true
  }

sitemap generated:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:news="http://www.google.com/schemas/sitemap-news/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns:mobile="http://www.google.com/schemas/sitemap-mobile/1.0" xmlns:image="http://www.google.com/schemas/sitemap-image/1.1" xmlns:video="http://www.google.com/schemas/sitemap-video/1.1">
  <url>
    <loc>https://hanzi.press/about</loc>
  </url>
  <url>
    <loc>https://hanzi.press/ar</loc>
  </url>
  <url>
    <loc>https://hanzi.press/card</loc>
  </url>
  <url>
    <loc>https://hanzi.press/story</loc>
  </url>
  <url>
    <loc>https://hanzi.press/surround</loc>
  </url>
  <url>
    <loc>https://hanzi.press/</loc>
  </url>
</urlset>

i have no idea whether it's about nuxt v2.13.0

@drewbaker
Copy link

Would love this to work with nuxt generate and the crawler

@colsen1991
Copy link

Second this! Although as a Netlify user, @mornir's comment about https://github.com/netlify-labs/netlify-plugin-sitemap seems to do the trick for now :)

@JohnFitz
Copy link

JohnFitz commented Oct 22, 2020

Until that's implemented, if you're not using Netlify and your requirements are modest, you could add this to your nuxt.config.js:

//nuxt.config.js

import fs from 'fs'
import path from 'path'

export default {
    hooks: {
        generate: {
            done: (generator) => {

                const hostname = 'https://pinkyandthebrain.com' // Your hostname

                // Array of regular expressions to exclude routes. If you don't need to exclude anything, just use an empty array
                const exclusionRegExps = ['\/admin\/.+', '\/secrets.*', 'planstotakeovertheworld'] 

                const sitemapFilePath = path.join(
                    generator.nuxt.options.generate.dir,
                    'sitemap.xml' // Customise sitemap name here
                )

                const filteredRoutes = Array.from(generator.generatedRoutes).filter((route) => {
                    return exclusionRegExps.some(rule => RegExp(rule).test(route)) ? false : true
                })

                const urlElements = filteredRoutes.map(route => {
                    return `<url><loc>${hostname}${route}</loc></url>`
                })
                const template = `<?xml version="1.0" encoding="UTF-8"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">${urlElements.join('')}</urlset>`
                fs.writeFileSync(sitemapFilePath, template)
            }
        }
    }
}

Caveats

  1. This is all of the code, i.e. this doesn't use the sitemap module at all. Because it's so simple, you don't get all of the nice features like gzipping, sitemap indexes, globbing etc., it just spits out a simple sitemap including your dynamic routes, minus whatever routes match the regular expressions in the exclusionRegExps array.
  2. Logic in the config smells bad, so I'm sure this isn't considered idiomatic Nuxt, but I'm new to Nuxt and I needed the feature fast so I had to cut corners. When dynamic routes can be added using the sitemap module, it'd probably be better to use it over this solution.
  3. This uses regex to exclude routes so as with anything regex, you have to look out for corner cases. Check the output to make sure.

Pros

It doesn't get any simpler than this, so it should be easy for anyone to customise 😉

@mornir
Copy link

mornir commented Nov 9, 2020

@timbenniks also wrote a blog post about it: https://timbenniks.dev/writings/easy-dynamic-routes-in-your-nuxt-sitemap/
I guess it shouldn't too hard now to have that code directly in this module.

@timbenniks
Copy link

@mornir thanks for the mention. It should indeed not be too complex to add my approach to the module. However, I don't know about its internals and potentially there is quite a bit to consider. Like multiple sitemaps etc.

@drewbaker
Copy link

@mornir Thanks for that! I just implemented it, but I realized you can use the default sitemap option filter to exclude routes, so you don't need to define them in the build module like you have it. Just keep all config in the nuxt.config file this way.

@drewbaker
Copy link

I tired making a PR for this, but kept getting Git commit issues. Never done a PR to an OSS project before.

Oh well I give up. So frustrating what has become of frontend DX these days.

Pretty sure this is the solution if someone wants to make a PR. Will need to add generatedRoutes to /lib/options.js too. And tests and documentation.

/lib/module.js:

const path = require('path')

const fs = require('fs-extra')

const { generateSitemaps } = require('./generator')
const logger = require('./logger')
const { registerSitemaps } = require('./middleware')
const { getStaticRoutes } = require('./routes')

module.exports = async function module(moduleOptions) {
  const nuxtInstance = this

  // Init options
  const options = await initOptions(nuxtInstance, moduleOptions)
  if (options === false) {
    logger.info('Sitemap disabled')
    return
  }

  // Init cache
  // a file "sitemap-routes.json" is written to "dist" dir on "build" mode
  const jsonStaticRoutesPath = !nuxtInstance.options.dev
    ? path.resolve(nuxtInstance.options.buildDir, path.join('dist', 'sitemap-routes.json'))
    : null
  const staticRoutes = fs.readJsonSync(jsonStaticRoutesPath, { throws: false })
  const globalCache = { staticRoutes }

  // Init static routes
  nuxtInstance.extendRoutes((routes) => {
    // Create a cache for static routes
    globalCache.staticRoutes = getStaticRoutes(routes)

    // On run cmd "build"
    if (!nuxtInstance.options.dev) {
      // Save static routes
      fs.outputJsonSync(jsonStaticRoutesPath, globalCache.staticRoutes)
    }
  })

  // On "generate" mode, generate static files for each sitemap or sitemapindex
  nuxtInstance.nuxt.hook('generate:done', async (context) => {
    await nuxtInstance.nuxt.callHook('sitemap:generate:before', nuxtInstance, options)
    logger.info('Generating sitemaps')
    await Promise.all(options.map((options) => generateSitemaps(options, globalCache, nuxtInstance)))

    // Add Nuxt's auto generated routes
    options.map((opts) => {
      const output = {
        ...opts,
      }
      if (opts.generatedRoutes) {
        const routes = Array.from(context.generatedRoutes)
        output.routes = [...routes]
      }
    })

    await nuxtInstance.nuxt.callHook('sitemap:generate:done', nuxtInstance)
  })

  // On "ssr" mode, register middlewares for each sitemap or sitemapindex
  options.forEach((options) => {
    registerSitemaps(options, globalCache, nuxtInstance)
  })
}

async function initOptions(nuxtInstance, moduleOptions) {
  if (nuxtInstance.options.sitemap === false || moduleOptions === false) {
    return false
  }

  let options = nuxtInstance.options.sitemap || moduleOptions

  if (typeof options === 'function') {
    options = await options.call(nuxtInstance)
  }

  if (options === false) {
    return false
  }

  return Array.isArray(options) ? options : [options]
}

module.exports.meta = require('../package.json')

@NicoPennec NicoPennec linked a pull request Dec 9, 2020 that will close this issue
@NicoPennec
Copy link
Member

Thank you all for your suggestions! I know this feature is eagerly awaited.

I just published a PR to handle the new static mode directly in the sitemap module.
You can test this proposal by installing it manually from the github branch, as follows:

$ npm install nuxt-community/sitemap-module#feat/static-crawler

Your test results and feedbacks would be really appreciated before to release it? 🙏

@NicoPennec NicoPennec added this to the 2.x milestone Dec 9, 2020
@mornir
Copy link

mornir commented Dec 12, 2020

@NicoPennec Thanks for implementing this feature! It works great.
Is it better to add this module to buildModules instead of modules, so that this module code doesn't end up in the production bundle?
I've added it to buildModules and there was no error.

@rutgerbakker95
Copy link
Author

@NicoPennec very nice! In my case it also works great.

@Velikolay
Copy link

@NicoPennec that's really cool, worked well on my side too.

@Tragio
Copy link

Tragio commented Jan 23, 2021

@NicoPennec thank you for the amazing work. What is missing to be merged? 😃

@Velikolay
Copy link

Velikolay commented Jan 27, 2021

@NicoPennec does the feature respect the exclude option? I couldn't make exclude work on routes generated with the crawler.

@LukaHarambasic
Copy link

@Velikolay as there is a new test added without a crawler and the old one still tests with an exclude I would say, yes it is expected to work.

https://github.com/nuxt-community/sitemap-module/pull/170/files#diff-f91577e8fb8dcd2f38cbffaf99bd21cae2be72d4ee60e0bbcecb11de6d15d171R743

@willin
Copy link

willin commented Mar 26, 2021

not compatible to @nuxt/content-theme-docs

nuxt config:

import theme from '@nuxt/content-theme-docs';

export default theme({
  loading: { color: '#00CD81' },
  i18n: {
    locales: () => [
      {
        code: 'zh',
        iso: 'zh-CN',
        file: 'zh-CN.js',
        name: '简体中文'
      }
    ],
    defaultLocale: 'zh'
  },
  buildModules: ['@nuxtjs/google-analytics', '@nuxtjs/google-adsense', '@nuxtjs/sitemap'],
  content: {
    liveEdit: false
  },
  components: true,
  pwa: {
    manifest: {
      name: 'RxJS教程'
    }
  },
  googleAnalytics: {
    id: 'UA-33096931-4'
  },
  'google-adsense': {
    id: 'ca-pub-5059418763237956',
    pageLevelAds: true
  },
  sitemap: {
    hostname: 'https://rx.js.cool'
  }
});

result: https://rx.js.cool/sitemap.xml

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:news="http://www.google.com/schemas/sitemap-news/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns:mobile="http://www.google.com/schemas/sitemap-mobile/1.0" xmlns:image="http://www.google.com/schemas/sitemap-image/1.1" xmlns:video="http://www.google.com/schemas/sitemap-video/1.1">
<url>
<loc>https://rx.js.cool/</loc>
</url>
<url>
<loc>https://rx.js.cool/releases</loc>
</url>
</urlset>

source: https://github.com/willin/rx.js.cool

@urbgimtam
Copy link

Hoping this PR gets released soon. Its a must for larger static websites.

@fabianwohlfart
Copy link

fabianwohlfart commented Apr 23, 2021

+1
Any News?

ps. Feature works great on nuxt 2.15.4 with content, i18n, 6 level deep urls, 200+ pages.

@patrickcate
Copy link

Is it better to add this module to buildModules instead of modules, so that this module code doesn't end up in the production bundle?

I'd like to know this as well.

@honeyamin
Copy link

honeyamin commented Oct 12, 2021

is is for static or work by ssr? also which version of nuxt?
and also can we get dynamic routes from apollo?? thanks every body that who read this and answer.
also thanks to all

@pixelomo
Copy link

pixelomo commented Nov 15, 2021

I gave up trying to get the crawler feature working, wasted 1-2 hours with this module then found a chrome extension that could generate a sitemap for SPAs with zero configuration - Sitemap Generator

When it finishes crawling save the sitemap.xml and move it your /static folder. When you run npm run build it will be added to your /dist folder

@marvinhuebner
Copy link

Thank you all for your suggestions! I know this feature is eagerly awaited.

I just published a PR to handle the new static mode directly in the sitemap module. You can test this proposal by installing it manually from the github branch, as follows:

$ npm install nuxt-community/sitemap-module#feat/static-crawler

Your test results and feedbacks would be really appreciated before to release it? 🙏

Is there any update on this and when will this be merged? Would be great to have that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.