Add dynamic routes from crawler in generate mode

[NicoPennec]

resolves #143

By default the sitemap-module will add all dynamic routes detected by the Nuxt crawler in generate mode, if the generate.crawler option is enabled in the nuxt.config.js (by default true on Nuxt >= v2.13):

$ nuxt generate --target static

A new boolean option crawler will be available in the sitemap config to disable it.

eg. of usage:

// disable on sitemap config
sitemap: {
  hostname: 'https://example.com/',
  crawler: false
},

// disable on sitemapindex config
sitemap: {
  hostname: 'https://example.com/',
  sitemaps: [{
    path: '/sitemap-foo.xml',
    crawler: true
  }, {
    path: '/sitemap-bar.xml',
    crawler: false
  }]
},

[LukaHarambasic]

Is there anyway I can support you on this PR to get it merged? :)

Thanks for your work ❤️

[itsmatteomanf]

Any update on this @NicoPennec? It would be very cool to have this integrated, doing it manually is a pain especially with this ready to go.

[finex]

Hi, does the sitemap crawler also put the last update date (lastmod sitemap field) from the crawled pages?

[itsmatteomanf]

It's been almost a year... any update on this? It's pretty important, as it breaks sitemap support with the latest Nuxt versions.

[iBobik]

Works well with Nuxt 2.15.8, tried also exclude option.