Add image checking to link crawler test

[holly-cummins]

Single crawl pass now checks both broken links and broken images. The crawl results are cached so both test methods reuse the same pass.

In order to be able to re-use the existing crawl and not crawl twice (sloooow), this needed a bit of a refactor of the current tests to allow the crawl results to be re-used.

The tests found one image which was semi-broken, but only semi – there was a .. at the start of the url, which would go out of the quarkus.io site, but all browsers normalise this to .. In fact, it's mandated by spec that they do so:

Silently collapsing /../ to / is a standard URL normalization technique defined in RFC 3986. This process, called "removing dot-segments," ensures paths don't result in directory traversal that breaks the intended hierarchy

So I've added this behaviour to the tests.

This PR will probably make a merge mess with #2713, but I'll see which merges first and then fix.

This PR has found 4 real dead images, so I'll have to fix those before it can merge.

• 404 http://localhost:8090/assets/images/che-icon-dark.svg found on: http://localhost:8090/blog/march-of-ides
• 404 http://localhost:8090/assets/images/posts/oidc-proxy/custom-gpt-approve-action.png found on: http://localhost:8090/blog/oidc-proxy
• 404 http://localhost:8090/assets/images/worldtour/2022/eventsband_worldtourlogo.png found on: http://localhost:8090/blog/quarkusworldtour-2022
• 404 http://localhost:8090/newsletter/60/index_files/content(10) found on: http://localhost:8090/newsletter/60

The normalisation change in the link crawler, as a side effect, found another real dead link that had snuck through previous versions of the test. It turns out that http://quarkus.io/faq linked to user stories, but the link was inappropriately relative and doesn't work in a browser, even though it passed in the previous versions of the tests.

[holly-cummins]

Ugh, worked on my machine. Will fix. :(

[holly-cummins]

Ah, seems like it's a side effect of the normalisation, and also a difference in behaviour between the vertx server and the serving solution used in gh pages.

The root cause: normalize() strips trailing slashes, so /newsletter/18/ becomes /newsletter/18. When URI.resolve("index_files/img.png") runs against /newsletter/18, RFC 3986 treats 18 as a filename and resolves relative to /newsletter/, giving /newsletter/index_files/img.png. The browser, having followed the redirect to /newsletter/18/, correctly resolves to /newsletter/18/index_files/img.png. Against quarkus.io this didn't matter because the production CDN/server handles these paths differently, but the local CI server serves from the filesystem where only the correct path works.

[gsmet]

Not sure why the build is failing here, I will let you sort it out :).

[holly-cummins]

Not sure why the build is failing here, I will let you sort it out :).

Turns out I made typos in two of my image fixes, which is why we need automates checks. :)

And my revisions to the link crawler caused it to pop up another, genuinely broken, dead link.

[github-actions]

🙈 The PR is closed and the preview is expired.