Skip to content

Comments

Add a /robots.txt to all sites#75

Merged
bloodearnest merged 1 commit intomainfrom
robots.txt
Feb 17, 2026
Merged

Add a /robots.txt to all sites#75
bloodearnest merged 1 commit intomainfrom
robots.txt

Conversation

@bloodearnest
Copy link
Member

This will hopefully reduce the amount of spurious logs noise we get from
crawlers.

Have added it to the disabled changelogs proxy config too, as it was the
worst offender, and we may need it again.

This will hopefully reduce the amount of spurious logs noise we get from
crawlers.

Have added it to the disabled changelogs proxy config too, as it was the
worst offender, and we may need it again.
@bloodearnest bloodearnest merged commit 527bb71 into main Feb 17, 2026
4 checks passed
@bloodearnest bloodearnest deleted the robots.txt branch February 17, 2026 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants