Giacomo

But in the end, you have to choose your battles, and I know I may have some impact on bots' configuration more easily than the global one :)

Yeah, considering also that normal users don't know anything about that and the configuration is often out of a user's control. For example, I've seen that Slack's webview browser uses its own app configuration and that may differ from my preference for browsing.

Your Accept-Language Redirects Could Be Blocking Search Engines and AI Crawlers | Merj Locale-adaptive redirects are one of those things that “worked fine” until the crawler ecosystem changed. Search engines generally don’t use...

While not common, if your infrastructure redirects fetch requests based on Accept-Language headers, please ensure bots are excluded. merj.com/blog/your-ac...