Set user agent for crawler #2
Description
Crawler should set a user agent so site owners know where the request is coming from.
Via https://twitter.com/benediktLEB/status/1037659559211556864
{{ message }}
Crawler should set a user agent so site owners know where the request is coming from.
Via https://twitter.com/benediktLEB/status/1037659559211556864