I want to prevent robots flooding my website and I'm thinking about a ban in these cases...
Can I assume that an empty user agent will always be a robot?
Information Security Stack Exchange is a question and answer site for information security professionals. It only takes a minute to sign up.Sign up to join this community
Almost always, but not always. It could be a user behind a misconfigured proxy, or a browser extension, or (more likely) someone using telnet/netcat:
netcat server 80 GET / HTTP/1.0 <enter> <enter>
You can ban users without
User-Agent set, but it's dead easy to set one.
python and almost every single command/library/framework out there will have an option to override the User-Agent string.
You'd better implement rate-limiting on sensible pages, captchas when the limit is reached, and ban after a number of captcha misses.