How to block foreign language UTF-8 characters in usernames?

Question

I am using UTF-8 for a new website, so I can do internationalisation on different pages, for example in greek (ελληνικά). I asked a question here earlier about possible security bugs this could bring and some people mentioned ones for example: "admin" (english alphabet) and "аdmin" (cyrillic "a") look absolutely the same however have different character codes and someone could register another account called "аdmin".

I am wondering what's the best way to completely filter out non-english alphabet characters. In the account information page I have a field for changing the username (for example, I don't actually), how would I go about removing them? Should I add a function checking with regex or something?

function changeUsername($newUsername)
{
    if($this->isValidCharacters($newUsername) !== 1) {
        send message saying invalid characters
    }    

    continue changin username
}

And if so, what would the regex look like, still allowing all symbols but NO unicode characters or other alphabet characters?

Or is there an even easier way of blocking all input coming in from being UTF-8? For example changing the default charset in php.ini back to the default and simply sending a header changing the charset when I'm outputting in another language? And if they do send input with another alphabet and my server doesn't have the UTF-8 charset on how does my server interpretate it? Do I still need to do checks?

(As I said in my other post, I have no idea what I'm doing when it comes down to character sets.)

Answer 1

Should I add a function checking with regex or something?

There are lots of unicode characters. Lots and lots of them. If you are concerned about weird characters in usernames, a blacklist will not solve the problem. You'll need to use a whitelist instead.

One solution is to go with alphanumeric plus some special characters like - and _. That could be done in a regex like this:

/^[a-zA-Z0-9_\-]+$/

If you want to be more permissive with the special characters you could allow all printable ASCII characters. They go from #32 () to #126 (~). So you would get this regex instead:

/^[ -~]+$/

Note that this will not give you any XSS protection - the username <script> would be allowed. Still, there are special characters this will forbid, like ¤ and £. If you want to include some, you can add them one by one to the whitelist.

For example changing the default charset in php.ini back to the default and simply sending a header changing the charset when I'm outputting in another language?

Messing around with multiple charsets at the same time is a sure fire way of opening up XSS vulnerabilities and similar by mistake. Don't do this. Just stick with UTF-8 all of the time.