1

I am using UTF-8 for a new website, so I can do internationalisation on different pages, for example in greek (ελληνικά). I asked a question here earlier about possible security bugs this could bring and some people mentioned ones for example: "admin" (english alphabet) and "аdmin" (cyrillic "a") look absolutely the same however have different character codes and someone could register another account called "аdmin".

I am wondering what's the best way to completely filter out non-english alphabet characters. In the account information page I have a field for changing the username (for example, I don't actually), how would I go about removing them? Should I add a function checking with regex or something?

function changeUsername($newUsername)
{
    if($this->isValidCharacters($newUsername) !== 1) {
        send message saying invalid characters
    }    

    continue changin username
}

And if so, what would the regex look like, still allowing all symbols but NO unicode characters or other alphabet characters?

Or is there an even easier way of blocking all input coming in from being UTF-8? For example changing the default charset in php.ini back to the default and simply sending a header changing the charset when I'm outputting in another language? And if they do send input with another alphabet and my server doesn't have the UTF-8 charset on how does my server interpretate it? Do I still need to do checks?

(As I said in my other post, I have no idea what I'm doing when it comes down to character sets.)

6
  • 2
    What's the problem with an account called "admin"? The system won't give it extra privileges just because the username looks like admin. Leave it as-is and use a separate indicator for administrative accounts (like the diamond here on Stack Exchange for example). Aug 13, 2016 at 1:16
  • Yes, however people can be frauded. Same with passwords, it can cause security holes using passwords with utf-8 passwords.
    – user120698
    Aug 13, 2016 at 2:00
  • I agree with the username part even though again, you could fix that without restricting UTF-8 (what about foreign names with accented chars?). On the other hand I don't understand why you say UTF-8 is bad for passwords. It increases the number of possibilities for each character, so it can actually improve security. Aug 13, 2016 at 2:46
  • UnfortunateIy I lack knowledge in php. But in Python one could at least perform the following (if not via an eventually more elegant way): (1) Convert the user input string to a byte-array, specifying thereby the coding to be UTF-8. (2) Check the bytes and let only those whose values are in a list of values corresponding to the characters of the English characters (upper and lower case, 52 items in all) in UTF-8 to be entered into a new byte-array. (3) Convert the new byte-array to a string. Aug 13, 2016 at 18:06
  • 3
    Just let other users report you if someone is spoothing names. From UX side, it will be way more acceptable than being rejected because "Dédé" is not an allowed name. Plus, you'll certainly forget a lot of other spoofing cases (like "administrator", "owner", "" [empty name], "moderator", "administrateur" [French version of administrator]...)
    – Xenos
    Apr 10, 2017 at 8:50

1 Answer 1

1

Should I add a function checking with regex or something?

There are lots of unicode characters. Lots and lots of them. If you are concerned about weird characters in usernames, a blacklist will not solve the problem. You'll need to use a whitelist instead.

One solution is to go with alphanumeric plus some special characters like - and _. That could be done in a regex like this:

/^[a-zA-Z0-9_\-]+$/

If you want to be more permissive with the special characters you could allow all printable ASCII characters. They go from #32 () to #126 (~). So you would get this regex instead:

/^[ -~]+$/

Note that this will not give you any XSS protection - the username <script> would be allowed. Still, there are special characters this will forbid, like ¤ and £. If you want to include some, you can add them one by one to the whitelist.

For example changing the default charset in php.ini back to the default and simply sending a header changing the charset when I'm outputting in another language?

Messing around with multiple charsets at the same time is a sure fire way of opening up XSS vulnerabilities and similar by mistake. Don't do this. Just stick with UTF-8 all of the time.

5
  • You're trying to reinvent the wheel here with RegEx when there is a function in PHP set to do this, with less likelihood of making a mistake and a greater readability: php.net/manual/en/filter.filters.sanitize.php
    – LTPCGO
    Sep 10, 2019 at 22:15
  • @LTPCGO Not sure how the link applies to this situation. OP wants to block foreign language chracters but allow special characters. A code small example would be helpfull.
    – Anders
    Sep 10, 2019 at 22:24
  • filter_var($string, array('filter' => FILTER_SANITIZE_STRING, 'flags' => FILTER_FLAG_STRIP_LOW | FILTER_FLAG_STRIP_HIGH | FILTER_REQUIRE_SCALAR)); 'NO unicode character' specified by OP indicates to me that it OP cares only about single-byte encodings, which the above will filter for.
    – LTPCGO
    Sep 10, 2019 at 22:55
  • 1
    @LTPCGO Not sure I would prefer that code. It's for filtering, not validation, so you have to check that input equals output. If you want to allow specific special characters, like the £ in my example, you can't easily do that. It filters out some special characters like < and >. May be a bug or a feature depending on your perspective. But I guess I agree with your desire to use the built ins when possible.
    – Anders
    Sep 11, 2019 at 6:05
  • Agree with Anders here. The first suggested regex is readable, explicit and describes a common and sensible character whitelist for usernames.
    – Arminius
    Feb 7, 2020 at 19:18

You must log in to answer this question.