As a rule, traceability is good. In some jurisdictions, you are supposed to keep logs for some minimum amount of time. So I would say: keep an entry for the banned user. Abusive behaviour may have legal consequences (depending on the kind of abuse, another user may wish to trigger legal retaliation, or law enforcement agencies might declare themselves interested on their own account). If you are not able to show logs and other traces, then you could be suspected of being an accomplice, which is rarely a good thing.
In any case, you most probably have other, technical reasons for keeping the banned user entry around. If your Web site is some kind of forum (a typical reason for banning being abusive behaviour when writing) then your forum database contains some messages which refer to the user as author. By deleting the user entry, you would make these references dangling links, which can only confuse your forum management software. Deleting the banned user would then entail deleting all traces of his activities on your site, i.e. pretending that the user never existed at all in the first place. This is probably hard to do, if at all possible.
Also, a user who gets banned is also a user who got bad reputation. You can delete forum posts and records, but you cannot delete reputation. If the user entry is destroyed, another user may register and happen to use the same name, thus unwillingly inheriting the dubious fame.