The "OT" in "HOTP" means "one-time". This is true only if the server indeed rejects passwords corresponding to a counter value which is not greater than the last confirmed value. Having "a little margin" here would mean accepting that a given password is accepted twice, the very thing that HOTP tries to prevent.
This is all based on the idea that the HOTP owner uses his passwords sequentially; there is never a situation where the owner would have any interest in using a password which is not the last one that his password-generating device produced. Correspondingly, if the server received at some point the password corresponding to counter value 1567, then the server knows that the owner's device has gone to value 1568, and any attempt at using, say, password 1565 would be a sure indication of foul play.
The "window" is toward the future, because the device owner may have inadvertently pushed on the button a few times without noticing it, so maybe the next password will correspond to counter value 1575, not 1568, and the server will still accept it.
In other words, the window behaviour as specified in RFC 4226 is the margin you need. There should be no need for any margin in the other direction; if there is such a need, then you are probably misusing HOTP badly.
Edit: if you have several servers consuming one-time passwords from the same source, then I strongly hope that you route all such passwords to a single authentication server, that maintains the counter value. (If this is not the case then you have much bigger issues.)
If you just fear that two such OTP may reach the authentication server "out of order" then you can indeed make things a bit looser without compromising security, but it requires some care. The important property of a one-time-passwords system is that each password value should ever be used once. The counter value, with the semantics explained in RFC 4226, is a cheap and efficient way to maintain these semantics: the server only has to remember the last seen counter, and that's all.
However, you can make the authentication server remember the actually used password values. This can be a "backward window": at any time, the server has a current counter N, which is the highest seen counter value, and remembers all see passwords in the N-9 to N range. With such a window, the server can receive passwords slightly out of order: a password will be accepted if its counter M is greater than N, or is in the N-9 to N range and was not seen yet. The programming on the server is slightly more complex, but not unsurpassingly so.
Important note: the "backward window" is safe in the case of HOTP, but that does not extend to just any "one-time password" scheme. This is safe for HOTP because HOTP is a MAC computed over a counter value -- it relies on the counter not to repeat itself, but not on the possible counter values being used in numerical order. The same would not be true for one-time password schemes which use hash chains (where each OTP is a hash of the next one). What I say here is only for HOTP.