In a traditional HOTP setup, the counter version is stored by the server, and only a small number of codes are accepted (those for the next N counters). This typically prevents replay attacks, since the server will typically not accept earlier codes. There may, however, be a small time window in which a code remains valid depending on the implementation of the server (due to things like database lag or such).
However, TOTP is essentially HOTP where the counter value is a timestamp. In this scenario, you can indeed replay values over a short period. For example, if you log into an unencrypted website on a coffee shop Wi-Fi, then anyone else on the network can snoop it and log in with your credentials shortly after you, and then your account is compromised. Using a secure channel, such as TLS or SSH, prevents this problem because TLS and SSH contain their own defenses against replay attacks and the TOTP value isn't visible to the attacker.
So while this is less likely with standard HOTP, since TOTP is essentially HOTP and it does suffer from this problem, this is a thing that you do need to handle in practice. Of course, you should have been using a secure channel anyway, because that's considered the minimal level of acceptable security for untrusted networks these days.
