The entire point of the IV is that it's not secret and not tied to the key, so you can generate a new one for each message. It is important that you never, ever re-use it for multiple messages with the same key. Especially for stream ciphers (or block ciphers in stream-like modes of operation, such as CTR), it is essential that you don't re-use it with the same key, because a given (cipher, IV, key) tuple will produce an identical "keystream" every time, and that keystream gets XORed with the plaintext to produce ciphertext (and vice versa; the encrypt and decrypt operations are identical). It's like a one-time pad, except instead of the entire pad being generated randomly each time, you have a long-term random secret (the key) and a single-use random public datum (the IV, or "nonce" as they're usually called in such cases, for "number used once"), and these two random fields are combined in an algorithmic was to generate an arbitrary-length pseudo-random stream of bits.
The problem is, you are reusing that stream of bits for multiple messages, which means anybody who sees two ciphertexts (for the same key+IV) can just XOR them together to get the difference of the two plaintexts, revealing what parts of the data changed, and how. It gets even worse, though. Suppose user A gets v1 of the data. User A then is removed from the service, so they'll never get future versions. However, user A intercepts the encrypted traffic as user B downloads the same v1 data. User A can now XOR the encrypted traffic with their own (decrypted) version of v1, and learn user B's keystream (up to the length of v1). When v2 ships, even though user B won't officially receive it, they can read the encrypted traffic to user B, XOR it with their computed value for user B's keystream, and decrypt v2 anyhow! They could also potentially do things like encrypt traffic to the server using user B's keystream, thus seemingly using user B's actual key (they aren't, they don't know that, but they don't need it) and impersonating user B.
Finally, since encryption and decryption of CTR traffic is just XOR, an attacker with the ability to intercept and modify traffic for another user can simply flip bits in the ciphertext - even if they don't know what the decrypted value should be - and flip exactly those bits (and no others) in the plaintext. Such "bit-flipping attacks" can be used for all sorts of mischief, from giving people incorrect information to maliciously tampering with trusted data such that when it gets processed, it exploits a vulnerability in the parser and gets code execution on the victim's machine.
Stuff like this is the entire reason the number one rule in applied cryptography is "never roll your own".
people have been questioning my choice (without more information than that) and I would like to know what you think
I think you should have listened to them. I didn't need most of the information you gave here to know that what you were doing was dangerously insecure; most of the details are just different ways you're doing it wrong.
Like I said above, never roll your own cryptography. Is there some reason you didn't just use TLS?
Am I delusional, or is 64bits entropy enough to counter the reuse?
I don't know about delusional, but you're absolutely wrong!
- Since in CTR mode every bit of message is independent of every other bit, changing the first 64 bits doesn't do a single thing for security of the rest of the message.
- Even if you were using a mode where changes to one part of the message propagate to other parts (like CBC), reusing IVs exposes information about which message is being transmitted.
- This still doesn't address bit-flipping or spoofing attacks at all.
