Is this generally wrong?
I see two problems with your implementation:
Using random prefix instead of IV
This technically could be done but it is not recommended or encouraged. Please see this for more details. Generate random IV for each file and save it along with ciphertext (as a prefix or suffix).
Using derived key directly as an encryption key.
In case a password is changed you will need to decrypt and encrypt all of the files again. To solve this, you should generate a single random encryption key that will be used for encrypting all of the files. You will then encrypt this key with a master key derived from user password and save it in a separate file. When password is changed (and consequently master key), you will only decrypt and encrypt the file containing the encryption key. For an example of a well known implementation that uses similar schema take a look at DPAPI and how it handles the keys.
Is a single password/derived key enough or
- Should I derive a new key for each file (I worry the time it takes)?
- Should I derive a new key for a bulk of files (e.g. every 100)?
This depends on what are you defending against. Considering my comments above, you will not derive each encryption key out of the user password but rather generate them randomly from a strong CSPRNG. If you have new random encryption key for each file you will need to store the encryption key encrypted with master key along with ciphertext and IV. If the file encryption key is compromised only that one file is compromised. You need to think if it is possible for a single encryption key to be compromised without compromising the master key (brute force?). If this scenario is not likely or you choose not to defend against it, you can use single encryption key for all files.
One more thing to consider when using single key for all of the files is how large data encrypted with single key can be before you need to change the key. For an answer to this question please see this and this.