Regardless; Could a malicious program theoretically steal data from primary memory (RAM)?
With sufficient permissions, absolutely. The way modern computers manage memory is a little complex, but the gist of it is easy to understand. An x86 computer uses two privilege "rings", called ring 0 and ring 3 (it has more, but they are unused by any popular operating system). Ring 0 is highly privileged and can do just about anything. Ring 3 is less privileged and must ask code running in ring 0 (the kernel) to do anything privileged on its behalf (this is called a system call). On all modern x86 processors, there is an additional protection, called the Memory Management Unit, or MMU. It allows each process to be given its own virtual address space, and if it tries to access any other memory, it will crash.
So this seems like all processes are isolated, right? Unfortunately it's not that simple. The kernel, running in ring 0, provides an interface for processes to communicate with each other, and even access each other's memory indirectly, using a system call. Whether or not the system call is permitted depends on the permissions of the process. A malicious process with high permissions can simply ask the kernel to give it access to another process' memory, and the kernel will happily oblige. Malicious processes without permissions are not able to do that. It is important to note, though, that on many systems, simply running as the same user as another process is considered a sufficient permission. Two processes that are running as the same user may thus read and write to each other's memory on some systems.
The first answer explained how this works on Windows. On Linux, the equivalent ways to read another process' memories are the system calls
process_vm_readv(), and the
/proc/<pid>/mem file. The former two check to ensure that the calling process is either running as the same user as the target, or that it has specific overriding permissions (except on some systems with additional security features). The latter file can only be read by the program that owns it.
Although the passwords.txt file is only 'stored' in permanent secondary memory on the USB, it will briefly appear within the RAM of the computer which I use to mount the USB, decrypt the data, and open the file. It is here that I suspect a weakness my reside.
Actually the key is in memory constantly, not just briefly. It will remain in memory for as long as the device is mounted. This is fundamentally needed for encryption/decryption to work. Note that it'll likely be a derivative of the password, not the password itself, but it is still sufficient to decrypt the drive. However it is present in the kernel which, being ring 0, cannot be accessed by normal processes.
(Side question): Aside from the obvious Anti-Virus solution to mitigate a malicious program, how may RAM be protected?
Assuming no exploitable bugs, two processes run as different users will not be able to access each other's memory. Running a user as another process provides better isolation. Do note that, on Windows and most Linux, if two graphical processes of different users are running simultaneously, they can still communicate via graphics APIs. Windows attempts to block this, but it has been repeatedly bypassed. Additionally, if the kernel is compromised (which requires advanced exploitation), all bets are off, as ring 0 code can access any and all memory in normal circumstances.
Antivirus does not help all that much, since it only tends to protect from old, poorly written, or already discovered and mass-produced malware. Specialized malware, even if fairly amateur, will still bypass antivirus with high likelihood.