If i'm correct, due ASLR we load libc into some random address. And then in order to make that happen without allowing write permissions of text pages within memory we use plt/got. Now I can simply jump back into some libc@plt function that is well-known before execution of the program and pretty much bypass it. So does return-to-plt make the entire concept useless?
Yes, if PIE is disabled. It is often said that ASLR's effectiveness is significantly reduced for applications which are not compiled with PIE (Position Independent Code) support. When PIE is not used, the program must rely on a fixed PLT, created during linking, to resolve the addresses of functions in shared libraries. When ASLR is used and PIE is enabled, code-reuse attacks in general are mitigated, though infoleaks are still so ubiquitous that ASLR can often be defeated with side-channel attacks, making it close to useless against local malicious code and limiting its effectiveness to attacks over the network and scriptless exploits. There are several other reasons ASLR without PIE is weaker and why less randomization is used, as explained in another answer.
return to plt and return to libc are slightly different attacks.
Return to libc
One of the ways to prevent buffer overflow is to use a non executable stack. To make non executable stack, from CPU & system level they use something called NX bit. If NX bit is set, that memory address is non executable. Even if we perform a buffer overflow and over write the stack to redirect the return pointer to stack where our shell code resides, the shell code wouldn't run because it would be in stack - which is a non-executable memory section.
This is when we use return to libc attack to spawn a shell. Instead of using the classic approach to overwrite the return address to the address of shell code, we use the address of libc system() call.
Return to PLT
ASLR or address space layout randomization is another method to control buffer overflows. Like I said in the previous section, people bypass NX by finding executables loaded on to memory by system.like libc. This was possible because it was easy to identify the location of system loaded executables. ASLR randomised the address of these executables and thus reduced the probability of spawning a shell by redirecting to them.
The reason why this attack is possible is that in executables with dynamically linked libraries the libc function address can be obtained from PLT and GOT. PLT address is not randomised and that's makes the attack easy.