Information Security is a very wide field, it consists of various sub-fields: infrastructure security, application security, network security and so on. From your question I believe the field you are interested in is Web Application Security - WebGoat and Gruyeres are two vulnerable applications dedicated to teach the most common vulnerabilities in Web Application Security. This is the only subject they mention and explain on.
In my honest opinion, the best way to start in web application security, is reading the OWASP top 10 list and explanations, and then continue to test web applications for vulnerabilities (of course only against your own QA machines or with the administrators written consent) .
As mentioned, the Fundstone (now Mcafee) Hacme series is very good, comes in many languages (so you might find a language you are familiar with) and comes with detailed tutorials on how to manipulate and break the Hacme applications.
For more complete list of vulnerable applications and virtual machines, you may want to try the Vulnerable Applications Market
Another excellent way to learn, although it is a bit old, is to pass the MSDN Security Labs which are free and teach a wide variety of subjects:
- Developer Starter Kit: Buffer Overflows
- Developer Starter Kit: Code Analysis
- Developer Starter Kit: Compiler Defenses
- Developer Starter Kit: Fuzz Testing
- Developer Starter Kit: Security Code Review
Developer Starter Kit: SQL Injection Vulnerabilities
Good Luck!