As said by Tom Leek, security is a HUGE area. It covers everything from security secrets, protocols, assets, to privacy, repudiation, availability, disasters recovery etc. I know you started the question about protocols, I am going ahead with a little bit more because I do not know if your perception is that just adding these will suffice for security, or if you have everything else already secured and this is the only area left.
In addition, security is across the entire stack, anywhere from ensuring that you do not have code bugs (such as buffer overflows) to all security patches applied, to overall secure design (how do users authenticate, access control, etc.).
You will either have to narrow specifically what you are trying to secure, or if you are looking to improve security in general, try these:
- Look at ISO 27034 and Microsoft's SDL both of which give prescriptive guidance on how to push security into development process (Disclaimer: I work for Microsoft, and SDL is one of my primary sources at work and in personal projects.)
- You can also look at certification books, such as those for CISSP or CCSK from Cloud Security Alliance, which will focus more on various applicable security principles. Of course, you can search for more certifications and read reviews to pick the best one for your purpose.
Both of these, esp. SDL from Microsoft (only because I have intimate knowledge of it) will give very prescriptive guidance, and has links to it. I have studied CISSP, but that's more of a Chief Security Office level. Both are valuable though, but if I could do just one, I would go with SDL, or some resource that targets ISO 27034, or something similar.
- Lastly, no matter how much of this stuff you read, you will have to stay on top of current security issues, and make sure you keep applying fixes. For example, if you have a website, there's a plethora of attacks around XSS, XSRF, SQL injection, XML bombs, etc. that you'll have to know and depending on what technologies you use, will have to ensure are not exposed in your service.
- Like these attacks, you will also have to make sure that you know what external software you are using (including the OS), and what are the latest security issues and whether you have applied all the patches, etc. This means apache, IIS, Linux, Windows, you name it. For example, if you used OpenSSL (even if you didn't know, but your installation for example had it), you had better applied a fix to guard against Heart Bleed vulnerability.
As far as protocols are concerned, usually it comes down to your design around who is going to access, what is being secured, etc., pretty much most of the questions that Tom Leek asked. Between some protocols (such as OAUTH and SAML), it is not so much that they will differ in security (depending on your needs), rather that different ones are supported by different parties you may want to integrate with. For example, most, if not all, social IDPs (facebook, google, Microsoft account, linkedin, even stackexchange) support OAUTH, and if you were to use either of them as your identity provider, you would be forced to consume OAUTH and token formats supported by any of these.
However, if you were acting as your own IDP, then you would have more choice and you would consider whether you want to use a directory (esp. if your organization already has one, such as AD), or build your own. A directory will typically provide a set of protocols it supports for authentication. If you already have a third party service your organization is using (e.g. SAP, Salesforce, Office online), it might be possible to use authentication from there.
Hope this helps.