The cipher function is just a building-block for encryption. It's an important one, no doubt, but it's just one single component that can be used both securely and insecurely. If you write the the crypto framework yourself and plug in ciphers, etc. where you think they go, you run a very real risk of doing something wrong.
- What's your cipher key? Are you using a key derivation function or just naively hashing your password?
- Initialization vector? Where does it come from and what do you do with it?
- Block chaining mode, which do you pick and why did you pick it? Did you say "CBC because everyone else uses it"? Does your block chaining mode affect your initialization vector choice?
- Message authentication; are you testing to ensure that the message hasn't been tampered with? Where do you put that authentication code?
- Flexibility: do you allow parameters such as cipher specification or chaining mode to be specified at run-time? (How) do you indicate your decision in your output?
- ...and dozens of other issues that don't spring immediately to mind.
There are lots of decisions to be made, and a huge number of ways to get it wrong. Typically there's an easy-but-horrifyingly-wrong answer to every question which you may stumble into using simply by not knowing that you had to make a decision.
Frameworks (hopefully) answer a lot of these questions for you, and (hopefully) do so in a way that is community-vetted and largely secure. Not all frameworks do things right, and many may punt the question back to you.
But (and here's the important point) if you go it alone and build your own framework, there's an overwhelmingly high probability that you'll come up with the wrong answer for some decision.
If you use a trusted, vetted, well-written framework, you up your odds a bit.