Well, one issue until very recently (i.e. last week!) was that implementations of Math.random() in most browsers were, frankly, rubbish. Both V8 (the Javascript engine behind both Chrome and NodeJS) and Spidermonkey (the Firefox engine) used PRNGs with too short periods, and there is evidence to suggest that IE and Edge also have broken PRNGs. This means that if there are any calls to them from anything in a layer that relies on them, there is going to be a limited set of possible results. Now, this can still be a lot of results (about 590 million, for the V8 case), but that is a lot less than the ideal 2^132 (5,444,517,870,735,015,415,413,993,718,908,291,383,296).
This means that any attack has a much higher chance of success - intercept data in transit, and then attack offline. Might take a while, but AES is fast on modern processors, and computers are cheap.
Other issues with this could be: how can you ensure that there are no extensions in use? The browser is out of your control. How can you be sure that there aren't undiscovered flaws in IPFS? Security professionals tend to prefer systems that have been subject to a lot of attacks, and not broken, to new systems that haven't been tested as much. Even if the mathematical theory is perfectly sound, is the implementation correct? One typo or incorrect addition could result in a massive problem - see Heartbleed.
There are also problems with other parts of the browser infrastructure that can affect some types of in-browser crypto. The FileWriter interface was deprecated, meaning that anything larger than the browser memory can't be reliably decrypted (there is no sensible way to write it to disk - oddly enough, FileReader works fine for encryption). Browser support for encryption operations is lacking - Windows and Linux can use native calls for AES nowadays, on modern processors, but it would take work to make browsers able to do the same.
We're getting there, but it's not yet a solved problem. If IPFS stands up to scrutiny, the browser manufacturers implement improved randomisation sources, and so on, it'll be close. The extension problem doesn't seem likely to go away though - people seem attached to their adblockers, and probably don't want to trade security through not having any extensions turned on for adverts!
SJCL
etc.: Did you ever meet a crypto lib author saying his own lib is not good? Right. The libs are as good as the authors, but latter factor varies.IPFS...Therefore all its contents can be trusted, nobody can inject malicious code or modify it.
Lol? ... Any reason you want to make your own thing instead of using established stuff?