Generally speaking, you can always assume that brute force attacks will always be possible. The trick is limiting how effective they can be. Increasing the length, and or complexity but preferably length, is a good way to slow down a brute force.
So we can discuss specifics, let's assume that the attacker has access to your password hash and cannot use any shortcuts such as hash lookup tables. The length of time necessary to succeed is pretty strongly dependent on the speed of the compute power available. That is, a computer that is capable of checking 1,000 passwords per second will take much longer than a computer that is capable of checking 100,000 passwords per second. As compute power increases on computer systems, the length of time to perform an off-line brute force will decrease. So in this case you would want your password, or passphrase, to be sufficiently complex to render the brute force time longer than the expected lifespan of the password. That is, if you change your password every 6 months you want it to take at least 7 months to break.
Another common brute force method is to attempt to log into a life system, this is what you'll see commonly in scans, such as Morto, that have a common password list and look for systems using some simple credentials. Here the same principles apply, except you have a little more control over how the process can be slowed down. For instance, you can lock out remote systems after a certain number of attempts, respond with the Bad Password message with increasing delays, or lock out accounts after a certain number of bad passwords. All of those have their downsides, in that they can be used as a mechanism for DoS attacks. They are, however, fairly common. In some cases, such as PCI compliance, account lock outs are actually required.