Enumerating through all possible password, having each one in memory at some point, is an extremely fast process. One can assume that a simple PC will be able to build one such password per clock cycle and per CPU core. So the PC will enumerate billions of passwords per second. If the figures you see are substantially less than that, then they include the password verification cost as well.
The password verification cost is highly variable. Normally, the system which wants to verify passwords stores salted hashed passwords. If the hash function is SHA-1, then an Intel Core2 x86 CPU, running at 2.4 GHz, will be able to try about 12 millions of password per second (and per core): this is what the speed at which the processor will be able to apply the hash function on each password (the password enumeration itself is less than 1% of that cost). My PC is a quad core, so that's 48 millions per second. I also have a not-too-bad Nvidia graphics card which can do password hashing, actually 160 millions per second with a simple raw SHA-1. At that speed, all 8-character passwords (with uppercase and lowercase letters, and digits) are checked in about a fortnight.
Good password storage schemes use iterated or concatenated hashes, in which the password is hashed thousands of times: this makes password verification thousands of times slower, which does not imply a user-noticeable slowdown, but makes the task much harder (namely, thousands of times much harder) for the attacker.