It largely depends, but there were some significant API changes between 2.1 and 4.2 which may change the way certain operations work on the device.
If you're looking at something like a music app, or a game, you're likely to find any vulnerabilities in operational code (credential handling, network stuff, buffers, etc.) rather than misuse of the API. However, if you're looking at a more in-depth app (e.g. an APK installer, a ROM manager, etc.) you're likely to find more unusual uses (or abuses) of the API, so that's a case where it's certainly worth checking on multiple versions and devices.
The exact same logic applies to iOS, though keep in mind a lot of apps will target a rather restricted set of iOS versions due to Apple's breaking changes between releases.