There are two main reasons for creating a new version of a program: (in this case a browser)
- To add new features (eg. viewing a video in the browser).
- To fix a problem, which may be:
- A minor issue (such as cmd-L not opening a new window when no window is available)
- A big issue (eg. a malicious web page can read data from other requests  )
Some changes are a bit harder to classify (improving performance is a bugfix or a feature?)
Most of the big issues are security issues solving some kind of vulnerability. A modern browser is as complex as an OS, it's extremely hard not to have errors.
There are also new features improving security that don't involve a vulnerability (but you should try to adopt).
Some programs (eg. a postcard creator) it's not so critical to have the latest version. Mainly because you only use it with trusted files (those you have created) so the risk of getting infected through it is quite low.
On the other hand, web browsers are used everyday to visit lots of untrusted sites. And by untrusted I really mean it. If you are a heavy internet user, it's very strange that you don't visit a website that could compromised for a few weeks. The online newspaper you read? Guess what can be done within its ad network. When you searched X on Google? That search result could have been an exploit, etc, etc.
There are many evil guys trying to infect you (and mainly through your browser or one of its plugins). Sometimes they attempt a single vuulnerability, others they try as much as they can (an exploit kit), attempting whatever that could allow them to control your machine.
As the web browsers are updated as soon as they know about the bug -and succeed on fixing it- (sometimes they discover it first, sometimes it had been abused for a long time before that), if you keep it updated it's much harder to compromise you, as an attacker would need a bug without a fix (called a 0-day). However, using an old version means that you are also vulnerable to all the problems fixed after that version, for which the exploit in most cases was not even written before it was solved! (cf. Microsoft Security Intelligence Report 16, page 24 and MSIR 15) Non-updated systems are many times more likely to get infected.
Mozilla Firefox releases a new version every six weeks (plus whenever a security problem forces a new one). If you want to keep the browser updated but are not interested in less important changes (such as toolbar changes), you can run an Extended Support Release, which is supported for 54 weeks (although you can still get a new version with minor fixes every six weeks, plus the when-needed security updates).
Many other vendors also provide LTS (Long Term Support) versions of their programs, to which they apply security fixes (only), even though it's not the latest one. For instance Debian supports (ie. fixes the old versions they are shipping) their packages -although they are a distributor- for ~3 years, and Red Hat Enterprise Linux provides support for 13 years.
However, although you phrased the question in a generic way, you were probably wondering however why Firefox had to update to Firefox 32.0.1 (12 September 2014) just a few days after installing Firefox 32.0 (2 September 2014). This is not constantly, though as last update was on July.
The explanation is -as always- in the release notes https://www.mozilla.org/en-US/firefox/32.0.1/releasenotes/ where we can that although Firefox 32 fixed some security issues, Firefox 32.0.1 was not due to discovery of new vulnerabilities, but because Firefox 32 "failed" on computers with multiple graphics cards (one of those non-security major issues).
32.0.1 - Stability issues for computers with multiple graphics cards
32.0.1 - Mixed content icon may be incorrectly displayed instead of lock icon for SSL sites
32.0.1 - WebRTC: setRemoteDescription() silently fails if no success callback is specified
So, as an exception to the general advice, updating to Firefox 32.0.1 if you are running Firefox 32.0 is not a pressing issue unless you use multiple graphic cards (or using Android). But when 32.0.2 appears, updating to it may be. You must be very careful each time about what exactly changed if you want to keep running non-latest versions of your browser (and safe).
If you are dealing with a single computer, updating probably takes less time than figuring it out ☺ (and we should be grateful to the last generation of auto-updaters for making them so seamless)