The answer is quite well explained on the FIPS 140-2 Wikipedia article.
The requirements for different levels of certification are as follows (quoting directly from the article; no infringement intended):
Security Level 1 provides the lowest level of security. Basic security requirements are specified for a cryptographic module (e.g., at least one Approved algorithm or Approved security function shall be used). No specific physical security mechanisms are required in a Security Level 1 cryptographic module beyond the basic requirement for production-grade components. An example of a Security Level 1 cryptographic module is a personal computer (PC) encryption board.
Security Level 2 improves upon the physical security mechanisms of a Security Level 1 cryptographic module by requiring features that show evidence of tampering, including tamper-evident coatings or seals that must be broken to attain physical access to the plaintext cryptographic keys and critical security parameters (CSPs) within the module, or pick-resistant locks on covers or doors to protect against unauthorized physical access.
In addition to the tamper-evident physical security mechanisms required at Security Level 2, Security Level 3 attempts to prevent the intruder from gaining access to CSPs held within the cryptographic module. Physical security mechanisms required at Security Level 3 are intended to have a high probability of detecting and responding to attempts at physical access, use or modification of the cryptographic module. The physical security mechanisms may include the use of strong enclosures and tamper detection/response circuitry that zeroes all plain text CSPs when the removable covers/doors of the cryptographic module are opened.
Security Level 4 provides the highest level of security. At this security level, the physical security mechanisms provide a complete envelope of protection around the cryptographic module with the intent of detecting and responding to all unauthorized attempts at physical access.
Penetration of the cryptographic module enclosure from any direction has a very high probability of being detected, resulting in the immediate deletion of all plaintext CSPs.
Security Level 4 cryptographic modules are useful for operation in physically unprotected environments. Security Level 4 also protects a cryptographic module against a security compromise due to environmental conditions or fluctuations outside of the module's normal operating ranges for voltage and temperature. Intentional excursions beyond the normal operating ranges may be used by an attacker to thwart a cryptographic module's defenses. A cryptographic module is required to either include special environmental protection features designed to detect fluctuations and delete CSPs, or to undergo rigorous environmental failure testing to provide a reasonable assurance that the module will not be affected by fluctuations outside of the normal operating range in a manner that can compromise the security of the module.
So, in summary:
- Level 1 is essentially just "it uses a FIPS certified cipher".
- Level 2 additionally requires tamper evidence, with some effort put towards physical containment.
- Level 3 additionally requires active responses to tampering attempts, e.g. deleting the access keys to data if someone attempts to open the device.
- Level 4 additionally requires environmental monitoring to further prevent the security of the device being compromised by movement, temperature, etc.
Each of these feature sets is more expensive, and the FIPS certification itself is not without cost, so it stands to reason that the higher level devices cost more.
As an anecdote, while I'm unsure whether they do indeed maintain any official FIPS certification, most chip & pin / EMV devices have anti-tamper features which are roughly in-line with levels 3 and 4. For example, most of them have two separate boards internally which are screwed to opposite sides of the chassis, with one board containing power circuitry and a battery, and the other having the firmware on a volatile storage medium (e.g. SDRAM). The two boards are interconnected using a conductive foam or rubber pad, and attempting to open the device separates the boards, causing the firmware and any key material to be lost.
EDIT: So it seems I answered a different question to the intended one.
A common trick in PCB manufacture is to manufacture one board which supports the maximum amount of functionality of a series of devices, then remove components for cut-down versions of the product. This is really common in consumer devices like routers where the same system-on-chip (SoC) design is common among a series of devices, but the feature sets differ. This is a cost-saving exercise, because you can spin up one board and get it produced in large numbers, then tailor the components based on demand of each edition. Why am I talking about PCBs and routers? Because HSMs likely do exactly the same thing: build a common chassis and common internal hardware set, then add the additional modules as and when needed. The different anti-tamper hardware devices and features can be added as and when needed. The practice is a variant of selective assembly.
