There are substantial flaws in the way phone routing works that makes this difficult.
Caller ID is specified by the outgoing call and anybody who has control over a phone branch exchange (PBX) can mess with it. (This is a "feature" so e.g. a support center can always have the same callback number as opposed to the number of the associate calling you.) I believe there is some limited ability to traverse the call routing, but that stops as soon as it hits a voice-over-IP provider (VoIP), which effectively provide a screen akin to reputation hijacking (an attacker gets lost in the noise of a shared pool of resources).
I am under the impression most spam calls come from VoIP, but my spam expertise is limited to email.
Happily, at least for the US, STIR/SHAKEN was just adopted, as announced by the Federal Communications Commission in Mandating STIR/SHAKEN to Combat Spoofed Robocalls.
With strict limitations on Caller ID forgery, endpoint anti-spam can then build blacklists of at least VoIP spammers.
From the FCC's news release PDF:
FCC MANDATES THAT PHONE COMPANIES IMPLEMENT CALLER ID AUTHENTICATION TO COMBAT SPOOFED ROBOCALLS
Industry-wide Deployment of STIR/SHAKEN Will Yield Substantial Benefits for American Consumers
WASHINGTON, March 31, 2020—The Federal
Communications Commission today adopted new rules requiring
implementation of caller ID authentication using technical standards
known as “STIR/SHAKEN.” These rules will further the FCC’s efforts to
protect consumers against malicious caller ID “spoofing,” which is
often used during robocall scam campaigns to trick consumers into
answering their phones.