I have a real concern they may somehow be able to access my contacts
Most of these CEO frauds spoof the
From: header to be the real address of the CEO, the envelope sender being something else. This means that they don't have access to your mail accounts. The contacts are more likely collected from the public information on your web site and similar sources. The scammers tend to do some research before the attack. The name of the CEO is usually public information, and it may be easy to find other names too.
firstname.lastname@example.org addresses are easy to extrapolate from names.
How do we get it shut down?
Technical perspective. Whereas Sender Policy Framework (SPF) protects your domain from SMTP level unauthorized use in
MAIL FROM envelope sender, it doesn't protect the content, including headers. A spam filter can be utilized to detect this, the action being either delete, moving to junk or even adding a line in the beginning, that this has been detected as a potential fraud.
However, it's not always clear that a mismatch between
From: header and an envelope sender unauthorized, a typical case being a mail list that preserves the headers but forwards the email. As these CEO scams works typically better from top to bottom, it might be better to start from top by protecting the CEO address and possibly all executive team addresses by creating custom rules. This limited group is easier to inform on the possible problems this may cause to them.
Legal perspective. Contacting the police is a correct thing to do, but reality is that their resources may be limited for handling these, because the offender is usually in a different country and not so easily traceable considering the low severity of a crime that remained an attempted fraud. Not reacting prevents the crime, which is what you should care about, rather than punishments. Pros: The crime record is used for statistics purposes giving valuable information on a larger scale, and occasionally someone even gets caught. I'd do it even when there's no direct benefit from it.