Firstly, I am clearly a non-expert on anything I am about to say so be cautious...

In the FSOSS world I live in, people are working on [reproducible builds](https://reproducible-builds.org/). The goal is that every binary downloaded could be verified by the user themselves by, well, building it. Many [distros and software projects](https://reproducible-builds.org/who/) are  interested in it. If I remember correctly, this idea was started by the bitcoin people and followed by the tor project.

From what I have heard, some people suggested after sufficiently many packages are reproducible, the hash could be shared in some p2p ways. So to answer your question, in my opinions, at loading time, a verifier can simply hash the binary and check if it matches the hash published by the others, although this may be slow depending on your internet connection speed (maybe the connection has to go through tor).

Also, I happen to know 2 distros more than others so I will elaborate more with them:<br>
From what I have heard, Debian will make [reproducible-build](https://wiki.debian.org/ReproducibleBuilds) a policy requirement after sufficiently many packages are reproducible.<br>
Guix implements [guix challenge](https://www.gnu.org/software/guix/manual/html_node/Invoking-guix-challenge.html) command so that one can verify the hash of your local build with the official build on Hydra.