If I were to configure a server that would be configured to only be accessible (HTTP) from a single IP address via a port, is it safe from unauthorized access?
Access from a single address does not make it magically safe if the attacker can use this IP address. This can be done directly if the attacker has compromised this specific system. It can be done indirectly if the attacker can cause a HTTP request from this IP address, for example by sending a mail with a link to the user on this system or by including some link (might be automatically submitted) inside a web page which the user on this IP visits. See also CSRF and XSS attacks and similar.
Apart from that an attacker might reside in the path of the connection. Thus even if the data seem to originate at the specific IP they could be modified or newly generated by some man in the middle (proper HTTPS protects against this though).
We cannot say "It is safe". But you are trying to reduce attack surface. But http and your server is still unsafe. There are many attacking vectors are available to attack your server.
You are going to achieve security by obscurity. It can be successful rarely but it is not recommended or standard way to protect your resources.
If you really want to protect your server, Use standard security mechanism based on your security requirement.
You can use IP restrictions as part of a defense in depth approach to reduce the likelihood of attack or compromise, but they are not a substitute for standard security measures such as TLS or other securely encrypted protocols.
When you use TLS properly, it provides confidentiality, authenticity, and integrity. Without TLS or a similar protocol (e.g., SSH), you don't have any of those things. Anyone who can sit in the middle of the connection can view and tamper with the connection at will. It is well known that many providers do this; for example, Verizon tampers with HTTP connections to insert advertising identifiers and Southwest has inserted text into all HTTP connections showing you how long your Wi-Fi connection will last. Neither of these will work with TLS.
If you used HTTPS, an IP restriction might be sufficient to prevent unauthorized access, but it depends. For example, if the IP address is your corporate NAT address and the site is a set of corporate policies, then that is probably sufficient to protect the information given the fact that your corporate policies, while private and sensitive, are probably similar to most other corporate policies and would likely not result in the company going out of business if exposed.
However, if your data is more important, you might want to institute a random shared password (e.g., stored in a shared password manager vault) in addition to HTTPS, or even add a full set of normal authentication mechanisms (username and password, 2FA, or SAML). Without knowing what the information is you want to protect and how important it is to you, it's hard to advise on how much security should be used. That being said, using TLS or another secure protocol is the baseline level of acceptable security these days, so you should definitely implement that.