It is possible to enumerate resources through distinctions such as these - if you return a 403 for pages that do exist, but the current user can't access, and 404 for pages that don't exist, an attacker can easily distinguish between valid and invalid resources.
Whether this matters depends on your application - if you reveal information in this circumstance that has value, it may be a bad thing: if
/user/73463 gives me a 404 but
/user/64500 gives me a 403, I can make an educated guess that 64500 is a valid user whose profile I can't see. On the other hand, if
/year/2016 give me a 403 but
/year/2020 give me a 404, all I can tell is that you've not created year folders off into the distant future, and possibly that you don't have anything from before 2003.
Ideally, you don't want to provide URLs to which the user has no access (from your question, I'm not sure if the URL being followed is on your site, or from elsewhere). You can only control this within your system though - if user B sends user A a link, user A can discover that the resource exists without caring about your response to them. That applies if a link is provided on a third party website too - no-one is likely to link to a non-existent resource (although they might have linked to a resource that no longer exists, in which case a 404 or 410 would be appropriate). For your own systems, you should aim to minimise the chances that a legitimate user can follow a link to a resource which they don't have permission to access.
Generally, if something can easily be enumerated (e.g. numeric identifier) it makes more sense to return a consistent response to all requests, whether the actual status is lacking authorisation or the resource not existing. If something can't be enumerated (e.g. long hash identifier), it might make sense to distinguish between the states, but you should be aware of the potential for someone to stumble across resources they shouldn't be able to detect, and take steps to prevent this (e.g. monitor for high request rates and throttle apparent attempts to enumerate for resources).