Can a VPN gateway enforce policies in form of checking whether a particular software or set of software is installed on the endpoint and based on that allow / deny the remote connection to be established ?
The Trusted Computing Group (TCG) has defined a set of standards that allow VPN gateways (and other endpoints) to do that in a secure way called Trusted Network Communications (TNC). Many of them have also been released as RFCs by the nea IETF working group and some related ones by the sacm working group.
The following image provides an overview of these protocols/interfaces:
The open source strongSwan IPsec VPN solution implements these protocols and provides many example use cases, for instance, endpoint assessment (e.g. with Linux or partially with Android), or creating software inventories together with the strongTNC application (and the swidGenerator utility). The latter also allows to define policies that are enforced by the VPN gateway (e.g. to only allow restricted VPN access if software updates are missing on a client or other irregularities are found).
There are commercial vendors that implement some of the protocols (e.g. IF-MAP) but they often rely on their own proprietary solutions. For instance, Microsoft provided the Network Access Protection framework, but that has been removed from the latest releases of their operating systems.