Windows systems tend to be chatty. For instance, at many times they will try to discover other Windows machines on the same LAN, or printers; and they will announce themselves similarly. This behaviour is largely muted if you inform Windows that you are on a "public network" (see this page). If you let your Windows behave as if it was at home, then it will begin to shout information across the LAN, including its name.
Even if you configure Windows to be discreet, the sysadmin may still, through passive eavesdropping, notice the presence of your machine, see its MAC address, observe the connection from your machine to your VPN server (and thus know not only that you use a VPN, but also the IP address of the VPN server). Your machine may also issue some out-of-VPN network accesses (e.g. to Windows Update), and the sysadmin will see those. If the sysadmin is a bit more nosy, he may begin to actively scan your machine for open network services, and analyse emitted packets to infer your operating system brand and version (e.g. with nmap).
The MAC address alone is (normally) already sufficient to track you, i.e. notice that it is still you if you disconnect and reconnect later on. The MAC address can be changed in software, but the other elements (IP address of the VPN server, OS type and version...) will most probably allow reliable tracking nonetheless.
If you use a VPN and the VPN is any good, then the nosy sysadmin will not be able to see the contents of your traffic; however, he may still infer things because he will know when you exchange data and how much data you exchange. E.g. he will be able to distinguish between Web browsing and online gaming. He will also be able to distinguish between accesses to Youtube and to Stack Exchange, if he already knows somehow that you are connecting to one of these two sites.
If your VPN is poorly configured (i.e. if your DNS requests are not sent through the VPN), then the sysadmin will know the names of all the sites you browse.