Let's break this down into 3 sections: Digital, Physical, and Human
I'd recommend giving them a company computer, whether they're working on- or off-site. This will allow you to restrict and monitor some of their activities. However, don't go so far as to install a keylogger. No one wants a keylogger on their system, even if it's a company PC. They'll need to use logins with passwords, which don't go well with keyloggers.
Most of your monitoring should be on the server-side. Monitor traffic from and to the server. Activities should be logged, whether or not they're an employee, if it tries to interact with your server. You can go as far as logging every single request (which is not that uncommon I believe), or just logging things that interact with the important parts of your system like direct access to the database.
Usual traffic may indicate malicious activities. For example, if you see 1GB of outbound data, you may wanna check it it's just your employee retrieving some data for work or someone trying to export your data. Seeing 500GB of data flow from/to a single IP should be an immediate red flag.
Any and all administrative work should be done from your internal network. This means they should have to be from inside to even see your admin portal. If you want to allow them to work remotely, I'd suggest you set up a VPN to allow your employee(s) to connect to your internal network remotely.
There are 2 main things you'll have to watch out for: access to the physical office and the work computer. I'll only discuss the company computer.
Well, actually, there's not much you can do when they have the physical computer in their hands. The best you can do would probably by to put seals on the computer, such that any attempt to open up the computer hardware would be visible to you or at least detected by a forensics professional.
This is law 3 of security:
If a bad guy has unrestricted physical access to your computer, it's not your computer anymore
This is the main factor here. As mentioned by @Anonymous, this is mostly a human problem.
First of all, hire someone you can trust. We have penetrations testers trying to break into systems all the time on purpose. One of the main reasons they're allowed to do this is because they have little reason to do bad things to your system. In order to qualify, we have to be thoroughly vetted by the hiring company. Trust and interest are the main things keeping people with access to your system from intentionally doing damage.
Notice I said intentionally. Insider threats aren't always intentionally malicious. The TSA keys were leaked no because someone wanted to do harm, but because of blunders by internal and third parties. An untrained employee can easily wipe out your entire database by mistake with a single command. Untrained employees are often vulnerable to social engineering attacks and can let an outsider in when they shouldn't.
To avoid such problems, you have to train your employees on both in system administration and security. Training will help with little accidents, but only thorough vetting can keep malicious actors from becoming part of your company.
Now to address your suggested ideas
Buy a work laptop for them
Encrypt the hard drive (like with Bitlocker)
This will protect data at rest. Great idea, but not gonna help much against an employee with access to the computer and the password(s).
Disable all USB ports
We use USB ports all the time. USB mouse, keyboards, webcams, etc. This will also be the main means of conveniently transporting data for most people. Instead of disabling them all, use software to prompt the user every time a USB device is plugged in. You'll also have to train your employee(s) not to plug some random USB they found on the street into the work computer. The most you can realistically do is train and whitelist their USB devices such that only registered devices can connect to the computer.
Create a non-admin / limited user account with no install permissions and just the IDEs (e.g. Visual Studio) installed. I use Windows 10 for most development with the exception of a Mac for the iOS portion of the app development.
Again, software preference is personal. This will be entirely your choice to restrict their work performance should you not allow them to use their preferred tools. Restricted non-administrative accounts is advisable, though you'll have to manage installing tools for them yourself.
Install some kind of employee logging software.
Most of this should be on the server-side rather than the employee's computer. Logging their local activities will make it easier to monitor them and sue them for malicious activities but be careful not to go too far. Children don't like their parents watching everything they do, and neither do your employees. Server-side logging shouldn't make them uncomfortable so it's a great way of monitoring.
Disable access to file-hosting websites.
This may be difficult to do, but I'm not sure how you can do this yourself.
Even if you can do it, they may need to access their personal drives for some work, but it's your choice to be restrictive.
Somehow detect and stop when a certain folder is being uploaded or copied somewhere?
Monitor traffic to and from your server. It's their choice to copy your files to a USB stick and upload it elsewhere. You can only monitor to detect unusual/suspicious traffic from/to your server.
Somehow make the git repository only accessible from that machine.
Use a VPN for this, as well as all your administrative portals.
Install some kind of remote admin management system? Azure Active Directory or something?
I mainly use Linux so I'm not well-versed on Windows, but Active Directory should be accessible only from the internal network. Again, use a VPN for remote access.