The most likely thing that happened here is a tech support scam. These scams are mostly simple and straightforward.
Without properly investigating, this is just an educated guess. This will apply to most people in this situation, but it might not fit yours. If anything is different about your situation, please ask in a comment for clarification or ask a new question, and/or hire someone who knows their way around computers.
0. How they get in
The victim receives a call, or an email telling them to call some number, or finds a number for support (of some kind) while googling online.
The victim then accepts whatever this support says, which will often involve installing some software like Teamviewer or Logmein or a similar program. (These programs are not malware, but like a hammer can be used to hurt someone, you can use this software for bad purposes also.)
This allows the person on the phone access to their computer.
How to defend against this
Incoming calls (caller ID) and email addresses ("From") can be spoofed pretty easily. But if you dial a number or send an email, it is much harder to intercept this. That is why calling back to a known-to-be-legitimate number, or email back to a known-to-be-legitimate email address, is what you should do. Just ask for the person's name (if on a call) so that you can ask for this person when calling back. If it is legitimate, this will be absolutely no problem.
When calling, make sure you have the right phone number. Use normal search terms so you get an obvious website (e.g. look for "toshiba", not "toshiba free repair help me"). Check if the search result you are clicking contain an "ad" label or something. Don't click ads...
Tech support will almost never ask you to install software on your computer. Especially for something like your bank, it never makes sense, because they are not the owners of your computer and you don't have an existing support contract with them.
Never ever let someone remote control your computer if you do not already know them and have an established support contract to do this.
Also, have backups. Anything you want to keep longer than until tomorrow, make sure you have a copy somewhere. Hard drives crash, houses get broken into, people get scammed.
1. What the scammers do
This varies and changes over time. There's a lot of info online about what the common scam methods are and how they work. That is not very IT security related, so I will not go into that here.
In terms of software modifications (although this might depend on the type of scam), they usually just install this remote software and perhaps another tool to maintain some level of access after disconnecting the live session. If you get them angry they might also try to break things (like set a password that you don't know) but this is not really useful for them.
It is extremely unlikely that a support scam has real, new vulnerabilities that they make use of. Usually they just ask nicely if they want access to something. Or they ask un-nicely while threatening with something (withholding money, file encryption, something like that).
They might also copy files, for example from passwords stored in a browser, or try to access network drives.
2. Fixing it afterwards
The best thing is: wipe the infected computer. Don't keep/copy any files because those might be infected, but instead restore from a clean backup (if that is available).
In practice, things like pictures are fairly safe, but there can be trickery to it looks like a picture but really it's an executable program (malware). It is safest to not keep any files, insofar that is possible in your situation.
Although they usually don't do much more than install the remote software, there is no way to be sure. You should not assume that you are safe merely after removing their software. (In a desperate case, this would be a quick first step to make it usable again, but I would not trust that computer ever again until it had a proper reinstall!)
3. Things they got access to
Having network access is not that important, unless you run special software on other systems in the network (e.g. XAMPP, file shares, ssh, or other things that are network-accessible). Your network is almost certainly not compromised and other computers on the network are very unlikely to be affected.
In fact, the network itself cannot really be compromised, just things connected to it (including your router, but again, as of 2022 they are almost never sophisticated enough to do something like exploit a router).
Check that all systems had the latest software. If the systems were up-to-date during the time of the access, then no known weaknesses are likely to have been abused. Malware scanners might also help here, but not necessarily (careful what you install from random websites).
Any files they had access to: you should consider those compromised. If you have a password manager, be sure the password is strong and it was never unlocked while they had access, else everything inside is likely already compromised. If you store passwords in a browser, those are almost certainly compromised as well. Those password in there will need changing. (To be clear: password managers are good, but don't give attackers access to your system and then enter the master password...)
4. Report to the authorities
It is well known that most police (or other authorities) do not have the knowledge or manpower to pick every individual case up. However, if nobody ever reports anything, there is also no pressure to learn or even try to catch these criminals, or to consolidate evidence on a single scam group.
I would always recommend reporting an (attempted) break-in to the police, no matter if it was digital or physical. Keep track of things like phone numbers and especially bank accounts used by the scammers. (Even if you can find the person yourself, do not assume that a bank account holder is really the perpetrator. They might be hacked and used without the knowledge of the owner.)
If you lost money, definitely report it. Your bank (or e.g. paypal, if the scam was done via paypal) might also be able to help recover the money.
In the specific situation from the question, where someone with an infected system connected to your network only (and briefly, it sounds like), you are probably fine. But of course it would be best to get someone involved who can check your systems and give more specific advice.