At home I have a connection to my provider via glass fiber that provides two VLANs: IPTV and Internet.
The structure of the network is exactly as shown in the following image, except for the phone connection, that I don't have. The router I use (in the image the one with antennas) is based on OpenWRT and kept updated.
As switch in front of the glass fiber I use a Netgear GS108Ev2 switch, that has no web access, but only access through the Netgear ProSafe utility with a simple password.
I set a password and I gave the router a static IP address in the local network, but I'm not aware of any setting that can restrict the access to the configuration to a specific VLAN, therefore I guess it would be possible to brute force the password from Internet and then change the IP settings of the router, to allow configuration via Internet side.
What are the risks I'm exposed to? Is there something malicious an attacker could do besides scrambling the settings and causing headaches/loss of service until I restore everything? I consider this alternative of no interest for attackers, especially since it requires brute forcing a password (assuming no other vulnerabilities in the router).
I think there are no workarounds, but please mention them, if they are simple (otherwise I'll open a new question). I also have a Netgear GS105E v2, in case it helps.