We have our dev & prod env split between AWS (databricks workspaces) and on-prem Linux boxes. Specifically, we have DB instances on-prem and have python code running inside our databricks workspace in AWS that reads/writes to the DB.
We just noticed that things (firewalls/VPCs/VPNs/instruments/??) are setup in such a way that I can run python code in our Dev workspace in AWS that reads/writes data into our prod DB instance.
There is some debate as to whether this is very bad or ignorably bad, because one would still need dev credentials to login to databricks web-ui and then need prod credentials be able to read/write the DB. So questions are how bad is it? and why?
Generally our company is pretty hyper about security due to our domain. E.g. one can not access prod DB using company laptop even if you're on VPN. To access prod DB we must RDP into VMs running on-prem.
My 2 cents: DB is secured by two basic things
- Network security by restricting incoming traffic: So that unwanted persons are unable to even try to connect to DB.
- DB security (authentication/authorization): If the DB is accessible you can try brute force or any other methods to log into the DB, and DB's security will prevent you.
With current situation we have 2 but not 1, which is very bad.
And someone else made a point that with this setup, someone with access to dev workspace in AWS can bring down prod by running bad queries. Maliciously or unwittingly.
For those not familiar with databricks:
- It's SaaS version of Apache Spark.
- It provides a web-UI to interact with your Spark clusters in your databricks workspace.
- Clusters consist of EC2 instances, that are inside some VPCs.
- EC2 instances run python code. And can run Java/R/Scala/SQL.
