Skip to main content
added 492 characters in body
Source Link
poolie
  • 303
  • 1
  • 8

This could be a reasonable approach in certain circumstances:

  1. The customer gets read-only access.

  2. They get read access to an entire database: it's either quasi-public data to all your customers, or it contains only their own data. In particular it must not contain user PII or data that's otherwise subject to regulatory controls.

  3. You don't mind them reading it as much as they want, or making copies. For example if it leaks and becomes entirely public, it's no more than a bit annoying.

  4. They don't access the live production system, but rather a write-behind mirror or data warehouse.

  5. You have adequately considered and addressed the risk of sensitive or customer-specific data leaking into the warehouse.

  6. The system's technically isolated from your real production systems. I'd look at perhaps creating a Google BigQuery service with a mirror of your data, and granting access to that.

  7. You have a good way to manage access grants, including revocation, abuse detection, and including letting customers manage internal delegation of access granted to them. Again, outsourcing that to an IaaS provider like BQ's IAM is probably much easier than bringing it up yourself.

  8. The customer wants to do complex operations on the data that are easily expressed in SQL, and they know how to write SQL.

  9. Your exported schema is stable enough, or your customers are tolerant enough, that changing the schema and breaking their queries isn't a big problem.

These conditions aren't totally black-and-white, but direct access to a live database containing information from many users gets increasing risky in ways other answers have described.

A hypothetical scenario where this might be reasonable is: you have a complex parts catalog of items for sale. The information about what parts you have and what their prices are isn't commercially sensitive and you're not too worried about people keeping copies of it. And it's more complex than just a simple list: perhaps there are complex relations around pricing or which parts work together.

If all those conditions apply, then a starting point is to just provide a download of the data as CSVs or JSON. If you're not comfortable to do that, then giving SQL access probably isn't right either. However there are a couple of cases where granting access to BQ would be better than providing downloads:

  • There are so many tables that managing the imports will be annoying for customers.

  • The data is very large (TBs?) and user's queries read relatively little of it.

  • The data's exported frequently so, again, batch downloads will be hard to keep fresh.

  • You want to provide canned examples of interesting queries and to control the query engine.

  • Your customers are technical enough to write SQL but don't want the hassle of running their own import system and database.

  • The schema changes often enough that their import automation would break, but not in ways that break their queries.


A nice example of this pattern is this 3+TB dataset from GitHub on BigQuery. Although all this information is available through the GitHub API, some queries will be much easier or faster in SQL. Other datasets include political advertising on Google, and film locations in San Francisco.

This could be a reasonable approach in certain circumstances:

  1. The customer gets read-only access.

  2. They get read access to an entire database: it's either quasi-public data to all your customers, or it contains only their own data. In particular it must not contain user PII or data that's otherwise subject to regulatory controls.

  3. You don't mind them reading it as much as they want, or making copies. For example if it leaks and becomes entirely public, it's no more than a bit annoying.

  4. They don't access the live production system, but rather a write-behind mirror or data warehouse.

  5. You have adequately considered and addressed the risk of sensitive or customer-specific data leaking into the warehouse.

  6. The system's technically isolated from your real production systems. I'd look at perhaps creating a Google BigQuery service with a mirror of your data, and granting access to that.

  7. You have a good way to manage access grants, including revocation, abuse detection, and including letting customers manage internal delegation of access granted to them. Again, outsourcing that to an IaaS provider like BQ's IAM is probably much easier than bringing it up yourself.

  8. The customer wants to do complex operations on the data that are easily expressed in SQL, and they know how to write SQL.

  9. Your exported schema is stable enough, or your customers are tolerant enough, that changing the schema and breaking their queries isn't a big problem.

These conditions aren't totally black-and-white, but direct access to a live database containing information from many users gets increasing risky in ways other answers have described.

A hypothetical scenario where this might be reasonable is: you have a complex parts catalog of items for sale. The information about what parts you have and what their prices are isn't commercially sensitive and you're not too worried about people keeping copies of it. And it's more complex than just a simple list: perhaps there are complex relations around pricing or which parts work together.

If all those conditions apply, then a starting point is to just provide a download of the data as CSVs or JSON. If you're not comfortable to do that, then giving SQL access probably isn't right either. However there are a couple of cases where granting access to BQ would be better than providing downloads:

  • There are so many tables that managing the imports will be annoying for customers.

  • The data is very large (TBs?) and user's queries read relatively little of it.

  • The data's exported frequently so, again, batch downloads will be hard to keep fresh.

  • You want to provide canned examples of interesting queries and to control the query engine.

  • Your customers are technical enough to write SQL but don't want the hassle of running their own import system and database.

  • The schema changes often enough that their import automation would break, but not in ways that break their queries.

This could be a reasonable approach in certain circumstances:

  1. The customer gets read-only access.

  2. They get read access to an entire database: it's either quasi-public data to all your customers, or it contains only their own data. In particular it must not contain user PII or data that's otherwise subject to regulatory controls.

  3. You don't mind them reading it as much as they want, or making copies. For example if it leaks and becomes entirely public, it's no more than a bit annoying.

  4. They don't access the live production system, but rather a write-behind mirror or data warehouse.

  5. You have adequately considered and addressed the risk of sensitive or customer-specific data leaking into the warehouse.

  6. The system's technically isolated from your real production systems. I'd look at perhaps creating a Google BigQuery service with a mirror of your data, and granting access to that.

  7. You have a good way to manage access grants, including revocation, abuse detection, and including letting customers manage internal delegation of access granted to them. Again, outsourcing that to an IaaS provider like BQ's IAM is probably much easier than bringing it up yourself.

  8. The customer wants to do complex operations on the data that are easily expressed in SQL, and they know how to write SQL.

  9. Your exported schema is stable enough, or your customers are tolerant enough, that changing the schema and breaking their queries isn't a big problem.

These conditions aren't totally black-and-white, but direct access to a live database containing information from many users gets increasing risky in ways other answers have described.

A hypothetical scenario where this might be reasonable is: you have a complex parts catalog of items for sale. The information about what parts you have and what their prices are isn't commercially sensitive and you're not too worried about people keeping copies of it. And it's more complex than just a simple list: perhaps there are complex relations around pricing or which parts work together.

If all those conditions apply, then a starting point is to just provide a download of the data as CSVs or JSON. If you're not comfortable to do that, then giving SQL access probably isn't right either. However there are a couple of cases where granting access to BQ would be better than providing downloads:

  • There are so many tables that managing the imports will be annoying for customers.

  • The data is very large (TBs?) and user's queries read relatively little of it.

  • The data's exported frequently so, again, batch downloads will be hard to keep fresh.

  • You want to provide canned examples of interesting queries and to control the query engine.

  • Your customers are technical enough to write SQL but don't want the hassle of running their own import system and database.

  • The schema changes often enough that their import automation would break, but not in ways that break their queries.


A nice example of this pattern is this 3+TB dataset from GitHub on BigQuery. Although all this information is available through the GitHub API, some queries will be much easier or faster in SQL. Other datasets include political advertising on Google, and film locations in San Francisco.

edited body
Source Link
poolie
  • 303
  • 1
  • 8

This could be a reasonable approach in certain circumstances:

  1. The customer gets read-only access.

  2. They get read access to an entire database: it's either quasi-public data to all your customers, or it contains only their own data. In particular it must not contain user PII or data that's otherwise subject to regulatory controls.

  3. You don't mind them reading it as much as they want, or making copies. For example if it leaks and becomes entirely public, it's no more than a bit annoying.

  4. They don't access the live production system, but rather a write-behind mirror or data warehouse.

  5. You have adequately considered and addressed the risk of sensitive or customer-specific data leaking into the warehouse.

  6. The system's technically isolated from your real production systems. I'd look at perhaps creating a Google BigQuery service with a mirror of your data, and granting access to that.

  7. You have a good way to manage access grants, including revocation, abuse detection, and including letting customers manage internal delegation of access granted to them. Again, outsourcing that to an IaaS provider like BQ's IAM is probably much easier than bringing it up yourself.

  8. The customer wants to do complex operations on the data that are easily expressed in SQL, and they know how to write SQL.

  9. Your exported schema is stable enough, or your customers are tolerant enough, that changing the schema and breaking their queries isn't a big problem.

These conditions aren't totally black-and-white, but direct access to a live database containing information from many users gets increasing risky in ways other answers have described.

A hypothetical scenario where this might be reasonable is: you have a complex parts catalog of items for sale. The information about what parts you have and what their prices are isn't commercially sensitive and you're not too worried about people keeping copies of it. And it's more complex than just a simple list: perhaps there are complex relations around pricing or which parts work together.

If all those conditions apply, then a starting point is to just provide a download of the data as CSVs or JSON. If you're not comfortable to do that, then giving SQL access probably isn't writeright either. However there are a couple of cases where granting access to BQ would be better than providing downloads:

  • There are so many tables that managing the imports will be annoying for customers.

  • The data is very large (TBs?) and user's queries read relatively little of it.

  • The data's exported frequently so, again, batch downloads will be hard to keep fresh.

  • You want to provide canned examples of interesting queries and to control the query engine.

  • Your customers are technical enough to write SQL but don't want the hassle of running their own import system and database.

  • The schema changes often enough that their import automation would break, but not in ways that break their queries.

This could be a reasonable approach in certain circumstances:

  1. The customer gets read-only access.

  2. They get read access to an entire database: it's either quasi-public data to all your customers, or it contains only their own data. In particular it must not contain user PII or data that's otherwise subject to regulatory controls.

  3. You don't mind them reading it as much as they want, or making copies. For example if it leaks and becomes entirely public, it's no more than a bit annoying.

  4. They don't access the live production system, but rather a write-behind mirror or data warehouse.

  5. You have adequately considered and addressed the risk of sensitive or customer-specific data leaking into the warehouse.

  6. The system's technically isolated from your real production systems. I'd look at perhaps creating a Google BigQuery service with a mirror of your data, and granting access to that.

  7. You have a good way to manage access grants, including revocation, abuse detection, and including letting customers manage internal delegation of access granted to them. Again, outsourcing that to an IaaS provider like BQ's IAM is probably much easier than bringing it up yourself.

  8. The customer wants to do complex operations on the data that are easily expressed in SQL, and they know how to write SQL.

  9. Your exported schema is stable enough, or your customers are tolerant enough, that changing the schema and breaking their queries isn't a big problem.

These conditions aren't totally black-and-white, but direct access to a live database containing information from many users gets increasing risky in ways other answers have described.

A hypothetical scenario where this might be reasonable is: you have a complex parts catalog of items for sale. The information about what parts you have and what their prices are isn't commercially sensitive and you're not too worried about people keeping copies of it. And it's more complex than just a simple list: perhaps there are complex relations around pricing or which parts work together.

If all those conditions apply, then a starting point is to just provide a download of the data as CSVs or JSON. If you're not comfortable to do that, then giving SQL access probably isn't write either. However there are a couple of cases where granting access to BQ would be better than providing downloads:

  • There are so many tables that managing the imports will be annoying for customers.

  • The data is very large (TBs?) and user's queries read relatively little of it.

  • The data's exported frequently so, again, batch downloads will be hard to keep fresh.

  • You want to provide canned examples of interesting queries and to control the query engine.

  • Your customers are technical enough to write SQL but don't want the hassle of running their own import system and database.

  • The schema changes often enough that their import automation would break, but not in ways that break their queries.

This could be a reasonable approach in certain circumstances:

  1. The customer gets read-only access.

  2. They get read access to an entire database: it's either quasi-public data to all your customers, or it contains only their own data. In particular it must not contain user PII or data that's otherwise subject to regulatory controls.

  3. You don't mind them reading it as much as they want, or making copies. For example if it leaks and becomes entirely public, it's no more than a bit annoying.

  4. They don't access the live production system, but rather a write-behind mirror or data warehouse.

  5. You have adequately considered and addressed the risk of sensitive or customer-specific data leaking into the warehouse.

  6. The system's technically isolated from your real production systems. I'd look at perhaps creating a Google BigQuery service with a mirror of your data, and granting access to that.

  7. You have a good way to manage access grants, including revocation, abuse detection, and including letting customers manage internal delegation of access granted to them. Again, outsourcing that to an IaaS provider like BQ's IAM is probably much easier than bringing it up yourself.

  8. The customer wants to do complex operations on the data that are easily expressed in SQL, and they know how to write SQL.

  9. Your exported schema is stable enough, or your customers are tolerant enough, that changing the schema and breaking their queries isn't a big problem.

These conditions aren't totally black-and-white, but direct access to a live database containing information from many users gets increasing risky in ways other answers have described.

A hypothetical scenario where this might be reasonable is: you have a complex parts catalog of items for sale. The information about what parts you have and what their prices are isn't commercially sensitive and you're not too worried about people keeping copies of it. And it's more complex than just a simple list: perhaps there are complex relations around pricing or which parts work together.

If all those conditions apply, then a starting point is to just provide a download of the data as CSVs or JSON. If you're not comfortable to do that, then giving SQL access probably isn't right either. However there are a couple of cases where granting access to BQ would be better than providing downloads:

  • There are so many tables that managing the imports will be annoying for customers.

  • The data is very large (TBs?) and user's queries read relatively little of it.

  • The data's exported frequently so, again, batch downloads will be hard to keep fresh.

  • You want to provide canned examples of interesting queries and to control the query engine.

  • Your customers are technical enough to write SQL but don't want the hassle of running their own import system and database.

  • The schema changes often enough that their import automation would break, but not in ways that break their queries.

Source Link
poolie
  • 303
  • 1
  • 8

This could be a reasonable approach in certain circumstances:

  1. The customer gets read-only access.

  2. They get read access to an entire database: it's either quasi-public data to all your customers, or it contains only their own data. In particular it must not contain user PII or data that's otherwise subject to regulatory controls.

  3. You don't mind them reading it as much as they want, or making copies. For example if it leaks and becomes entirely public, it's no more than a bit annoying.

  4. They don't access the live production system, but rather a write-behind mirror or data warehouse.

  5. You have adequately considered and addressed the risk of sensitive or customer-specific data leaking into the warehouse.

  6. The system's technically isolated from your real production systems. I'd look at perhaps creating a Google BigQuery service with a mirror of your data, and granting access to that.

  7. You have a good way to manage access grants, including revocation, abuse detection, and including letting customers manage internal delegation of access granted to them. Again, outsourcing that to an IaaS provider like BQ's IAM is probably much easier than bringing it up yourself.

  8. The customer wants to do complex operations on the data that are easily expressed in SQL, and they know how to write SQL.

  9. Your exported schema is stable enough, or your customers are tolerant enough, that changing the schema and breaking their queries isn't a big problem.

These conditions aren't totally black-and-white, but direct access to a live database containing information from many users gets increasing risky in ways other answers have described.

A hypothetical scenario where this might be reasonable is: you have a complex parts catalog of items for sale. The information about what parts you have and what their prices are isn't commercially sensitive and you're not too worried about people keeping copies of it. And it's more complex than just a simple list: perhaps there are complex relations around pricing or which parts work together.

If all those conditions apply, then a starting point is to just provide a download of the data as CSVs or JSON. If you're not comfortable to do that, then giving SQL access probably isn't write either. However there are a couple of cases where granting access to BQ would be better than providing downloads:

  • There are so many tables that managing the imports will be annoying for customers.

  • The data is very large (TBs?) and user's queries read relatively little of it.

  • The data's exported frequently so, again, batch downloads will be hard to keep fresh.

  • You want to provide canned examples of interesting queries and to control the query engine.

  • Your customers are technical enough to write SQL but don't want the hassle of running their own import system and database.

  • The schema changes often enough that their import automation would break, but not in ways that break their queries.