I try to design a system which reads events comming from devices, stores them and exposes API to various applications. Originally I wanted to introduce a module Id translator, so I would store only anonymous device id in a database. It would be then protected from unauthorized data harvesting.

Then I wanted to have another module called Roles Registry responsible for handling API permissions for each application. I had an idea in mind that application wishing to work with concrete device id would need different permission than application using anonymous identifiers.

Another module Event Access API would be single entry point for data access and it would evaluate if the API permission was granted to the application.

But as I start to think of it I think that this concept is weak. It is hard to explain because myriads What if questions arrive in my mind, I consider some as probable, I reject others etc. I need to clearly specify my requirements and find consequences.

I want

  1. anonymous database so neither admin can search for data of specific event id
  2. search by concrete device id would require different permission set than anonymous search by other criteria

But I see main problem with returned data:

  1. If API returns artificial id then caller can save it and then he can use anonymous API next time
  2. If API returns real device id then API needs access to Id Translator database (really big data, fast access) and then admin can easily match anonymous data with concrete device id

It may be a unsolvable and I shall just skip the first requirement (admin protection).

I would be gratefull for

  1. Any insights how to design such system and API
  2. What is the best practice?
  3. Shall data be anonymized in database?
  4. Shall API return anonymous db identifier or real device id?



I was reading about the anonymizeation problem just the other day. As it happens, the first edition of what I was reading is on line, at https://www.cl.cam.ac.uk/~rja14/Papers/

This is from Ross Anderson at Cambridge, who is well-known in the community. You want one of the Security Engineering chapters, starting at SE-01.pdf, but the're other stuff there in his blog.

| improve this answer | |

You want to use dynamic data masking. There are different products in that space. Have a look at Oracle VPD (Virtual Private Database), Informatica DDM, or Axiomatics Data Access Filter which all provide means to define policies as to what you can see / cannot see.

These tools are meant to protect data, which seems to be along the lines of what you want to achieve.

If you want a good list of capabilities, vendors, open source alternatives, check out the Gartner reports - in particular the Magic Quadrant which Informatica gives you complimentary access to. See here.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.