I have implemented encryption for certain columns in a database. In some cases it may be necessary to search by these columns, either for exact matches or for substrings. At the moment, there is less than 10000 rows to search through (at most). However this will probably change in the future so I am anticipating efficency problems.
The data will remain encrypted in the database, but when retrieved by the application, it can then be decrypted. This means that the retrieval does not have to be 100% accurate, it can retrieve more records than actually match the query, and the application itself can discard the non-matching records.
With this in mind I came up with the following solutions:
For exact text match, hash the encrypted data and modulo this with a relatively low number (2^16 perhaps), and store that value in an additional column. When querying the database, it is only necessary to perform the same operation on the input string, and retrieve all records with matching hash value.
For finding substrings, hash each letter and mod 32 the result. Then set the corresponding bit in an integer. Perform the same operation on the query string. Thus it is only necessary to retrieve records where the result of a binary AND on the query with the stored value is greater than 0. This could also be done using each pair of letters, perhaps with a 64-bit or longer integer to allow for more records to be discarded.
My question is, would either of these techniques have a real impact on the security of the data in the encrypted columns? The encryption scheme is AES-256.