2

I need to receive emails and put them in different "inboxes". Identifying and validating an inbox involves a database hit in order to search for the to-address.

I would like to not hit the db if I know that the to-address is not valid. As of now the inbox names are generated as random strings with only smallcaps and 1-9 (email address friendly).

Would it be possible to generate signed strings so I could check them without a db-hit, much like I handle JWTs, i.e. no db-hit is necessary?

This is what I need:

  • random strings with only smallcaps and 0-9
  • preferrably about 10 char long
  • will need at most a couple of 100,000 unique strings

The collision risk is already handled. I'm generating new strings until they won't collide with existing strings.


edit: I thought I'd take the golden hammer on a test-run. If I haven't done something stupid in my test, it seems like it's plenty fast.

According to the my code (node.js) below it takes about 250 nano seconds to calculate one hmac on my MackBook Pro. The roundtrip to the database for a simple lookup is of course way slower.

Even if HMAC is way overkill I like the solution, becuase I don't need to implement the following:

  • b-tree building and searching
  • notification listening from the database so I know when new inboxes are created
  • adding new inboxes to the b-tree upon db notification
const {createHmac} = require('crypto');
const NS_PER_SEC = 1e9;

const time = process.hrtime();

for(i=0; i< 1000; i++) {
  const hmac = createHmac('sha256', 'secret');
  hmac.update('email-address');
  hmac.digest('hex').substr(0,5);
}

const diff = process.hrtime(time);
console.log(`Benchmark took ${diff[0] * NS_PER_SEC + diff[1]} nanoseconds`);

edit again: My first benchmark was flawed. I forgot to digest. Since digest can only be called once, I need to recreate the hmac every time.

Instead of 250 nanoseconds I'm now at about 10,000 nanoseconds. Still orders of magnitude less than a database roundtrip.

7
  • Why exactly would you need to sign them?
    – user163495
    Commented Mar 18, 2021 at 12:45
  • When I receive an email I would like to check that the to-address is valid before hitting the database to identify the inbox. If someone decides to bombard me with emails I can disregard them by just looking at the to-address. I realise that the short length of the strings will make it possible to forge addresses. But this is just to optimize..
    – Michael
    Commented Mar 18, 2021 at 13:12
  • MD5 on the address and a b-tree in memory would be way faster.
    – ThoriumBR
    Commented Mar 18, 2021 at 13:31
  • Thanks. That's interesting. But I'm going for the HMAC signature since it's a drop in solution in my code. ATM I'm just doing regex to get the "inbox-id" part of the email address. I've actually already implemented the hmac check after the regex so I'm moving forward in the project. Also, a b-tree would need to be updated everytime I add an inbox to the database.
    – Michael
    Commented Mar 18, 2021 at 13:38
  • Now replace HMAC with plain MD5 on your test. You don't need HMAC here.
    – ThoriumBR
    Commented Mar 18, 2021 at 15:15

2 Answers 2

1

Yes, but with your requirements it is not really going to be very secure. With 10 characters and 100,000 valid values, the search space is just too small. However, since this seems only to be an optimization step, it may be acceptable.

Simply sign the value, base-36 encode the signature and truncate it. So you would call HMAC(key, inbox), convert its output to alphanumeric and take as many characters as you want.

When creating the inbox identifiers, you could take five random characters and append a signature of five characters. When validating, you recalculate the signature over the first five characters and check that against the last five characters.

0
0

Use a b-tree or hash table. Or even a binary tree.

Don't use a golden hammer. You don't need to sign anything. HMAC here is way too much overkill. Use the right tool for the job, and a simple job of searching if a mailbox exists is a good use case for a b-tree.

You will do way more searches than inserts. And inserting any entry on a b-tree takes way less time than calculating a HMAC.

If you hash the email before inserting on the tree, the b-tree will be very close to be balanced, because the distribution of values after a hash is balanced.

And with a fast hash like MD5, you don't need to care about collisions, as you algorithm to detect and treat collisions will take way more time than just ignoring them. If a mail address isn't on your database but is a collision, the only thing that happens is that your mail server will try to deliver mail to a non-existent mailbox, and I guess you won't see any collision on a century or more.

4
  • Thanks. I have updated my question with some more arguments. I hope they hold up (i.e. my code is ok).
    – Michael
    Commented Mar 18, 2021 at 14:46
  • Couldn't OP just add a database index to the right column of his database, instead of implementing a btree himself?
    – Sjoerd
    Commented Mar 18, 2021 at 14:53
  • OP wants specifically to not consult the database.
    – ThoriumBR
    Commented Mar 18, 2021 at 14:55
  • My benchmark was flawed. Updating now..
    – Michael
    Commented Mar 18, 2021 at 15:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .