3

I write Python scripts for clients to "algo trade" at currency/stock exchange websites. My clients are typically running my scripts on traditional personal desktop PC's. Usually using these PCs for web browsing activities as well. The environment is always Linux; usually Debian. In the industry; Python is quite standard for algo trading in this manner; both institutionally and privately.

However, I can't help but see a flaw in the security model.

Each exchange has a slightly different authentication method, but in short, there is:

USER INPUTS:

api['secret']       # private key from exchange

USER CONFIG FILE CONTAINS:

api['key']          # public key from exchange
api['exchange']     # name of exchange; ie "binance"
api['symbol']       # market pair symbol in format BTC:USD
api['uri']          # the url up to .com/

FROM USER INPUTS SCRIPT BUILDS REQUEST SPECIFIC:

api['nonce']        # time.time() at beginning of request
api['endpoint']     # path/to/server/resource
api['url']          # uri + endpoint
api['method']       # GET, POST, or DELETE
api['params']       # dict with request specific parameters
api['data']         # str with request specific parameters
api['headers']      # authentication signature specific to the request

These requests are of the sort:

POST BUY/SELL
DELETE BUY/SELL (CANCEL)
GET ACCOUNT BALANCES
GET OPEN ORDERS
WITHDRAW FUNDS

Here are some examples of the authentication methods I've written to some currency exchanges; they're all quite standard in the forex/stock/crypto trading industry. Mostly in the generalized format:

api["header"] = {"signature": HMAC(SHA256(the_request_parameters))}

examples:

def signed_request(api, signal):
    """
    Remote procedure call for authenticated exchange operations
    api         : dict with keys for building external request
    signal      : multiprocessing completion relay
    """
    api = lookup_url(api)
    api["data"] = ""
    if api["exchange"] == "coinbase":
        api["data"] = json_dumps(api["params"]) if api["params"] else ""
        api["params"] = None
        message = (
            str(api["nonce"]) + api["method"] + api["endpoint"] + api["data"]
        ).encode("ascii")
        secret = b64decode(api["secret"])
        signature = hmac.new(secret, message, hashlib.sha256).digest()
        signature = b64encode(signature).decode("utf-8")
        api["headers"] = {
            "Content-Type": "Application/JSON",
            "CB-ACCESS-SIGN": signature,
            "CB-ACCESS-TIMESTAMP": str(api["nonce"]),
            "CB-ACCESS-KEY": api["key"],
            "CB-ACCESS-PASSPHRASE": api["passphrase"],
        }
    elif api["exchange"] == "poloniex":
        api["params"]["nonce"] = int(api["nonce"] * 1000)
        message = urlencode(api["params"]).encode("utf-8")
        secret = api["secret"].encode("utf-8")
        signature = hmac.new(secret, message, hashlib.sha512).hexdigest()
        api["headers"] = {
            "Content-Type": "application/x-www-form-urlencoded",
            "Key": api["key"],
            "Sign": signature,
        }
    elif api["exchange"] == "binance":
        api["params"]["timestamp"] = int(api["nonce"] * 1000)
        api["params"]["signature"] = signature
        message = urlencode(api["params"].items()).encode("utf-8")
        secret = bytes(api["secret"].encode("utf-8"))
        signature = hmac.new(secret, message, hashlib.sha256).hexdigest()
        api["headers"] = {"X-MBX-APIKEY": api["key"]}
    elif api["exchange"] == "bittrex":
        api["params"]["apikey"] = api["key"]
        api["params"]["nonce"] = int(api["nonce"] * 1000)
        message = api["url"] + api["endpoint"] + urlencode(api["params"])
        message = bytearray(message, "ascii")
        secret = bytearray(api["secret"], "ascii")
        signature = hmac.new(secret, message, hashlib.sha512).hexdigest()
        api["headers"] = {}
    elif api["exchange"] == "kraken":
        api["data"] = api["params"][:]
        api["params"] = {}
        data["nonce"] = int(1000 * api["nonce"])
        api["endpoint"] = "/2.1.0/private/" + api["endpoint"]
        message = (str(data["nonce"]) + urlencode(data)).encode("ascii")
        message = api["endpoint"].encode("ascii") + hashlib.sha256(message).digest()
        secret = b64decode(api["secret"])
        signature = b64encode(hmac.new(secret, message, hashlib.sha512).digest())
        api["headers"] = {
            "User-Agent": "krakenex/2.1.0",
            "API-Key": api["key"],
            "API-Sign": signature,
        }
    elif api["exchange"] == "bitfinex":
        nonce = str(int(api["nonce"] * 1000))
        api["endpoint"] = path = "v2/auth/r/orders"
        api["data"] = json.dumps(api["params"])
        api["params"] = {}
        message = ("/api/" + api["endpoint"] + nonce + api["data"]).encode("utf8")
        secret = api["secret"].encode("utf8")
        signature = hmac.new(secret, message, hashlib.sha384).hexdigest()
        api["headers"] = {
            "bfx-nonce": nonce,
            "bfx-apikey": api["key"],
            "bfx-signature": signature,
            "content-type": "application/json",
        }

    url = api["url"] + api["endpoint"]
    ret = requests.request(
        method=api["method"],
        url=url,
        data=api["data"],
        params=api["params"],
        headers=api["headers"],
    )
    response = ret.json()

My clients often ask me, where to store the api["secret"]. In the config file? In an environmental variable? Enter it manually at every restart and store it on physically on paper? I have no good answer and anything suggested... I quickly facepalm.

I set out to write an app - in python - to store the API keys:

primary feature:

  • given URL and key input:

  • writes secret to clipboard w/ xclip

  • auto clears clipboard in 10 seconds

security features:

  • reads/writes AES CBC encrypted password JSON to a text file

  • salt is 16 byte shake256 and generated in crypto secure manner w/ os.urandom

  • new salt after every return to the main menu and exit to prevent a dictionary attack

  • master password stretched to 400 megabytes to prevent GPU/FPGA attack

  • master password hashed iteratively 1,000,000 times via traditional salted pbkdf sha512 to prevent a brute attack

  • only 3rd party module is "pycryptodome"; raises an exception if deprecated "pycrypto" is found

  • sudo system password required to edit the script

My thought was my users could use this app to secure their keys and in a scripting cryptographic sense... I've dotted my I's and crossed my t's... when they're stored they're stored. Period. I'm quite convinced you cannot break my encryption scheme.

You can find my app here or by googling: litepresence/CypherVault

But I still facepalm. I opened a Reddit/r/security thread here to discuss my app and my facepalm was quickly validated.

In the end of things... no matter how I handle the api["secret"] it ends up on RAM which can be dumped by malware and uploaded to an attacker.

When the user enters the secret to be encrypted... its on RAM. When the script decrypts the secret to sign a transaction... its on RAM.

Then hocus pocus... "my script" caused somebody to lose money because it was not "secure"; I have a sense of responsibility.

How can this be avoided? How can one securely store exchange API secrets for composing financial transactions - by a bot - in scripting languages, on desktop machines, without at some point exposing them to your RAM, and potentially worse... your SWAP?

  • 4
    If you think that your RAM is compromised, you have already lost. – MechMK1 Dec 30 '19 at 17:20
  • 1
    You should post your code snippet on the companion codereview site. They will have lots of helpful suggestions for you. – Conor Mancone Dec 30 '19 at 17:27
  • 2
    I'm pretty sure its effectively impossible to both keep something out of RAM and use it, at least not without custom hardware solutions. I think you'll be helped by taking the time to work through a threat modelling session, and considering exactly what kind of threats you are attempting to protect the user from. The reason I suggest this is because (although I'm just repeating @MechMK1), anyone who can read secrets out of your RAM most likely has far more control over the machine, and your chances of keeping things secure is likely 0 anyway. – Conor Mancone Dec 30 '19 at 17:33
  • It is the client's fault if their credentials are compromised on their local machine. They shouldn't be running untrusted programs in the same context as something sensitive, if they are really concerned. – multithr3at3d Dec 30 '19 at 23:51
  • Really interesting question. One thing that might reduce the likelihood of the threat models you describe is to give your customers the option to run the entire application in a more secluded environment, such as a VM. One interesting approach would be packaging it in Vagrant. Obv the host machine could still be targeted, but this would be a less likely attack given the additional level of packaging of the sensitive data. – pw_v Jan 1 at 19:08
3

no matter how I handle the api["secret"] it ends up on RAM which can be dumped by malware and uploaded to an attacker.

...

How can this be avoided?

It can't be. There is a fundamental contradiction between the requirement to use the password for a computer-based authentication step, and yet to want the computer not to have access to the password. There's no third way.

You can encrypt it when not in use (sounds like you've done this). You can overwrite it in memory when not in use. But when the computer needs to use it, it's going to be in memory.

Depending on your threat model, this may not be a big deal. Government-level physical attack is the most common concern for memory contents, but few people need to gear their defenses against that.

| improve this answer | |
  • ,,,or inject the secret on a proxy. In itself, this just moves the vulnerability elsewhere, but you can use a second, per client secret to authenticate to the proxy and rotate it on each request and send it back to the client in the response, thereby disallowing re-use. Still not completely water-tight though/ – symcbean Dec 31 '19 at 1:15
0

This is exactly the problem that hardware security modules (HSM's) aim to solve. An HSM has its own embedded processor, and contains one or more private keys, which never leave the HSM. Hence, the private keys on the HSM are inaccessible to programs running on the machine that the HSM is connected to.

The HSM can be used to authenticate with a server, by proving that it has possession of the private key corresponding to the public key for the user. The HSM does this by computing a value using the private key and its onboard processor, such as a digital signature or a shared secret. The digital signature or shared secret are based on ephemeral values provided by the server, so the digital signature/shared secret cannot be reused by an attacker who is able to intercept these (i.e. the solution is resistant to replay attacks).

Of course, the service that you are using must support this type of authentication.

| improve this answer | |
  • all this does is substitute one secret for another secret - which then needs to be read back into memory to be injected into the request. – symcbean Dec 31 '19 at 1:12
  • Yes, but in this case the secret that is read back into memory and injected into the request cannot be reused. – mti2935 Dec 31 '19 at 1:17
  • So your method is predicated on rewriting the authentication software used by the exchanges providing the service. I think I still see a flaw there. – symcbean Dec 31 '19 at 1:19
  • Thanks for your comments. I've added some detail to the answer to clarify why this solution is resistant to the type of attack that you describe. With regard to the method being predicated on rewriting the authentication software used by the exchanges - yes, this is correct. The service has to support this type of authentication. But, this is basically how yubikey and FIDO work, and these solutions/standards are quickly being adopted. – mti2935 Dec 31 '19 at 1:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.