I write Python scripts for clients to "algo trade" at currency/stock exchange websites. My clients are typically running my scripts on traditional personal desktop PC's. Usually using these PCs for web browsing activities as well. The environment is always Linux; usually Debian. In the industry; Python is quite standard for algo trading in this manner; both institutionally and privately.
However, I can't help but see a flaw in the security model.
Each exchange has a slightly different authentication method, but in short, there is:
USER INPUTS:
api['secret'] # private key from exchange
USER CONFIG FILE CONTAINS:
api['key'] # public key from exchange
api['exchange'] # name of exchange; ie "binance"
api['symbol'] # market pair symbol in format BTC:USD
api['uri'] # the url up to .com/
FROM USER INPUTS SCRIPT BUILDS REQUEST SPECIFIC:
api['nonce'] # time.time() at beginning of request
api['endpoint'] # path/to/server/resource
api['url'] # uri + endpoint
api['method'] # GET, POST, or DELETE
api['params'] # dict with request specific parameters
api['data'] # str with request specific parameters
api['headers'] # authentication signature specific to the request
These requests are of the sort:
POST BUY/SELL
DELETE BUY/SELL (CANCEL)
GET ACCOUNT BALANCES
GET OPEN ORDERS
WITHDRAW FUNDS
Here are some examples of the authentication methods I've written to some currency exchanges; they're all quite standard in the forex/stock/crypto trading industry. Mostly in the generalized format:
api["header"] = {"signature": HMAC(SHA256(the_request_parameters))}
examples:
def signed_request(api, signal):
"""
Remote procedure call for authenticated exchange operations
api : dict with keys for building external request
signal : multiprocessing completion relay
"""
api = lookup_url(api)
api["data"] = ""
if api["exchange"] == "coinbase":
api["data"] = json_dumps(api["params"]) if api["params"] else ""
api["params"] = None
message = (
str(api["nonce"]) + api["method"] + api["endpoint"] + api["data"]
).encode("ascii")
secret = b64decode(api["secret"])
signature = hmac.new(secret, message, hashlib.sha256).digest()
signature = b64encode(signature).decode("utf-8")
api["headers"] = {
"Content-Type": "Application/JSON",
"CB-ACCESS-SIGN": signature,
"CB-ACCESS-TIMESTAMP": str(api["nonce"]),
"CB-ACCESS-KEY": api["key"],
"CB-ACCESS-PASSPHRASE": api["passphrase"],
}
elif api["exchange"] == "poloniex":
api["params"]["nonce"] = int(api["nonce"] * 1000)
message = urlencode(api["params"]).encode("utf-8")
secret = api["secret"].encode("utf-8")
signature = hmac.new(secret, message, hashlib.sha512).hexdigest()
api["headers"] = {
"Content-Type": "application/x-www-form-urlencoded",
"Key": api["key"],
"Sign": signature,
}
elif api["exchange"] == "binance":
api["params"]["timestamp"] = int(api["nonce"] * 1000)
api["params"]["signature"] = signature
message = urlencode(api["params"].items()).encode("utf-8")
secret = bytes(api["secret"].encode("utf-8"))
signature = hmac.new(secret, message, hashlib.sha256).hexdigest()
api["headers"] = {"X-MBX-APIKEY": api["key"]}
elif api["exchange"] == "bittrex":
api["params"]["apikey"] = api["key"]
api["params"]["nonce"] = int(api["nonce"] * 1000)
message = api["url"] + api["endpoint"] + urlencode(api["params"])
message = bytearray(message, "ascii")
secret = bytearray(api["secret"], "ascii")
signature = hmac.new(secret, message, hashlib.sha512).hexdigest()
api["headers"] = {}
elif api["exchange"] == "kraken":
api["data"] = api["params"][:]
api["params"] = {}
data["nonce"] = int(1000 * api["nonce"])
api["endpoint"] = "/2.1.0/private/" + api["endpoint"]
message = (str(data["nonce"]) + urlencode(data)).encode("ascii")
message = api["endpoint"].encode("ascii") + hashlib.sha256(message).digest()
secret = b64decode(api["secret"])
signature = b64encode(hmac.new(secret, message, hashlib.sha512).digest())
api["headers"] = {
"User-Agent": "krakenex/2.1.0",
"API-Key": api["key"],
"API-Sign": signature,
}
elif api["exchange"] == "bitfinex":
nonce = str(int(api["nonce"] * 1000))
api["endpoint"] = path = "v2/auth/r/orders"
api["data"] = json.dumps(api["params"])
api["params"] = {}
message = ("/api/" + api["endpoint"] + nonce + api["data"]).encode("utf8")
secret = api["secret"].encode("utf8")
signature = hmac.new(secret, message, hashlib.sha384).hexdigest()
api["headers"] = {
"bfx-nonce": nonce,
"bfx-apikey": api["key"],
"bfx-signature": signature,
"content-type": "application/json",
}
url = api["url"] + api["endpoint"]
ret = requests.request(
method=api["method"],
url=url,
data=api["data"],
params=api["params"],
headers=api["headers"],
)
response = ret.json()
My clients often ask me, where to store the api["secret"]. In the config file? In an environmental variable? Enter it manually at every restart and store it on physically on paper? I have no good answer and anything suggested... I quickly facepalm.
I set out to write an app - in python - to store the API keys:
primary feature:
given
URLandkeyinput:writes
secretto clipboard w/ xclipauto clears clipboard in 10 seconds
security features:
reads/writes AES CBC encrypted password JSON to a text file
salt is 16 byte shake256 and generated in crypto secure manner w/ os.urandom
new salt after every return to the main menu and exit to prevent a dictionary attack
master password stretched to 400 megabytes to prevent GPU/FPGA attack
master password hashed iteratively 1,000,000 times via traditional salted pbkdf sha512 to prevent a brute attack
only 3rd party module is "pycryptodome"; raises an exception if deprecated "pycrypto" is found
sudo system password required to edit the script
My thought was my users could use this app to secure their keys and in a scripting cryptographic sense... I've dotted my I's and crossed my t's... when they're stored they're stored. Period. I'm quite convinced you cannot break my encryption scheme.
You can find my app here or by googling: litepresence/CypherVault
But I still facepalm. I opened a Reddit/r/security thread here to discuss my app and my facepalm was quickly validated.
In the end of things... no matter how I handle the api["secret"] it ends up on RAM which can be dumped by malware and uploaded to an attacker.
When the user enters the secret to be encrypted... its on RAM.
When the script decrypts the secret to sign a transaction... its on RAM.
Then hocus pocus... "my script" caused somebody to lose money because it was not "secure"; I have a sense of responsibility.
How can this be avoided? How can one securely store exchange API secrets for composing financial transactions - by a bot - in scripting languages, on desktop machines, without at some point exposing them to your RAM, and potentially worse... your SWAP?
