Reading through this article, I hit a little wall of understanding. The author states:

This is because my laptop doesn’t have the secret weapon. Once you get this up and running on an EC2 instance, these temporary security credentials will be automatically retrieved from the instance metadata. If the concept of temporary credentials is new to you, here’s an excerpt from the documentation (emphasis mine)

I don't understand what he means by these temporarily security credentials will be automatically retrieved from the instance metadata. The argument is that there are temporary credentials being sent to authenticate the application's access to a given secret. He shows a demo script which doesn't send any credentials at all, so I'm not understanding how the demo application actually authenticates to the API Gateway in this model?

import re
import urlparse

# To install the required packages run: 
# pip install requests boto3 aws-requests-auth
import requests
from aws_requests_auth.aws_auth import AWSRequestsAuth
from aws_requests_auth import boto_utils

# We got this URL from our provisioned stack's output. This should be passed as a configuration variable, 
# but since there's no secrets in there, you could hard-code this
GITHUB_PROXY_URL = 'https://w974f1rs6e.execute-api.us-east-1.amazonaws.com/dev/repos/PokaInc/test-github-api/issues'

def get_aws_auth(url):
    # These next variables are needed for the signing process
    api_gateway_netloc = urlparse.urlparse(url).netloc
    api_gateway_region = re.match(

    return AWSRequestsAuth(
        # This is how we query the temporary credentials of the EC2 instance, as simple as that

list_issues_response = requests.get(

print list_issues_response.json()

As shown in the script, there is no authentication other than the script querying a proxy URL.


This is an interesting post, but on your specific question though, the answer is rather complicated (as all of AWS is).

Firstly it's important to understand that nearly every service in AWS is accessible via a authenticated REST API calls. These calls are made by your SDK (e.g. boto3 for python) which provides high level abstraction for your code.

So first let’s delve into how your AWS SDK gets the necessary credentials to authenticate itself to AWS. If you write a bit of code to access a file in S3 for example, your code needs to call the get-object method on the S3 API, and in order to do this, you'll need to sign that request with a key/token belonging to a IAM role that has the privileges to do so.

Fortunately AWS provide a very nice SDK to make this call for you, and all you have to do is supply it with credentials, which it will then use to automatically sign the request for you. The SDK looks for these credentials in 3 different places.

  • A specific environment variable
  • In a file located at ~/.aws/credentials
  • The metadata service

Once the SDK locates the credentials it will sign the request and you’ll be able to get the file from S3.

Most people understand files and environment variables, but trip up at the metadata service ... like what is that?! I didn’t even know it existed until I read the capitalone breach. This service exists on EC2 instances.

When you spin up an EC2 with an assigned IAM role, the instance comes with a metadata service that the SDK can request credentials from. The difference is that it uses STS tokens rather than Long-lived API keys. The end effect is still the same, as the SDK abstracts this complexity from your code. But in the background, your SDK continually requests tokens from the service, and use those tokens to sign the outgoing requests to AWS APIs.

The metadata service exist on a private IP on the EC2 instance (169.x.x.x), hence is accessible only inside the instance.

So what the author suggested was using an EC2 with a script that uses the metadata service to gain access to the API Gateway. That way the script will only use temporary tokens from STS rather than any Long-lived credentials like manually generated api keys. He could not do it on his local machine -- because it didn't have the metadata service.

| improve this answer | |

boto_utils.get_credentials() is adding aws_access_key, aws_secret_access_key, and aws_token to AWSRequestsAuth() by use of the two star operator **kwargs.

Thus, the credentials are being sent by the boto_utils library call which automatically pulls them from the instance metadata / IAM role.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.