Connecting Two Clouds - Accessing Private Resources In Aws From Azure Through A Lambda Proxy
We were trying to integrate an Azure Function to one of our services in AWS, on our development environment. Unfortunately, this environment is in a private VPC that can be accessed only from our network. Of the few solutions that we considered (lambda proxy, creating a VPN, opening the endpoint to public), using a lambda proxy seemed the fairer.
- It’s cheap
- It’s fast
- It’s flexible
- It doesn’t create a single point of failure
- It offers good security options
- It’s relatively easy to setup
The setup looks like that:
graph LR;
subgraph Azure
Function[Azure Function]
end
subgraph AWS
Gateway[API Gateway]
Lambda
subgraph Private VPC
Service[EC2 Service]
end
end
Function --> Gateway
Gateway --> Lambda
Lambda --> Service
To implement that model, you’ll need to:
- Create a Gateway resource
- (optional) Do a body mapping in Gateway so you can extract headers (if you need to)
- Setup the lambda to proxy the API calls to the private service
- Configure the lambda to gain access to the VPC
Setup the lambda
Create a new blank lambda function with Node.js.
Use the proxy.js
javascript you can find here. This basically pipes the body you receive into the private service and back, and maps whatever headers you need to the private service.
Alternatively you can use multiproxy.js
from the same source, which allows you to access multiple endpoints in the same VPC with only one lambda function.
proxy.js
gives you access to one precise endpoint in one service. Limiting to one endpoint reduces the chance that you’re opening something you didn’t want to by mistake.multiproxy.js
gives you access to the whole VPC, and requires that you set the host and path in the API Gateway endpoint. This allows one lambda function to act as a proxy to multiple endpoints, so you can create multiple endpoints in Gateway. Setup of that script is described in README of the GitHub repo.
You need to set the environment variables up, so the lambda function calls the correct endpoint in your VPC (not necessary when using multiproxy.js
):
TARGET_HOSTNAME
: Hostname of the proxied serviceTARGET_PATH
: resource path of the proxied servicesTARGET_METHOD
: POST, GET, PUT, etc.HEADERS
: a JSON list of headers to forward from the request, e.g.:["Authorization", "Content-Type"]
Then you need to configure the connection to the VPC. Go to the “configuration” screen of the lambda, then in the Advanced settings you need to set the VPC
, Subnets
and Security Group
the lambda should access to.
Then set the test event to something like this (will need to include target
information for multiproxy.js
):
{
"headers": {
"Authorization": "Bearer eyj0blabla...",
"content-type": "application/json"
// Whatever your client is adding
},
"body": {
// The actual body you'll receive from the client.
}
}
Then save + test. Hopefully it’s going to work. Else:
- Check the headers
- Check the environment variables
- Check the security settings are correctly set
Setup Gateway
Create a new API Gateway (or add a resource to one). Pick “Lambda function” as integration type, then select the lambda function you created in step 1.
Then you’ll have to setup the method request (what is exposed to the outer world) and the integration request (what is sent to the inner world).
Method request: It’s up to you how you want to secure your call, but if you don’t want the endpoint to be public you’ll need to set one. If you goal is to connect two clouds, API keys are an appropriate solution. You’ll just need to provide an additional header with all requests to authenticate them.
You can also go crazy and do some validation on your inputs and request type. Given that this is supposed to be a proxy, I would advocate that the less it changes the data, the less likely you’ll have to scratch your head later and try to figure why-o-why the messages don’t go through anymore.
Integration request: Unfortunately lambdas don’t expose headers to the execution code. You need to setup a body mapping in the integration request, to remap headers into the body. In the “Body Mapping Templates” section, select “Request Body Passthrough: Never”, then set a mapping for type application/json
. It goes like this (and it’s slightly different when using multiproxy.js
- refer to README):
{
"body" : $input.json('$'),
"headers": {
#foreach($header in $input.params().header.keySet())
"$header": "$util.escapeJavaScript($input.params().header.get($header))" #if($foreach.hasNext),#end
#end
}
}
There you go. You’ll need to define a stage (QA / prod / etc) and setup your API keys and Usage Plan, and finally “Deploy” the API from the “Actions” menu in the Resources of the API.
Then you can test invoking the function through the gateway using your favourite REST client. If it doesn’t work:
- Try making the lambda “dumb” (i.e. return static text, or the event object)
- Try using the Cloud Watch console on the lambda to verify it’s invoked
- Abandon everything and go live your life on a far, far away country, cultivating coca for a living, remarry with the daughter or son of a local car dealer1 and get chased by the international police2, change identity again and move to a random place in Africa, become a valued, but envied counsel to the Democratic Revolutionary Government of the People’s Republic, get booted out when the
dictatordemocratically elected president discovers your liaison with one of his wives, then come back to your ex and ask for forgiveness, which you won’t obtain since she also remarried, gain power in the debate for gun control and become a minor celebrity invited to discuss the issue on the TV shows, develop a problem with alcohol that leads you to an addiction to crack-cocaine, fight the hardest fight of your life to get back out of your misery, go to all the people you hurt and this time ask for genuine forgiveness, figure out what your password was, it’s been so long, and then try to fix the API Gateway or the lambda. - Seriously, I don’t know, maybe something doesn’t work.
Creating other endpoints to the same VPC
If you opted for multiproxy.js
, adding new endpoints is done by adding resources in the gateway and repeating “Setup the gateway”.
Configuring the Azure Function
The last step is to configure the Azure Function to call the API Gateway. The only thing to do is set the host and path to that we just created, and add a header x-api-key
to the request.
Notes
1 Make an educated choice based on the gender you assimilate to and your general preferences and tastes but please go with the one that makes you happy rather that the one that social pressure forces you into, you’re on the run chasing a better life after all, might as well making it happen