Invoking Google Cloud Platform (GCP) API from AWS Lambda

4 min readAug 7, 2019

Invoking Google Cloud Platform (GCP) API from AWS Lambda

Interestingly I was trying to experiment text detection, OCR, with various tools in the open source and was trying to compare their results and performances in various kind of documents both digital and handwritten. During this process I was inspired by the way google cloud vision out performed other text detection tools that were available.

As I was developing my entire engine in AWS Lambda I was curious to call google cloud vision API from AWS Lambda. I did hit a lot of roadblocks and finally was successful to call google cloud services from AWS which motivated me to write this article.

This article assumes that the reader is familiar with AWS, Lambda and serverless concepts.

Text Detection — Function

The Google Cloud Vision API enables a variety of detection like objects, text, OCR etc. It takes complex machine learning models focused around image recognition and formats it in a simple REST API interface that can be called from anywhere with proper authentication. Internally the Cloud Vision API uses a pre-trained model trained on a large dataset of images.

For the purpose of this article, we will contain ourselves on text detection capability of Google Cloud Vision API and provide you with steps to invoke the same from AWS lambda. The python code below can be used for text detection using google cloud vision API.

function.pyfrom google.cloud import vision
from google.cloud.vision import types
import io
from PIL import Image
from PIL import ImageDraw
from enum import Enum
import os
import jsonimage_file  = "test.jpg"
image  = Image.open(image_file)os.environ["GOOGLE_APPLICATION_CREDENTIALS"]="first.json"def lambda_handler(event, context):
    client = vision.ImageAnnotatorClient()
    with io.open(image_file, 'rb') as image_file1:
         content = image_file1.read()    content_image=types.Image(content=content)
    response=client.document_text_detection(image=content_image)
    texts=[text.description for text in response.text_annotations]
    print(texts[0])

Authentication

Authentication, as per google, refers to the process of determining a users or client’s identity. Authorization refers to the process of determining what permissions an authenticated client has for a set of resources. That is, authentication refers to who you are, and authorization refers to what you can do.

For authentication, google recommends using something called “service account”.Refer to https://cloud.google.com/docs/authentication/getting-started for creating service accounts in Google Cloud Platform. Its a Google account that is associated with your GCP project, as opposed to a specific user. Service accounts can be used for authentication regardless of where your code runs (locally, Compute Engine, App Engine, on premises, etc.).

After creating service account, download a corresponding JSON credentials file and setting the environmental variable of JSON file path to GOOGLE_APPLICATION_CREDENTIALS

os.environ[“GOOGLE_APPLICATION_CREDENTIALS”]=”Authenticate.json”

After setting the environment variable, you don’t need to explicitly specify your credentials in code when using a Google Cloud Client Library. The client library can determine your credentials implicitly.If the request succeeds, authentication works.

Deployment Package for AWS Lambda

A deployment package is nothing but a zip archive that contains our function code and dependencies. Lambda Console editor manages the deployment package when we author our function by default. This method is perfectly fine as long as we don’t need to add any libraries..In our case we need to invoke google cloud vision API from Lamba like (python),

from google.cloud import vision
from google.cloud.vision import types

Deployment package can be created if you plan to use the Lambda API to manage functions or if you need to include libraries and dependencies other than the AWS SDK. The deployment package can be directly uploaded to lambda or S3 bucket.

https://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html#python-package-venv elaborates the various methods of creating deployment package in python. I launched a EC2 instance to create my deployment package. As i was using python3.7, the built in venv module can be used to create a virtual environment instead of virtualenv as show below. The steps involved are,

Create virtual environment
Activate the environment
Install libraries with pip [install all libraries that are required for import]
Deactivate the virtual environment
Create zip archive with contents of library
Add your function code to the archive

~/my-testfunction$ python3 -m venv gcpenv~/my-testfunction$ source gcpenv/bin/activate (gcpenv)~/my-testfunction$ pip install google-cloud-vision(gcpenv)~/my-testfunction$ deactivate~/my-testfunction$ cd ../gcpenv/lib/python3.7/site-packages~/my-testfunction$ cd ../gcpenv/lib/python3.7/site-packages~/gcpenv/lib/python3.7/site-packages$ zip -r9 ${OLDPWD}mytestproject.zip .~/gcpenv/lib/python3.7/site-packages$ cd $OLDPWD~/my-testfunction$ zip -g mytestproject.zip function.py

Depending on the library, dependencies may appear in either site-packages or dist-packages, and the first folder in the virtual environment may be lib or lib64. You can use the pip show command to locate a specific package.

Deploy in Lambda

Once the deployment package is created, on the AWS console or CLI update the function code. I have used AWS console as shown below. The Handler tab highlighted below, takes the format of The filename.handler-method value in your function. For example, “function.handler” calls the handler method defined in function.py.

I hope this article helps with understanding the Google Cloud API using Python and AWS Lambda.If you’ve tried the above and have any feedback, let me know what you think poornima narasimhan and here are some links to everything I’ve discussed in this post:

Written by poornima narasimhan