Close

Introduction to ImageLabeller


ImageLabeller is a simple demo application that uses machine learning to apply labels to images. It uses AWS Lambda, AWS DynamoDB, AWS S3, AWS SageMaker, AWS API Gateway, AWS CloudWatch, and AWS CloudFormation, and is written in Go, and Python. This guide covers how to use Atlassian Open DevOps tools to deploy, and monitor a subset of these components.

ImageLabeller Architecture

Diagram

How it works

A user invokes AWS Lambda SubmitImage, passing a valid image URL as a parameter. SubmitImage downloads the image, uploads it to an AWS S3 bucket open-devops-images-, inserts a new tuple into an AWS DynamoDB table ImageLabels, and returns an image ID token to the user. The image ID token is used to correlate the image in open-devops-images-, and the tuple in ImageLabels.

The AWS Lambda InvokeLabeller has an AWS DynamoDB event trigger. InvokeLabeller downloads the corresponding image from open-devops-images-, calls a pre-trained AWS SageMaker Inception V3 model to generate image labels, and updates the tuple in ImageLabels when a new tuple is inserted into ImageLabels.

The user invokes AWS Lambda GetImageLabel, passing the image ID token as a parameter. GetImageLabel looks up the labels associated with the image ID token in ImageLabels, and returns them to the user. The user gets a NOT_CLASSIFIED label if the AWS SageMaker model hasn’t finished processing an image.

ImageLabeller is deployed to five environments. A Test environment in us-west-1, a Staging environment in us-east-2, and three Production environments in us-west-2, us-east-1, and ca-central-1.

Components

The AWS S3 buckets and AWS DynamoDB tables that are part of ImageLabeller are deployed using AWS CloudFormation templates. This guide provides CI/CD configuration information for how to deploy the AWS S3 buckets. The process for deploying the AWS DynamoDB tables is almost identical and is left as an exercise for you to complete on your own.

There are three AWS Lambda in ImageLabeller: SubmitImage, InvokeLabeller, and GetImageLabel. This guide provides CI/CD configuration information for how to deploy SubmitImage. The process for deploying GetImageLabel is almost identical and is also left for you to complete on your own.

InvokeLabeller is a python-based AWS Lambda and requires minor updating before it can be published. Specifically, InvokeLabeller is dependent on AWS SageMaker configuration which is done through the AWS console.

The system tests included in the ImageLabeller project test ImageLabeller as a whole. The tests will fail if any component of ImageLabeller is not present or working as expected. SubmitImage, GetImageLabel, and InvokeLabeller deployments all execute the system tests as part of their respective CI/CD pipelines.

For the initial deployment of SubmitImage, GetImageLabel, and InvokeLabeller you need to comment out system test execution steps. Once the deployments have successfully run once, you can uncomment the system test execution steps, and redeploy.

Order to deploy components

Deploy AWS SageMaker pre-trained model manually through the AWS console

Deploy AWS infrastructure repositories

  1. s3_infra
  2. dynamodb_infra

Deploy AWS Lambdas

  1. SubmitImage
  2. GetImageLabel
  3. InvokeLabeller

Deploy System Tests

Deploy CloudWatchAlarms

Setup AWS SageMaker Pre-Trained Model

Details how to setup the AWS SageMaker Pre-Trained Model.

Jira integration with CI/CD

Connect your planning, repository, CI/CD, and monitoring tools with Atlassian Open DevOps.

Warren Marusiak
Warren Marusiak

Warren is a Canadian developer from Vancouver, BC with over 10 years of experience. He came to Atlassian from AWS in January of 2021.


Share this article

Recommended reading

Bookmark these resources to learn about types of DevOps teams, or for ongoing updates about DevOps at Atlassian.

Devops illustration

DevOps community

Devops illustration

DevOps learning path

Map illustration

Get started for free

Sign up for our DevOps newsletter

Thank you for signing up