Articles

To demonstrate how to develop, deploy, and manage applications using Jira Software and various connected tools, our team created ImageLabeller, a simple demo application built on AWS that uses machine learning to apply labels to images.

This page covers how to deploy ImageLabeller with GitHub. Before you begin, we recommend reading the ImageLabeller architecture and AWS SageMaker setup pages for context.

Prerequisites

If you don’t already have a GitHub organization, follow the steps in this GitHub guide to create one from scratch.

Public facing GitHub repositories with ImageLabeller code

https://github.com/AtlassianOpenDevOpsGuides

Jira GitHub integration demo video

Watch video

Integrate Jira and GitHub

From Jira click Board, then Apps, then GitHub.

Click Get it now.

Click Apps, then Manage apps, and expand GitHub.

Click Get started.

Click Get started.

Install Jira plugin in GitHub

Click Install Jira to install the Jira plugin to GitHub. This step is necessary if the Jira plugin is not already installed in GitHub.

connect github organization to jira image

Jira will start to sync data from GitHub.

When the sync is complete the Sync Status changes to COMPLETE.

Setup SSH access

Set up GitHub SSH keys for your account by following the instructions here. Additional reference material is present here.

Setup a Personal Access Token

Setup a Personal Access Token by following this guide. The PAT is used to clone the SystemTest repo during integration testing steps. Give the PAT repo and workflow access.

Create a repository for AWS S3 infrastructure

A standard developer loop typically has a developer picking up a task from Jira, moving it to work in progress, and then doing the development work. The Jira issue ID is the key which ties the development work to the Jira issue. It is the core integration component between the two systems.

Go to Jira, and create a new issue for adding an AWS S3 infrastructure repository to GitHub. Make note of the issue ID. IM-9 in this example.

Go to GitHub and click New. Choose the appropriate organization for Owner. Click Create repository to proceed.

Add an AWS access key repository variables

Click Settings, then Secrets. Click New repository secret, and add AWS access key ID, and AWS secret access key. Give the IAM user associated with the AWS access key AdministratorAccess. You can opt to use more fine grained access control by choosing individual AWS access policies, but the details are left to the reader.

In your terminal go to your s3_infra repository, and run the following to push to GitHub.

git add --all
git commit -m "IM-9 add s3_infra repository to github"
git remote add origin git@github.com:PmmQuickStartGuides01/s3_infra.git
git branch -m mainline
git push -u origin mainline

GitHub actions for deploying to AWS

Go to your s3_infra repository in your terminal, create a branch named after your Jira issue ID, and create a .github/workflows directory.

git checkout -b IM-9
mkdir -p .github/workflows && cd .github/workflows

Create deploy-test-staging.yml with the following yaml in the new .github/workflows directory. This defines a deployment workflow for your Test, and Staging environments that runs during pushes to branches other than mainline.

name: deploy-s3-infra-test-staging
on:
  push:
    branches:
      - '*'
      - '!mainline'

jobs:
  deploy-us-west-1:
    runs-on: ubuntu-latest
    outputs:
      env-name: ${{ steps.env-name.outputs.environment }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Configure AWS credentials
        id: creds
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: "us-west-1"
      - name: Deploy to AWS CloudFormation
        uses: aws-actions/aws-cloudformation-github-deploy@v1
        with:
          name: OpenDevOpsS3Infra
          template: template.yml
          no-fail-on-empty-changeset: "1"

  deploy-us-east-2:
    runs-on: ubuntu-latest
    needs: deploy-us-west-1
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Configure AWS credentials
        id: creds
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: "us-east-2"
      - name: Deploy to AWS CloudFormation
        uses: aws-actions/aws-cloudformation-github-deploy@v1
        with:
          name: OpenDevOpsS3Infra
          template: template.yml
          no-fail-on-empty-changeset: "1"

Then create deploy-prod.yml with the following yaml. This defines a deployment workflow for your Production environments that runs when a pull request merges changes into mainline.

name: deploy-s3-infra-prod
on:
  pull_request:
    branches:
      - mainline

jobs:
  deploy-us-west-2:
    runs-on: ubuntu-latest
    outputs:
      env-name: ${{ steps.env-name.outputs.environment }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Configure AWS credentials
        id: creds
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: "us-west-2"
      - name: Deploy to AWS CloudFormation
        uses: aws-actions/aws-cloudformation-github-deploy@v1
        with:
          name: OpenDevOpsS3Infra
          template: template.yml
          no-fail-on-empty-changeset: "1"

  deploy-ca-central-1:
    runs-on: ubuntu-latest
    needs: deploy-us-west-2
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Configure AWS credentials
        id: creds
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: "ca-central-1"
      - name: Deploy to AWS CloudFormation
        uses: aws-actions/aws-cloudformation-github-deploy@v1
        with:
          name: OpenDevOpsS3Infra
          template: template.yml
          no-fail-on-empty-changeset: "1"

  deploy-us-east-1:
    runs-on: ubuntu-latest
    needs: deploy-ca-central-1
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Configure AWS credentials
        id: creds
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: "us-east-1"
      - name: Deploy to AWS CloudFormation
        uses: aws-actions/aws-cloudformation-github-deploy@v1
        with:
          name: OpenDevOpsS3Infra
          template: template.yml
          no-fail-on-empty-changeset: "1"

Understanding GitHub actions

Triggers

Declare events that trigger the workflow at the top of these files. In deploy-test-staging.yml the event is pushes to all branches except mainline.

name: deploy-s3-infra-test-staging
on:
  push:
    branches:
      - '*'
      - '!mainline'

Many events can trigger workflows. See documentation here for more information.

Jobs

A workflow contains many jobs that run when the triggering event occurs. Each job has a set of steps that run when the job is executed. There is a step to checkout the repository code, a step to configure AWS credentials, and a step to deploy to AWS using AWS CloudFormation in this example.

jobs:
  deploy-us-west-1:
    runs-on: ubuntu-latest
    outputs:
      env-name: ${{ steps.env-name.outputs.environment }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Configure AWS credentials
        id: creds
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: "us-west-1"
      - name: Deploy to AWS CloudFormation
        uses: aws-actions/aws-cloudformation-github-deploy@v1
        with:
          name: OpenDevOpsS3Infra
          template: template.yml
          no-fail-on-empty-changeset: "1"

Read more about jobs here, and more about steps here.

Enforcing order with needs

You can specify an order to your jobs using needs. GitHub runs all steps in parallel by default. Use needs to make one step depend on another.

deploy-us-east-1:
    runs-on: ubuntu-latest
    needs: deploy-ca-central-1
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Configure AWS credentials
        id: creds
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: "us-east-1"
      - name: Deploy to AWS CloudFormation
        uses: aws-actions/aws-cloudformation-github-deploy@v1
        with:
          name: OpenDevOpsS3Infra
          template: template.yml
          no-fail-on-empty-changeset: "1"

Pushing to a feature branch

From the command line run the following to push your code to the IM-9 branch of your s3_infra repository.

git add --all
git commit -m "IM-9 add GitHub actions to s3_infra"
git push -u origin IM-9

Click Actions to see running workflows.

Creating a pull request

Click Pull requests followed by Create pull request.

Choose your feature branch as the source branch, then click Create pull request.

Review the code changes, then click Merge pull request to accept the changes.

Click Actions to see that the Production deployment has started. GitHub runs the jobs in deploy-prod.yml because the branch is mainline.

Create a repository for SubmitImage AWS Lambda

Go to Jira and create a Jira issue for adding a SubmitImage AWS Lambda repository to GitHub. Take note of the Jira issue ID. IM-8 in this example.

Go to GitHub and click New. Choose the appropriate organization for Owner. Click Create repository to proceed.

Click Settings, then Secrets. Add your Personal Access Token as ACCESS_KEY, your AWS access key id as AWS_ACCESS_KEY_ID, your AWS secret access key as AWS_SECRET_ACCESS_KEY, and your AWS account ID as AWS_ACCOUNT_ID.

In your terminal go to your SubmitImage repository, and run the following to push your code to GitHub.

git add --all
git commit -m "IM-8 add SubmitImage to github"
git remote add origin git@github.com:PmmQuickStartGuides01/submitImage.git
git branch -m mainline
git push -u origin mainline

GitHub actions for deploying to AWS

Go to your SubmitImage repository in your terminal, create a branch named after your Jira issue ID, and create a .github/workflows directory.

git checkout -b IM-8
mkdir -p .github/workflows && cd .github/workflows

name: deploy-submit-image-test-staging
on:
  push:
    branches:
      - '*'
      - '!mainline'

env:
  aws_account_id: ${{secrets.AWS_ACCOUNT_ID}}

jobs:
  run-unit-tests:
    runs-on: ubuntu-latest
    steps:
      - name: Install Go
        uses: actions/setup-go@v2
        with:
          go-version: 1.16.x
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Test
        run: |
          cd ${{ github.workspace }}/submitImage
          ls
          go test ./opendevopslambda...

  deploy-us-west-1:
    runs-on: ubuntu-latest
    needs: run-unit-tests
    outputs:
      env-name: ${{ steps.env-name.outputs.environment }}
    steps:
      - name: Install Go
        uses: actions/setup-go@v2
        with:
          go-version: 1.16.x
      - uses: actions/checkout@v2
      - name: Configure AWS credentials
        id: creds
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: "us-west-1"
      - name: SAM Validate
        run: |
          sam --version
          sam validate
      - name: SAM Build
        run: |
          sam build
      - name: SAM Deploy
        continue-on-error: true
        run: |
          sam deploy --template-file .aws-sam/build/template.yaml \
            --stack-name "OpenDevOpsSubmitImage" \
            --s3-bucket "open-devops-code-us-west-1-${aws_account_id}" \
            --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM

            #  integration-test-us-west-1:
            #    runs-on: ubuntu-latest
            #    needs: deploy-us-west-1
            #    steps:
            #      - name: Pull systemTests repo
            #        uses: actions/checkout@master
            #        with:
            #          repository: PmmQuickStartGuides01/systemTests
            #          token: ${{ secrets.ACCESS_KEY }}
            #      - name: Configure AWS credentials
            #        id: creds
            #        uses: aws-actions/configure-aws-credentials@v1
            #        with:
            #          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
            #          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
            #          aws-region: "us-west-1"
            #      - name: Install Go
            #        uses: actions/setup-go@v2
            #        with:
            #          go-version: 1.16.x
            #      - name: Run integration tests
            #        run: go test -v ./... -aws_region=us-west-1

  deploy-us-east-2:
    runs-on: ubuntu-latest
    needs: deploy-us-west-1
    outputs:
      env-name: ${{ steps.env-name.outputs.environment }}
    steps:
      - name: Install Go
        uses: actions/setup-go@v2
        with:
          go-version: 1.16.x
      - uses: actions/checkout@v2
      - name: Configure AWS credentials
        id: creds
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: "us-east-2"
      - name: SAM Validate
        run: |
          sam --version
          sam validate
      - name: SAM Build
        run: |
          sam build
      - name: SAM Deploy
        continue-on-error: true
        run: |
          sam deploy --template-file .aws-sam/build/template.yaml \
            --stack-name "OpenDevOpsSubmitImage" \
            --s3-bucket "open-devops-code-us-east-2-${aws_account_id}" \
            --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM

            #  integration-test-us-east-2:
            #    runs-on: ubuntu-latest
            #    needs: deploy-us-east-2
            #    steps:
            #      - name: Pull systemTests repo
            #        uses: actions/checkout@master
            #        with:
            #          repository: PmmQuickStartGuides01/systemTests
            #          token: ${{ secrets.ACCESS_KEY }}
            #      - name: Configure AWS credentials
            #        id: creds
            #        uses: aws-actions/configure-aws-credentials@v1
            #        with:
            #          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
            #          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
            #          aws-region: "us-east-2"
            #      - name: Install Go
            #        uses: actions/setup-go@v2
            #        with:
            #          go-version: 1.16.x
            #      - name: Run integration tests
            #        run: go test -v ./... -aws_region=us-east-2

Then create deploy-prod.yml with the following yaml. This defines a deployment workflow for your Production environments that runs when a pull request merges changes into mainline.

name: deploy-submit-image-prod
on:
  pull_request:
    branches:
      - mainline

env:
  aws_account_id: ${{secrets.AWS_ACCOUNT_ID}}

jobs:
  deploy-us-west-2:
    runs-on: ubuntu-latest
    outputs:
      env-name: ${{ steps.env-name.outputs.environment }}
    steps:
      - name: Install Go
        uses: actions/setup-go@v2
        with:
          go-version: 1.16.x
      - uses: actions/checkout@v2
      - name: Configure AWS credentials
        id: creds
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: "us-west-2"
      - name: SAM Validate
        run: |
          sam --version
          sam validate
      - name: SAM Build
        run: |
          sam build
      - name: SAM Deploy
        continue-on-error: true
        run: |
          sam deploy --template-file .aws-sam/build/template.yaml \
            --stack-name "OpenDevOpsSubmitImage" \
            --s3-bucket "open-devops-code-us-west-2-${aws_account_id}" \
            --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM

            #  integration-test-us-west-2:
            #    runs-on: ubuntu-latest
            #    needs: deploy-us-west-2
            #    steps:
            #      - name: Pull systemTests repo
            #        uses: actions/checkout@master
            #        with:
            #          repository: PmmQuickStartGuides01/systemTests
            #          token: ${{ secrets.ACCESS_KEY }}
            #      - name: Configure AWS credentials
            #        id: creds
            #        uses: aws-actions/configure-aws-credentials@v1
            #        with:
            #          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
            #          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
            #          aws-region: "us-west-2"
            #      - name: Install Go
            #        uses: actions/setup-go@v2
            #        with:
            #          go-version: 1.16.x
            #      - name: Run integration tests
            #        run: go test -v ./... -aws_region=us-west-2

  deploy-us-east-1:
    runs-on: ubuntu-latest
    needs: deploy-us-west-2
    outputs:
      env-name: ${{ steps.env-name.outputs.environment }}
    steps:
      - name: Install Go
        uses: actions/setup-go@v2
        with:
          go-version: 1.16.x
      - uses: actions/checkout@v2
      - name: Configure AWS credentials
        id: creds
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: "us-east-1"
      - name: SAM Validate
        run: |
          sam --version
          sam validate
      - name: SAM Build
        run: |
          sam build
      - name: SAM Deploy
        continue-on-error: true
        run: |
          sam deploy --template-file .aws-sam/build/template.yaml \
            --stack-name "OpenDevOpsSubmitImage" \
            --s3-bucket "open-devops-code-us-east-1-${aws_account_id}" \
            --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM

            #  integration-test-us-east-1:
            #    runs-on: ubuntu-latest
            #    needs: deploy-us-east-1
            #    steps:
            #      - name: Pull systemTests repo
            #        uses: actions/checkout@master
            #        with:
            #          repository: PmmQuickStartGuides01/systemTests
            #          token: ${{ secrets.ACCESS_KEY }}
            #      - name: Configure AWS credentials
            #        id: creds
            #        uses: aws-actions/configure-aws-credentials@v1
            #        with:
            #          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
            #          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
            #          aws-region: "us-east-1"
            #      - name: Install Go
            #        uses: actions/setup-go@v2
            #        with:
            #          go-version: 1.16.x
            #      - name: Run integration tests
            #        run: go test -v ./... -aws_region=us-east-1

  deploy-ca-central-1:
    runs-on: ubuntu-latest
    needs: deploy-us-east-1
    outputs:
      env-name: ${{ steps.env-name.outputs.environment }}
    steps:
      - name: Install Go
        uses: actions/setup-go@v2
        with:
          go-version: 1.16.x
      - uses: actions/checkout@v2
      - name: Configure AWS credentials
        id: creds
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: "ca-central-1"
      - name: SAM Validate
        run: |
          sam --version
          sam validate
      - name: SAM Build
        run: |
          sam build
      - name: SAM Deploy
        continue-on-error: true

        run: |
          sam deploy --template-file .aws-sam/build/template.yaml \
            --stack-name "OpenDevOpsSubmitImage" \
            --s3-bucket "open-devops-code-ca-central-1-${aws_account_id}" \
            --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM

            #  integration-test-ca-central-1:
            #    runs-on: ubuntu-latest
            #    needs: deploy-ca-central-1
            #    steps:
            #      - name: Pull systemTests repo
            #        uses: actions/checkout@master
            #        with:
            #          repository: PmmQuickStartGuides01/systemTests
            #          token: ${{ secrets.ACCESS_KEY }}
            #      - name: Configure AWS credentials
            #        id: creds
            #        uses: aws-actions/configure-aws-credentials@v1
            #        with:
            #          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
            #          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
            #          aws-region: "ca-central-1"
            #      - name: Install Go
            #        uses: actions/setup-go@v2
            #        with:
            #          go-version: 1.16.x
            #      - name: Run integration tests
            #        run: go test -v ./... -aws_region=ca-central-1

The execution of the integration tests is commented out for now. The system tests will only pass when the entire application is deployed. Uncomment the integration test steps in your repository, and do another push to run the deployment pipeline after all components of ImageLabeller are deployed. Update the needs section to run the steps in order. You must update the git clone line for SystemTests to be your SystemTests repository.

needs: deploy-us-east-1

Understanding GitHub actions

This job uses AWS SAM to deploy your SubmitImage AWS Lambda to us-west-2.

deploy-us-west-2:
    runs-on: ubuntu-latest
    outputs:
      env-name: ${{ steps.env-name.outputs.environment }}
    steps:
      - name: Install Go
        uses: actions/setup-go@v2
        with:
          go-version: 1.16.x
      - uses: actions/checkout@v2
      - name: Configure AWS credentials
        id: creds
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: "us-west-2"
      - name: SAM Validate
        run: |
          sam --version
          sam validate
      - name: SAM Build
        run: |
          sam build
      - name: SAM Deploy
        continue-on-error: true
        run: |
          sam deploy --template-file .aws-sam/build/template.yaml \
            --stack-name "OpenDevOpsSubmitImage" \
            --s3-bucket "open-devops-code-us-west-2-${aws_account_id}" \
            --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM

This job clones the SystemTests repository, and runs integrations tests in us-west-2.

integration-test-us-west-2:
    runs-on: ubuntu-latest
    needs: deploy-us-west-2
    steps:
      - name: Pull systemTests repo
        uses: actions/checkout@master
        with:
          repository: PmmQuickStartGuides01/systemTests
          token: ${{ secrets.ACCESS_KEY }}
      - name: Configure AWS credentials
        id: creds
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: "us-west-2"
      - name: Install Go
        uses: actions/setup-go@v2
        with:
          go-version: 1.16.x
      - name: Run integration tests
        run: go test -v ./... -aws_region=us-west-2

This step uses the Personal Access Token to clone the SystemTests repository.

    - name: Pull systemTests repo
        uses: actions/checkout@master
        with:
          repository: PmmQuickStartGuides01/systemTests
          token: ${{ secrets.ACCESS_KEY }}

Pushing to a feature branch

Run the following from the command line to push your changes to the IM-8 branch of your SubmitImage repository. Include the Jira issue ID in commit messages, and branch names to enable the Jira GitHub integration to keep track of what is happening in your project.

git add --all
git commit -m "IM-8 add github actions to SubmitImage"
git push -u origin IM-8

Click Actions to see running workflows.

Notice that the unit test job has passed, and deployment to Test us-west-1 has started.

The system tests defined earlier are run as part of the integration-test-us-west-1, and integration-test-us-east-2 jobs.

Create a pull request

To create a pull request click Pull requests, then New pull request.

Choose to merge from your feature branch.

Click Create pull request.

Merge the pull request, and delete the feature branch. Click Actions to monitor the Production deployment.

Create a repository for InvokeLabeller AWS Lambda

Go to Jira and create a Jira issue for adding a InvokeLabeller AWS Lambda repository to GitHub. Take note of the Jira issue ID. IM-11 in this example.

Go to GitHub and click New. Choose the appropriate organization for Owner. Click Create repository to proceed.

In your terminal go to your InvokeLabeller repository, and run the following to push your code to GitHub.

git add --all
git commit -m "IM-11 add InvokeLabeller to github"
git remote add origin git@github.com:PmmQuickStartGuides01/InvokeLabeller.git
git branch -m mainline
git push -u origin mainline

GitHub actions for deploying to AWS

Go to your InvokeLabeller repository in your terminal, create a branch named after your Jira issue ID, and create a .github/workflows directory.

git checkout -b IM-11
mkdir -p .github/workflows && cd .github/workflows

name: deploy-invoke-labeller-test-staging
on:
  push:
    branches:
      - '*'
      - '!mainline'

env:
  aws_account_id: ${{secrets.AWS_ACCOUNT_ID}}

jobs:
  run-unit-tests:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - uses: actions/setup-python@v2
        with:
          python-version: '3.x'
      - name: Test
        run: |
          pip3 install pytest
          pip3 install moto
          pip3 install -r tst/requirements.txt --user
          python3 -m pytest -v tst/unit --junitxml=test-reports/report.xml

  deploy-us-west-1:
    runs-on: ubuntu-latest
    needs: run-unit-tests
    outputs:
      env-name: ${{ steps.env-name.outputs.environment }}
    steps:
      - name: Install Go
        uses: actions/setup-go@v2
        with:
          go-version: 1.16.x
      - uses: actions/checkout@v2
      - name: Configure AWS credentials
        id: creds
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: "us-west-1"
      - name: SAM Validate
        run: |
          sam --version
          sam validate
      - name: SAM Build
        run: |
          sam build
      - name: SAM Deploy
        continue-on-error: true
        run: |
          sam deploy --template-file .aws-sam/build/template.yaml \
            --stack-name "OpenDevOpsImageLabeller" \
            --s3-bucket "open-devops-code-us-west-1-${aws_account_id}" \
            --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM

            #  integration-test-us-west-1:
            #    runs-on: ubuntu-latest
            #    needs: deploy-us-west-1
            #    steps:
            #      - name: Pull systemTests repo
            #        uses: actions/checkout@master
            #        with:
            #          repository: PmmQuickStartGuides01/systemTests
            #          token: ${{ secrets.ACCESS_KEY }}
            #      - name: Configure AWS credentials
            #        id: creds
            #        uses: aws-actions/configure-aws-credentials@v1
            #        with:
            #          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
            #          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
            #          aws-region: "us-west-1"
            #      - name: Install Go
            #        uses: actions/setup-go@v2
            #        with:
            #          go-version: 1.16.x
            #      - name: Run integration tests
            #        run: go test -v ./... -aws_region=us-west-1

  deploy-us-east-2:
    runs-on: ubuntu-latest
    needs: deploy-us-west-1
    outputs:
      env-name: ${{ steps.env-name.outputs.environment }}
    steps:
      - name: Install Go
        uses: actions/setup-go@v2
        with:
          go-version: 1.16.x
      - uses: actions/checkout@v2
      - name: Configure AWS credentials
        id: creds
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: "us-east-2"
      - name: SAM Validate
        run: |
          sam --version
          sam validate
      - name: SAM Build
        run: |
          sam build
      - name: SAM Deploy
        continue-on-error: true
        run: |
          sam deploy --template-file .aws-sam/build/template.yaml \
            --stack-name "OpenDevOpsImageLabeller" \
            --s3-bucket "open-devops-code-us-east-2-${aws_account_id}" \
            --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM

            #  integration-test-us-east-2:
            #    runs-on: ubuntu-latest
            #    needs: deploy-us-east-2
            #    steps:
            #      - name: Pull systemTests repo
            #        uses: actions/checkout@master
            #        with:
            #          repository: PmmQuickStartGuides01/systemTests
            #          token: ${{ secrets.ACCESS_KEY }}
            #      - name: Configure AWS credentials
            #        id: creds
            #        uses: aws-actions/configure-aws-credentials@v1
            #        with:
            #          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
            #          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
            #          aws-region: "us-east-2"
            #      - name: Install Go
            #        uses: actions/setup-go@v2
            #        with:
            #          go-version: 1.16.x
            #      - name: Run integration tests
            #        run: go test -v ./... -aws_region=us-east-2

Then create deploy-prod.yml with the following yaml. This defines a deployment workflow for your Production environments that runs when a pull request merges changes into mainline.

name: deploy-invoke-labeller-prod
on:
  pull_request:
    branches:
      - mainline

env:
  aws_account_id: ${{secrets.AWS_ACCOUNT_ID}}

jobs:
  deploy-us-west-2:
    runs-on: ubuntu-latest
    outputs:
      env-name: ${{ steps.env-name.outputs.environment }}
    steps:
      - name: Install Go
        uses: actions/setup-go@v2
        with:
          go-version: 1.16.x
      - uses: actions/checkout@v2
      - name: Configure AWS credentials
        id: creds
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: "us-west-2"
      - name: SAM Validate
        run: |
          sam --version
          sam validate
      - name: SAM Build
        run: |
          sam build
      - name: SAM Deploy
        continue-on-error: true
        run: |
          sam deploy --template-file .aws-sam/build/template.yaml \
            --stack-name "OpenDevOpsImageLabeller" \
            --s3-bucket "open-devops-code-us-west-2-${aws_account_id}" \
            --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM

            #  integration-test-us-west-2:
            #    runs-on: ubuntu-latest
            #    needs: deploy-us-west-2
            #    steps:
            #      - name: Pull systemTests repo
            #        uses: actions/checkout@master
            #        with:
            #          repository: PmmQuickStartGuides01/systemTests
            #          token: ${{ secrets.ACCESS_KEY }}
            #      - name: Configure AWS credentials
            #        id: creds
            #        uses: aws-actions/configure-aws-credentials@v1
            #        with:
            #          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
            #          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
            #          aws-region: "us-west-2"
            #      - name: Install Go
            #        uses: actions/setup-go@v2
            #        with:
            #          go-version: 1.16.x
            #      - name: Run integration tests
            #        run: go test -v ./... -aws_region=us-west-2

  deploy-us-east-1:
    runs-on: ubuntu-latest
    needs: deploy-us-west-2
    outputs:
      env-name: ${{ steps.env-name.outputs.environment }}
    steps:
      - name: Install Go
        uses: actions/setup-go@v2
        with:
          go-version: 1.16.x
      - uses: actions/checkout@v2
      - name: Configure AWS credentials
        id: creds
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: "us-east-1"
      - name: SAM Validate
        run: |
          sam --version
          sam validate
      - name: SAM Build
        run: |
          sam build
      - name: SAM Deploy
        continue-on-error: true
        run: |
          sam deploy --template-file .aws-sam/build/template.yaml \
            --stack-name "OpenDevOpsImageLabeller" \
            --s3-bucket "open-devops-code-us-east-1-${aws_account_id}" \
            --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM

            #  integration-test-us-east-1:
            #    runs-on: ubuntu-latest
            #    needs: deploy-us-east-1
            #    steps:
            #      - name: Pull systemTests repo
            #        uses: actions/checkout@master
            #        with:
            #          repository: PmmQuickStartGuides01/systemTests
            #          token: ${{ secrets.ACCESS_KEY }}
            #      - name: Configure AWS credentials
            #        id: creds
            #        uses: aws-actions/configure-aws-credentials@v1
            #        with:
            #          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
            #          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
            #          aws-region: "us-east-1"
            #      - name: Install Go
            #        uses: actions/setup-go@v2
            #        with:
            #          go-version: 1.16.x
            #      - name: Run integration tests
            #        run: go test -v ./... -aws_region=us-east-1

  deploy-ca-central-1:
    runs-on: ubuntu-latest
    needs: deploy-us-east-1
    outputs:
      env-name: ${{ steps.env-name.outputs.environment }}
    steps:
      - name: Install Go
        uses: actions/setup-go@v2
        with:
          go-version: 1.16.x
      - uses: actions/checkout@v2
      - name: Configure AWS credentials
        id: creds
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: "ca-central-1"
      - name: SAM Validate
        run: |
          sam --version
          sam validate
      - name: SAM Build
        run: |
          sam build
      - name: SAM Deploy
        continue-on-error: true

        run: |
          sam deploy --template-file .aws-sam/build/template.yaml \
            --stack-name "OpenDevOpsImageLabeller" \
            --s3-bucket "open-devops-code-ca-central-1-${aws_account_id}" \
            --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM

            #  integration-test-ca-central-1:
            #    runs-on: ubuntu-latest
            #    needs: deploy-ca-central-1
            #    steps:
            #      - name: Pull systemTests repo
            #        uses: actions/checkout@master
            #        with:
            #          repository: PmmQuickStartGuides01/systemTests
            #          token: ${{ secrets.ACCESS_KEY }}
            #      - name: Configure AWS credentials
            #        id: creds
            #        uses: aws-actions/configure-aws-credentials@v1
            #        with:
            #          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
            #          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
            #          aws-region: "ca-central-1"
            #      - name: Install Go
            #        uses: actions/setup-go@v2
            #        with:
            #          go-version: 1.16.x
            #      - name: Run integration tests
            #        run: go test -v ./... -aws_region=ca-central-1

needs: deploy-us-east-1

Update src/app.py with AWS SageMaker endpoint

Open InvokeLabeller’s src/app.py file and look for query_endpoint. Change the endpoint_name, and client region_name to match your AWS SageMaker notebook.

def query_endpoint(img):
  endpoint_name = 'jumpstart-dft-image-labeller-endpoint'
  client = boto3.client(service_name='runtime.sagemaker', region_name='us-west-1')
  response = client.invoke_endpoint(EndpointName=endpoint_name, ContentType='application/x-image', Body=img)
  model_predictions = json.loads(response['Body'].read())['predictions'][0]
  return model_predictions

Pushing to a feature branch

Run the following from the command line to push your changes to the IM-11 branch of your InvokeLabeller repository. Include the Jira issue ID in commit messages, and branch names to enable the Jira GitHub integration to keep track of what is happening in your project.

git add --all
git commit -m "IM-11 add github actions to InvokeLabeller"
git push -u origin IM-11

Click Actions to see running workflows. The system tests defined earlier are run as part of the integration-test-us-west-1, and integration-test-us-east-2 jobs.

Create a pull request

To create a pull request click Pull requests, then New pull request. Choose to merge from your feature branch.

Click Actions to monitor the Production deployment.

Create a repository for SystemTests

Go to Jira and create a Jira issue for adding a SystemTests repository to GitHub. Take note of the Jira issue ID. In this example it is IM-7.

Go to GitHub and click New. Choose the appropriate organization for Owner. Click Create repository to proceed.

In your terminal go to your SystemTests repository, and run the following to push your code to GitHub.

git add --all
git commit -m "IM-7 add SystemTests repository to GitHub"
git remote add origin git@github.com:PmmQuickStartGuides01/systemTests.git
git branch -M mainline
git push -u origin mainline

The SystemTests repository doesn’t need GitHub actions. It has no pipeline of its own since it provides tests for other pipelines to run. The integration test steps of the CI/CD workflow files can be uncommented, committed, and pushed once all components of the ImageLabeller application are deployed. The tests will only pass if all components of the application are working properly.

Take note of your SystemTests' remote url. SubmitImage, GetImageLabel, and InvokeLabeller CI/CD pipelines will clone the SystemTests repository during testing steps. You will need to update the gitlab-ci.yml of later repositories with the correct url.

If you’ve made this far, congratulations! You just deployed ImageLabeller. The next step is to set up monitoring ImageLabeller with Opsgenie.

Warren Marusiak

Warren is a Canadian developer from Vancouver, BC with over 10 years of experience. He came to Atlassian from AWS in January of 2021.

Share this article

Next Topic

Monitor with Opsgenie

Featured

Jira

Confluence

Jira Service Management

Trello

Rovo NEW

Jira Product Discovery NEW

Compass NEW

Guard NEW

Loom NEW

Developers

Jira

Bitbucket

Compass NEW

Product Managers

Jira

Confluence

Jira Product Discovery NEW

IT professionals

Jira Service Management

Guard NEW

Business Teams

Jira

Confluence

Trello

Loom NEW

Leadership Teams

Jira

Confluence

Loom NEW

Jira Align

Teams

Software

Marketing

IT

solution

By team size

By industry

Why Atlassian

Integrations

Customers

FedRAMP

Resilience

Platform

Trust center

Resources

Customer Support

Find Partners

Migration Program

University

support

learn

Articles

Tutorials

Interactive Guides

Deploy ImageLabeller with GitHub

Warren Marusiak

Prerequisites

Public facing GitHub repositories with ImageLabeller code

Jira GitHub integration demo video

Integrate Jira and GitHub

Install Jira plugin in GitHub

Setup SSH access

Setup a Personal Access Token

Create a repository for AWS S3 infrastructure

Add an AWS access key repository variables

GitHub actions for deploying to AWS

Understanding GitHub actions

Triggers

Jobs

Enforcing order with needs

Pushing to a feature branch

Creating a pull request

Create a repository for SubmitImage AWS Lambda

GitHub actions for deploying to AWS

Understanding GitHub actions

Pushing to a feature branch

Create a pull request

Create a repository for InvokeLabeller AWS Lambda

GitHub actions for deploying to AWS