Articles

To demonstrate how to develop, deploy, and manage applications using Jira Software and various connected tools, our team created ImageLabeller, a simple demo application built on AWS that uses machine learning to apply labels to images.

This page covers how to deploy ImageLabeller with Bitbucket. Before you begin, we recommend reading the ImageLabeller architecture and AWS SageMaker setup pages for context.

Prerequisites

If you don’t already have SSH configured for your Bitbucket account, follow these instructions.

Public facing GitHub repositories with ImageLabeller code

https://github.com/AtlassianOpenDevOpsGuides

Bitbucket pipelines demo video

Watch video

Create a repository for AWS S3 infrastructure

A standard developer loop typically has a developer picking up a task from Jira, moving it to work in progress, and then completing the development work. The Jira issue ID is the key which ties the development work to the Jira issue. It is the core integration component between the two systems.

From Jira, create a new issue for adding an AWS S3 infrastructure repository to Bitbucket. Take note of the issue ID. IM-5 in this example.

Screenshot showing Jira issue to add AWS S3 infrastructure repository to Bitbucket

Go to Bitbucket > Create > Repository.

Select the appropriate Workspace and Project. Set the default branch name to mainline. Click Create repository to proceed.

Pop-up window to create a new repository in Bitbucket

In your terminal, go to your s3_infra repository and run the following to push your AWS CloudFormation template.yml file to Bitbucket.

git add --all
git commit -m "IM-5 add s3_infra repository to Bitbucket"
git remote add origin git@bitbucket.org:pmmquickstartguides01/s3_infra.git
git branch -m mainline
git push -u origin mainline

Enable Bitbucket pipelines

Go to Repository settings > Settings > Enable Pipelines.

Add an AWS access key repository variables

Go to Repository variables, enter your AWS access key ID, and click Add. Then enter your AWS secret access key and click Add.

Set up deployment environments

Click Deployments.

Deployments settings screen in Bitbucket

Click add environment to add new environments. There is a Test environment in US-WEST-1, a Staging environment in US-EAST-2, and three Production environments in US-WEST-2, US-EAST-1, and CA-CENTRAL-1 in this example.

Adding deployment environments in Bitbucket

bitbucket-pipelines.yml for deploying to AWS

Go to your s3_infra repository in your terminal and create a branch named after your Jira issue ID.

git checkout -b IM-5

Create a bitbucket-pipelines.yml file with the following yaml. This defines a deployment workflow for your Test, Staging, and Production environments.

definitions:
  steps:
    - step: &deploy-test-usw1
        name: Deploy Test us-west-1
        script:
        - pipe: atlassian/aws-cloudformation-deploy:0.10.0
          variables:
            AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
            AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
            AWS_DEFAULT_REGION: 'us-west-1'
            STACK_NAME: 'OpenDevOpsS3Infra'
            CAPABILITIES: ['CAPABILITY_IAM', 'CAPABILITY_AUTO_EXPAND']
            WAIT: 'true'
            TEMPLATE: 'template.yml'
    - step: &deploy-staging-use2
        name: Deploy Staging us-east-2
        script:
        - pipe: atlassian/aws-cloudformation-deploy:0.10.0
          variables:
            AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
            AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
            AWS_DEFAULT_REGION: 'us-east-2'
            STACK_NAME: 'OpenDevOpsS3Infra'
            CAPABILITIES: ['CAPABILITY_IAM', 'CAPABILITY_AUTO_EXPAND']
            WAIT: 'true'
            TEMPLATE: 'template.yml'
    - step: &deploy-production-usw2
        name: Deploy Production us-west-2
        script:
        - pipe: atlassian/aws-cloudformation-deploy:0.10.0
          variables:
            AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
            AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
            AWS_DEFAULT_REGION: 'us-west-2'
            STACK_NAME: 'OpenDevOpsS3Infra'
            CAPABILITIES: ['CAPABILITY_IAM', 'CAPABILITY_AUTO_EXPAND']
            WAIT: 'true'
            TEMPLATE: 'template.yml'
    - step: &deploy-production-use1
        name: Deploy Production us-east-1
        script:
        - pipe: atlassian/aws-cloudformation-deploy:0.10.0
          variables:
            AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
            AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
            AWS_DEFAULT_REGION: 'us-east-1'
            STACK_NAME: 'OpenDevOpsS3Infra'
            CAPABILITIES: ['CAPABILITY_IAM', 'CAPABILITY_AUTO_EXPAND']
            WAIT: 'true'
            TEMPLATE: 'template.yml'
    - step: &deploy-production-cac1
        name: Deploy Production ca-central-1
        script:
        - pipe: atlassian/aws-cloudformation-deploy:0.10.0
          variables:
            AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
            AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
            AWS_DEFAULT_REGION: 'ca-central-1'
            STACK_NAME: 'OpenDevOpsS3Infra'
            CAPABILITIES: ['CAPABILITY_IAM', 'CAPABILITY_AUTO_EXPAND']
            WAIT: 'true'
            TEMPLATE: 'template.yml'
pipelines:
  default:
    - step:
        <<: *deploy-test-usw1
        deployment: Test us-west-1
    - step:
        <<: *deploy-staging-use2
        deployment: Staging us-east-2
  branches:
    mainline:
      - step:
          <<: *deploy-production-usw2
          deployment: Production us-west-2
      - step:
          <<: *deploy-production-use1
          deployment: Production us-east-1
      - step:
          <<: *deploy-production-cac1
          deployment: Production ca-central-1

Understanding a bitbucket-pipelines.yml file

Definitions and steps

Define a set of steps in the definitions section. Each step has an alias that is referenced throughout the bitbucket-pipelines.yml file, a name that shows up in the Bitbucket deployment screen, and a script. A script is a collection of one or more commands.

definitions:
  steps:
    - step: &deploy-test-usw1
        name: Deploy Test us-west-1
        script:
        - pipe: atlassian/aws-cloudformation-deploy:0.10.0
          variables:
            AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
            AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
            AWS_DEFAULT_REGION: 'us-west-1'
            STACK_NAME: 'OpenDevOpsS3Infra'
            CAPABILITIES: ['CAPABILITY_IAM', 'CAPABILITY_AUTO_EXPAND']
            WAIT: 'true'
            TEMPLATE: 'template.yml'

Pipes

This step uses the atlassian/aws-cloudformation-deploy pipe to deploy the AWS S3 buckets defined in the s3_infra template.yml file.

- pipe: atlassian/aws-cloudformation-deploy:0.10.0

The set of available pipes is found here. The documentation for the atlassian/aws-cloudformation-deploy pipe is found here.

Pipelines

Pipelines run a set of steps. The default pipeline is the set of steps that is run for branches that are not explicitly named under pipelines. This default pipeline runs steps that deploy to Test and Staging environments.

pipelines:
  default:
    - step:
        <<: *deploy-test-usw1
        deployment: Test us-west-1
    - step:
        <<: *deploy-staging-use2
        deployment: Staging us-east-2

Deployments enable integration between Jira and Bitbucket.

deployment: Test us-west-1

You can define a set of steps that run for specific, named branch under branches. The snippet below defines a set of steps for the mainline branch.

branches:
    mainline:
      - step:
          <<: *deploy-production-usw2
          deployment: Production us-west-2
      - step:
          <<: *deploy-production-use1
          deployment: Production us-east-1
      - step:
          <<: *deploy-production-cac1
          deployment: Production ca-central-1

Read this reference article for more information on this topic.

Pushing to a feature branch

Run the following from the command line to push your changes to the IM-5 branch of your s3_infra repository. Include the Jira issue ID in commit messages, and branch names to enable the Jira Bitbucket integration to keep track of what is happening in your project.

git add --all
git commit -m "IM-5 add bitbucket-pipelines.yml"
git push -u origin IM-5

Click Pipelines, then IM-5 to see the running pipeline.

Click the pipeline itself to see details of the execution. The pipeline ran steps toscreenshot_s deploy to a Test environment in us-west-1 and a Staging environment in us-east-2.

Create a pull request

To create a pull request click Pull requests, then Create pull request.

Choose your feature branch as the source branch, check the Close branch checkbox, then click Create pull request.

Creating a pull request modal in Bitbucket.

Review the code changes, then Approve and Merge the pull request.

Approve and merge pull request in Bitbucket

Clicking Merge opens the merge pull request screen. Check the Transition issue check box and click Merge.

Check "transition issue" checkbox when merging pull request in Bitbucket

Click Pipelines to monitor the mainline pipeline.

Monitoring the main pipeline in Bitbucket

The IM-5 branch is gone. The mainline branch is left, and a pipeline is running. Click the pipeline.

Mainline branch pipeline screen in Bitbucket

The pipeline ran steps to deploy to Production environments in us-west-2, us-east-1, and ca-central-1. It is possible to rerun a pipeline by clicking the Rerun button if a pipeline fails.

Rollback a bad deployment

To rollback a deployment click Deployments.

Click the environment you want to rollback to get a list of historical deployments, choose the version you want to rollback to, then click Redeploy.

Selecting an environment, then version you want to rollback to in Bitbucket

Verify the change is correct, and click Redeploy.

Verify your changes are correct before selecting "redeploy" in Bitbucket

Only the chosen environment is redeployed.

Pipeline modal in Bitbucket showing only the chosen environment being redeployed

Create a repository for SubmitImage AWS Lambda

Go to Jira and create a Jira issue for adding a SubmitImage repository to Bitbucket. Take note of the Jira issue ID. In this example it is IM-6.

Creating an issue in Jira Software for adding a SubmitImage repository to Bitbucket

Go to Bitbucket, and click Create, then Repository.

Select the appropriate Workspace, and Project. Set the Default branch name to mainline. Click Create repository to proceed.

Pop-up modal when creating a new repository in Bitbucket

In your terminal go to your SubmitImage repository, and run the following to push your AWS Lambda code to Bitbucket. Include the Jira issue ID in commit messages, and branch names to enable the Jira Bitbucket integration to keep track of what is happening in your project.

git add --all
git commit -m "IM-6 add SubmitImage to Bitbucket"
git remote add origin git@bitbucket.org:pmmquickstartguides01/submitimage.git
git branch -m mainline
git push -u origin mainline

Enable pipelines

Go to Repository settings, click Settings, then Enable pipelines.

Add repository variables

Click Repository variables to add your AWS access key ID, AWS secret access key, and AWS account ID. Give the IAM user associated with the AWS access key AdministratorAccess. You can opt to use more fine grained access control by choosing individual AWS access policies.

Add deployment environments

Click Deployments, then add environment to add new environments. There is a Test environment in us-west-1, a Staging environment in us-east-2, and three Production environments in us-west-2, us-east-1 and ca-central-1 in this example.

Create SSH keys

Go to your SubmitImage repository in Bitbucket, and click Repository settings, then SSH keys, then Generate keys.

You will need this SSH key when you create the SystemTests repository.

bitbucket-pipelines.yml for deploying to AWS

Go to your SubmitImage repository in your terminal, and create a branch named after your Jira issue ID.

git checkout -b IM-6

Create a bitbucket-pipelines.yml file with the following yaml. This defines a Bitbucket pipeline for your Test, Staging, and Production environments. You must update the git clone line for SystemTests to be your SystemTests repository.

definitions:
  steps:
    -step: &rununittests
        name: run unit tests
        image: golang:buster
        script:
          - cd submitImage
          - go test ./opendevopslambda/...
    -step: &deploy-test-usw1
        name: Deploy Test us-west-1
        image: amazon/aws-sam-cli-build-image-provided
        script:
          - curl https://dl.google.com/go/go1.16.3.linux-amd64.tar.gz -o go1.16.3.tar.gz
          - rm -rf /usr/local/go
          - tar -C /usr/local -xzf go1.16.3.tar.gz
          - export PATH=$PATH:/usr/local/go/bin
          - go version
          - make
          - ls -lah /opt/atlassian/pipelines/agent/build/build/
          - pipe: atlassian/aws-sam-deploy:1.2.0
            variables:
              AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
              AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
              AWS_DEFAULT_REGION: 'us-west-1'
              STACK_NAME: 'OpenDevOpsSubmitImage'
              CAPABILITIES: [ 'CAPABILITY_IAM', 'CAPABILITY_NAMED_IAM', 'CAPABILITY_AUTO_EXPAND' ]
              TEMPLATE: 'https://s3.amazonaws.com/open-devops-code-us-west-1-${AWS_ACCOUNT_ID}/submit-image-packaged.yml'
              WAIT: 'true'
              DEBUG: 'true'
              S3_BUCKET: 'open-devops-code-us-west-1-${AWS_ACCOUNT_ID}'
              SAM_TEMPLATE: 'build/template.yaml'
    -step: &integration-test-usw1
      name: Integration test usw1
      image: golang:buster
      script:
        - git clone git@bitbucket.org:pmmquickstartguides01/systemtests.git
        - cd systemtests
        - go test -v ./... -aws_region=us-west-1
    -step: &deploy-staging-use2
        name: Deploy Staging us-east-2
        image: amazon/aws-sam-cli-build-image-provided
        script:
          - curl https://dl.google.com/go/go1.16.3.linux-amd64.tar.gz -o go1.16.3.tar.gz
          - rm -rf /usr/local/go
          - tar -C /usr/local -xzf go1.16.3.tar.gz
          - export PATH=$PATH:/usr/local/go/bin
          - go version
          - make
          - ls -lah /opt/atlassian/pipelines/agent/build/build/
          - BITBUCKET_PIPE_SHARED_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes"
          - chmod -R o+rw $BITBUCKET_PIPE_SHARED_STORAGE_DIR
          - pipe: atlassian/aws-sam-deploy:1.2.0
            variables:
              AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
              AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
              AWS_DEFAULT_REGION: 'us-east-2'
              STACK_NAME: 'OpenDevOpsSubmitImage'
              CAPABILITIES: [ 'CAPABILITY_IAM', 'CAPABILITY_NAMED_IAM', 'CAPABILITY_AUTO_EXPAND' ]
              TEMPLATE: 'https://s3.amazonaws.com/open-devops-code-us-east-2-${AWS_ACCOUNT_ID}/submit-image-packaged.yml'
              WAIT: 'true'
              DEBUG: 'true'
              S3_BUCKET: 'open-devops-code-us-east-2-${AWS_ACCOUNT_ID}'
              SAM_TEMPLATE: 'build/template.yaml'
    -step: &integration-test-use2
      name: Integration test use2
      image: golang:buster
      script:
        - git clone git@bitbucket.org:pmmquickstartguides01/systemtests.git
        - cd systemtests
        - go test -v ./... -aws_region=us-east-2
    -step: &deploy-production-usw2
        name: Deploy Production us-west-2
        image: amazon/aws-sam-cli-build-image-provided
        script:
          - curl https://dl.google.com/go/go1.16.3.linux-amd64.tar.gz -o go1.16.3.tar.gz
          - rm -rf /usr/local/go
          - tar -C /usr/local -xzf go1.16.3.tar.gz
          - export PATH=$PATH:/usr/local/go/bin
          - go version
          - make
          - ls -lah /opt/atlassian/pipelines/agent/build/build/
          - pipe: atlassian/aws-sam-deploy:1.2.0
            variables:
              AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
              AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
              AWS_DEFAULT_REGION: 'us-west-2'
              STACK_NAME: 'OpenDevOpsSubmitImage'
              CAPABILITIES: [ 'CAPABILITY_IAM', 'CAPABILITY_NAMED_IAM', 'CAPABILITY_AUTO_EXPAND' ]
              TEMPLATE: 'https://s3.amazonaws.com/open-devops-code-us-west-2-${AWS_ACCOUNT_ID}/submit-image-packaged.yml'
              WAIT: 'true'
              DEBUG: 'true'
              S3_BUCKET: 'open-devops-code-us-west-2-${AWS_ACCOUNT_ID}'
              SAM_TEMPLATE: 'build/template.yaml'
    -step: &integration-test-usw2
      name: Integration test usw2
      image: golang:buster
      script:
        - git clone git@bitbucket.org:pmmquickstartguides01/systemtests.git
        - cd systemtests
        - go test -v ./... -aws_region=us-west-2
    -step: &deploy-production-use1
        name: Deploy Production us-east-1
        image: amazon/aws-sam-cli-build-image-provided
        script:
          - curl https://dl.google.com/go/go1.16.3.linux-amd64.tar.gz -o go1.16.3.tar.gz
          - rm -rf /usr/local/go
          - tar -C /usr/local -xzf go1.16.3.tar.gz
          - export PATH=$PATH:/usr/local/go/bin
          - go version
          - make
          - ls -lah /opt/atlassian/pipelines/agent/build/build/
          - BITBUCKET_PIPE_SHARED_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes"
          - chmod -R o+rw $BITBUCKET_PIPE_SHARED_STORAGE_DIR
          - pipe: atlassian/aws-sam-deploy:1.2.0
            variables:
              AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
              AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
              AWS_DEFAULT_REGION: 'us-east-1'
              STACK_NAME: 'OpenDevOpsSubmitImage'
              CAPABILITIES: [ 'CAPABILITY_IAM', 'CAPABILITY_NAMED_IAM', 'CAPABILITY_AUTO_EXPAND' ]
              TEMPLATE: 'https://s3.amazonaws.com/open-devops-code-us-east-1-${AWS_ACCOUNT_ID}/submit-image-packaged.yml'
              WAIT: 'true'
              DEBUG: 'true'
              S3_BUCKET: 'open-devops-code-us-east-1-${AWS_ACCOUNT_ID}'
              SAM_TEMPLATE: 'build/template.yaml'
    -step: &integration-test-use1
      name: Integration test use1
      image: golang:buster
      script:
        - git clone git@bitbucket.org:pmmquickstartguides01/systemtests.git
        - cd systemtests
        - go test -v ./... -aws_region=us-east-1
    -step: &deploy-production-cac1
        name: Deploy Production ca-central-1
        image: amazon/aws-sam-cli-build-image-provided
        script:
          - curl https://dl.google.com/go/go1.16.3.linux-amd64.tar.gz -o go1.16.3.tar.gz
          - rm -rf /usr/local/go
          - tar -C /usr/local -xzf go1.16.3.tar.gz
          - export PATH=$PATH:/usr/local/go/bin
          - go version
          - make
          - ls -lah /opt/atlassian/pipelines/agent/build/build/
          - BITBUCKET_PIPE_SHARED_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes"
          - chmod -R o+rw $BITBUCKET_PIPE_SHARED_STORAGE_DIR
          - pipe: atlassian/aws-sam-deploy:1.2.0
            variables:
              AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
              AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
              AWS_DEFAULT_REGION: 'ca-central-1'
              STACK_NAME: 'OpenDevOpsSubmitImage'
              CAPABILITIES: [ 'CAPABILITY_IAM', 'CAPABILITY_NAMED_IAM', 'CAPABILITY_AUTO_EXPAND' ]
              TEMPLATE: 'https://s3.amazonaws.com/open-devops-code-ca-central-1-${AWS_ACCOUNT_ID}/submit-image-packaged.yml'
              WAIT: 'true'
              DEBUG: 'true'
              S3_BUCKET: 'open-devops-code-ca-central-1-${AWS_ACCOUNT_ID}'
              SAM_TEMPLATE: 'build/template.yaml'
    -step: &integration-test-cac1
      name: Integration test cac1
      image: golang:buster
      script:
        - git clone git@bitbucket.org:pmmquickstartguides01/systemtests.git
        - cd systemtests
        - go test -v ./... -aws_region=ca-central-1
pipelines:
  default:
    - step: *rununittests
    - step:
        <<: *deploy-test-usw1
        deployment: Test us-west-1
#    - step: *integration-test-usw1
    - step:
        <<: *deploy-staging-use2
        deployment: Staging us-east-2
#    - step: *integration-test-use2
  branches:
    mainline:
      - step:
          <<: *deploy-production-usw2
          deployment: Production us-west-2
#      - step: *integration-test-usw2
      - step:
          <<: *deploy-production-use1
          deployment: Production us-east-1
#      - step: *integration-test-use1
      - step:
          <<: *deploy-production-cac1
          deployment: Production ca-central-1
#      - step: *integration-test-cac1

The execution of the integration tests is commented out for now. The system tests will only pass when the entire application is deployed. Uncomment the integration test steps in your repository, and do another push to run the deployment pipeline after all components of ImageLabeller are deployed. You must update the git clone line for SystemTests to be your SystemTests repository.

Understanding a bitbucket-pipelines.yml file

This step runs unit tests from the SubmitImage code base.

-step: &rununittests
        name: run unit tests
        image: golang:buster
        script:
          - cd submitImage
          - go test ./opendevopslambda/...

This step uses AWS SAM to deploy your SubmitImage AWS Lambda to us-west-2. The documentation for this pipe can be found here.

-step: &deploy-usw2-prod
    name: deploy us-west-2 prod
    image: amazon/aws-sam-cli-build-image-provided
    script:
      - curl https://dl.google.com/go/go1.16.3.linux-amd64.tar.gz -o go1.16.3.tar.gz
      - rm -rf /usr/local/go
      - tar -C /usr/local -xzf go1.16.3.tar.gz
      - export PATH=$PATH:/usr/local/go/bin
      - go version
      - make
      - pipe: atlassian/aws-sam-deploy:1.2.0
        variables:
          AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
          AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
          AWS_DEFAULT_REGION: 'us-west-2'
          STACK_NAME: 'OpenDevOpsSubmitImage'
          CAPABILITIES: [ 'CAPABILITY_IAM', 'CAPABILITY_NAMED_IAM', 'CAPABILITY_AUTO_EXPAND' ]
          TEMPLATE: 'https://s3.amazonaws.com/open-devops-code-us-west-2-${AWS_ACCOUNT_ID}/submit-image-packaged.yml'
          WAIT: 'true'
          DEBUG: 'true'
          S3_BUCKET: 'open-devops-code-us-west-2-${AWS_ACCOUNT_ID}'
          SAM_TEMPLATE: 'build/template.yaml'

This step clones the SystemTests repository, and runs integrations tests in us-west-2. You must update the git clone line for SystemTests to be your SystemTests repository.

-step: &integration-test-usw2
      name: Integration test usw2
      image: golang:buster
      script:
        - git clone git@bitbucket.org:pmmquickstartguides01/systemtests.git
        - cd systemtests
        - go test -v ./... -aws_region=us-west-2

Pushing to a feature branch

From the command line run the following to push your changes to the IM-8 branch of your SubmitImage repository.

git add --all
git commit -m "IM-6 add bitbucket-pipelines.yml to SubmitImage"
git push -u origin IM-6

Click Pipelines, then IM-6 to see the running pipeline.

Create a pull request

To create a pull request click Pull requests, then Create pull request. Finish the pull request, and click Pipelines to see the Production deployment.

Production deployment pipeline in Bitbucket cloud

Create a repository for InvokeLabeller AWS Lambda

Go to Jira and create a Jira issue for adding an InvokeLabeller repository to Bitbucket. Take note of the Jira issue ID. In this example it is IM-10.

Screenshot displaying Jira issues on a Jira board

Go to Bitbucket, and click Create, then Repository. Select the appropriate Workspace, and Project. Set the Default branch name to mainline. Click Create repository to proceed.

Create invokelabeller repository in Bitbucket

In your terminal go to your InvokeLabeller repository, and run the following to push your AWS Lambda code to Bitbucket. Include the Jira issue ID in commit messages, and branch names to enable the Jira Bitbucket integration to keep track of what is happening in your project.

git add --all
git commit -m "IM-10 add InvokeLabeller to Bitbucket"
git remote add origin git@bitbucket.org:pmmquickstartguides01/invokelabeller.git
git branch -m mainline
git push -u origin mainline

Enable pipelines

Go to Repository settings, click Settings, then Enable pipelines.

Add repository variables

Add deployment environments

Create SSH keys

Go to your SubmitImage repository in Bitbucket, and click Repository settings, then SSH keys, then Generate keys.

bitbucket-pipelines.yml for deploying to AWS

Go to your InvokeLabeller repository in your terminal, and create a branch named after your Jira issue ID.

git checkout -b IM-10

definitions:
  steps:
    -step: &rununittests
        name: run unit tests
        image: python:rc-buster
        script:
          - pip3 install pytest
          - pip3 install moto
          - pip3 install -r tst/requirements.txt --user
          - python3 -m pytest -v tst/unit --junitxml=test-reports/report.xml
    -step: &deploy-usw1-test
        name: deploy us-west-1 test
        image: amazon/aws-sam-cli-build-image-python3.8
        script:
          - pipe: atlassian/aws-sam-deploy:1.2.0
            variables:
              AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
              AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
              AWS_DEFAULT_REGION: 'us-west-1'
              STACK_NAME: 'OpenDevOpsImageLabeller'
              CAPABILITIES: [ 'CAPABILITY_IAM', 'CAPABILITY_NAMED_IAM', 'CAPABILITY_AUTO_EXPAND' ]
              TEMPLATE: 'https://s3.amazonaws.com/open-devops-code-us-west-1-${AWS_ACCOUNT_ID}/image-labeller-packaged.yml'
              WAIT: 'true'
              DEBUG: 'true'
              S3_BUCKET: 'open-devops-code-us-west-1-${AWS_ACCOUNT_ID}'
              SAM_TEMPLATE: 'template.yml'
    -step: &integration-test-usw1
      name: integration test usw1
      image: golang:buster
      script:
        - git clone git@bitbucket.org:pmmquickstartguides01/systemtests.git
        - cd systemtests
        - go test -v ./... -aws_region=us-west-1
    -step: &deploy-use2-staging
        name: deploy us-east-2 staging
        image: amazon/aws-sam-cli-build-image-python3.8
        script:
          - BITBUCKET_PIPE_SHARED_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes"
          - chmod -R o+rw $BITBUCKET_PIPE_SHARED_STORAGE_DIR
          - pipe: atlassian/aws-sam-deploy:1.2.0
            variables:
              AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
              AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
              AWS_DEFAULT_REGION: 'us-east-2'
              STACK_NAME: 'OpenDevOpsImageLabeller'
              CAPABILITIES: [ 'CAPABILITY_IAM', 'CAPABILITY_NAMED_IAM', 'CAPABILITY_AUTO_EXPAND' ]
              TEMPLATE: 'https://s3.amazonaws.com/open-devops-code-us-east-2-${AWS_ACCOUNT_ID}/image-labeller-packaged.yml'
              WAIT: 'true'
              DEBUG: 'true'
              S3_BUCKET: 'open-devops-code-us-east-2-${AWS_ACCOUNT_ID}'
              SAM_TEMPLATE: 'template.yml'
    -step: &integration-test-use2
      name: integration test use2
      image: golang:buster
      script:
        - git clone git@bitbucket.org:pmmquickstartguides01/systemtests.git
        - cd systemtests
        - go test -v ./... -aws_region=us-east-2
    -step: &deploy-usw2-prod
        name: deploy us-west-2 prod
        image: amazon/aws-sam-cli-build-image-python3.8
        script:
          - pipe: atlassian/aws-sam-deploy:1.2.0
            variables:
              AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
              AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
              AWS_DEFAULT_REGION: 'us-west-2'
              STACK_NAME: 'OpenDevOpsImageLabeller'
              CAPABILITIES: [ 'CAPABILITY_IAM', 'CAPABILITY_NAMED_IAM', 'CAPABILITY_AUTO_EXPAND' ]
              TEMPLATE: 'https://s3.amazonaws.com/open-devops-code-us-west-2-${AWS_ACCOUNT_ID}/image-labeller-packaged.yml'
              WAIT: 'true'
              DEBUG: 'true'
              S3_BUCKET: 'open-devops-code-us-west-2-${AWS_ACCOUNT_ID}'
              SAM_TEMPLATE: 'template.yml'
    -step: &integration-test-usw2
      name: integration test usw2
      image: golang:buster
      script:
        - git clone git@bitbucket.org:pmmquickstartguides01/systemtests.git
        - cd systemtests
        - go test -v ./... -aws_region=us-west-2
    -step: &deploy-use1-prod
        name: deploy us-east-1 prod
        image: amazon/aws-sam-cli-build-image-python3.8
        script:
          - BITBUCKET_PIPE_SHARED_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes"
          - chmod -R o+rw $BITBUCKET_PIPE_SHARED_STORAGE_DIR
          - pipe: atlassian/aws-sam-deploy:1.2.0
            variables:
              AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
              AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
              AWS_DEFAULT_REGION: 'us-east-1'
              STACK_NAME: 'OpenDevOpsImageLabeller'
              CAPABILITIES: [ 'CAPABILITY_IAM', 'CAPABILITY_NAMED_IAM', 'CAPABILITY_AUTO_EXPAND' ]
              TEMPLATE: 'https://s3.amazonaws.com/open-devops-code-us-east-1-${AWS_ACCOUNT_ID}/image-labeller-packaged.yml'
              WAIT: 'true'
              DEBUG: 'true'
              S3_BUCKET: 'open-devops-code-us-east-1-${AWS_ACCOUNT_ID}'
              SAM_TEMPLATE: 'template.yml'
    -step: &integration-test-use1
      name: integration test use1
      image: golang:buster
      script:
        - git clone git@bitbucket.org:pmmquickstartguides01/systemtests.git
        - cd systemtests
        - go test -v ./... -aws_region=us-east-1
    -step: &deploy-cac1-prod
        name: deply ca-central-1 prod
        image: amazon/aws-sam-cli-build-image-python3.8
        script:
          - BITBUCKET_PIPE_SHARED_STORAGE_DIR="/opt/atlassian/pipelines/agent/build/.bitbucket/pipelines/generated/pipeline/pipes"
          - chmod -R o+rw $BITBUCKET_PIPE_SHARED_STORAGE_DIR
          - pipe: atlassian/aws-sam-deploy:1.2.0
            variables:
              AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
              AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
              AWS_DEFAULT_REGION: 'ca-central-1'
              STACK_NAME: 'OpenDevOpsImageLabeller'
              CAPABILITIES: [ 'CAPABILITY_IAM', 'CAPABILITY_NAMED_IAM', 'CAPABILITY_AUTO_EXPAND' ]
              TEMPLATE: 'https://s3.amazonaws.com/open-devops-code-ca-central-1-${AWS_ACCOUNT_ID}/image-labeller-packaged.yml'
              WAIT: 'true'
              DEBUG: 'true'
              S3_BUCKET: 'open-devops-code-ca-central-1-${AWS_ACCOUNT_ID}'
              SAM_TEMPLATE: 'template.yml'
    -step: &integration-test-cac1
      name: integration test cac1
      image: golang:buster
      script:
        - git clone git@bitbucket.org:pmmquickstartguides01/systemtests.git
        - cd systemtests
        - go test -v ./... -aws_region=ca-central-1
pipelines:
  default:
    - step: *rununittests
    - step:
        <<: *deploy-usw1-test
        deployment: us-west-1 Test
#    - step: *integration-test-usw1
    - step:
        <<: *deploy-use2-staging
        deployment: us-east-2 Staging
#    - step: *integration-test-use2
  branches:
    mainline:
      - step:
          <<: *deploy-usw2-prod
          deployment: us-west-2 Prod
#      - step: *integration-test-usw2
      - step:
          <<: *deploy-use1-prod
          deployment: us-east-1 Prod
#      - step: *integration-test-use1
      - step:
          <<: *deploy-cac1-prod
          deployment: ca-central-1 Prod
#      - step: *integration-test-cac1

Understanding a bitbucket-pipelines.yml file

This step runs unit tests from the InvokeLabeller code base.

-step: &rununittests
        name: run unit tests
        image: golang:buster
        script:
          - cd submitImage
          - go test ./opendevopslambda/...

Update src/app.py with AWS SageMaker endpoint

Open InvokeLabeller’s src/app.py file and look for query_endpoint. Change the endpoint_name, and client region_name to match your AWS SageMaker notebook.

def query_endpoint(img):
  endpoint_name = 'jumpstart-dft-image-labeller-endpoint'
  client = boto3.client(service_name='runtime.sagemaker', region_name='us-west-1')
  response = client.invoke_endpoint(EndpointName=endpoint_name, ContentType='application/x-image', Body=img)
  model_predictions = json.loads(response['Body'].read())['predictions'][0]
  return model_predictions

Pushing to a feature branch

From the command line run the following to push yourchanges to the IM-10 branch of your InvokeLabeller repository.

git add --all
git commit -m "IM-10 add bitbucket-pipelines.yml to InvokeLabeller"
git push -u origin IM-10

Click Pipelines, then IM-10 to see the running pipeline.

See running pipelines in Bitbucket cloud

Create a pull request

To create a pull request click Pull requests, then Create pull request. Finish the pull request, and click Pipelines to see the Production deployment.

Viewing production deployment in Bitbucket pipelines

Create a repository for System Tests

Go to Jira, and create a new issue for adding a SystemTests repository to Bitbucket. Make note of the issue ID. IM-7 in this example.

Viewing a newly created Jira issue on Jira board

Go to Bitbucket, and click Create, then Repository.

Select the appropriate Workspace, and Project. Set the Default branch name to mainline. Click Create repository to proceed.

Entering information when creating a new repository in Bitbucket

In your terminal go to your SystemTests repository, and run the following to push your code to Bitbucket.

git add --all
git commit -m "IM-7 add SystemTests repository to Bitbucket"
git remote add origin git@bitbucket.org:pmmquickstartguides01/systemtests.git
git branch -M mainline
git push -u origin mainline

The systemTests repository doesn’t need a bitbucket-pipelines.yml file. It has no pipeline of its own since it provides tests for other pipelines to run. Take note of your SystemTests' remote url. SubmitImage, GetImageLabel, and InvokeLabeller CI/CD pipelines will clone the SystemTests repository during testing steps. You will need to update the bitbucket-pipelines.yml of later repositories with the correct url.

Add SSH keys from SubmitImage, GetImageLabel, and InvokeLabeller

Click Repository settings, then Access keys.

Click Add Key, paste the SSH key copied from SubmitImage, GetImageLabel, or InvokeLabeller, then click Add SSH key.

If you’ve made this far, congratulations! You just deployed ImageLabeller. The next step is to set up monitoring ImageLabeller with Opsgenie.

Warren Marusiak

Warren is a Canadian developer from Vancouver, BC with over 10 years of experience. He came to Atlassian from AWS in January of 2021.

Share this article

Next Topic

Monitor with Opsgenie

Featured

Jira

Confluence

Jira Service Management

Trello

Rovo NEW

Jira Product Discovery NEW

Compass NEW

Guard NEW

Loom NEW

Developers

Jira

Bitbucket

Compass NEW

Product Managers

Jira

Confluence

Jira Product Discovery NEW

IT professionals

Jira Service Management

Guard NEW

Business Teams

Jira

Confluence

Trello

Loom NEW

Leadership Teams

Jira

Confluence

Loom NEW

Jira Align

Teams

Software

Marketing

IT

solution

By team size

By industry

Why Atlassian

Integrations

Customers

FedRAMP

Resilience

Platform

Trust center

Resources

Customer Support

Find Partners

Migration Program

University

support

learn

Articles

Tutorials

Interactive Guides

Deploy ImageLabeller with Bitbucket

Warren Marusiak

Prerequisites

Public facing GitHub repositories with ImageLabeller code

Bitbucket pipelines demo video

Create a repository for AWS S3 infrastructure

Enable Bitbucket pipelines

Add an AWS access key repository variables

Set up deployment environments

bitbucket-pipelines.yml for deploying to AWS

Understanding a bitbucket-pipelines.yml file

Definitions and steps

Pipes

Pipelines

Pushing to a feature branch

Create a pull request

Rollback a bad deployment

Create a repository for SubmitImage AWS Lambda

Enable pipelines

Add repository variables

Add deployment environments

Create SSH keys

bitbucket-pipelines.yml for deploying to AWS

Understanding a bitbucket-pipelines.yml file

Pushing to a feature branch