Using Serverless Secrets in Jenkinsfile Pipelines

This blog is going to explain how to use secrets as environment variables in Serverless applications, store those credentials in Jenkins, and have a Jenkinsfile pipeline deploy a Serverless application using Docker that references those newly defined variables.

The example application used to explain these concepts will be the serverless-slack-app1 by John Agan on GitHub. This blog assumes introductory level knowledge of Git, Docker, Serverless, Jenkins, and Jenkinsfile pipelines.

To summarize, here are the goals of this tutorial:

  1. Create a basic Jenkinsfile with Docker agents that deploys a Serverless application.
  2. Convert custom variables in serverless.yml to reference environment variables.
  3. Store secrets in Jenkins for the pipeline to use.

1. Serverless Jenkinsfile using Docker

Example Application

If you want to DIY as opposed to just reading, feel free to start with the example application. Keep in mind you’ll need a Jenkins instance with Docker available and an Amazon Web Services account that may incur charges for deploying the Serverless application. If you want to spin up a Jenkins container that has Docker and Jenkinsfile pipelines already configured, feel free to use Liatrio’s Alpine-Jenkins image.

$ npm install -g serverless
$ serverless install --url https://github.com/johnagan/serverless-slack-app
$ cd serverless-slack-app/

Agent

For this Serverless application, we’re just going to use one Docker image as the agent. Since we want to keep it lightweight, we’re going with the Alpine version of Node which comes in at roughly 71.1MB. Of course, if you aren’t using Docker with Jenkins you can refer to the example on the Jenkins CI GitHub wiki that doesn’t use Docker.

We’re also going to need to be running as root in the container to properly run Serverless commands, so we’re going to specify to run as root in the arguments.

Since we’re only using one image throughout the whole pipeline, we can define it just once in the beginning of the pipeline.

  agent {
    docker {
      image 'node:alpine'
      args '-u 0:0'
    }
  }

Environment

For the environment of the pipeline, we’re going to reference Amazon Web Services credentials from Jenkins. For details on how to add credentials to Jenkins, you can refer to the Adding Docker Hub Credentials to Jenkins section from Building with Docker in Jenkins Pipelines. You can also refer to the official documentation.

environment {
  AWS_ACCESS_KEY_ID = credentials('aws-secret-key-id')
  AWS_SECRET_ACCESS_KEY = credentials('aws-secret-access-key')
}

These environment variables will be referenced automatically when the Serverless application is deployed. However, they will only be referenced automatically if a profile is not defined in serverless.yml. So, go ahead and delete this line in the example application’s serverless.yml file.

profile: serverless

With this line removed, the Serverless deployment will search for the default AWS profile which is what we have defined in the Jenkinsfile.

Simple Serverless Jenkinsfile using Docker

To keep things simple, we’re only going to define a Build stage and a Deploy stage for the Jenkinsfile. However, a Test stage could easily be added that ran an npm test command. Here’s the resulting Jenkinsfile that will successfully build and deploy our example application.

pipeline {
  agent {
    docker {
      image 'node:alpine'
      args '-u 0:0'
    }
  }
  environment {
    AWS_ACCESS_KEY_ID = credentials('aws-secret-key-id')
    AWS_SECRET_ACCESS_KEY = credentials('aws-secret-access-key')
  }
  stages {
    stage('Build') {
      steps {
        sh 'npm install'
      }
    }
    stage('Deploy') {
      steps {
        sh 'npm install -g serverless'
        sh 'serverless deploy'
      }
    }
  }
}

Great. This deploys our Serverless application! However, this only works if the Slack secrets are defined as text in the serverless.yml file. Let’s fix that.

2. Set Custom Serverless Variables

Reference Environment Variables in Serverless Configuration

As it stands, our example application has required Slack variables defined as custom variables in serverless.yml. We need to change that so the variables are obtained from the environment. Luckily, the Serverless framework has a way to do that outlined in their official docs.

Before, we had something like this where the x’s represented secrets.

  slack_verification_token: "xxxxxxxxxxxxx"
  ... 
  slack_client_id: "xxxxxxxxxx.xxxxxxxxxxxx" 
  slack_client_secret: "xxxxxxxxxxxxxxxxxx"

We can convert these into environment variable references.

 slack_verification_token: ${env:SLACK_VERIFICATION_TOKEN}
 ...
 slack_client_id: ${env:SLACK_CLIENT_ID}
 slack_client_secret: ${env:SLACK_CLIENT_SECRET}

Great! Now our secrets are never defined in code.

Add Credentials to Jenkins

We need to add the SLACK_VERIFICATION_TOKEN, SLACK_CLIENT_ID, and SLACK_CLIENT_SECRET variables as secret text variables in Jenkins so that we can reference them in our Jenkinsfile. If you’re unsure how to do this, refer to the Environment section above.

3. Put it all Together

Now, we need to add the newly defined credentials into our Jenkinsfile pipeline environment. We can do this the same way we defined our AWS credentials.

environment {
  SLACK_VERIFICATION_TOKEN = credentials('slack-verification-token')
  SLACK_CLIENT_ID = credentials('slack-client-id')
  SLACK_CLIENT_SECRET = credentials('slack-client-secret')
  AWS_ACCESS_KEY_ID = credentials('aws-secret-key-id')
  AWS_SECRET_ACCESS_KEY = credentials('aws-secret-access-key')
 }

End Result

We’ve successfully created a Jenkinsfile that automatically deploys a Serverless application using custom secrets stored in Jenkins. If you want to see all of the code implemented in this tutorial, feel free to check out the repository on GitHub. You can view the complete serverless.yml and Jenkinsfile there.

If you run into any issues, you can leave a comment below or tweet me @shanemacbride or @liatrio.

Liatrio is a DevOps Consulting firm focussing on helping enterprises get better at software delivery using DevOps & Rapid Release philosophies. We work as “boots on the ground change agents” helping our clients re-imagine their daily work and get better at delivery one day at a time. Liatrio is also hiring! If you enjoy being a part of a team that is solving challenges around software delivery automation, deployment pipelines, and large-scale transformations, reach out to us via our contact page on our website.

1LICENSE

Leave a Reply

Your email address will not be published. Required fields are marked *