How to automate Inspec tests on AWS

Chef Inspec is an open source testing framework designed for automated compliance and security checks on infrastructure. Combining it with State Manager on AWS you can ensure EC2 instances in your account are in a state you define. In this blog post I will explain how I automated this setup with cloudformation.

Solution Architecture

Key concepts

SSM Documents

AWS Systems Manager documents are JSON or YAML documents that define actions that get executed in managed instances. They can be custom or AWS managed. There are a number of pre-configured documents you can choose out of the shelf including one for Chef Inspec. Since AWS-RunInspecCheck document (AWS managed document) installs chef-cdk on each execution and uninstalls it after, I had to write a custom SSM document to run Inspec profile.

An SSM document has mainly a schemaVersion, parameters and mainSteps. Each action defined under mainSteps form individual steps. In the below example parameters are sourceType and sourceInfo. Source type can be either s3 or Github. Depending on the source type source info parameter need varying information. aws:downloadContent and aws:runShellScript are the two actions.

aws:downloadContent downloads Inspec profile from s3 bucket defined in parameter sourceInfo to SSM directory in a managed instance.

aws:runShellScript computes region from ec2 metadata. Further downloads compliance file from an AWS defined s3 bucket from the computed region. Finally it runs Inspec exec command executing Inspec profile and return results to Systems Manager.

 schemaVersion: '2.2'
 description: Run a single InSpec test or an InSpec profile on a group of managed instances.
 parameters:
          sourceType:
            description: "(Required) Specify the source type."
            type: String
            allowedValues:
            - GitHub
            - S3
          sourceInfo:
            description: '(Required) Specify the information required to access the resource
              from the source. If source type is GitHub, then you can specify any of the following:
              ''owner'', ''repository'', ''path'', ''getOptions'', ''tokenInfo''. If source
              type is S3, then you can specify ''path''. Example github parameters: {"owner":"awslabs","repository":"amazon-ssm","path":"Compliance/InSpec/PortCheck","getOptions":"branch:master"}'
            type: StringMap
            displayType: textarea
            default: {}
        mainSteps:
        - action: aws:downloadContent
          name: downloadContent
          inputs:
            sourceType: "{{ sourceType }}"
            sourceInfo: "{{ sourceInfo }}"
        - precondition:
            StringEquals:
            - platformType
            - Linux
          action: aws:runShellScript
          name: runInSpecLinux
          inputs:
            runCommand:
              - "#!/bin/bash"
              - "export HOME=/root"
              - "export CHEF_LICENSE=accept-no-persist"
              - "if ! which curl &> /dev/null; then"
              - "echo 'curl is missing from the instance! Exiting.'"
              - "exit 1"
              - "fi"
              - 'TOKEN=`curl -sS -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`'
              - 'region=`curl -sS -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/placement/availability-zone | sed ''s/\(.*\)[a-z]/\1/''`'
              - "complianceFile='Report-Compliance-20200225'"
              - "s3Prefix='https://s3.dualstack.'"
              - "s3Suffix=''"
              - curl -sS "$s3Prefix$region.amazonaws.com$s3Suffix/aws-ssm-$region/statemanagerdocumentspayload/AWS-RunInspecChecks/$complianceFile" -o "$complianceFile"
              - "if [ $? -ne 0 ] ; then"
              - "echo 'Failed to download inspec compliance file from S3. Exiting.'"
              - "exit 1"
              - "fi"
              - "inspec exec . --reporter json | ruby ./Report-Compliance-20200225"
              - "if [ $? -ne 0 ] ; then"
              - "echo 'Failed to run Inspec checks. Exiting.'"
              - "exit 1"
              - "fi"

SSM Associations

An association maps instances to a document. In the given cloudformation example for a SSM association, document execution is scheduled. It also specifies the target instances by defining tags.

  InspecCheckLinuxAssociation:
    Type: AWS::SSM::Association
    Properties:
      Name: !Ref InspecCheckDocumentLinux
      AssociationName: "inspecCheckAgentsLinux"
      Parameters:
        sourceType: ["S3"]
        sourceInfo: [!Sub '{"path": "${InspecProfilePath}"}']
      ScheduleExpression: "cron(0 23 * * ? *)"
      Targets:
      - Key: tag:OS
        Values: 
        - Linux

Prerequisites

• EC2 instances should be SSM enabled.
• Chef-cdk and aws-sdk-ssm gem are installed.

Implementation

The stack creates a custom SSM document, an SSM Association and an S3 bucket to store Inspec profile. A sample Inspec profile from awslabs can be a reference to create your own profile. https://github.com/awslabs/aws-systems-manager/blob/master/Compliance/InSpec/PortCheck/

Testing

1. copy its contents to the s3 bucket created. Make sure that the path specified in sourceInfo parameter matches the path where Inspec profile is located. A valid Inspec profile has a inspec.yaml file and controls folder with test files.
2. Apply SSM association from AWS console for a quick test.
3. Navigate to AWS Systems Manager > State Manager > Association ID : {association ID}
4. Choose an execution ID to get execution success/errors of each instance
5. Visit AWS Systems Manager > Managed Instances > Instance ID: {Instance ID} to view a list of compliance tests that passed/failed under Configuration Compliance

References

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ssm-association.html

https://github.com/dev-sec/linux-baseline/blob/master/inspec.yml

https://trevorsullivan.net/2019/04/03/errors-and-fixes-using-aws-cloudformation-to-deploy-shell-scripts-via-aws-systems-manager-associations/

https://github.com/awslabs/aws-systems-manager/blob/master/Compliance/InSpec/PortCheck/