What’s in Your Container?

When it comes to security, what is inside your container matters. Applications and infrastructures are composed of readily available components, many of which are open source packages such as, the Linux operating system, JBoss Web Server, PostgreSQL, and Node.js.

Containerized versions of these packages are now also readily available, so that you do not have to build your own. But, as with any code you download from some external source, you need to know where they originally came from, who built them, and whether there is any malicious code inside them. Ask yourself:

  • Will what is inside the containers compromise your infrastructure?

  • Are there known vulnerabilities in the application layer?

  • Are the runtime and OS layers up to date?

Further Reading

  • OpenShift Origin Using Images

    • Reference documentation on framework, database, and service container images provided by Red Hat for use on OpenShift Origin

Scanning Container Content

To help answer these questions, container scanning tools can leverage continuously updated vulnerability databases to be sure you always have the latest information on known vulnerabilities for your container content. Because the list of known vulnerabilities is constantly evolving, you need to check the contents of your container images when you first download them and continue to track vulnerability status over time for all your approved and deployed images.

RHEL provides a pluggable API to supports multiple scanners. Red Hat CloudForms can also be used with OpenSCAP to scan container images for security issues.

In addition, OpenShift Origin gives you the ability to leverage such scanners with your CI/CD process. For example, you can integrate static code analysis tools that test for security flaws in your source code and software composition analysis tools that identify open source libraries in order to provide metadata on those libraries such as known vulnerabilities. This is covered in more detail in Build Process.

Integrating External Scanning Tools with OpenShift

OpenShift makes use of object annotations to extend functionality. External tools, such as vulnerability scanners, may annotate image objects with metadata to summarize results and control pod execution. This section describes the recognized format of this annotation so it may be reliably used in consoles to display useful data to users.

Image Metadata

There are different types of image quality data, including package vulnerabilities and open source software (OSS) license compliance. Additionally, there may be more than one provider of this metadata. To that end, the following annotation format has been reserved:

quality.images.openshift.io/<qualityType>.<providerId>: {}
Table 1. Annotation key format
Component Description Acceptable values

qualityType

metadata type

vulnerability, license, operations

providerId

provider ID string

redhatcloudforms, redhatcatalog, redhatinsights, blackduck, jfrog

Example Annotation Keys

quality.images.openshift.io/vulnerability.blackduck: {}
quality.images.openshift.io/vulnerability.jfrog: {}
quality.images.openshift.io/license.blackduck: {}
quality.images.openshift.io/vulnerability.redhatcloudforms: {}

The value of the image quality annotation is structured data that must adhere to the following format:

Table 2. Annotation value format
Field Required? Description Type

name

yes

provider display name

string

timestamp

yes

scan timestamp

int

description

no

short description

string

reference

no

URL for detailed information

string

compliant

no

compliance pass/fail

boolean

summary

no

summary of vulnerabilities found

list (see table below)

The summary field must adhere to the following format:

Table 3. Summary field value format
Field Description Type

label

display label for component, e.g. critical, important, moderate, low or freshness.

string

score

score for this component, e.g. count of vulnerabilities found

int

severityIndex

component score index allowing for ordering and assigning graphical representation. The value is a range where 0=low or none.

int

Example Annotation Values

In this example, a Red Hat CloudForms annotation for an image with vulnerability summary data and a compliance boolean.

Red Hat CloudForms Annotation
{
  name: CloudForms,
  description: vulnerability score,
  timestamp: 2147483647,
  reference: null,
  compliant: true,
  summary: [
    { label: critical, score: 4, severityIndex: 3 },
    { label: important, score: 12, severityIndex: 2 },
    { label: moderate, score: 8, severityIndex: 1 },
    { label: low, score: 26, severityIndex: 0 }
  ]
}

In this example, a Red Hat Container Catalog annotation for an image with vulnerability summary data with an external URL for additional details.

Red Hat Container Catalog Annotation
{
  name: Red Hat Container Catalog,
  description: Container freshness score,
  timestamp: 2147483647,
  reference: https://access.redhat.com/errata/RHBA-2016:1566,
  compliant: null,
  summary: [
    { label: Freshness, score: 4, severityIndex: 4 }
  ]
}

Annotating Image Objects

While imagestream objects are what an end-user of OpenShift operates against, image objects are annotated with security metadata. Image objects are cluster-scoped, pointing to a single image that may be referenced by many imagestreams and tags.

Example annotate CLI command

Replace IMAGE with an image name, e.g. sha256:fec8a395afe3e804b3db5cb277869142d2b5c561ebb517585566e160ff321988.

$ oc annotate image IMAGE \
    quality.images.openshift.io/vulnerability.redhatcatalog='{
    "name": "Red Hat Container Catalog",
    “description”: “Container freshness score”,
    “timestamp”: 2147483647,
    “compliant”: null,
    "reference": “https://access.redhat.com/errata/RHBA-2016:1566”,
    "summary": “[
      { “label”: “Freshness”, “score”: 4, “severityIndex”: 4 } ]" }'

Controlling Pod Execution

To programmatically control if an image may be run, the images.openshift.io/deny-execution image policy may be used. See image policy for more information.

Example annotation

annotations:
  images.openshift.io/deny-execution: true

Integration Reference

In most cases, external tools such as vulnerability scanners will develop a script or plugin that watches for image updates, performs scanning and annotate the associated image object with the results. Typically this automation calls the OpenShift REST API to write the annotation. See REST API documentation for general information on the REST API and PATCH call to update images.

Example REST API Call

This example call using cURL will overwrite the value of the annotation. Be sure to replace the values for TOKEN, OPENSHIFT_SERVER, image ID and IMAGE_ANNOTATION.

Patch API call
$ curl -X PATCH \
  -H "Authorization: Bearer TOKEN" \
  -H "Content-Type: application/merge-patch+json" \
  https://OPENSHIFT_SERVER:8443/oapi/v1/images/sha256:fec8a395afe3e804b3db5cb277869142d2b5c561ebb517585566e160ff321988 \
  --data '{ IMAGE_ANNOTATION }'

Below is example PATCH payload data.

Patch call data
{
"metadata": {
  "annotations": {
    "quality.images.openshift.io/vulnerability.redhatcatalog":
       "{ 'name': 'Red Hat Container Catalog', 'description': 'Container freshness score', 'timestamp': 2147483647, 'compliant': null, 'reference': 'https://access.redhat.com/errata/RHBA-2016:1566', 'summary': [{'label': 'Freshness', 'score': 4, 'severityIndex': 4}] }"
    }
  }
}

Due to the complexity of this API call and challenges with escaping characters, an API developer tool such as Postman may assist in creating API calls.

Further Reading