Supply Chain Security: Cosign & SBOM
- Image Integrity: Supply chain security ensures that the container images running in your cluster are authentic, untampered, and built by trusted pipelines. Without verification, an attacker who compromises your registry can swap any image with a malicious one.
- Image Signing (Cosign/Sigstore): Cosign, part of the Sigstore project, digitally signs container images using either key-pair cryptography or keyless signing with OIDC identities. Admission controllers verify these signatures at deploy time, blocking unsigned or untrusted images.
- Software Bill of Materials (SBOM): SBOMs provide a comprehensive, machine-readable inventory of every library, dependency, and component within a container image. When a new CVE is discovered, SBOMs enable instant identification of affected images without rescanning every running container.
- SLSA Framework: Supply-chain Levels for Software Artifacts (SLSA) defines progressive security levels (L1-L4) for build integrity, from basic provenance to hermetic, reproducible builds. It provides a maturity model for improving your build pipeline's security posture.
- Defense in Depth: A comprehensive supply chain security strategy combines image signing, SBOM generation, vulnerability scanning, admission policy enforcement, and secure CI/CD pipeline configuration into multiple overlapping layers of protection.
Securing your Kubernetes cluster goes beyond RBAC, NetworkPolicies, and runtime security. You must also guarantee that the code running in your cluster is exactly what your developers built and that it does not contain known vulnerabilities. A compromised CI/CD pipeline, a hijacked base image, or a poisoned dependency can inject malicious code that passes every other security check. Supply chain security addresses this attack surface.
1. The Supply Chain Attack Surface
The path from source code to running container crosses multiple trust boundaries:
- Source code: Compromised developer accounts or malicious pull requests can inject backdoors.
- Dependencies: Third-party libraries may contain vulnerabilities or intentional malware (dependency confusion attacks).
- Build pipeline: A compromised CI system can modify the build output without changing the source.
- Container registry: An attacker with registry access can replace a legitimate image tag with a malicious image.
- Deployment: Without verification, Kubernetes will run whatever image the manifest specifies.
Each boundary needs its own controls. Supply chain security tools provide these controls.
2. Image Signing with Cosign and Sigstore
Cosign is the standard tool for signing container images. It is part of the Sigstore project, a Linux Foundation initiative that provides free signing infrastructure.
Key-Pair Signing
The traditional approach: generate a key pair, sign images with the private key, and verify with the public key.
# CI/CD pipeline: sign the image after building
# Step 1: Build and push the image
# docker build -t myregistry.io/api-server:v2.4.1 .
# docker push myregistry.io/api-server:v2.4.1
# Step 2: Sign the image with Cosign
# cosign sign --key cosign.key myregistry.io/api-server:v2.4.1
# The signature is stored as an OCI artifact alongside the image in the registry.
# No separate signature storage is needed.
Keyless Signing (Recommended)
Keyless signing eliminates the burden of managing long-lived signing keys. Instead, Cosign uses short-lived certificates issued by the Sigstore Fulcio CA, tied to an OIDC identity (GitHub Actions, GitLab CI, Google Cloud Build).
# Keyless signing in GitHub Actions — no keys to manage
# The OIDC token from GitHub Actions is used to obtain a signing certificate
# cosign sign myregistry.io/api-server:v2.4.1
# The signature includes the OIDC identity:
# Issuer: https://token.actions.githubusercontent.com
# Subject: https://github.com/org/repo/.github/workflows/build.yml@refs/heads/main
This means you can verify not just that an image was signed, but who signed it and which workflow produced it. The signing event is recorded in the Sigstore Rekor transparency log, providing a tamper-evident audit trail.
3. Software Bill of Materials (SBOM)
An SBOM is a machine-readable list of every component inside a container image: OS packages, application libraries, and their versions. Two standard formats exist: SPDX and CycloneDX.
Generating SBOMs
# Generate an SBOM with Syft (Anchore)
# syft myregistry.io/api-server:v2.4.1 -o spdx-json > sbom.spdx.json
# Generate an SBOM with Trivy (Aqua Security)
# trivy image --format spdx-json myregistry.io/api-server:v2.4.1 > sbom.spdx.json
Attaching SBOMs to Images
Cosign can attach SBOMs directly to container images as OCI artifacts, so the SBOM travels with the image through any registry.
# Attach the SBOM to the image
# cosign attach sbom --sbom sbom.spdx.json myregistry.io/api-server:v2.4.1
# Or use cosign attest to create a signed attestation containing the SBOM
# cosign attest --predicate sbom.spdx.json --type spdxjson myregistry.io/api-server:v2.4.1
Using SBOMs for Vulnerability Response
When a critical CVE is announced (like Log4Shell), you can scan all your SBOMs without pulling or re-scanning every container:
# Scan an SBOM for vulnerabilities
# grype sbom:sbom.spdx.json
# Or use Trivy
# trivy sbom sbom.spdx.json
This enables rapid triage: within minutes of a CVE announcement, you know exactly which images and pods are affected.
4. Image Scanning
Image scanning examines container images for known vulnerabilities by matching installed packages against CVE databases.
Scanning Tools
| Tool | Publisher | Key Features |
|---|---|---|
| Trivy | Aqua Security | Open source, scans OS packages + language deps, supports SBOM output |
| Grype | Anchore | Open source, fast CLI scanner, integrates with Syft for SBOMs |
| Snyk | Snyk | Commercial, deep language-level scanning, IDE integration |
Integrating Scanning into CI/CD
# GitHub Actions workflow: scan before push, fail on critical vulnerabilities
name: Build and Scan
on: push
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Build image
run: docker build -t myregistry.io/api-server:${{ github.sha }} .
- name: Scan with Trivy
uses: aquasecurity/trivy-action@master
with:
image-ref: myregistry.io/api-server:${{ github.sha }}
format: table
exit-code: 1 # Fail the pipeline on findings
severity: CRITICAL,HIGH # Only block on critical and high
ignore-unfixed: true # Skip CVEs without available fixes
- name: Sign with Cosign
if: success() # Only sign if scan passed
run: cosign sign myregistry.io/api-server:${{ github.sha }}
5. Admission Control for Verified Images
Signing images is useless without enforcement. Admission controllers verify signatures at deploy time and reject images that fail verification.
Kyverno verifyImages
# Kyverno policy: only allow images signed by our CI/CD pipeline
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: verify-image-signatures
spec:
validationFailureAction: Enforce # Block non-compliant pods
background: false
rules:
- name: verify-cosign-signature
match:
any:
- resources:
kinds:
- Pod
verifyImages:
- imageReferences:
- "myregistry.io/*" # Apply to all images from our registry
attestors:
- entries:
- keyless:
issuer: "https://token.actions.githubusercontent.com"
subject: "https://github.com/org/repo/*"
rekor:
url: "https://rekor.sigstore.dev"
attestations:
- type: "https://spdx.dev/Document" # Require SBOM attestation
conditions:
- all:
- key: "{{ len(creationInfo.creators) }}"
operator: GreaterThan
value: "0"
Connaisseur
Connaisseur is a lightweight admission controller dedicated to image signature verification. It supports Cosign, Notary v2, and static delegation lists.
Policy Controller (Sigstore)
The Sigstore Policy Controller is purpose-built for Cosign signature verification. It integrates natively with the Sigstore ecosystem and supports both key-pair and keyless verification.
6. Base Image Management
Base images are the foundation of your container supply chain. A compromised base image affects every application built on top of it.
Best practices:
- Pin base images by digest, not tag. Tags are mutable; digests are immutable.
FROM nginx:1.25@sha256:abc123...ensures you always get the exact same image. - Maintain internal golden images: Build approved base images from minimal sources (distroless, Alpine), scan them, sign them, and store them in your private registry.
- Automate base image updates: Use tools like Renovate or Dependabot to detect when base images have new versions or security patches, and automatically create pull requests.
- Minimize image contents: Every additional package is an additional attack surface. Use multi-stage builds and distroless final images.
7. SLSA Framework
SLSA (Supply-chain Levels for Software Artifacts, pronounced "salsa") is a security framework from Google that defines four progressive levels of build integrity:
| Level | Requirement | What It Proves |
|---|---|---|
| SLSA L1 | Build provenance exists | You know what was built and where |
| SLSA L2 | Signed provenance from a hosted build service | Provenance was not tampered with |
| SLSA L3 | Hardened build platform, non-falsifiable provenance | The build ran on a trusted platform |
| SLSA L4 | Hermetic, reproducible builds with two-party review | The build is fully reproducible and auditable |
Most organizations should aim for SLSA L2 (signed provenance from CI/CD) as a practical first milestone. GitHub Actions and Google Cloud Build can generate SLSA L3 provenance natively using the SLSA GitHub generator.
8. In-Toto Attestations
In-toto is a framework for securing the entire software supply chain by verifying that each step in the pipeline was performed by the expected actor and produced the expected output. Cosign attestations use the in-toto envelope format.
# Create an in-toto attestation for a vulnerability scan result
# cosign attest --predicate scan-results.json \
# --type vuln \
# myregistry.io/api-server:v2.4.1
# The attestation is signed and stored alongside the image
# Admission controllers can verify both the signature and the predicate content
Attestations allow admission controllers to enforce not just "is this image signed?" but "was this image scanned, did it pass, and was the SBOM generated?" -- creating a verifiable chain of evidence from source to deployment.
9. Registry Security
Private Registries
- Use private registries (ECR, ACR, GCR, Harbor) instead of public Docker Hub for production images.
- Enable immutable tags to prevent image tag overwriting.
- Configure vulnerability scanning at the registry level (most cloud registries offer this natively).
- Use Kubernetes
imagePullSecretsto authenticate to private registries.
# Image pull secret for a private registry
apiVersion: v1
kind: Secret
metadata:
name: registry-credentials
namespace: production
type: kubernetes.io/dockerconfigjson
data:
.dockerconfigjson: <base64-encoded-docker-config>
---
# Reference the secret in a ServiceAccount for automatic injection
apiVersion: v1
kind: ServiceAccount
metadata:
name: app-service-account
namespace: production
imagePullSecrets:
- name: registry-credentials # All pods using this SA get the credentials
Harbor as an Enterprise Registry
Harbor, a CNCF graduated project, provides vulnerability scanning, image signing, replication across registries, RBAC, and audit logging. It serves as a central control point for container image governance.
10. Vulnerability Management Workflow
A mature vulnerability management workflow integrates scanning, SBOMs, and admission control into a continuous process:
- Build time: Scan images during CI/CD. Block builds with critical vulnerabilities. Generate and attach SBOMs.
- Admission time: Verify image signatures and attestations. Reject unsigned or unscanned images.
- Runtime: Continuously scan running workloads against updated CVE databases. Alert when new vulnerabilities affect deployed images.
- Response: Use SBOM data to identify affected workloads within minutes of a CVE announcement. Prioritize remediation based on exploitability and exposure.
Common Pitfalls
- Signing but not enforcing: Signing images without deploying an admission controller that verifies signatures provides zero security. The enforcement point is what makes signing valuable.
- Using mutable tags: Referencing images by tags like
:latestor:v2means the image content can change without your knowledge. Always pin by digest in production manifests. - Ignoring transitive dependencies: Your application may be clean, but a nested dependency three levels deep could contain a critical vulnerability. SBOM generation must include the full dependency tree.
- Scanning only at build time: CVE databases are updated daily. An image that was clean last week may have critical vulnerabilities today. Implement continuous scanning of deployed images.
- Not handling key rotation: If you use key-pair signing and your private key is compromised, you need a plan to rotate keys and re-sign images. Keyless signing avoids this problem entirely.
- Blocking all unscanned images immediately: Rolling out strict admission policies without a transition period will break existing deployments. Use
Auditmode first, remediate, then switch toEnforce.
What's Next?
- Implement keyless signing with Cosign in your CI/CD pipeline using OIDC identity federation.
- Generate SBOMs for every production image using Syft or Trivy and attach them as signed attestations.
- Deploy Kyverno or the Sigstore Policy Controller to enforce signature verification at admission time.
- Adopt the SLSA framework as a roadmap for improving build pipeline security incrementally.
- Set up continuous scanning of running images using Trivy Operator or Grype to detect newly discovered vulnerabilities.
- Deploy Harbor as your private registry with built-in scanning, signing, and replication capabilities.