StageX Container Images

Overview

StageX provides minimal, reproducible, zero-trust container base images for the IPCEI-CIS Edge Developer Platform. Built from source using the StageX upstream methodology, these images contain no package manager and compose dependencies exclusively via the COPY --from= OCI pattern.

Repository: edp.buildth.ing/DevFW-CICD/stagex
Stakeholder KPI Report: Confluence — KPI e1 & e2 D62/D64

Key Properties

• Zero-trust — no package manager, no shell in production images, minimal attack surface
• Reproducible — all images pinned by sha256 digest; SOURCE_DATE_EPOCH ensures deterministic timestamps
• Minimal — musl libc, LLVM/Clang toolchain; production runtime images as small as 4.86 MB
• Signed — cosign key-based signing with Kyverno N-of-M policy verification
• Scanned — dual-scanner pipeline: Trivy (app-level) + Grype CPE (OS-level NVD matching)
• Efficient — 100% layer efficiency, 0 wasted bytes; 37% smaller Forgejo Runner image achieved

Image Size Comparison

StageX images are significantly smaller than standard enterprise base images:

For compiled Go/Rust binaries using stagex-base-static:

• 96% smaller than debian:bookworm-slim
• 94% smaller than ubuntu:24.04
• Fewer bytes = faster pulls, less storage, less network I/O, smaller attack surface

Available Images

All images are published to the EDP container registry:

Registry path: edp.buildth.ing/devfw-cicd/stagex-{base,base-go,base-nodejs,base-python,base-rust,base-java,base-ruby,base-static}

Vulnerability Scanning

StageX uses a dual-scanner pipeline because no single scanner covers all layers:

The cpes.txt file lists CPE 2.3 entries for all packages included in StageX images (musl 1.2.5, busybox 1.37.0, openssl 3.4.1, etc.) and is updated when package digests are bumped.

CI gate: Trivy hard-fails on CRITICAL vulnerabilities. Grype CPE runs as advisory (upstream-controlled packages).

Build Methodology

StageX images are composed without a package manager. Dependencies are layered using the OCI COPY --from= pattern:

FROM stagex/core-busybox AS busybox
FROM stagex/core-musl AS musl
FROM stagex/core-openssl AS openssl

FROM scratch
COPY --from=musl / /
COPY --from=openssl / /
COPY --from=busybox / /

All upstream StageX packages are built from source with:

• musl libc (not glibc)
• LLVM/Clang toolchain
• PGP quorum signing for upstream package verification
• Stage0 bootstrap chain — from minimal hex assembler (hex0, ~500 bytes) through increasingly complex compilers to full LLVM/Clang. No pre-compiled binary is trusted.

CI Pipeline

The build pipeline (.forgejo/workflows/build-sign.yaml) performs:

1. Build — multi-stage Docker build with SOURCE_DATE_EPOCH for reproducibility
2. Scan — Trivy (app-level) + Grype CPE (OS-level NVD matching) + Malcontent (behavioral analysis)
3. SBOM — generates SPDX and CycloneDX Software Bill of Materials
4. Push — publish to EDP registry with digest pinning
5. Sign — cosign key-based signing with git SHA and provenance annotations
6. Verify — automated signature verification post-push

Supply Chain Verification

Cosign Signing

All images are signed using cosign with key-based signatures (not keyless — Forgejo OIDC is not in Fulcio’s trust root):

cosign verify --key cosign.pub edp.buildth.ing/devfw-cicd/stagex-base:latest

Signature annotations include: git.sha, git.ref, build.workflow, source.repo.

Kyverno Admission Control

A Kyverno ClusterPolicy (policies/kyverno-require-signatures.yaml) enforces 2-of-3 multi-party signature verification at pod admission:

• CI pipeline key (automated)
• Security team key (manual review)
• Release manager key (release approval)

Unsigned or insufficiently signed images are rejected at the Kubernetes API level:

kubectl run test --image=debian:bookworm-slim --dry-run=server
# Error: admission webhook "validate.kyverno.svc" denied the request:
# Only signed StageX base images are allowed

Requirements

• Docker v25+ with containerd image store enabled (required for OCI image handling)
• amd64 architecture (arm64 designed for but pending upstream support)

Completed Work