Deployment

Platform-level component provisioning via Stacks - Orchestrating the platform infrastructure itself

Overview

Platform Orchestration refers to the automation and management of the platform infrastructure itself. This includes the provisioning, configuration, and lifecycle management of all components that make up the Internal Developer Platform (IDP).

In the context of IPCEI-CIS, Platform Orchestration means:

Platform Bootstrap: Initial setup of Kubernetes clusters and core services
Platform Services Management: Deployment and management of ArgoCD, Forgejo, Keycloak, etc.
Infrastructure-as-Code: Declarative management using Terraform and GitOps
Multi-Cluster Orchestration: Coordination across different Kubernetes clusters
Platform Stacks: Reusable bundles of platform components (CNOE concept)

Target Audience

Platform Orchestration is primarily aimed at:

Platform Engineering Teams: Teams that build and operate the IDP
Infrastructure Architects: Those responsible for the platform architecture
SRE Teams: Teams responsible for reliability and operations

Key Features

Declarative Platform Definition

The entire platform is defined declaratively as code:

GitOps-First: Everything is versioned in Git and traceable
Reproducibility: The platform can be rebuilt at any time
Environment Parity: Consistency between Dev, Test, and Production
Auditability: Complete history of all changes

Self-Bootstrapping

The platform can bootstrap itself:

Initial Bootstrap: Minimal tool (like idpbuilder) starts the platform
Self-Management: After bootstrap, ArgoCD takes over management
Continuous Reconciliation: Platform is continuously reconciled with Git state
Self-Healing: Automatic recovery on deviations

Stack-based Composition

Platform components are organized as reusable stacks (CNOE concept):

Modularity: Components can be updated individually
Reusability: Stacks can be used across different environments
Composability: Compose complex platforms from simple building blocks
Versioning: Stacks can be versioned and tested

In IPCEI-CIS: The stacks concept from CNOE is the core organizational principle for platform components.

Multi-Cluster Support

Platform Orchestration supports different cluster topologies:

Control Plane + Worker Clusters: Centralized control, distributed workloads
Hub-and-Spoke: One management cluster manages multiple target clusters
Federation: Coordination across multiple independent clusters

Purpose in EDP

Platform Orchestration is the foundation of the IPCEI-CIS Edge Developer Platform. It enables:

Foundation for Developer Self-Service

Platform Orchestration ensures all services are available that developers need for self-service:

GitOps Engine (ArgoCD) for continuous deployment
Source Control (Forgejo) for code and configuration management
Identity Management (Keycloak) for authentication and authorization
Observability (Grafana, Prometheus) for monitoring and logging
CI/CD (Forgejo Actions/Pipelines) for automated build and test

Consistency Across Environments

Through declarative definition, consistency is guaranteed:

Development, test, and production environments are identically configured
No “configuration drift” between environments
Predictable behavior across all stages

Platform as Code

The platform itself is treated like software:

Version Control: All changes are versioned in Git
Code Review: Platform changes go through review processes
Testing: Platform configurations can be tested
Rollback: Easy rollback on problems

Reduced Operational Overhead

Automation reduces manual effort:

No manual installation steps
Automatic updates and patching
Self-healing on failures
Standardized deployment processes

Repository

CNOE Reference Implementation: cnoe-io/stacks

CNOE idpbuilder: cnoe-io/idpbuilder

Documentation: CNOE.io Documentation

Getting Started

Prerequisites

Docker: For local Kubernetes clusters (Kind)
kubectl: Kubernetes CLI tool
Git: For repository management
idpbuilder: CNOE bootstrap tool

Quick Start

Platform Orchestration with CNOE Reference Implementation:

# 1. Install idpbuilder
curl -fsSL https://cnoe.io/install.sh | bash

# 2. Bootstrap platform
idpbuilder create \
  --use-path-routing \
  --package-dir https://github.com/cnoe-io/stacks//ref-implementation

# 3. Wait for platform ready (ca. 10 minutes)
kubectl get applications -A

Verification

Verify the platform is running correctly:

# Get platform secrets (credentials)
idpbuilder get secrets

# Check all ArgoCD applications
kubectl get applications -n argocd

# Expected: All applications "Synced" and "Healthy"

Access URLs (with path-routing):

ArgoCD: https://cnoe.localtest.me:8443/argocd
Forgejo: https://cnoe.localtest.me:8443/gitea
Keycloak: https://cnoe.localtest.me:8443/keycloak

Usage Examples

Use Case 1: Platform Bootstrap

Initial bootstrapping of a new platform instance:

idpbuilder create \
  --use-path-routing \
  --package-dir https://github.com/cnoe-io/stacks//ref-implementation \
  --log-level debug

# Workflow:
# 1. Creates Kind cluster
# 2. Installs ingress-nginx
# 3. Clones and installs ArgoCD
# 4. Installs Forgejo
# 5. Waits for core services
# 6. Creates technical users
# 7. Configures Git repositories
# 8. Installs remaining stacks via ArgoCD

After approximately 10 minutes, the platform is fully deployed.

Use Case 2: Adding New Platform Components

Add new platform components via ArgoCD:

# Create ArgoCD Application for new component
cat <<EOF | kubectl apply -f -
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: external-secrets
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://charts.external-secrets.io
    targetRevision: 0.9.9
    chart: external-secrets
  destination:
    server: https://kubernetes.default.svc
    namespace: external-secrets-system
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
EOF

Use Case 3: Platform Updates

Update platform components:

# 1. Update via Git (GitOps)
cd your-platform-config-repo
git pull

# 2. Update stack version
vim argocd/applications/component.yaml
# Change targetRevision to new version

# 3. Commit and push
git add .
git commit -m "Update component to v1.2.3"
git push

# 4. ArgoCD will automatically sync
# 5. Monitor the update
argocd app sync component --watch

Integration Points

ArgoCD Integration

Bootstrap: ArgoCD is initially installed via idpbuilder
Self-Management: After bootstrap, ArgoCD manages itself via Application CRD
Platform Coordination: ArgoCD orchestrates all other platform components
Health Monitoring: ArgoCD monitors health status of all platform services

Forgejo Integration

Source of Truth: Git repositories contain all platform definitions
GitOps Workflow: Changes in Git trigger platform updates
Backup: Git serves as backup of platform configuration
Audit Trail: Git history documents all platform changes
CI/CD: Forgejo Actions can automate platform operations

Terraform Integration

Infrastructure Provisioning: Terraform provisions cloud resources for platform
State Management: Terraform state tracks infrastructure
Integration: Terraform can be triggered via Forgejo pipelines
Multi-Cloud: Support for multiple cloud providers

Architecture

Platform Orchestration Flow

┌─────────────────┐
│   idpbuilder    │  Bootstrap Tool
│  (Initial Run)  │
└────────┬────────┘
         │
         ▼
┌─────────────────────────────────────────────────────┐
│              Kubernetes Cluster                      │
│                                                      │
│  ┌──────────────┐         ┌──────────────┐         │
│  │   ArgoCD     │────────▶│   Forgejo    │         │
│  │  (GitOps)    │         │  (Git Repo)  │         │
│  └──────┬───────┘         └──────────────┘         │
│         │                                            │
│         │  Monitors & Syncs                         │
│         │                                            │
│         ▼                                            │
│  ┌──────────────────────────────────────┐          │
│  │     Platform Stacks                  │          │
│  │                                      │          │
│  │  ┌──────────┐  ┌──────────┐        │          │
│  │  │Forgejo   │  │Keycloak  │        │          │
│  │  └──────────┘  └──────────┘        │          │
│  │  ┌──────────┐  ┌──────────┐        │          │
│  │  │Observ-   │  │Ingress   │        │          │
│  │  │ability   │  │          │        │          │
│  │  └──────────┘  └──────────┘        │          │
│  └──────────────────────────────────────┘          │
└─────────────────────────────────────────────────────┘

Platform Bootstrap Sequence

The idpbuilder executes the following workflow:

Create Kind Kubernetes cluster
Install ingress-nginx controller
Install ArgoCD
Install Forgejo Git server
Wait for services to be ready
Create technical users in Forgejo
Create repository for platform state in Forgejo
Push platform stacks to Forgejo
Create ArgoCD Applications for all stacks
ArgoCD takes over continuous synchronization

Deployment Architecture

The platform is deployed in different namespaces:

argocd: ArgoCD and its components
gitea: Forgejo Git server
keycloak: Identity and access management
observability: Prometheus, Grafana, etc.
ingress-nginx: Ingress controller

Configuration

idpbuilder Configuration

Key configuration options for idpbuilder:

# Path-based routing (recommended for local development)
idpbuilder create --use-path-routing

# Custom package directory
idpbuilder create --package-dir /path/to/custom/packages

# Custom Kind cluster config
idpbuilder create --kind-config custom-kind.yaml

# Enable debug logging
idpbuilder create --log-level debug

ArgoCD Configuration

Important ArgoCD configurations for platform orchestration:

# argocd-cm ConfigMap
data:
  # Enable automatic sync
  application.instanceLabelKey: argocd.argoproj.io/instance

  # Repository credentials
  repositories: |
    - url: https://github.com/cnoe-io/stacks
      name: cnoe-stacks
      type: git

  # Resource exclusions
  resource.exclusions: |
    - apiGroups:
      - cilium.io
      kinds:
      - CiliumIdentity

Platform Stack Configuration

Configuration of platform stacks via Kustomize:

# kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: platform-system

resources:
  - argocd-app.yaml
  - forgejo-app.yaml
  - keycloak-app.yaml

patches:
  - target:
      kind: Application
    patch: |-
      - op: add
        path: /spec/syncPolicy
        value:
          automated:
            prune: true
            selfHeal: true

Troubleshooting

Platform not reachable

Problem: After idpbuilder create, platform services are not reachable

Solution:

# 1. Check if all pods are running
kubectl get pods -A

# 2. Check ArgoCD application status
kubectl get applications -n argocd

# 3. Check ingress
kubectl get ingress -A

# 4. Verify DNS resolution
nslookup cnoe.localtest.me

# 5. Check idpbuilder logs
idpbuilder get logs

ArgoCD Applications not synchronized

Problem: ArgoCD Applications show status “OutOfSync”

Solution:

# 1. Check application details
argocd app get <app-name>

# 2. View sync status
argocd app sync <app-name> --dry-run

# 3. Force sync
argocd app sync <app-name> --force

# 4. Check for errors in ArgoCD logs
kubectl logs -n argocd deployment/argocd-application-controller

Git Repository Connection Issues

Problem: ArgoCD cannot access Git repository

Solution:

# 1. Verify repository configuration
argocd repo list

# 2. Test connection
argocd repo get https://your-git-repo

# 3. Check credentials
kubectl get secret -n argocd

# 4. Re-add repository with correct credentials
argocd repo add https://your-git-repo \
  --username <user> \
  --password <token>