Terralist

Private Terraform Module and Provider Registry with OAuth authentication

Overview

Terralist is an open-source private Terraform registry for modules and providers that implements the HashiCorp registry protocol. As part of the Edge Developer Platform, Terralist enables teams to securely store, version, and distribute internal Terraform modules and providers with built-in authentication and documentation capabilities.

The Terralist stack deploys a self-hosted instance with OAuth2 authentication, persistent storage, and integrated ingress for secure access.

Key Features

Private Module Registry: Securely host and distribute confidential Terraform modules and providers
HashiCorp Protocol Compatible: Works seamlessly with terraform CLI and standard registry workflows
OAuth2 Authentication: Integrated OIDC authentication supporting terraform login command
Documentation Interface: Web UI to visualize artifacts with automatic module documentation
Flexible Storage: Supports local storage or remote cloud buckets with presigned URLs
Git Integration: Works with mono-repositories while leveraging Terraform version attributes
API Management: RESTful API for programmatic module and provider management

Repository

Code: Terralist Stack Templates

Documentation:

Getting Started

Prerequisites

Kubernetes cluster with ArgoCD installed (provided by core stack)
Ingress controller configured (provided by otc stack)
cert-manager for TLS certificate management (provided by otc stack)
Domain name configured via DOMAIN_GITEA environment variable
OAuth2 provider configured (Dex or external provider)

Quick Start

The Terralist stack is deployed as part of the EDP installation process:

Trigger Deploy Pipeline
- Go to Infra Deploy Pipeline
- Click on Run workflow
- Enter a name in “Select environment directory to deploy”. This must be DNS Compatible. (if you enter test-me then the domain will be terralist.test-me.t09.de)
- Execute workflow
ArgoCD Synchronization ArgoCD automatically deploys:
- Terralist application (Helm chart v0.8.1)
- Persistent volume for module storage
- Ingress configuration with TLS
- OAuth2 credentials and configuration

Verification

Verify the Terralist deployment:

# Check ArgoCD application status
kubectl get application terralist -n argocd

# Verify Terralist pods are running
kubectl get pods -n terralist

# Check persistent volume claim
kubectl get pvc -n terralist

# Verify ingress configuration
kubectl get ingress -n terralist

Access the Terralist web interface at https://terralist.{DOMAIN_GITEA}.

Architecture

Component Architecture

The Terralist stack consists of:

Terralist Application:

Web interface for module and provider management
REST API for programmatic access
OAuth2 authentication handler
Module documentation renderer

Storage Layer:

SQLite database for metadata and configuration
Local filesystem storage for modules and providers
Persistent volume with 10Gi capacity on csi-disk storage class
Optional cloud bucket integration for remote storage

Networking:

Nginx ingress with TLS termination
cert-manager integration for automatic certificate management
OAuth2 callback endpoint configuration

Configuration

Environment Variables

The Terralist application is configured through environment variables in values.yaml:

OAuth2 Configuration:

TERRALIST_AUTHORITY_URL: OIDC provider authority URL (from terralist-oidc-secrets secret)
TERRALIST_CLIENT_ID: OAuth2 client identifier
TERRALIST_CLIENT_SECRET: OAuth2 client secret
TERRALIST_TOKEN_SIGNING_SECRET: Secret for token signing and validation

Storage Configuration:

SQLite database at /data/database.db
Module storage at /data/modules

Helm Chart Configuration

Key Helm values configured in stacks/terralist/terralist/values.yaml:

controllers:
  main:
    strategy: Recreate
    containers:
      main:
        env:
          - name: TERRALIST_AUTHORITY_URL
            valueFrom:
              secretKeyRef:
                name: terralist-oidc-secrets
                key: authority_url
          - name: TERRALIST_CLIENT_ID
            valueFrom:
              secretKeyRef:
                name: terralist-oidc-secrets
                key: client_id

ingress:
  main:
    enabled: true
    className: nginx
    hosts:
      - host: "terralist.{DOMAIN_GITEA}"
        paths:
          - path: /
            service:
              identifier: main
    annotations:
      cert-manager.io/cluster-issuer: main
    tls:
      - secretName: terralist-tls-secret
        hosts:
          - "terralist.{DOMAIN_GITEA}"

persistence:
  data:
    enabled: true
    size: 10Gi
    storageClass: csi-disk
    accessMode: ReadWriteOnce

ArgoCD Application Configuration

Registry Application (template/registry/terralist.yaml):

Name: terralist-reg
Manages the Terralist stack directory
Automated sync with prune and self-heal enabled

Stack Application (template/stacks/terralist/terralist.yaml):

Name: terralist
Deploys Terralist Helm chart v0.8.1 from https://github.com/terralist/helm-charts
Automated self-healing enabled
Creates namespace automatically
References values from stacks-instances repository

Usage Examples

Authenticating with Terralist

Configure Terraform CLI to use your private registry:

# Authenticate using OAuth2
terraform login terralist.${DOMAIN_GITEA}

# This opens a browser window for OAuth2 authentication
# After successful login, credentials are stored in ~/.terraform.d/credentials.tfrc.json

Publishing a Module

Publish a module to your private registry:

Create Module Structure

my-module/
├── main.tf
├── variables.tf
├── outputs.tf
└── README.md

Tag and Push via API

# Package module
tar -czf my-module-1.0.0.tar.gz my-module/

# Upload to Terralist (requires authentication token)
curl -X POST https://terralist.${DOMAIN_GITEA}/v1/modules/my-org/my-module/my-provider/1.0.0 \
  -H "Authorization: Bearer ${TERRALIST_TOKEN}" \
  -F "file=@my-module-1.0.0.tar.gz"

Consuming Private Modules

Use modules from your private registry in Terraform configurations:

# Configure Terraform to use private registry
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

# Reference module from private registry
module "vpc" {
  source  = "terralist.${DOMAIN_GITEA}/my-org/vpc/aws"
  version = "1.0.0"

  cidr_block = "10.0.0.0/16"
  environment = "production"
}

Browsing Module Documentation

Access the Terralist web interface to view module documentation:

# Open Terralist UI
open https://terralist.${DOMAIN_GITEA}

# Browse available modules
# - View module versions
# - Read generated documentation
# - Access module sources
# - Copy usage examples

Managing Modules via API

# List all modules
curl -H "Authorization: Bearer ${TERRALIST_TOKEN}" \
  https://terralist.${DOMAIN_GITEA}/v1/modules

# Get specific module versions
curl -H "Authorization: Bearer ${TERRALIST_TOKEN}" \
  https://terralist.${DOMAIN_GITEA}/v1/modules/my-org/my-module/my-provider

# Delete a module version
curl -X DELETE -H "Authorization: Bearer ${TERRALIST_TOKEN}" \
  https://terralist.${DOMAIN_GITEA}/v1/modules/my-org/my-module/my-provider/1.0.0

Integration Points

Core Stack: Depends on ArgoCD for deployment orchestration
OTC Stack: Requires ingress-nginx controller and cert-manager for external access and TLS
Dex (SSO): Integrates with platform OAuth2 provider for authentication
Forgejo Stack: Modules can be sourced from platform Git repositories
Observability Stack: Application metrics can be collected by platform monitoring tools

Troubleshooting

Terralist Pod Not Starting

Problem: Terralist pod remains in Pending or CrashLoopBackOff state

Solution:

Check persistent volume claim status:

kubectl get pvc -n terralist
kubectl describe pvc data-terralist-0 -n terralist

Verify OAuth2 credentials secret:

kubectl get secret terralist-oidc-secrets -n terralist
kubectl describe secret terralist-oidc-secrets -n terralist

Check Terralist logs:

kubectl logs -n terralist -l app.kubernetes.io/name=terralist

Cannot Access Terralist UI

Problem: Terralist web interface is not accessible at configured URL

Solution:

Verify ingress configuration:

kubectl get ingress -n terralist
kubectl describe ingress -n terralist

Check TLS certificate status:

kubectl get certificate -n terralist
kubectl describe certificate terralist-tls-secret -n terralist

Verify DNS resolution:
```
nslookup terralist.${DOMAIN_GITEA}
```

OAuth2 Authentication Fails

Problem: terraform login or web authentication fails

Solution:

Verify OAuth2 configuration in secret:

kubectl get secret terralist-oidc-secrets -n terralist -o yaml

Check OAuth2 provider (Dex) is accessible:

curl https://dex.${DOMAIN_GITEA}/.well-known/openid-configuration

Verify callback URL is correctly configured in OAuth2 provider:

Expected callback: https://terralist.${DOMAIN_GITEA}/auth/cli/callback

Check Terralist logs for authentication errors:

kubectl logs -n terralist -l app.kubernetes.io/name=terralist | grep -i auth

Module Upload Fails

Problem: Cannot upload modules via API or UI

Solution:

Verify authentication token is valid:

# Test token with API call
curl -H "Authorization: Bearer ${TERRALIST_TOKEN}" \
  https://terralist.${DOMAIN_GITEA}/v1/modules

Check persistent volume has available space:

kubectl exec -n terralist -it terralist-0 -- df -h /data

Verify module package format is correct:

# Module should be a gzipped tar archive
tar -tzf my-module-1.0.0.tar.gz

Review upload logs:

kubectl logs -n terralist -l app.kubernetes.io/name=terralist --tail=50

Terraform Cannot Download Modules

Problem: terraform init fails to download modules from private registry

Solution:

Verify authentication credentials exist:

cat ~/.terraform.d/credentials.tfrc.json

Re-authenticate if needed:

terraform logout terralist.${DOMAIN_GITEA}
terraform login terralist.${DOMAIN_GITEA}

Test module availability via API:

curl -H "Authorization: Bearer ${TERRALIST_TOKEN}" \
  https://terralist.${DOMAIN_GITEA}/v1/modules/my-org/my-module/my-provider

Check module source URL format in Terraform configuration:

# Correct format
source = "terralist.${DOMAIN_GITEA}/org/module/provider"

# Not: https://terralist.${DOMAIN_GITEA}/...

Terralist

Overview

Key Features

Repository

Getting Started

Prerequisites

Quick Start

Verification

Architecture

Component Architecture

Configuration

Environment Variables

Helm Chart Configuration

ArgoCD Application Configuration

Usage Examples

Authenticating with Terralist

Publishing a Module

Consuming Private Modules

Browsing Module Documentation

Managing Modules via API

Integration Points

Troubleshooting

Terralist Pod Not Starting

Cannot Access Terralist UI

OAuth2 Authentication Fails

Module Upload Fails

Terraform Cannot Download Modules

Additional Resources