Package Cache on Porch Documentation

Cache Design

Mon, 01 Jan 0001 00:00:00 +0000

Cache Interface

The Package Cache provides a unified interface that abstracts the underlying storage mechanism. Both the CR Cache and DB Cache implementations conform to this common interface, allowing the CaDEngine to work with either implementation without knowing which is in use.

Repository Operations:

Open repository connections from Repository CR specifications
Close repositories and clean up resources
Retrieve list of all open repositories
Get specific repository by key
Update repository metadata
Check repository connectivity

Repository Interface Delegation:

Custom Resource Cache

Mon, 01 Jan 0001 00:00:00 +0000

Overview

The Custom Resource (CR) Cache is the default cache implementation in Porch. It stores package metadata using a Kubernetes Custom Resources while keeping repository data in memory. This implementation is designed for standard Porch deployments where Kubernetes etcd provides sufficient storage and performance.

Key characteristics:

Default implementation: Used when no cache type is explicitly configured
Hybrid storage: In-memory repository cache + CR-based metadata storage
RDBMS-independent: No RDBMS (such as Posgres) required, leverages the Kubernetes API, etcd, and Git for persistence
Suitable for: Small to medium deployments with moderate package counts
No external dependencies: Only requires a Kubernetes cluster
Git interaction: Interacts with Git at every stage of package revision lifecycle for presistence

Implementation Details

Storage Architecture

The CR Cache uses a two-tier storage model:

Database Cache

Mon, 01 Jan 0001 00:00:00 +0000

Overview

The Database (DB) Cache is an alternative cache implementation in Porch designed for larger deployments. It stores both package metadata and repository data in a PostgreSQL database rather than in memory or Kubernetes Custom Resources. This implementation provides better scalability and persistence characteristics for production environments with high package counts.

Key characteristics:

Alternative implementation: Used when explicitly configured with database connection details
Database-backed storage: All repository, package, and package revision data stored in PostgreSQL
External dependency: Requires PostgreSQL database instance
Suitable for: Large deployments with thousands of packages and package revisions
Better persistence: Survives Porch server restarts without re-fetching from Git
Git interaction: By default, interacts with Git only during approval/publish and sync operations. Configurable with --db-push-drafts-to-git flag to push drafts

Implementation Details

Storage Architecture

The DB Cache uses a database-centric storage model:

Cache Interactions

Mon, 01 Jan 0001 00:00:00 +0000

Overview

The Package Cache acts as an intermediary layer that coordinates between the CaDEngine, Repository Adapters, and external Git repositories. It provides caching, synchronization, and change notification services while maintaining a clean separation of concerns.

High-Level Architecture

┌─────────────────────────────────────────────────────────┐
│ Package Cache Layer │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────┐ │
│ │ CaDEngine │ ───> │ Cache │ ───> │ Repo │ │
│ │ Requests │ │ Operations │ │Adapt │ │
│ │ │ │ │ │ ers │ │
│ └──────────────┘ └──────────────┘ └──────┘ │
│ ↑ │ │ │
│ │ ↓ ↓ │
│ │ ┌──────────────┐ ┌──────┐ │
│ └──────────────│ Watcher │ │ Git │ │
│ │ Notifier │ │Repos │ │
│ └──────────────┘ └──────┘ │
└─────────────────────────────────────────────────────────┘

Repository Adapter Integration

The cache wraps repository adapters to provide caching and synchronization services: