How It Works

glci sits between your .gitlab-ci.yml and the official gitlab-runner. Instead of talking to a real GitLab instance, the runner talks to a mock GitLab API server running as a daemon-scoped container. Jobs execute in real Docker containers with the same shell scripts, image pulls, and lifecycle that GitLab CI uses in production — no simulation, no approximation.

Component overview#

Data flow#

The mock server is the central hub — the only component both the daemon and the runner communicate with. Jobs push/pull artifacts, cache, and container images directly to/from the mock server via extra_hosts (host-gateway) with FF_NETWORK_PER_BUILD.

Infrastructure#

Daemon-managed (shared across pipelines)#

• glci-mock — mock server container (API + embedded OCI registry on port 39741 by default, configurable via mock_server_port). Started once by the daemon; jobs reach it via extra_hosts (host-gateway) with FF_NETWORK_PER_BUILD.

• Runner containers — persistent gitlab/gitlab-runner containers managed by RunnerManager. Three tiers:

  • Shared runner (glci-runner-{suffix}) — handles projects with default config; jobs without matching named-runner tags go here.
  • Per-project runner (glci-runner-{suffix}-{project}) — created when a project defines a custom [runner] config_template. Isolated from the shared runner.
  • Named runner (glci-runner-{suffix}-{project}-{name}) — one per [runners.<name>] config entry. Jobs whose tags match the runner name are routed here via run_untagged=false.

  Each runner container polls the mock server for jobs via tag-based dispatch. Config is written to a Docker volume and reloaded via SIGHUP. When a config_template produces multiple [[runners]] blocks, glci derives sub-tokens and registers each runner separately.

All containers are long-lived. They start on first use and stop when the daemon shuts down. If a container is missing (manually removed, Docker restart), the daemon recreates it on demand. Per-project and named runners are created lazily when a pipeline first needs them.

Per-pipeline#

• glci-net-{id} — Docker bridge network. The shared mock and runner containers are connected to it for the pipeline’s lifetime.
• DinD jobs are serialized via the _glci_dind resource group to prevent concurrent DinD jobs from conflicting on the docker service alias.
• runner-* — Transient job execution containers, spawned by the runner’s Docker executor. One per job, destroyed after the job finishes.

Pipeline execution lifecycle#

This is the sequence of events when you run glci run [jobs...]:

Detailed steps#

1. CLI sends StartPipeline to daemon via Unix socket (daemon auto-starts if not running). If another pipeline is already being prepared for the same directory, the request is queued instead of rejected — the daemon holds the HTTP connection open and starts the queued pipeline automatically when the current preparation finishes. Cancelling the request (Ctrl+C) dequeues it. A maximum queue depth of 3 prevents resource exhaustion.
2. Daemon parses .gitlab-ci.yml — offline parser resolves includes, extends, !reference tags; evaluates workflow/job rules; expands parallel: matrix:; builds execution plan (stages, DAG, auto-include upstream deps)
3. Daemon creates per-pipeline Docker network (glci-net-{id})
4. Daemon prepares source code — creates a bare git repo from the working directory (including uncommitted/untracked files in dirty mode), tar.gz for upload
5. Daemon ensures mock container is running — connects to pipeline network, registers per-pipeline state (secret), configures S3 cache credentials, uploads git repo
6. Daemon ensures runner container is running — connects to pipeline network, Docker socket or DOCKER_HOST for remote Docker
7. Scheduler dispatches jobs — concurrent within stages, DAG-aware, respects resource_group exclusions. Per job: queue on mock server with HMAC token, persistent runner polls for job, clones source, downloads artifacts, executes scripts, uploads artifacts, streams trace, reports completion
8. Trigger jobs spawn child pipelines sharing the same mock and runner containers (max depth: 2)
9. Daemon persists artifacts to ~/.glci/projects/{key}/pipelines/{id}/
10. EventBus streams events to all connected CLI clients

Mock server API#

The mock server (pkg/mockserver/) is the central hub. It implements the subset of the GitLab API that gitlab-runner needs, plus infrastructure services and an internal control API consumed by the daemon.

GitLab runner API#

These endpoints are consumed by the runner, which thinks it is talking to a real GitLab instance:

Infrastructure services#

Internal API (daemon to mock server)#

These endpoints are consumed by the daemon to manage pipelines. They are not authenticated via the runner protocol — they go through the docker exec transport:

Docker exec transport#

The daemon communicates with the mock server container exclusively through docker exec, which enables remote Docker daemon support (TCP via DOCKER_HOST, Docker contexts, SSH tunnels, cloud Docker).

Three transport implementations exist in pkg/daemon/transport.go:

The persistentExecTransport keeps a single subprocess alive, multiplexing all requests over shared stdin/stdout with the muxproto binary protocol. This avoids the overhead of spawning a new docker exec for every API call.

Muxproto binary protocol#

pkg/muxproto/ implements a length-prefixed binary protocol for multiplexing HTTP-like request/response pairs over a byte stream:

Request:  int64(methodLen) method | int64(pathLen) path | int64(ctLen) contentType | int64(bodyLen) body
Response: int32(statusCode) | int64(bodyLen) body

All integers are big-endian. Max frame size is 1 GB (matching the artifact upload limit). The transport auto-restarts on stream corruption.

Event streaming#

The mock server emits events on /internal/events as a hybrid binary/JSON stream:

• Control events (job_result, job_dispatched) are JSON lines
• Trace data uses a binary frame format to avoid JSON overhead for high-volume output: \x00 | uint64(pipelineID) | uint64(jobID) | uint32(offset) | uint32(dataLen) | data

Both event types flow on a single FIFO channel per subscriber (buffer 4096), so a job_result never overtakes an earlier trace frame. Trace drops are tolerated and reconciled via GET /internal/trace. Control drops fall back to /internal/status polling. The daemon also polls /internal/status every 5 seconds as a safety net for results missed during event stream reconnection.

EventBus#

The daemon’s EventBus (pkg/daemon/eventbus.go) manages in-memory event storage and streaming for CLI consumers:

• Two-tier storage: Status events (stage_start, job_status, pipeline_done) are kept permanently. Log events (job_log) are bounded by byte size (default 10 MB) with slice compaction.
• SubscribeWithReplay: Atomically returns replay history and registers a subscriber under a write lock, preventing event loss between replay and subscription.
• Unbounded subscriber queues: Each subscriber gets an eventQueue backed by a mutex-protected slice and a drain goroutine. Publish never blocks and never drops events. The drain goroutine feeds a 64-element output channel the consumer reads from.

Scheduler#

The scheduler (pkg/scheduler/) provides DAG-aware concurrent job dispatch with concurrency limits and resource_group support.

Scheduling modes#

Job lifecycle states#

Jobs move through these states: pending -> running -> done (or skipped). Manual jobs (when: manual) start in a separate manual state and move to pending only when triggered.

Skip logic#

• on_success jobs are skipped when the pipeline has failed and their dependencies are complete
• on_failure jobs are skipped when the pipeline has not failed and no running jobs remain that could cause a failure
• Cascading skips: if a job’s dependency was skipped, the job itself is skipped (DAG mode only)
• resource_group: only one job per resource group runs at a time

Authentication#

Pipeline-level security without a real GitLab instance:

1. Pipeline secret: 32 bytes from crypto/rand, generated per pipeline
2. Job tokens: HMAC-SHA256(secret, jobID) — each job gets a unique token
3. Runner dispatch: job routed to the runner presenting the matching token (prevents FIFO dispatch races when multiple jobs are queued)
4. Git auth: HTTP Basic with job token
5. Registry auth: Docker-standard token auth
6. Cache (S3): AWS Signature V4 with daemon-secret-derived credentials (stable across pipeline runs)

Crash recovery#

The daemon performs recovery on startup (pkg/daemon/recovery.go):

1. Find recoverable pipelines: Scan ~/.glci/active-workdirs/ markers, read execution state from ~/.glci/projects/{key}/pipelines/{id}/execution.json, check if both mock and runner containers are still alive via docker inspect.
2. Preserve resources: Build sets of container names, volume names, and network names belonging to recoverable pipelines.
3. Clean orphaned resources: Remove any glci-* containers, volumes, and networks not in the preserve sets. Also clean stale temp files.
4. Finalize stuck pipelines: Mark pipelines without surviving containers as failed in history.
5. Resume: Reconnect to containers via docker exec, query ground-truth state via GET /internal/status, reconcile with persisted state, and re-dispatch pending jobs.

Execution state is written atomically (temp file + rename) on every job state transition. It contains container names, job states, runner tokens, pipeline secret, and the original StartPipelineRequest for re-parsing CI config during resume.

CI config parsing#

glci includes a full offline .gitlab-ci.yml parser as the primary parsing path (pkg/config/). It implements GitLab’s processing pipeline in Go:

1. Load YAML with yaml.Node-based parser preserving !reference tags
2. Interpolate inputs ($[[ inputs.key ]]) before and after include resolution
3. Resolve all includes — local: (disk), remote: (HTTP), template: (GitLab CDN), project: (API), component: (API). Component version selectors (@~latest, @~N, @~N.M) are resolved to the latest matching semver tag via the GitLab Tags API before fetching. Includes are cached and deduplicated.
4. Resolve extends: — recursive deep merge with circular/depth detection (10 levels max)
5. Resolve !reference tags — path lookup with cycle detection, sequence flattening
6. Inject edge stages (.pre/.post)
7. Apply default: inheritance with inherit: control (true/false/[keys])
8. Parse workflow: rules with auto_cancel: support
9. Parse all job fields into enriched types (trigger, release, retry, environment, hooks, id_tokens, etc.)
10. Expand parallel: matrix: jobs

Also handles: rules: changes: (git diff matching), rules: exists: (filesystem glob), YAML anchors/aliases, multi-document YAML.

API-assisted mode (optional)#

Set GLCI_PREFER_API=1 to use the GitLab CI Lint API for YAML resolution. This requires a GitLab token but delegates include:, extends:, and !reference resolution to the server. The offline parser is the default because it requires no network access and handles the vast majority of CI configurations.

API Lint response — field inventory#

When using API-assisted mode, glci calls POST /projects/:id/ci/lint with include_jobs: true, dry_run: false. The API returns merged_yaml (fully resolved YAML) and a jobs array. These fields are available from each source:

Daemon API#

The daemon runs as a background process and exposes an HTTP/JSON API over a Unix socket (~/.glci/daemon.sock). The CLI is the primary consumer.

The daemon auto-starts on the first glci run and auto-stops after 30 minutes of idle. A version guard auto-restarts it if the CLI/daemon build versions mismatch.