Troubleshooting

Start with glci doctor — it runs pre-flight checks on the container engine (Docker or Podman), the daemon, your GitLab token, CI config, and git repository in one command:

glci doctor

It always prints the full report, and exits 1 when a check hard-failed — engine not reachable, Podman API socket not reachable, CI config file missing, CI config does not parse, not a git repository, or an error while checking daemon status — so you can gate on it:

glci doctor && glci run

A stopped daemon, a missing GitLab token, and ignored project-config keys are reported as warnings (~) and keep exit 0: the daemon starts automatically on first use, a token is only needed for remote include: and private registries, and dropped keys are already neutralized. Remediation for a failing row is printed underneath it as an unsymboled → ... continuation line, so it never reads as an extra warning. The report is colored only when stderr is a terminal and NO_COLOR is unset. See glci doctor.

Daemon logs#

Check the daemon log first when something goes wrong:

glci daemon logs        # last 50 lines
glci daemon logs -F     # follow in real time
glci daemon logs -n 0   # full log

Or read directly:

cat ~/.glci/daemon.log
tail -f ~/.glci/daemon.log

Daemon won’t start#

If glci run fails with “daemon did not start within 30s”:

Force a clean restart:

glci daemon stop --force
glci daemon start

Daemon crashes or misbehaves#

The daemon automatically recovers from crashes on startup. If issues persist:

glci daemon status
glci daemon stop --force
glci daemon start

# Nuclear option: clean all daemon state
glci daemon stop --force
rm -rf ~/.glci/daemon.pid ~/.glci/daemon.sock ~/.glci/daemon.log
glci daemon start

A resumed pipeline warns about the commit it dispatches against#

daemon: resume: warning: no commit recorded for this pipeline, dispatching against "..." from CI_COMMIT_SHA; jobs may fail to check it out
daemon: resume: warning: recorded commit "..." is not a commit id, dispatching against "..." from CI_COMMIT_SHA; jobs may fail to check it out
daemon: resume: warning: recorded ref "..." is not a ref name, dispatching against "..." from CI_COMMIT_SHA; jobs may fail to check it out

glci does not hand the runner your working tree’s HEAD. It serves a rewritten repository, so the commit jobs check out is a synthetic one, and the daemon records which commit and ref that was in order to resume against the same ones.

Any form of the warning means that record is unusable: the pipeline was prepared by a glci old enough not to write it, its bare repository was never built, or one of the recorded values was rejected as malformed. The commit and the ref are restored together or not at all, so a bad ref discards the recorded commit as well. The resumed jobs fall back to CI_COMMIT_SHA, which the served repository may not contain, in which case they fail in Getting source from Git repository with fatal: unable to read tree.

Start the pipeline again with glci run instead of resuming it.

Docker image missing or stale#

If glci run fails with glci Docker image not available:

make docker

After upgrading, always re-run make docker so the local image matches your binary. glci does not verify that the image matches the binary’s commit — it only picks the first image it finds, per the order below — so a stale image is not detected for you.

How the image is resolved#

glci picks its own container image in this order, using the first one it finds:

1. registry.gitlab.com/gitlab-org/ci-cd/runner-tools/glci:<commit> — the commit the running binary was built from, locally then pulled. Skipped entirely when the binary carries no commit (a plain go build) or a -dirty one (see below). Released binaries normally stop here, since CI publishes a tag per commit.
2. glci:local — a locally built image from make docker (local only; never pulled). Preferred over :latest because it matches the tree it was built from, while :latest can lag behind a new glci internal subcommand the in-container daemon depends on.
3. :latest — locally, then pulled.

Which step you land on depends on how the binary was built:

• make build / make install stamp the commit, so resolution starts at step 1. make docker tags the image with that same commit, so a make docker && make install pair matches exactly at step 1 — meaning docker ps shows the registry-qualified tag, not glci:local.
• A modified working tree is stamped <sha>-dirty. The stamp uses git status --porcelain, so untracked files count as dirty too — a single new file is enough. glci treats a -dirty stamp as unpublished and skips step 1 altogether: it never probes or pulls a -dirty tag, it goes straight to your glci:local, which is the image built from that same tree. No warning is logged, because that is the intended outcome. (make docker in a dirty tree does also tag its output <registry>:<sha>-dirty, but resolution never consults that tag.)
• Plain go build / go install leave the commit empty, so resolution starts at step 2.
• Released binaries normally match at step 1. If their commit tag is unreachable — offline, registry down, tag garbage-collected — they fall through to step 2 and will use a glci:local left over from an earlier make docker, which may be much older than the binary. glci logs daemon: warning: ... falling back to glci:local, which may not match this binary to ~/.glci/daemon.log when this happens. Only a binary stamped with a non-dirty commit can trigger it.

To force the published image instead of a local build, note that make docker applies three tags to the same image: glci:local, <registry>:<commit>, and <registry>:latest. docker rmi glci:local only drops one of them, so resolution finds the identical local build again at step 3 (or step 1, if the commit tag is still there). Untag all of them, then restart the daemon (the resolved image is cached for the daemon’s lifetime):

docker rmi $(docker image inspect glci:local --format '{{join .RepoTags " "}}')
glci daemon stop   # the next `glci run` starts a fresh one and pulls a published image

A bare docker pull <registry>:latest moves the :latest tag to the published image but leaves glci:local and the commit tag pointing at the local build, and both of those win over step 3 — so pulling alone does not switch you over. Under Podman, use podman rmi / podman image inspect; glci drives whichever engine [docker] engine selects, and its own error hints are printed with that engine’s binary name.

Pointing the image somewhere else#

The order above assumes registry.gitlab.com is reachable. Where it is not, set [images] in ~/.glci/config.toml rather than retagging images by hand:

[images]
registry = "proxy.corp.internal/mirror"     # rewrites the host, keeps the chain
# glci   = "proxy.corp.internal/tools/glci:latest"   # or name it outright

registry only swaps the registry host, so the commit-exact tag is still preferred. A per-image glci key is used verbatim — no tag guessing, no :latest fallback, and no glci:local.

The resolved sidecar reference is cached for the daemon’s lifetime, so run glci daemon stop after changing [images] glci or [images] registry — the next glci run starts a fresh daemon and re-resolves. ([images] upstreams is the exception: changing it recreates the mock container on the next run by itself.)

A digest-pinned [images] glci is accepted for Linux but refused when a named runner needs the Windows variant, since a digest cannot name it — use a tag there.

The same section covers the other four images glci pulls for itself (the runner, the helper, the utility image, and binfmt). Retagging a mirrored image to the reference glci expects does still work — glci checks locally before pulling — but it is undone by glci system prune, has to be repeated on every Docker host, and a leftover glci:local silently shadows the published image on the next upgrade. [images] has none of those failure modes. Run glci doctor to see what each of the five resolved to and why.

glci refuses to start: [images] ... is unusable#

Some [images] values are refused outright rather than warned about, because resolving them produces a plausible-looking reference pointing somewhere you did not mean — and glci runs these images with the container engine’s socket mounted. The run aborts before anything is pulled:

[images] in the global config is unusable: images.registry: "acmecorp" has no registry host, so it would resolve against Docker Hub — write it as a host and path, e.g. "proxy.corp.internal/mirror"

These are separate from the [images] warnings, which are shown by glci doctor and glci config but do not stop a run:

Podman#

glci can drive Podman instead of Docker. Select the engine in ~/.glci/config.toml ([docker] engine = "podman", or per-runner [runners.<name>] engine) — there is no --engine flag, and the key is ignored in a project .glciconfig.toml. glci doctor reports the active engine (Docker or Podman); for Podman it also checks the Docker-compatible API socket separately and hints how to start it if it isn’t running.

“cannot connect to Podman” / socket not running#

What you see: Pipelines fail to start with connection-refused errors, or glci doctor reports Podman is selected but unreachable.

Cause: glci talks to Podman over its Docker-compatible API socket, which is not running by default. podman info succeeds without it, which is why glci doctor probes the socket as its own check. Start podman system service:

# Rootless (recommended)
systemctl --user enable --now podman.socket

# Rootful
sudo systemctl enable --now podman.socket

# macOS / Windows
podman machine start

The socket lives at $XDG_RUNTIME_DIR/podman/podman.sock (rootless) or /run/podman/podman.sock (rootful); glci mounts it into the runner at /var/run/docker.sock. On Linux, glci doctor dials every candidate path (an explicit CONTAINER_HOST, then the rootless path, then the rootful one) and reports whichever is reachable, so a rootless setup is never pointed at a rootful socket it doesn’t use. The exception is a remote endpoint: when the resolved Podman endpoint is not a local unix:// path — CONTAINER_HOST, or failing that the default podman system connection, pointing at ssh:// or tcp:// — the socket is on the far side, so nothing on this machine can dial it and doctor reports “cannot verify from host” (a warning, not a failure) even on Linux.

Podman on macOS / Windows (podman machine)#

podman machine is supported. glci resolves the default connection from podman system connection list and exports both CONTAINER_HOST (the ssh:// URI) and CONTAINER_SSHKEY (that connection’s SSH identity). The identity matters: setting CONTAINER_HOST overrides the named connection podman would otherwise read the key from, so without CONTAINER_SSHKEY every later podman call against a healthy machine fails as “not reachable” (exit 125).

The socket glci mounts into containers is the path the machine connection reports — the path on the side the containers actually run on. A default podman machine is rootless inside the VM, so that is /run/user/<uid>/podman/podman.sock. The rootful /run/podman/podman.sock is only used as a fallback and only exists with podman machine init --rootful; mounting it on a rootless machine fails the container with statfs /run/podman/podman.sock: permission denied.

glci doctor reports the Podman socket as “cannot verify from host” on macOS and Windows — and on Linux too whenever the resolved endpoint is remote (ssh://, tcp://), since the socket is equally unreachable from this machine. That is expected, not an error: the check is a warning and does not affect doctor’s exit status. Verify the machine itself with podman machine list.

Docker unreachable but Podman is installed#

What you see: glci doctor fails the engine check with Docker not reachable and warns that Podman is installed.

Cause: Docker is the selected engine (explicitly, or because the running daemon resolved it at startup) but its socket is down, while the podman binary is on PATH. With the engine unset, auto-fallback only kicks in when Podman is actually reachable — an installed Podman whose API socket is down still lands here.

Fix: Either start Docker, or switch engines:

# ~/.glci/config.toml
[docker]
engine = "podman"

Then make sure the Podman API socket is running (see above). If Podman is not installed, doctor instead hints to start Docker (Docker Desktop, colima start, or sudo systemctl start docker).

glci doctor warns about the engine config#

The daemon resolves its engine once at startup, from the global config only. If [docker] engine in ~/.glci/config.toml changed after the daemon started, doctor warns that the running daemon still uses the old one — run glci daemon stop and it re-starts with the new engine on the next command.

A command stops with loading glci config: ... toml: line N#

What you see:

loading glci config: /path/to/repo/.glciconfig.toml: toml: line 2 (last key "skip.jobs"): expected a comma (',') or array terminator (']'), but got end of file

Cause: the named config file is not valid TOML. glci refuses to run rather than proceed with the skip rules, pipeline presets, job overrides and [gitlab] settings the file asked for silently missing.

The same message names ~/.glci/config.toml when the global file is the broken one.

Fix: correct the reported line. For the project file, glci config edit opens it even while it is broken; the global ~/.glci/config.toml has to be edited directly, since config edit never opens it. glci config show prints the same error until it parses, and glci doctor reports it as a failing Project config or Global config row and exits 1.

A broken project file no longer takes a valid global one down with it: whatever parsed is kept, so a dropped error can no longer revert your ~/.glci/config.toml settings (mock server port, Docker host) to built-in defaults behind your back. Resuming an interrupted pipeline stops as well, rather than running the remaining jobs without the [token], [skip] and [manual] rules the project file was holding them to. See Malformed config files.

glci says it is “ignoring” keys from .glciconfig.toml#

What you see: a line like

glci: warning: .glciconfig.toml: ignoring [docker] engine, [docker] privileged, [runners.gpu] docker_host — these settings are only honored in ~/.glci/config.toml, because a project file travels with the repository

glci doctor shows the same text as a ~ Project config row instead of a stray line.

Cause: the keys that decide which container endpoint glci talks to, how far it trusts that endpoint, and how much of the host it exposes to containers are global-config-only. A project config travels with the repository, so honoring them would let a cloned repo silently redirect glci to a container daemon of its choosing — along with your working tree, CI variables, and job token — or bind-mount your filesystem into every job. The affected keys are:

privileged applies to the runner container that holds the engine socket, and the three [paths] container dirs are pasted into gitlab-runner’s [runners.docker] volumes list, which accepts host:container binds — so a project container_cache_dir = "/:/hostroot" would mount your whole filesystem into every job.

Fix: move those keys to ~/.glci/config.toml. Everything else in the project config still applies. See Container-endpoint keys.

paths.container_cache_dir: ... is not a container path#

What you see:

paths.container_cache_dir: "/:/hostroot" is not a container path — it must be an absolute path such as "/cache"; a "host:container" value would bind-mount a host directory into every job container

Cause: container_builds_dir, container_cache_dir, and container_certs_dir end up in gitlab-runner’s [runners.docker] volumes list, which also accepts Docker’s host:container bind syntax. A value containing a colon is therefore a host bind mount, not a mount point, and is rejected wherever it came from.

Fix: use a plain absolute container path (/cache, /builds). container_certs_dir also accepts the Docker named-volume form (certs-vol:/certs), and Windows container paths (C:\builds) are accepted as-is.

A named runner still uses the global endpoint or template#

What you see: a project .glciconfig.toml redefines a runner that also exists in ~/.glci/config.toml, but its jobs keep going to the global docker_host/docker_context — and no key was reported as ignored.

Cause: for a name present in both files every field the global entry owns is carried over — the endpoint fields (engine, docker_host, docker_context, container_socket, tls_cert_path, tls_verify) and the config template — so a project entry cannot erase them either. A bare [runners.gpu] in a project file would otherwise move gpu-tagged jobs onto the local engine with no ignored key to warn about. A project config_template is used only where the global entry has none; when both set one, the global wins and the project’s is named in the ignored-keys warning.

Fix: change the endpoint in ~/.glci/config.toml, or use a different runner name for the project-local runner. See Runner defined in both config files.

Token forwarding rules in .glciconfig.toml are ignored#

What you see: [token] forward_host_token, [token] jobs/job_patterns/stages, or [pipelines.<name>.token] appear in the ignoring warning.

Cause: [token] decides whether your real GitLab token is substituted into CI_JOB_TOKEN for scripts the repository wrote, so a project config may only narrow it. A project forward_host_token = false is always honored; a project true is dropped when your global config explicitly set false; project filters are honored only when your global config sets none (two sets of regexes cannot be intersected). A project pipeline preset’s [token] section is dropped entirely if it would turn forwarding back on, since a selected preset’s [token] replaces the top-level one.

Fix: set the forwarding rules you want in ~/.glci/config.toml, or use --no-token for a one-off run. See Token forwarding in a project config.

Engine selection and auto-fallback#

With the engine unset, glci tries Docker first and auto-falls back to Podman when Docker is unreachable but Podman is. The fallback notice is printed to the terminal the first time the daemon starts (and logged in the daemon log); set engine = "podman" explicitly to silence it. An explicit engine = "podman" never falls back to Docker — if the Podman socket is down, the pipeline fails until you start it.

Note that engine is separate from [defaults] executor: the generated gitlab-runner config always uses executor = "docker" even under Podman, because gitlab-runner has no Podman executor. Do not set executor = "podman".

Docker-in-Docker under Podman#

Rootful Podman runs DinD jobs fine (privileged mode defaults on). Rootless Podman DinD is best-effort and unsupported — privileged nested containers and the docker service alias may not work. Use rootful Podman (or Docker) for pipelines that rely on DinD.

Segmentation faults on Apple Silicon (Colima)#

If jobs crash with signal: segmentation fault (core dumped) from Go toolchain binaries, Colima needs Rosetta enabled:

colima stop
colima delete  # needed if changing --vm-type
colima start --vm-type=vz --vz-rosetta --cpu 12 --memory 16

Pipeline hangs or won’t cancel#

glci stop <pipeline-id>

glci stop <id> handles orphaned pipelines automatically — if the daemon lost track of a pipeline (e.g., after a restart), it force-removes leftover containers and the pipeline network, then marks it as canceled in history. Restarting the daemon is only needed if force-stop itself doesn’t resolve the issue:

glci daemon stop
glci daemon start

A log line appears twice when attaching to a running job#

What you see: glci log <pipeline-id> <job-name> on a job that is still running prints one line (rarely a few) twice, right where the catch-up output ends and live output begins.

Cause: glci prints the trace recorded so far, then switches to the live stream. A line written at that exact moment is in both.

This is normal, and deliberate: the alternative ordering would drop such a line instead of repeating it. Reading the log again once the job has finished prints the stored trace, which has no such seam.

“Waiting for pipeline preparation to finish…”#

What you see: Running glci run shows Waiting for pipeline preparation to finish... and does not start immediately.

Cause: The daemon serializes pipeline preparation per directory. Another glci run in the same project is already being prepared, so your request is queued until it finishes.

This is normal. The pipeline will start automatically once the earlier preparation completes. Press Ctrl+C to cancel if you don’t want to wait.

Config template errors#

Template parse error#

What you see: runner.config_template: invalid Go template: ... or runners.<name>.config_template: invalid Go template: ... in glci config output.

Cause: The Go template syntax is invalid (unclosed braces, unknown functions, etc.).

Fix: Check your template syntax. Common mistakes:

Template render error#

What you see: rendering custom template for runner "<name>": ... in daemon logs.

Cause: The template references a field that doesn’t exist in the template context.

Fix: Use only fields from the template variable reference. Common fields: .URL, .Executor, .DefaultImage, .PullPolicy.

Key not allowed in a project config template#

What you see: the pipeline fails with

daemon: /path/to/repo/.glciconfig.toml [runners.gpu] config_template sets [runners.docker] host, which a project config may not change because it controls where and how jobs run; move this setting to ~/.glci/config.toml

The message names the config file, the section and the key you wrote, so it points at the exact line to edit. For the default runner the section reads [runner] config_template; glci never says runner "default" or [runners.default], which are internal names you cannot edit.

Cause: a .glciconfig.toml travels with the repository, so its config_template is untrusted input. After rendering, it may not set keys that choose the container endpoint, grant host or privilege access, share namespaces, redirect name resolution, or run code outside the job script. The config_template_file form is checked identically — a path in the project file loads project-controlled content just the same.

Fix: move that key to your own ~/.glci/config.toml, where templates are unrestricted, or drop it. The full list is in restricted keys in a project config template.

Named runner name invalid#

What you see: runners.<name>: name must match [a-zA-Z0-9][a-zA-Z0-9_-]* in glci config output.

Fix: Runner names must start with a letter or digit and contain only letters, digits, hyphens, and underscores.

Jobs not routing to named runner#

What you see: A job runs on the default runner instead of the named runner.

Cause: None of the job’s CI tags: match any named runner name. Tag matching is exact and case-sensitive.

Fix: Ensure the job has a tags: entry that exactly matches the runner name defined in [runners.<name>]:

# .gitlab-ci.yml
my-job:
  tags: [gpu]    # must match [runners.gpu] in config
  script: echo "runs on GPU runner"

Docker networking issues#

Each build gets its own Docker network via FF_NETWORK_PER_BUILD. Job containers reach the mock server via extra_hosts (host-gateway). The runner and mock containers share the per-pipeline network for Docker DNS resolution. Problems here usually surface as connection timeouts or host resolution failures inside jobs.

Container can’t reach the mock server#

What you see: Jobs fail with errors like connection refused, no such host, or could not resolve host when trying to reach $CI_SERVER_URL or $CI_REGISTRY.

Debugging steps:

# List pipeline networks
docker network ls --filter name=glci-net-

# Inspect a specific pipeline network — look for Containers section
docker network inspect glci-net-<pipeline-id>

# Check if the mock server container is running and attached
docker ps --filter name=glci-mock

DNS resolution failures inside jobs#

What you see: getaddrinfo or DNS resolution failed errors for external hosts (e.g., registry.gitlab.com, github.com).

Cause: The job container’s DNS resolver can’t reach an upstream DNS server. Common with custom Docker networks and restricted host DNS configs.

Fix: Check the host’s /etc/resolv.conf or Docker’s DNS settings. If using Colima, restart with --dns to override:

colima start --dns 8.8.8.8 --dns 1.1.1.1

Or configure extra hosts in .glciconfig.toml to bypass DNS for known hosts:

[network.extra_hosts]
entries = ["internal-registry.corp:10.0.0.50"]

Port conflicts#

What you see: address already in use errors in the daemon log when starting a pipeline.

Cause: The mock server requires port 39741 (default) on the Docker host. This port may be taken by another process or a previous daemon that didn’t shut down cleanly. Registry listeners can also conflict if their bind addresses overlap.

Fix:

# Find what's using the port
lsof -i :39741

# Force-restart the daemon to clean up stale listeners
glci daemon stop --force
glci daemon start

You can change the mock server port or registry bind addresses in ~/.glci/config.toml:

[network]
mock_server_port = 39741           # default; change requires daemon restart
registry_bind = "127.0.0.1:0"      # HTTPS listener
registry_http_bind = "0.0.0.0:0"   # HTTP listener

Note: Changing mock_server_port requires a full daemon restart to take effect.

Token & authentication failures#

GitLab token not found#

What you see: gitlab token not set (set GITLAB_TOKEN env var or use --token) or no GitLab token or project configured, using offline parser.

Cause: No token is available, so glci falls back to the offline parser. This means include: project:, include: component:, and remote CI/CD variable fetching are all disabled. The same applies to every config glci parses, not just .gitlab-ci.yml – a child pipeline’s config and a cross-project trigger’s target config resolve their own includes too, and lose the same ones (see Child pipeline is missing jobs).

Fix: Configure a token using any of these methods (first match wins):

# Option 1: environment variable
export GITLAB_TOKEN="glpat-xxxxxxxxxxxxxxxxxxxx"

# Option 2: glab CLI (token is picked up automatically)
glab auth login

# Option 3: config file
cat >> .glciconfig.toml <<'EOF'
[gitlab]
token = "$GITLAB_TOKEN"   # env var references are expanded
EOF

# Option 4: command-line flag
glci run --token "glpat-xxxxxxxxxxxxxxxxxxxx"

To see which includes actually resolved, run glci merged – it prints the fully-resolved config on stdout and reports every skipped include on stderr.

For self-managed GitLab instances, also set the URL:

export GITLAB_URL="https://gitlab.example.com"
# or in .glciconfig.toml:
# [gitlab]
# url = "https://gitlab.example.com"

A URL set in the project config only gets a token that the same project config supplies (or an explicit --token) — tokens from GITLAB_TOKEN, glab, and ~/.glci/config.toml are withheld from it (next entries). Set both keys together in .glciconfig.toml, or set the URL via GITLAB_URL / ~/.glci/config.toml / --gitlab-url instead.

Child pipeline is missing jobs#

What you see: a child pipeline runs and passes, but jobs a shared template defines never appear. glci jobs -f child.yml lists them; the child pipeline does not. Or the trigger job fails with extends target ".template-job" not found (referenced by "job").

Cause: the child’s own config uses include: project: or include: component: and no token is configured, so the include is skipped. Whatever it defined is gone – silently if nothing referenced it, as a parse failure if a job extends: one of its definitions.

Both cases log a warning, in the trigger job’s output and in the daemon log:

skipping project include my-group/my-templates: no GitLab token configured

Fix: configure a token (above). If a token is configured and the include still skips, the URL may be withheld from it – see the next entry. To confirm what the child should contain, parse it at top level:

glci jobs -f child.yml

glci merged -f child.yml shows what the child’s includes resolved to, and names the skipped ones on stderr.

GitLab token withheld: “url comes from the project config”#

What you see: glci: warning: [gitlab] url "..." comes from the project config, which travels with the repository, so the token from your <source> was withheld, token = <withheld> in glci config --gitlab, and no include: project:/component: resolution. The <source> is whichever one was dropped: environment (GITLAB_TOKEN / GITLAB_PRIVATE_TOKEN), glab credential-helper, glab CLI config, or global config file (~/.glci/config.toml).

Cause: [gitlab] url is set in the project .glciconfig.toml. That file arrives with the repository, so glci sends a token to that URL only when the repository named the token too — [gitlab] token in the same project file — or when you named it yourself with --token. Everything else is withheld, including [gitlab] token from ~/.glci/config.toml: that is your credential for your own instance, so sending it to a repository-chosen host leaks it just as an environment or glab token would. This is deliberate, not a resolution bug.

Fix: opt in explicitly for that instance, or move the URL to your own config:

# Option 1: name the token for this invocation
glci run --token "$GITLAB_TOKEN"

# Option 2: pair url and token in the config file
cat >> .glciconfig.toml <<'EOF'
[gitlab]
url = "https://gitlab.example.com"
token = "$GITLAB_TOKEN"
EOF

# Option 3: set the URL yourself instead of letting the repo set it
export GITLAB_URL="https://gitlab.example.com"   # or ~/.glci/config.toml, or --gitlab-url

Check the URL against the instance you expect before doing any of this — the warning is how you find out that a repository tried to redirect glci. Option 2 pairs the project URL with an env-ref token, which glci reports separately (next entry). See [gitlab] url in a project config.

“the project config sends $VAR to [gitlab] url”#

What you see:

glci: warning: the project config sends $GITLAB_TOKEN to [gitlab] url "https://gitlab.example.com" — both the variable and the URL come from this repository, so check that you trust it with that value.

Cause: the project .glciconfig.toml sets both [gitlab] url and an env-ref token (token = "$GITLAB_TOKEN"), so the repository picked the host variable to read and where its value is sent. Nothing is blocked — this is the documented way to keep secrets out of the file, and a team’s own config legitimately does it — the request is authenticated as normal. The variable name is printed; its value never is.

Fix: nothing, if the config is your team’s own. Otherwise verify the URL before trusting it with that variable, and drop the [gitlab] block from the project config (set the URL and token in ~/.glci/config.toml, or pass --gitlab-url/--token) if you do not. See [gitlab] url in a project config.

CI_JOB_TOKEN permissions differ from production#

What you see: API calls inside jobs succeed locally but fail in real CI (or vice versa), because CI_JOB_TOKEN in glci is actually your personal access token (with different scopes).

Cause: glci forwards your host GitLab token as CI_JOB_TOKEN. In production, CI_JOB_TOKEN is a short-lived token scoped to the job.

Fix:

# Disable token forwarding entirely
glci run --no-token

# Or test with reduced secrets
glci run --secrets none

“API returned 401” or “API returned 403” when fetching variables#

What you see: Warnings in daemon logs like could not fetch project variables: API returned 401.

Instance-level CI/CD variables aren’t applied#

What you see: A variable defined as an instance-level CI/CD variable on a self-managed GitLab instance isn’t set in your jobs, or an include: that references it (e.g. project: $_GITLAB_TEMPLATES_REPO) fails with HTTP 404 because the variable wasn’t expanded.

Cause: glci fetches instance variables from GET /admin/ci/variables, which requires an administrator token. With a non-admin token GitLab returns 403 and glci skips them (could not fetch instance variables in the daemon log). Project and group variables are unaffected. Separately, remote variables are fetched in parallel with config parsing, so even an admin-fetched instance variable is not available for include: path expansion.

Fix: Supply the variable locally so it is available everywhere, including include resolution:

# .glci.env (gitignored), or --env KEY=VALUE, or a pipeline preset
_GITLAB_TEMPLATES_REPO=project/gitlab_templates
_GITLAB_TEMPLATES_REF=1.0.x

Instance variables are fetched under --secrets all (the default for glci run). See Variables & Secrets for the full precedence order and include-expansion behavior.

Secrets cache is stale#

What you see: Variable values don’t match what you see on GitLab, even after updating them.

Cause: Remote variables are cached in daemon memory for 6 hours by default.

Fix:

# Force a fresh fetch for this run
glci run --refresh-secrets

Or change the TTL in .glciconfig.toml:

[gitlab]
secrets_ttl = "0"   # disable caching entirely

Registry & image issues#

Push fails with “unknown blob” or “manifest invalid”#

What you see: docker push $CI_REGISTRY_IMAGE fails with errors about unknown blobs or invalid manifests.

Cause: The embedded registry lost its blob storage (e.g., after a docker volume rm glci-registry or glci system prune --all).

Fix:

# Rebuild and re-push — the registry volume was wiped
glci registry clean
glci run

“image blobs not found in local registry”#

What you see: glci registry pull fails with image blobs not found in local registry (image may have been proxied from upstream without caching blobs — re-push the image to persist it).

Cause: The image was pulled through the registry as a read-through cache hit. The manifest is stored locally but the blobs were streamed directly from upstream. Only images explicitly pushed to $CI_REGISTRY have their blobs stored.

Fix: Re-push the image from your pipeline (use docker push $CI_REGISTRY_IMAGE/...) so blobs are stored locally.

Insecure registry / TLS certificate errors#

What you see: x509: certificate signed by unknown authority or server gave HTTP response to HTTPS client when pulling from or pushing to the embedded registry.

Cause: The embedded registry uses a self-signed CA. glci automatically configures trust, but some scenarios break it:

Cross-platform image issues#

What you see: exec format error when running containers, or build failures with wrong-architecture binaries.

Cause: The image was built for a different CPU architecture (e.g., amd64 image on Apple Silicon).

Fix:

# Ensure QEMU binfmt handlers are registered (glci does this automatically)
docker run --privileged --rm docker.io/tonistiigi/binfmt:latest --install all

# For Colima with Rosetta (preferred for Apple Silicon)
colima stop
colima delete
colima start --vm-type=vz --vz-rosetta --cpu 12 --memory 16

If glci daemon logs shows warning: could not install QEMU binfmt handlers, QEMU registration failed. Run the docker run --privileged command above manually.

Note on glci’s own image: glci images built before this was fixed shipped an amd64 binary on every platform, because the Dockerfile put defaults on the BUILDPLATFORM and TARGETARCH build args and those defaults won over the requested platform. Two variants of the same bug:

• Published images — the linux/arm64 entry of a multi-arch tag contained the amd64 binary despite the manifest advertising arm64, so on arm64 hosts it ran under emulation or failed outright. Re-pull :latest to pick up a corrected build:

  docker rmi registry.gitlab.com/gitlab-org/ci-cd/runner-tools/glci:latest
  docker pull registry.gitlab.com/gitlab-org/ci-cd/runner-tools/glci:latest
  

  Re-pulling only helps if you track a moving tag. A pinned :<sha> or release tag is immutable — the image published for it stays amd64 forever, so move to a newer tag (or build locally with make docker) instead.

• Local make docker images — same cause, no registry involved: make docker runs a plain docker build with no --platform, so on Apple Silicon it also produced an amd64 glci:local. Re-running make docker now produces a native image.

To check any image yourself — 62 is x86-64, 183 is AArch64:

docker run --rm --entrypoint od \
  registry.gitlab.com/gitlab-org/ci-cd/runner-tools/glci:latest \
  -An -tu1 -j18 -N1 /usr/local/bin/glci

That inspects the variant your daemon picks for your own platform. Add --platform linux/amd64 or --platform linux/arm64 to inspect a specific entry of a multi-arch tag — running a non-native one needs the QEMU binfmt registration above.

$CI_REGISTRY images fail with wrong platform#

What you see: Jobs that use $CI_REGISTRY/... as their image fail with no matching manifest for linux/arm64/v8 (or another platform) even though the image exists on the real registry.

Pulling docker image 127.0.0.1:32768/group/project/image:v1.0 ...
ERROR: Job failed: failed to pull image "127.0.0.1:32768/group/project/image:v1.0"
  with specified policies [if-not-present]: Error response from daemon:
  no matching manifest for linux/arm64/v8 in the manifest list entries

Cause: $CI_REGISTRY resolves to the embedded registry (127.0.0.1:<port>), which proxies the image from upstream. When the upstream image is a multi-arch manifest list, the local Docker daemon selects its native platform (e.g., linux/arm64 on Apple Silicon), but the image may only provide linux/amd64 manifests.

This is common with projects that reference $CI_REGISTRY images to avoid external dependencies.

Fix: Two approaches:

1. Push multi-arch images — if you control the upstream images, build and push them as multi-arch manifests (e.g., with docker buildx build --platform linux/amd64,linux/arm64). This fixes the problem at the source for all consumers.

2. Use per-job field overrides to force the correct platform without touching .gitlab-ci.yml:

# .glciconfig.toml

# Nested form
[jobs."renovate_validate".image.docker]
platform = "linux/amd64"

# Or flat dotted key form (equivalent)
[jobs."renovate_validate"]
"image.docker.platform" = "linux/amd64"

# Or override the image entirely to bypass the registry proxy
[jobs."renovate_validate".image]
name = "registry.gitlab.com/group/project/image:v1.0"

# Or apply to every job in the project with a glob pattern
[jobs."*".image.docker]
platform = "linux/amd64"

Push-through mirror failures#

What you see: docker push succeeds locally but the image doesn’t appear on the upstream registry, or pushes fail with setting push-through config: HTTP 4xx/5xx.

# .glciconfig.toml
[registry]
push_through = true

[registry.upstream]
username = "deploy-token"
password = "$REGISTRY_WRITE_TOKEN"

Remote Docker issues#

When DOCKER_HOST points to a remote machine (TCP or SSH), the daemon and Docker daemon are on different hosts. This changes how bind mounts, networking, and port forwarding work.

Relay proxy containers#

When a named runner targets a Docker daemon on a different machine, glci deploys a relay proxy container (glci-proxy-*) on the remote host to bridge mock server communication. The relay runs whichever image resolution settled on — normally the registry-qualified :<commit> or :latest tag, which glci pulls on the remote daemon if it isn’t there. If resolution landed on glci:local, nothing can be pulled: that tag exists only on the machine that built it and no registry serves it.

What you see: Jobs on the remote runner fail with connection refused or response is not application/json when trying to reach the mock server.

Possible causes:

• The glci image is missing on the remote host. A registry-qualified tag is pulled automatically; a glci:local has to be copied over: docker save glci:local | docker --context <remote> load. Alternatively build a commit-stamped binary (make install) at a pushed commit so glci resolves a pullable tag in the first place
• The relay container crashed — check docker ps -a on the remote host for exited glci-proxy-* containers
• Protocol mismatch after upgrading glci — old relay containers may use an incompatible muxproto format. Clean up all glci containers on the remote host: docker rm -f $(docker ps -aq --filter name=glci-)

Cleanup: If relay containers are left behind after a crash, remove them on the remote host:

docker --context <remote> rm -f $(docker --context <remote> ps -aq --filter name=glci-proxy-)

Bind mount failures#

What you see: Job containers start but files are missing, or mounts fail with no such file or directory.

Cause: Bind mounts reference paths on the Docker host, not your local machine. When Docker is remote, /Users/you/project doesn’t exist on the remote host.

Fix: glci works around this by uploading your project as a tarball into the mock server container, so standard job execution works. But if you have custom volume mounts in your CI config, they will reference remote paths.

Localhost port forwarding doesn’t work#

What you see: Jobs try to reach 127.0.0.1:<port> for the mock server or registry, but get connection refused.

Cause: 127.0.0.1 inside a container on the remote Docker host refers to that container’s loopback, not your local machine. The mock server runs as a container on the remote host and is reachable by container name on the pipeline network, not via localhost.

Fix: This should work automatically — glci connects mock and runner containers to the same Docker network. If it doesn’t, check glci daemon logs for network errors.

Wrong Docker endpoint#

glci resolves the Docker host at daemon startup with this priority:

1. [docker] host in ~/.glci/config.toml (highest; ignored in a project .glciconfig.toml — see Container-endpoint keys)
2. DOCKER_HOST environment variable
3. Docker context (docker context use / DOCKER_CONTEXT)
4. Default Docker socket

DOCKER_HOST outranks the active context, matching the docker CLI — so an endpoint left over in your shell wins over docker context use. Under Podman the order is [docker] host, then CONTAINER_CONNECTION, then CONTAINER_HOST, then the default connection (Podman itself ranks the connection above the host variable).

If glci reaches an unexpected daemon:

glci daemon status                      # engine endpoint + where it was resolved from
env | grep -E 'DOCKER_HOST|DOCKER_CONTEXT|CONTAINER_HOST|CONTAINER_CONNECTION'
unset DOCKER_HOST && glci daemon stop   # fall back to the context; stop applies it

The glci daemon stop is required: the running daemon keeps the endpoint it resolved at startup, so unsetting the variable alone changes nothing. glci warns when it notices the mismatch — including the case where you removed a selection the daemon still has — but it never restarts the daemon for you, because that would let two shells with different DOCKER_HOST values fight over one daemon.

Set the host explicitly in config to override all auto-detection:

# ~/.glci/config.toml
[docker]
host = "ssh://my-server"

glci config show --network prints the configured [docker] host value only — it does not reflect DOCKER_HOST, the active context, or the engine default. Use glci daemon status or glci doctor for the endpoint actually in use.

Variable resolution issues#

Variables not resolving#

What you see: Job scripts contain literal $MY_VARIABLE instead of its value, or variables are empty.

Cause: The variable isn’t defined at any level, or it’s defined at a lower-precedence level and overridden to empty.

Variable precedence (lowest to highest):

Debugging steps:

# Override a specific variable for testing
glci run --env MY_VARIABLE=test_value

# Use --secrets none to test without remote variables
glci run --secrets none --env MY_VARIABLE=test_value

Dotenv variables not appearing in downstream jobs#

What you see: A producer job creates a dotenv report artifact but the consumer job does not have the expected variables.

Note: needs: { job: producer, artifacts: false } blocks file artifact downloads but dotenv variables still propagate — matching GitLab CI behavior.

.glci.env not loading#

What you see: Variables defined in .glci.env are not available in jobs.

Example .glci.env:

MY_SECRET=s3cr3t
DEPLOY_TOKEN=glpat-xxxx
DB_PASSWORD=hunter2

--secrets none still shows some variables#

What you see: Variables like CI_REGISTRY, CI_PROJECT_PATH, etc. are present even with --secrets none.

Cause: --secrets none only disables fetching remote variables from the GitLab API (project and group variables). CI-derived variables (git info, registry URLs) and YAML-defined variables are always resolved.

Child pipeline & include resolution#

Child pipeline include not found#

What you see: the trigger job fails with resolving child includes for "<job>": child pipeline include "child.yml": stat /<repo>/child.yml: no such file or directory, or child pipeline include "...": not a path in the repository ..., for a file that is clearly there next to the CI file. The path in the message is the one glci actually tried.

Cause: trigger: include: paths resolve from the repository root, which is where GitLab reads them from — not from the directory holding the CI file or the one you ran glci in. A CI file in a subdirectory that names child.yml is asking for <repo-root>/child.yml.

Fix: write the path as GitLab reads it, from the repository root:

# apps/web/.gitlab-ci.yml
child:
  trigger:
    include: apps/web/ci/child.yml   # not ci/child.yml

Leading slashes are stripped and mean the same thing (/apps/web/ci/child.yml). A path that climbs out of the repository is rejected, because GitLab has no such file to read either — so a pipeline that relied on it was already failing upstream.

A wildcard include merges fewer files than on GitLab#

What you see: include: local: 'ci/**/*.yml' (or a trigger: include: wildcard) resolves to fewer jobs locally than the same pipeline produces on GitLab, with no error — the pipeline just runs a smaller set.

Cause: one of three, in rough order of likelihood.

• The file is gitignored. Wildcards resolve against the project’s non-ignored files, so a generated fragment written into a .gitignored path is not includable. GitLab resolves includes out of the repository and would not see it either — meaning the pipeline is already smaller upstream than the ignored file suggests.
• The file is inside a submodule. Git reports a submodule as a single entry, not as its contents, so nothing under it can be included. GitLab cannot include across a submodule boundary either.
• The pattern used ?, [ or { expecting shell-glob behaviour. Only * is a wildcard in an include path; the rest are literal filename characters, so ci/[ab].yml names one file called exactly that. See wildcards in include paths.

Fix: check what actually got merged:

glci merged

glci merged prints the fully-resolved configuration, so a file you expected is either in there or it never matched. If it is gitignored, commit it or stop ignoring it — that is also what makes it work on GitLab.

Include path must end in .yml or .yaml#

What you see: CI configuration is invalid: parsing CI config: resolving includes: local include "ci/child.txt": path must end in .yml or .yaml, for a file that exists and holds perfectly good YAML.

Cause: GitLab accepts an include location only when its last path segment ends in .yml or .yaml (YAML_ALLOWLIST_EXTENSION, checked before the file is read). A config naming ci/child.txt cannot run on GitLab, so glci rejects it too rather than reporting a green pipeline you cannot push. The rule is case-insensitive (ci/child.YML is fine) but needs at least one character before the dot, so a file literally named .yml is rejected.

Fix: rename the file to .yml/.yaml and update the include. Three cases catch people out:

• A wildcard is expanded before it is checked, so each match is validated individually and the pattern itself never is. local: 'ci/*' fails when ci/ holds a README.md; constrain it to local: 'ci/*.yml' and the same README.md is ignored. The message names both, so you can find the pattern in your config: local include "ci/*" matched "ci/README.md": path must end in .yml or .yaml.
• A remote URL’s query string or fragment is part of the name GitLab checks. remote: https://.../f.yml?ref_type=heads — what the GitLab UI’s Open raw button gives you — is rejected despite ending in .yml to the eye. The message tells you to strip from the ? on, and only says so when stripping actually fixes it.
• Three kinds of include are exempt. component:, whose location is FQDN/project/name@version and never ends in .yml; anything carrying rules:, because GitLab discards a rules-gated include before it validates it; and any location still holding an unexpanded $VAR, which glci reports through the fetcher’s own error instead.

This covers top-level include: only. A trigger: include: location is checked when the child pipeline is created, so glci lint passes and the trigger job fails — which is how GitLab behaves too.

include: project: fails instead of being skipped#

What you see: project include "my/group" file "ci/child.txt": path must end in .yml or .yaml for an include: project:, even with no GitLab token configured. glci used to print skipping project include ...: no GitLab token configured and carry on.

Cause: GitLab validates an include’s location before it looks at its content or at your access to it, so the extension is wrong whether or not the file could have been fetched.

Fix: rename the file. A token is not the problem here — setting GITLAB_TOKEN will not clear this message, and the same config fails on GitLab.

CI configuration errors#

A script: line contains ": " and is rejected#

What you see: CI configuration is invalid: jobs:<name>:script config should be a string or a nested array of strings up to 10 levels deep, for a script line that looks perfectly ordinary:

job:
  script:
    - curl -H "Authorization: Bearer $TOKEN" https://example.com
    - echo "Error: deployment failed"

Cause: this is a YAML rule, not a glci one. A colon followed by a space is the block-mapping indicator, and it is not allowed inside a plain (unquoted) scalar — so YAML reads those two entries as mappings rather than strings:

{'curl -H "Authorization': 'Bearer $TOKEN" https://example.com'}
{'echo "Error': 'deployment failed"'}

The double quotes in the line quote nothing, because they are not the first character of the scalar. GitLab refuses the whole file for this, so glci lint reports it too. Earlier versions accepted the file and dropped the entry without a word, so the command never ran and the job still reported success.

Fix: quote the whole entry, with the quote as the first character. Either form works, so pick whichever does not collide with the quoting inside the command:

job:
  script:
    - 'curl -H "Authorization: Bearer $TOKEN" https://example.com'
    - "echo 'Error: deployment failed'"

Two things that are not the problem:

• A colon with no space after it is fine. echo "ratio 1:2" needs no quoting — 1:2 is not a mapping indicator.
• An anchor alias is fine. - *shared_steps expands to a nested list, which GitLab flattens rather than rejecting, up to ten levels deep.

The same rule covers before_script:, after_script:, default:before_script, default:after_script, services:command: and hooks:pre_get_sources_script — GitLab validates all of them with the same Entry::Commands rule, and so does glci. The message names the exact entry, so default:before_script config should be… means the default: block rather than any one job.

This is reported by glci lint, not enforced by glci run. Like every other config diagnostic glci produces, it does not stop a run today: a pipeline started without linting first still executes the job with the line missing, and still passes. On GitLab the same file is refused at pipeline creation and nothing runs at all. Lint a config you have just edited before you run it.

Rule evaluation issues#

The pipeline did not run#

What you see: run, jobs, show or variables fails with

the pipeline did not run: no workflow:rules rule matched under CI_PIPELINE_SOURCE=merge_request_event, CI_MERGE_REQUEST_IID=1 [rule 1 if: $CI_PIPELINE_SOURCE == "schedule" → if: evaluated to false] — GitLab would not have created this pipeline either; simulate another context with --context, or review the workflow:rules configuration

Cause: the project’s workflow: rules: did not accept the simulated context, so there is no pipeline to plan — the same refusal GitLab answers a push with. The default context is merge_request, which sets CI_PIPELINE_SOURCE=merge_request_event and no CI_COMMIT_BRANCH, so a workflow block written for branch pipelines rejects it.

Fix: run with the context the pipeline expects, and check the bracketed rule list in the message to see which condition failed:

glci run --context branch=main
glci run --context tag=v1.0

To keep from typing it every time, name the context in .glciconfig.toml — a [contexts.<name>] entry you pass as --context <name>, or a context = inside [pipelines.<name>] that applies when you run with --pipeline <name>. See workflow rules.

Jobs unexpectedly skipped#

What you see: no jobs to run: requested [job-name] but none matched after rules evaluation or a job you expected is missing from glci show.

Cause: Rules evaluated to when: never in the simulated context. By default, glci simulates a merge_request context.

A job that declares needs: optional: true on a rules-excluded job is no longer one of the reasons — that edge is dropped and the dependent runs, as Optional needs describes.

Debugging steps:

# See which jobs are included in the default context
glci show

# Compare with a different context
glci show --context branch=main
glci show --context tag=v1.0

# Check a specific job
glci jobs   # lists all jobs and their when: status

Common reasons jobs are excluded:

An if: expression that does not parse is no longer one of the reasons — glci rejects that config outright, the way GitLab does, rather than treating the rule as false and dropping the job. glci lint names the job and exits non-zero, and glci run refuses to start. Long conditions written as a YAML block scalar (if: >, if: >-, if: |) parse normally; whitespace between tokens, newlines included, is ignored.

Jobs unexpectedly included#

What you see: A job that should be skipped (e.g., deploy jobs) runs anyway.

Cause: The default merge_request context may match rules that your real CI wouldn’t. Or rules: changes: matches everything because glci has no diff context.

Fix:

# Simulate the exact context you want
glci run --context branch=feature-x

# Skip specific jobs by name
glci run --skip "deploy*"

Context simulation not matching real CI#

What you see: Jobs appear in glci show that don’t appear in the GitLab pipeline (or vice versa).

Cause: glci evaluates rules locally with the simulated context. Some differences from real CI:

• $CI_PIPELINE_SOURCE is set to push (for branch/tag) or merge_request_event (for merge_request context) — your real pipeline may have a different source
• rules: changes: may behave differently because glci uses local git diff while GitLab compares against the target branch
• rules: exists: evaluates your local working tree minus gitignored files (tracked + untracked), while GitLab evaluates the committed tree at the pipeline SHA — so uncommitted/untracked files satisfy exists: locally but won’t on GitLab until committed
• Protected/unprotected branch distinctions are not enforced locally

Fix: Use --context and --env together to match your real CI environment as closely as possible:

glci show --context merge_request --mr-source feature --mr-target main

Performance tips#

Slow image pulls#

The embedded registry acts as a pull-through cache – the first pull is slow but subsequent pulls are instant.

Slow startup due to secrets fetch#

Remote variable fetching adds latency to pipeline startup. If you don’t need remote secrets:

glci run --secrets none      # skip all remote variable fetching
glci run --secrets project   # skip group variables (slower due to pagination)

Reducing disk usage#

glci system df               # check what glci is using
glci system cache clean      # wipe CI cache
glci system prune            # clean unused containers, networks, volumes
glci system prune --all      # also remove registry data and history

Known limitations#

• CI_JOB_TOKEN is forwarded from the host’s GitLab token — permissions may differ from production. Use --no-token to disable.
• include: project: and include: component: require a GitLab API token (other include types work offline). Component version selectors (@~latest, @~N, @~N.M) are supported and resolve to the latest matching semver release tag.
• Local include: paths resolve from the repository root (nearest .git above the CI file), matching GitLab — not from the CI file’s directory. Outside a git repository, resolution falls back to the CI file’s own directory, so a subdirectory CI file referencing repo-root paths only works inside a git checkout.
• Protected variables are fetched regardless of branch protection status.
• Kubernetes executor is not supported — only Docker.
• Git submodules — served from the mock for relative URLs and for absolute HTTP(S) URLs on the project’s own GitLab host (SSH URLs only under GIT_SUBMODULE_FORCE_HTTPS); anything on another host is left to the network and needs credentials in the container. Objects come from your checkout, a --project-dir/[projects] mapping, or a clone from GitLab limited to the superproject’s own namespace or below. A submodule glci cannot reach is a daemon: warning: in glci daemon logs and the job’s checkout fails. Child and cross-project pipelines are served too, each against its own project path, and a project path is served once per pipeline prefix — the first walk wins. Up to 10 levels deep, 64 repositories per walk. See Git submodules.
• retry: does not cover job timeouts — glci only re-runs a job after the runner reported a result, so retry: when: job_execution_timeout and retry: when: runner_system_failure never fire locally. A runner that picks a job up and then goes silent does trip the mock server’s stuck timer, so retry: when: stuck_or_timeout_failure works. allow_failure is honoured for all of these. See Tolerated failures.
• Child pipelines — trigger: include: supports up to 2 levels of nesting (matching GitLab). A child’s own config resolves include: (including project:/component:) against the same GitLab instance and token as the parent, and from the repository root, whatever directory glci was run from.
• CI_DEFAULT_BRANCH comes from refs/remotes/origin/HEAD and falls back to main when the clone has none — so on a repository whose default branch is master and whose clone never recorded it, $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH rules diverge from GitLab. git remote set-head origin -a fixes it. The served repository also carries the default branch as a second name for the pipeline’s commit, so git diff origin/<default>...HEAD in a job is empty rather than a real diff. See Context simulation.
• Dynamic child pipelines — trigger:include:artifact runs a child pipeline whose YAML a generator job produced. The generator must run before the trigger job (earlier stage or needs:); otherwise the trigger fails with artifact include "…": job "…" produced no artifacts. Each artifact-sourced config file is capped at 5 MiB and an empty file is rejected.
• Cross-project triggers — trigger: project: resolves from local directories or clones from GitLab. A cloned target has only its .yml/.yaml files extracted (the only thing an include: can name), so its rules: exists: sees no other file and its config must live at .gitlab-ci.yml; map the project locally when either matters. More than 64 MiB of YAML is refused.
• parallel: matrix: declaration order is read from the YAML, so shard names and CI_NODE_INDEX match GitLab. Two shapes carry no order for glci to read and fall back to alphabetical variable order: !reference to a path that is not a job’s parallel: matrix: (matrix: !reference [.matrices, deploy], where the target is a bare list — !reference [.template, parallel, matrix] keeps its order), and a matrix set through parallel in .glciconfig.toml, where TOML tables are unordered. Inline, extends:, include:, YAML anchors and merge keys (<<:) all keep their declared order. See Parallel job variables.
• A merge key (<<:) written after a key it also defines keeps the locally written value in glci, which is what the YAML specification prescribes and what the Go YAML parser implements. GitLab’s parser (Psych) applies the merge at the position of the <<, so there the merged value wins instead. Affects any key, not just parallel: matrix:; write <<: first — as nearly every config does — and both agree.
• Some parallel: shapes GitLab rejects are accepted locally: parallel: 2.5 (truncated to 2 shards), parallel: "3", parallel: {}, and an unknown key beside matrix: are ignored rather than reported. The diagnostics glci does mirror are the count range (1-200), the 200-job matrix cap, a matrix that is not a list of mappings, an entry with no variables, and a value that is not a string or an integer.
• Crash recovery requires both mock server and runner containers to survive. Secrets are held only in daemon memory (mlocked, never written to disk) and are lost on daemon restart, so resumed pipelines always re-fetch remote variables from the GitLab API. Remote include: files are the exception: those come from ~/.glci/cache/includes on resume rather than being re-fetched.