Debugging & Inspection

Variable precedence debugging#

When a job behaves unexpectedly, the variable stack is often the cause. See Variables & Secrets for the full precedence table.

Start by listing what each job actually resolves to and where each value came from:

glci variables my-job                # job/rules vars + per-rule evaluation trace
glci variables my-job --all          # also include predefined CI_* variables
glci run --show-variables my-job      # fully resolved set (incl. secrets) at run time

glci variables resolves locally without running, so it also shows jobs that rules: excluded — see Inspecting resolved variables. To isolate which variable source is causing a problem:

glci run --secrets none my-job       # run without remote variables
glci run --secrets project my-job    # project-level only (skip group)
glci run --env MY_VAR=debug my-job   # override a specific variable
glci run --refresh-secrets my-job    # force refresh cached API variables

Debugging rule evaluation#

Rules evaluation works largely the same as production GitLab CI (docs), with one intentional difference for exists: noted below. glci supports if:, changes: (with compare_to:), and exists: conditions. Both changes: and exists: accept ** globs that match at any depth (e.g. exists: ['**/*.php'] matches .php files in the project root and any subdirectory) and brace expansion ({a,b}, e.g. ['*.{php,inc}']), mirroring GitLab’s FNM_EXTGLOB. exists: matches regular files only, never directories – though a trailing slash (exists: ['src/']) is a directory-presence check that matches when any file lives under that directory. See workflow rules for the full glob reference.

exists: evaluates the project’s non-ignored files — everything git tracks plus untracked files, but excluding anything matched by .gitignore (git ls-files --cached --others --exclude-standard). This mirrors glci’s dirty mode, which sends your working tree (minus gitignored files) to the runner, so an exists: rule sees the same files your job will. Build artifacts such as node_modules/, target/, and other gitignored output therefore do not satisfy an exists: rule, while files you have created but not yet committed do.

Difference from GitLab: GitLab evaluates exists: against the committed tree at the pipeline SHA (tracked files only). glci evaluates your local working tree minus gitignored files, so uncommitted/untracked files match locally where they would not on GitLab until committed. When the project directory is not a git repository, glci falls back to scanning the entire working tree (gitignore can’t be consulted).

Like GitLab, exists: glob matching has a comparison budget (50,000 path×glob comparisons). On a project large enough to exceed it, glci — like GitLab — assumes a match (fails open) rather than risk dropping a job, and prints a warning. Literal paths and **/*.ext extension globs are matched directly and are not subject to the budget.

See exactly which rule matched (or why none did) for a job, including the variable values each condition was evaluated against:

glci variables my-job                          # per-rule trace + resolved variables
glci variables my-job --context branch=main

Compare the pipeline across contexts to understand which rules matched:

glci jobs --context merge_request
glci jobs --context branch=main
diff <(glci show --json --context merge_request) \
     <(glci show --json --context branch=main)

If glci show returns no jobs, workflow: rules: may be rejecting the entire pipeline. Try a different context.

The fully-resolved configuration#

A real pipeline is a root config plus include: entries, extends: chains, !reference splices and a default: block. When a job does not look like you expect, the first question is what the job actually is after all of that is merged. glci merged prints it:

glci merged                               # merged YAML
glci merged --json | jq '.["build-job"]'  # one job, machine-readable
glci merged --input env=staging           # with pipeline inputs

This is the local equivalent of GitLab’s “Full configuration” view — glci’s own fully-resolved view, no push required, and it works on a config that only exists on your machine. include: and extends: keys are gone, !reference is expanded in place, default: and global variables: have been applied to the jobs that inherit them, and .pre/.post are in stages: when the config declares a stages: key. It is not byte-for-byte identical to GitLab’s merged YAML — see glci merged for the divergences to normalize before diffing the two.

The output is the config before rules: are evaluated, so it is context-independent — pair it with glci variables for rule traces and glci show for what would run in a given context.

Keys are sorted and the output is byte-stable, so it diffs cleanly:

glci merged > /tmp/after.yml
git stash push -- .gitlab-ci.yml && glci merged > /tmp/before.yml
git stash pop && diff /tmp/before.yml /tmp/after.yml   # what did my refactor change?

(A bare git stash on a clean tree stashes nothing and exits 0, which makes the later git stash pop fail — hence the explicit pathspec. Keep the outputs in /tmp/ so they stay out of the tree being diffed.)

Only the config goes to stdout; warnings such as glci: skipping project include ...: no GitLab token configured go to stderr, so a redirect keeps the file clean while still telling you something was omitted.

The merged output is sensitive: it carries variable values from every included file, including private project:/component: includes fetched with your token, plus any --input values. Include-resolution warnings and errors on stderr can additionally echo values interpolated into an include: path, so --env/.glci.env secrets can appear there too. Review both before pasting into an issue or committing.

Daemon logs#

The glci daemon writes logs to ~/.glci/daemon.log with details not visible in job output: mock server requests, runner container lifecycle, child pipeline events, cross-project trigger resolution, and scheduler decisions.

tail -f ~/.glci/daemon.log     # follow in real time
glci system logs info          # check file size
glci system logs clean         # clear the log file (safe while daemon runs)

Log lines are prefixed with their source: daemon: (orchestration), mock-server: (API handling), runner: (gitlab-runner output).

Job logs#

glci log                       # all logs from latest pipeline
glci log 5 build-job           # specific job from pipeline #5
glci history                   # find child pipeline IDs
glci log 7 child-job-name      # child pipeline job log

glci log streams live output for running jobs and reads from disk for completed jobs.

Common issues#

Job is unexpectedly skipped#

Check which context is active (default is merge_request). Compare glci jobs --context merge_request vs glci jobs --context branch=main. If the job appears in one but not the other, the issue is in rules:. Also check workflow: rules: which can reject the entire pipeline.

Job fails with “variable not found”#

Isolate the source: glci run --secrets none my-job (remove remote vars), then glci run --secrets all my-job (add them back). Override with --env MISSING_VAR=value to confirm.

Trigger job fails immediately#

Child pipeline runs but is missing jobs#

A child pipeline’s config resolves its own include: entries. If one names a project:/component: and no token is configured, the include is skipped and every job it defined is simply absent – the child still passes. Look for skipping project include <path>: no GitLab token configured in the trigger job’s output or the daemon log, and compare against glci jobs -f child.yml, which parses the same file at top level.

Include resolution fails#

Tips#

Esc