Debugging & Inspection
Variable precedence debugging#
When a job behaves unexpectedly, the variable stack is often the cause. See Variables & Secrets for the full precedence table.
Start by listing what each job actually resolves to and where each value came from:
glci variables my-job # job/rules vars + per-rule evaluation trace
glci variables my-job --all # also include predefined CI_* variables
glci run --show-variables my-job # fully resolved set (incl. secrets) at run time
glci variables resolves locally without running, so it also shows jobs that rules: excluded — see Inspecting resolved variables. To isolate which variable source is causing a problem:
glci run --secrets none my-job # run without remote variables
glci run --secrets project my-job # project-level only (skip group)
glci run --env MY_VAR=debug my-job # override a specific variable
glci run --refresh-secrets my-job # force refresh cached API variables
Debugging rule evaluation#
Rules evaluation works largely the same as production GitLab CI (docs), with one intentional difference for exists: noted below. glci supports if:, changes: (with compare_to:), and exists: conditions. Both changes: and exists: accept ** globs that match at any depth (e.g. exists: ['**/*.php'] matches .php files in the project root and any subdirectory) and brace expansion ({a,b}, e.g. ['*.{php,inc}']), mirroring GitLab’s FNM_EXTGLOB. exists: matches regular files only, never directories – though a trailing slash (exists: ['src/']) is a directory-presence check that matches when any file lives under that directory. See workflow rules for the full glob reference.
exists: evaluates the project’s non-ignored files — everything git tracks plus untracked files, but excluding anything matched by .gitignore (git ls-files --cached --others --exclude-standard). This mirrors glci’s dirty mode, which sends your working tree (minus gitignored files) to the runner, so an exists: rule sees the same files your job will. Build artifacts such as node_modules/, target/, and other gitignored output therefore do not satisfy an exists: rule, while files you have created but not yet committed do.
Difference from GitLab: GitLab evaluates
exists:against the committed tree at the pipeline SHA (tracked files only). glci evaluates your local working tree minus gitignored files, so uncommitted/untracked files match locally where they would not on GitLab until committed. When the project directory is not a git repository, glci falls back to scanning the entire working tree (gitignore can’t be consulted).
Like GitLab, exists: glob matching has a comparison budget (50,000 path×glob comparisons). On a project large enough to exceed it, glci — like GitLab — assumes a match (fails open) rather than risk dropping a job, and prints a warning. Literal paths and **/*.ext extension globs are matched directly and are not subject to the budget.
See exactly which rule matched (or why none did) for a job, including the variable values each condition was evaluated against:
glci variables my-job # per-rule trace + resolved variables
glci variables my-job --context branch=main
Compare the pipeline across contexts to understand which rules matched:
glci jobs --context merge_request
glci jobs --context branch=main
diff <(glci show --json --context merge_request) \
<(glci show --json --context branch=main)
If glci show returns no jobs, workflow: rules: may be rejecting the entire pipeline. Try a different context.
The fully-resolved configuration#
A real pipeline is a root config plus include: entries, extends: chains, !reference splices and a default: block. When a job does not look like you expect, the first question is what the job actually is after all of that is merged. glci merged prints it:
glci merged # merged YAML
glci merged --json | jq '.["build-job"]' # one job, machine-readable
glci merged --input env=staging # with pipeline inputs
This is the local equivalent of GitLab’s “Full configuration” view — glci’s own fully-resolved view, no push required, and it works on a config that only exists on your machine. include: and extends: keys are gone, !reference is expanded in place, default: and global variables: have been applied to the jobs that inherit them, and .pre/.post are in stages: when the config declares a stages: key. It is not byte-for-byte identical to GitLab’s merged YAML — see glci merged for the divergences to normalize before diffing the two.
The output is the config before rules: are evaluated, so it is context-independent — pair it with glci variables for rule traces and glci show for what would run in a given context.
Keys are sorted and the output is byte-stable, so it diffs cleanly:
glci merged > /tmp/after.yml
git stash push -- .gitlab-ci.yml && glci merged > /tmp/before.yml
git stash pop && diff /tmp/before.yml /tmp/after.yml # what did my refactor change?
(A bare git stash on a clean tree stashes nothing and exits 0, which makes the later git stash pop fail — hence the explicit pathspec. Keep the outputs in /tmp/ so they stay out of the tree being diffed.)
Only the config goes to stdout; warnings such as glci: skipping project include ...: no GitLab token configured go to stderr, so a redirect keeps the file clean while still telling you something was omitted.
The merged output is sensitive: it carries variable values from every included file, including private project:/component: includes fetched with your token, plus any --input values. Include-resolution warnings and errors on stderr can additionally echo values interpolated into an include: path, so --env/.glci.env secrets can appear there too. Review both before pasting into an issue or committing.
Daemon logs#
The glci daemon writes logs to ~/.glci/daemon.log with details not visible in job output: mock server requests, runner container lifecycle, child pipeline events, cross-project trigger resolution, and scheduler decisions.
tail -f ~/.glci/daemon.log # follow in real time
glci system logs info # check file size
glci system logs clean # clear the log file (safe while daemon runs)
Log lines are prefixed with their source: daemon: (orchestration), mock-server: (API handling), runner: (gitlab-runner output).
Job logs#
glci log # all logs from latest pipeline
glci log 5 build-job # specific job from pipeline #5
glci history # find child pipeline IDs
glci log 7 child-job-name # child pipeline job log
glci log streams live output for running jobs and reads from disk for completed jobs.
Common issues#
Job is unexpectedly skipped#
Check which context is active (default is merge_request). Compare glci jobs --context merge_request vs glci jobs --context branch=main. If the job appears in one but not the other, the issue is in rules:. Also check workflow: rules: which can reject the entire pipeline.
Job fails with “variable not found”#
Isolate the source: glci run --secrets none my-job (remove remote vars), then glci run --secrets all my-job (add them back). Override with --env MISSING_VAR=value to confirm.
Trigger job fails immediately#
- Child pipelines: verify the include file exists on disk.
- Cross-project triggers: ensure
--project-dir group/project=../projectis set. extends target "..." not foundfrom a trigger job usually means the child’s owninclude: project:/component:was skipped for want of a token; look forskipping project includejust above it in the job output.- Check
tail -20 ~/.glci/daemon.logfor resolution errors.
Child pipeline runs but is missing jobs#
A child pipeline’s config resolves its own include: entries. If one names a project:/component: and no token is configured, the include is skipped and every job it defined is simply absent – the child still passes. Look for skipping project include <path>: no GitLab token configured in the trigger job’s output or the daemon log, and compare against glci jobs -f child.yml, which parses the same file at top level.
Include resolution fails#
include: project:requires a GitLab token – runglci doctorto verify. This applies to a child pipeline’s or cross-project target’s config just as it does to.gitlab-ci.yml.include: component:with version selectors (@~latest,@~N,@~N.M) requires a GitLab token to query the Tags API. If the token is missing or lacks read access, you will see errors likefailed to resolve version selector: unauthorizedorno matching tags found for selector ~latest.- Max nesting depth is 10 levels.
glci mergedshows what the includes did resolve to, and reports on stderr the ones that were skipped.
Tips#
- Quick validation loop:
glci show --watchre-renders the pipeline graph on every save. - Environment health check:
glci doctorchecks the container engine (Docker or Podman), daemon, token, CI config, and git. - Simulation mode:
glci run --simulatereplaces scripts with echo commands and produces dummy artifacts. - Effective config:
glci config,glci config --network,glci config --gitlab. - Effective CI config:
glci merged(add--jsonto query it withjq). - Diffing against GitLab:
glci mergedis a useful starting point when glci and GitLab disagree about a config, but normalize the known divergences (key order,default:/variables:placement,extends:) before comparing line by line.