Git Submodules

glci serves your project’s git submodules from its mock GitLab server, so a pipeline that sets GIT_SUBMODULE_STRATEGY checks them out the same way it does on GitLab.com — without needing credentials for your real GitLab instance inside the job container.

Nothing needs to be configured for the common case. When any job in the pipeline asks for submodules, glci reads .gitmodules from the commit it is about to serve, finds each submodule’s objects, and publishes them alongside the main repository.

variables:
  GIT_SUBMODULE_STRATEGY: recursive

test:
  script:
    - ls submod/

The strategy is read from anywhere a variable can come from — the CI config, --env, an env file, a pipeline preset, or CI/CD variables fetched from GitLab.

Where submodule objects come from#

For each submodule, glci looks in this order:

1. The submodule’s checkout in your working tree (<project>/<submodule path>). This is the normal case after git submodule update --init.
2. The superproject’s .git/modules/<name>, which still holds the objects after git submodule deinit.
3. A directory you mapped with --project-dir group/submod=/path/to/checkout, or a [projects."group/submod"] entry in .glciconfig.toml — the same mapping cross-project pipelines use. A mapping is authoritative: if the directory you named does not hold the pinned commit, glci reports that rather than looking elsewhere.
4. A clone from your real GitLab instance, when glci knows which instance that is (from the project’s origin remote or config). A token is used when one is available; public projects clone without one. Only projects in the superproject’s own namespace or below it are cloned automatically — acme/team/project reaches acme/team/*, not all of acme/*. Cloning uses your personal token, so anything further afield has to be pointed at explicitly with --project-dir.

Whatever the source, glci serves the exact commit your superproject pins, not the tip of a branch.

Which submodule URLs are supported#

An absolute URL only counts as “yours” when glci knows your project’s GitLab host. For HTTP(S) the port has to match too; for SSH it is ignored, since a server’s SSH port is unrelated to its web port. Outside a GitLab-connected checkout every absolute URL is treated as foreign.

How the rewrite reaches the job#

glci adds a pre_get_sources_script hook to the job, which gitlab-runner runs after exporting the job’s variables and before it clones. The hook configures a git credential helper for the mock server and one insteadOf entry per absolute submodule URL. The mock’s token is passed to the job in GLCI_MOCK_TOKEN — masked in the trace — and stays unexpanded in the generated git config, so the value itself is never written there.

Your own hooks:pre_get_sources_script still runs, appended after glci’s lines so it can override them. When glci serves no repository at all, its own lines are skipped and only your hook is emitted.

pre_get_sources_script is the only hook GitLab accepts under hooks: in .gitlab-ci.yml. post_get_sources_script is a gitlab-runner config.toml setting (and an internal job-spec hook name), not a CI keyword — GitLab’s lint API rejects a pipeline that puts it under hooks:, and glci lint reports the same error so you find out before pushing:

jobs:build:hooks config contains unknown keys: post_get_sources_script

glci run still executes the pipeline, ignoring the unknown key as it always has; glci lint is what tells you the push would be rejected.

To run commands after the clone and submodule update, use before_script — the first script you control that runs once sources are in place. (Cache restore and artifact download happen in between.)

Recursion and limits#

GIT_SUBMODULE_STRATEGY: recursive makes glci walk nested .gitmodules files too, up to 10 levels deep, 64 served repositories per walk, and 1024 entries per .gitmodules. Each submodule project is served once, at the first pinned commit encountered; two submodules pinning the same project at different commits are not both served. A child pipeline that walks for itself gets its own 64-repository budget.

GIT_SUBMODULE_PATHS is passed through to the runner unchanged, but glci ignores it when deciding what to serve: it only ever narrows what a job fetches, so preparing the full set is always safe.

Child pipelines#

Child pipelines get submodules too, on the same terms as the pipeline you invoke.

A trigger: include: child checks out the same repository as its parent, so it inherits everything the parent served. It walks the repository again only when its own jobs ask for more than the parent’s did — the parent runs without submodules, the child wants recursive where the parent had normal, or the child sets GIT_SUBMODULE_FORCE_HTTPS and the parent did not.

A cross-project trigger runs a different project, so its submodules are read from that project’s commit, and its relative URLs (../submod) resolve against its path, not the parent’s. The source order above applies unchanged, with one caveat: a target resolved by cloning from GitLab has no checkout on your machine, so sources 1 and 2 cannot apply and its submodules come from a --project-dir mapping or a clone. Auto-cloning is bounded by the namespace of the project being triggered, so a target in other/team reaches other/team/* — map anything further afield with --project-dir.

Each pipeline serves its repositories under its own prefix on the mock, and the pipelines it triggers directly share that prefix. Within one prefix a project path is served once, by whichever walk got there first: a later walk reuses that repository rather than replacing it, so a job cloning it is never pulled out from under. If a parent and a triggered project pin the same submodule project at different commits, the second pin is not served and the daemon log says so — the job that needs the other commit fails on a missing object. Map one of them with --project-dir (pointing at a checkout that holds both commits) when both revisions have to exist at once.

When a submodule cannot be served#

A submodule glci cannot reach is reported as a warning in the daemon log and skipped — the rest of the pipeline runs normally. The job’s checkout then fails with git’s own error. Check the daemon log for the reason:

glci daemon logs

Every message is prefixed daemon: warning:. Typical ones and their fix:

A submodule on a foreign host, or an SSH one without GIT_SUBMODULE_FORCE_HTTPS, is not a warning — glci deliberately leaves those to the network. They are summarized in the daemon log on a line without the warning: prefix.