Status: current · self-hosted runner: RTX 4090 (sm_89), CUDA 13.2, Ubuntu 22.04
Static checks run on GitHub-hosted runners. The GPU jobs run on one self-hosted workstation, and they are the only jobs that install or execute anything. One hosted job, acpp-compile, compiles every target with AdaptiveCpp. CI gates one configuration (CUDA DPC++ on sm_89) and no other row of the Tested platforms table. .github/workflows/ci.yml is the authority on what runs.
| Covered | Where | Notes |
|---|---|---|
| CMake lists parse; CMakePresets.json validates | hosted | cmake --list-presets; no configure |
| Exported package has no host paths (source mode) | hosted | Sees only literal paths in the list files |
| Public headers do not include src/; no unprefixed <blas/…>, <util/…>, <internal/…>; no type in the consumer's global namespace | hosted | Text scans; no build sees the last one |
| No C++ source over the 18% comment-density ceiling | hosted | check_comment_density.py. Doxygen doc comments in include/batchlas/ are exempt; see Writing API documentation |
| Every evidence: pointer resolves; cited slugs are unique across docs/ | hosted | check_evidence_anchors.py |
| No Markdown outside docs/; no raw content under an LFS path | hosted | check_markdown_locations.py, check_lfs_pointers.py |
| Docs site builds with zero Doxygen warnings; published on push to main | hosted | docs and docs-deploy; see The docs jobs |
| Every tuned/<op>.<device>.txt equals its ledger, byte for byte | hosted | tuned-tables-ledgers: sweep_to_table.py --self-test and --check, fetching only the ledger LFS objects (generation and the check) |
| Stale tables reported | hosted | tuned-tables: check_tuned_tables.py --self-test, then the staleness report; warning only, never fails |
| Configure and compile every target with AdaptiveCpp 25.10, generic target, CUDA off | hosted | acpp-compile: catches DPC++-only spellings; no CUDA-backend code, nothing runs. See The acpp-compile job |
| Configure with real CUDA DPC++ | GPU, per PR | -DBATCHLAS_ENABLE_CUDA=ON, then grep for Using SYCL targets: … nvidia_gpu_sm_89 |
| Compile every component library and test binary | GPU, per PR | cmake --build --preset dev-tests |
| Run the suite on a GPU | GPU, per PR | ctest --preset dev-tests -LE slow: every test but the slow suites and consumer_package_tests (ctest -N counts them). The only place wrong numbers, launch failures and device aborts are caught |
| Package in package mode | GPU, per PR | cmake --install to a temp prefix, then run_local_checks.sh <prefix> |
| Four slow suites; packaging end to end | GPU, nightly | Full ctest, with consumer_package_tests under --require |
| Vendor-free build configures, compiles, links | GPU, nightly | Report-only (continue-on-error); not green, no ledger |
Not covered: AMD/ROCm; CPU-only DPC++ and icpx builds (see Why there is no hosted compile job); macOS and Windows; Python bindings and benchmarks (both off in dev-tests, so a benchmark that stops compiling passes); performance (nothing is timed); other NVIDIA architectures, CUDA versions and DPC++ builds; multi-GPU behaviour; flakiness (one run, one verdict); and anything a skipped test would have covered (A green run that tested nothing).
The gate is a single self-hosted workstation. If it is off, wedged or busy with a human's build, every pull request queues behind it.
| Job | Runs on | Trigger | Does |
|---|---|---|---|
| Static checks (cmake-lint, exported-package, public-headers, comment-density, lfs-pointers) | ubuntu-latest | every trigger | Scripts under .github/ci/; all run locally with .github/ci/run_local_checks.sh |
| gpu-build-test | [self-hosted, linux, x64, cuda] | push to main, non-fork PRs | Configure, assert the CUDA target, build, ctest -LE slow, compare_failures.py, install and check the export. Timeout 180 min |
| gpu-nightly-full | same | 03:17 UTC, workflow_dispatch with run_nightly | Full ctest, then a vendor-free configure, build and ctest that reports without gating. Timeout 360 min |
| acpp-compile | ubuntu-24.04 | every trigger | Builds AdaptiveCpp 25.10 (cached), then configures BatchLAS with it and compiles library, tests and benchmarks. Timeout 90 min |
| tuned-tables | ubuntu-latest | every trigger | check_tuned_tables.py --self-test, then the stale-table report (never fails) |
| tuned-tables-ledgers | ubuntu-latest | every trigger | Fetches the ledger LFS objects (cached by their pointers' oids), then sweep_to_table.py --self-test and --check |
| docs | ubuntu-latest | every trigger | Builds the site (below) |
| docs-deploy | ubuntu-latest | push to main, after docs | Publishes the Pages artifact |
Note: a repository admin must enable GitHub Pages once (Settings → Pages → Source: "GitHub Actions"). Until then docs-deploy fails and docs stays green. To bump Doxygen, see Building the site.
Phase P6 of Two SYCL implementations. --acpp-targets=generic compiles device code to target-independent IR and JIT-compiles it at first launch, so a hosted runner with no GPU and no CUDA toolkit compiles every TU. There is no device link.
| Step | What |
|---|---|
| Toolchain | clang-20 llvm-20-dev libclang-20-dev libomp-20-dev from the Ubuntu 24.04 archive (20.1.2, the package the threadripper02 install uses), netlib liblapacke-dev liblapack-dev libblas-dev, ccache |
| AdaptiveCpp | .github/ci/build_adaptivecpp.sh: tag v25.10.0, which must resolve to 9f842c70; generic compiler and OpenMP runtime only, every GPU backend off. actions/cache keyed on tag, commit, llvm-20-dev version and the script's hash. 14 MB install |
| BatchLAS | .github/ci/acpp_compile.sh: the acpp-tests preset with CMAKE_CXX_COMPILER=<install>/bin/acpp, Release, -Werror=unknown-attributes, BATCHLAS_ENABLE_CUDA=OFF, benchmarks on. Fails unless configure printed Using AdaptiveCpp targets: generic, Using AdaptiveCpp for SYCL: and Found LAPACKE:. Builds all |
| Completeness | .github/ci/check_built_tests.py: every registered ctest executable must exist (all but the three tests that run cmake -P, python3 or a shell script) |
| ccache | ~/.ccache-acpp in actions/cache, per commit, restored from the newest earlier entry |
Measured on threadripper02 with the same scripts and a clean environment:
| Result | |
|---|---|
| AdaptiveCpp build | 58 s wall at -j4 |
| BatchLAS, cold | 269 TUs, 2,614 CPU-seconds, 133 s wall at -j32; max RSS 1.5 GB per TU |
| BatchLAS, warm ccache in a second tree | 269 of 269 hits, 6 s wall; cache 45 MB |
Deliberate breaks, each in the scratch copy the job compiles:
| Break | Result |
|---|---|
| q.get_backend() in src/ops/gemm/gemm.cc | red: no member named 'get_backend' in 'hipsycl::sycl::queue' |
| sycl::backend::ext_oneapi_cuda in tests/util_span_tests.cc | red: no member named 'ext_oneapi_cuda' |
| q.ext_oneapi_submit_barrier() in benchmarks/gemm_benchmark.cc | red |
| raw [[intel::max_work_group_size]] or [[sycl::reqd_sub_group_size(32)]] in src/ops/getrf/getrf.cc | red only through -Werror=unknown-attributes (acpp_compile.sh); acpp alone warns |
| q.get_backend() in src/backends/cublas.cc | green: not compiled with CUDA off |
Not covered (CUDA off). src/backends/cublas.cc, cusolver.cc, cusparse.cc, tools/tune and tune_race_gpu_tests are not built, and every #if BATCHLAS_HAS_CUDA_BACKEND block in the other TUs is skipped. That includes the acpp branch of impl::run_native, which only vendor TUs instantiate. Closing it needs a CUDA toolkit with cuBLAS, cuSOLVER and cuSPARSE on the runner (enable_language(CUDA) needs nvcc too), and an acpp configure path that accepts BATCHLAS_ENABLE_CUDA=ON without a device: generic code has no architecture to detect.
A CPU-only build (either implementation) did not compile before this job existed. Fixed with it: tests/{gesvdj_cta,bdsqr,bdsdc}_tests.cc compile to zero tests without BATCHLAS_HAS_CUDA_BACKEND, stein_tests.cc and stebz_tests.cc without a GPU backend; sytrd_sy2sb_tests.cc lost a default template argument naming test_utils::gpu_backend; the CUDA gemm seam in trsm_tests.cc, factor_bench and iluk_convergence_stats are guarded; tools/tune and tune_race_gpu_tests are built only with a GPU backend (BATCHLAS_BUILD_TUNE_TOOL).
Run it locally:
Warning: no runner has been installed and observed working end to end (~/actions-runner does not exist; gh is absent). The prerequisites are measured; the assembly is not. Expect to iterate once on the first run.
config.sh needs a registration token (not a personal access token; not stored on this machine). Get it from GitHub → the repository → Settings → Actions → Runners → New self-hosted runner. It is valid for one hour. The page prints the release version and download URL to use in the commands below.
config.sh adds self-hosted, Linux and X64 itself; cuda is the distinguishing label. Label matching in runs-on: is case-insensitive.
Install the service rather than using ./run.sh. An interactive run.sh inherits the login-shell environment that makes the project build, so it works while the service does not.
loginctl show-user jonaslacour reports Linger=no, so a systemd --user unit would not survive logout; the service must be a system service. A service running as root, or as a user whose HOME is not /home/jonaslacour, loses all ccache sharing (ccache is cold, or silently useless).
A systemd service inherits none of ~/.bashrc. It gets /etc/environment: the stock PATH, no /opt/dpcpp-cuda, no CUDA, no LD_LIBRARY_PATH. No /etc/profile.d script or ld.so.conf.d entry supplies them, and the installed binaries carry neither RPATH nor RUNPATH.
Under the default BATCHLAS_ENABLE_CUDA=AUTO, a naively installed runner detects no GPU, prints No specific GPU architectures detected, using default JIT compilation, and then configures, compiles, installs and passes ctest on a CPU-only library that never emitted NVPTX. This is the pitfall in the TL;DR of the agent guide.
| Variable | Value | Why |
|---|---|---|
| PATH | prepend /opt/dpcpp-cuda/bin, /usr/local/cuda-13.2/bin | cmake/BatchLASCompilerBootstrap.cmake finds sycl-ls first, then the compiler beside it |
| LD_LIBRARY_PATH | /opt/dpcpp-cuda/lib (mandatory), /usr/local/cuda-13.2/lib64, /opt/intel/oneapi-tbb-2022.3.0/lib/intel64/gcc4.8, /usr/lib/x86_64-linux-gnu | Nothing SYCL runs without the first entry |
| CUDA_PATH | /usr/local/cuda-13.2 | Fallback for --cuda-path when find_package(CUDAToolkit) misses |
| HOME | /home/jonaslacour | ccache is looked up in $HOME/.local/bin. Set by the service user, not the workflow |
Must not be set:
A failure here is silent, so ci.yml asserts it three ways (keep all three):
main is not green. Four gtest cases fail stably on a clean tree:
| ctest test | gtest case |
|---|---|
| lanczos_tests | LanczosTestBase.LanczosTest |
| lanczos_tests | LanczosTestBase.ToeplitzEigenpairs |
| steqr_tests | SteqrTest/0.StressExtremeMagnitudesN32 (float, Backend::NETLIB) |
| steqr_tests | SteqrTest/1.StressExtremeMagnitudesN32 (double, Backend::NETLIB) |
The gate is not ctest's exit code. tests/known-failures.txt is the ledger: expected failures, one per line, each with a comment saying why. It is a debt ledger, not an allowlist. .github/ci/compare_failures.py judges the run's machine-readable reports against it.
The script needs both report layers: per-case gtest XML (the only view inside a binary) and ctest's JUnit (the only place consumer_package_tests, a bash script with no gtest XML, and never-built executables appear).
Other report sections: SKIPPED (binaries in which no case executed, a vacuous run); LISTED BUT NOT IN THIS REPORT (ledger entries the run never reached); [COARSE: …] (an entry matched only at ctest-test level, so the specific case is unverified).
Fix the bug if you can. Otherwise, add a line keyed <ctest-test>::<Suite>.<Case>, for example lanczos_tests::LanczosTestBase.LanczosTest, with the reason in a # comment. For a typed test, pin the type: steqr_tests::SteqrTest/0.StressExtremeMagnitudesN32;type_param=<spelling>. Run the suite once; the script prints the observed spelling. A typed entry without the pin warns, and a mismatched pin is a hard error.
On NEWLY PASSING, confirm it is a real fix (LISTED BUT NOT IN THIS REPORT means the case did not run), delete the line in the same commit, and if the case is gone (renamed, deleted, not instantiated), say so in the commit message.
Static checks take seconds and need no toolchain:
The wrapper builds the docs into build/docs when Doxygen 1.18+ is on PATH (not strict; set BATCHLAS_DOCS_STRICT=1 to match the docs job). Run cmake --install build/presets/dev-tests --prefix /tmp/inst first and pass the prefix to check the package before pushing anything that touches the install rules.
Full gate, as the nightly runs it (the preset sets execution.jobs=1; do not add -j):
ctest -N counts the tests; ctest -LE slow takes about 95 s on the 4090. Do not run the full suite on every edit; tests/README.md has the scoping table. scripts/ctest_gpus.sh was validated only on threadripper02; CI and local full gates stay serial.
Two scopes are wrong for a pre-push gate: ctest -LE slow excludes the four slow suites and consumer_package_tests; BATCHLAS_TEST_TARGET_SET=smoke (the fast-dev preset) runs only the smoke binaries, with none of the four baseline failures. It is an edit-loop tool only.
This section is about DPC++. The AdaptiveCpp build has one: The acpp-compile job.
BATCHLAS_CPU_TARGET=none does not disable the device pass; it only skips appending native_cpu / spir64_x86_64 to the SYCL target list. -fsycl is unconditional, so on a GPU-less runner the driver falls back to its default spir64 device pass (verified with clang++ -fsycl -### and the __CLANG_OFFLOAD_BUNDLE sections in the object). There is no host-only mode.
A hosted job would pay several GB of oneAPI apt and a device pass over 95 TUs, to build a CPU-only library that cannot catch a CUDA-backend compile error. It becomes worthwhile with a second self-hosted machine or a prebuilt toolchain container with a warm ccache. Reasoning: the comment at the foot of ci.yml.
In ctest JUnit, only status="run" with no child is executed-and-passed. Skips are status="notrun" with a <skipped> child, which includes never-built binaries. The root summary attributes are unreliable.
1.18.0
