Backends & support
This matrix describes the implemented pre-alpha runtime, not a promise of production support. All operations are synchronous; tensors are contiguous and CPU/Metal/CUDA expose index 0; Vulkan has indexed routes with one physical GPU validated.
| Capability | CPU | Metal | CUDA | Vulkan (optional) |
|---|---|---|---|---|
| Float32/int32/bool tensor copy | Yes | Yes | Yes | Yes |
Explicit .to() placement / forced same-device copy |
Yes | Yes | Yes | Yes |
| Float32 fill, add, multiply | Yes | Yes | Yes | Yes |
| Int32 fill, add, multiply | Yes | Yes | Yes | Yes |
| Float32 subtract, divide, negate and scalar arithmetic | Yes | Yes | Yes | Yes |
| Int32 subtract, negate and scalar add/subtract/multiply | Yes | Yes | Yes | Yes |
| Contiguous reshape (float32/int32/bool) | Yes | Yes | Yes | Yes |
| Transpose to contiguous copy (float32/int32/bool) | Yes | Yes | Yes | Yes |
| Squeeze / expand dims views (float32/int32/bool) | Yes | Yes | Yes | Yes |
| Basic indexing / slicing copies (float32/int32/bool) | Yes | Yes | Yes | Yes |
| Concat / stack / split copies (float32/int32/bool) | Yes | Yes | Yes | Yes |
| Explicit float32/int32/bool conversion | Yes | Yes | Yes | Yes |
Comparisons and where (float32/int32/bool) |
Yes | Yes | Yes | Yes |
| Bool fill, logic, any/all and mask selection | Yes | Yes | Yes | Yes |
| Binary arithmetic broadcasting | Yes | Yes | Yes | Yes |
| Float32 batched/vector matmul | Yes | Custom + optional optimized path | Custom path | Custom path |
| Float32 sum, max, mean | Yes | Yes | Yes | Yes |
| Int32 sum, max | Yes | Yes | Yes | Yes |
Sum/max/mean keepdims |
Yes | Yes | Yes | Yes |
| All-axis / multi-axis sum, max, mean | Yes | Yes | Yes | Yes |
| Float32 log, sqrt | Yes | Yes | Yes | Yes |
| Float32/int32 min, argmax | Yes | Yes | Yes | Yes |
| Float32/int32 abs, clip, topk | Yes | Yes | Yes | Yes |
| Float32 linear, affine norms, embedding, attention | Yes | Yes | Yes | Yes |
| Float32 exp, GELU, SiLU | Yes | Yes | Yes | Yes |
| Float32 softmax, RMSNorm, LayerNorm | Yes | Yes | Yes | Yes |
| Generated kernels | Optional MLIR subset | Experimental MSL subset | Optional MLIR subset | Optional guarded MLIR/SPIR-V subset |
empty allocation |
CPU storage | Existing CPU staging/copy | Direct device allocation | Direct device allocation |
| DLPack exchange (contiguous float32/int32/bool) | Yes | Not implemented | Device 0 | Not implemented |
| Explicit shared allocation | Same-device aliases | Same-device aliases | Matching Vulkan GPU | Matching CUDA device 0 |
| Installation and operation/dtype diagnostics | Yes | Availability/capabilities | Yes | Yes |
| Memory snapshots | System physical memory | Not implemented | Device-wide free/total | Heap totals + separate owned-allocation diagnostics |
| Native host profiling | Bounded spans | Not instrumented | Bounded spans | Bounded spans |
| Optional GPU timing | No | Not instrumented | Kernel events | Copy/compute timestamps |
| Persistent compiled-code cache | Optional MLIR | Experimental MSL | Optional MLIR | Optional MLIR/SPIR-V |
| Bounded buffer pool | Not implemented | Not implemented | Explicit opt-in scope | Explicit opt-in scope + command reuse |
empty has unspecified contents on every backend. CUDA skips CPU payload
allocation/upload and can reuse idle storage in an explicit pool. See
memory reuse and profiling and the
feature overview for the remaining scope.
Explicit accelerator-to-accelerator placement uses synchronous CPU staging;
same-device copy=True allocates independent storage. See
placement and storage sharing.
The separate shared-allocation API
creates native CUDA/Vulkan aliases on compatible matching Linux hardware, in
either allocation direction. This does not establish actual two-GPU execution.
The mandatory reference backend. It runs without a GPU and is the baseline for device comparison tests. Start here for the simplest installation and debugging workflow. CPU-only CI covers Python 3.11, 3.12, and 3.13.
Apple Silicon is the primary Metal target. Building requires Xcode’s metal
and metallib tools. Static kernels implement tensor operations; matmul can use
an optional Apple optimized path or the custom correctness-first kernel.
cx.matmul(a, b, backend="custom") selects the custom implementation.
backend="optimized" requires the enabled Apple optimized path. macOS CI has
configurations with that path enabled and disabled. A job that skips device
tests does not establish hardware execution.
CUDA is an opt-in backend. Its recorded real-device validation is limited to a
GTX 980 Ti (sm_52) on Linux x86_64 with CUDA 12.4 and GCC 13. Tests compare
outputs with CPU references and include native contracts and GPU sanitizer runs.
Additional NVIDIA architectures and toolchain combinations still need their own validation. The hosted CUDA build job tests compilation and no-device behavior; it does not run on an NVIDIA GPU. A local push gate exercises the validated GPU. Repository-wide GPU acceptance still needs an isolated runner.
The CUDA backend supports int32 copies/casts, empty/zeros/ones, wrapping
add/subtract/multiply/negate, integer scalars and sum/max reductions. Division
and mean require float32. The math extension also supports int32
abs, min, argmax, clip and topk. Float32 matmul supports auto and custom, not optimized.
Vulkan / SPIR-V
Section titled “Vulkan / SPIR-V”The optional Linux backend supports GPU copies/factories/views, casts,
transpose, basic indexing/joins, comparisons/masks, boolean/numeric reductions and
broadcast float32/int32 arithmetic and float32 vector/batched matmul.
It has real GTX 980 Ti validation, with no
claim of other-GPU acceptance. Basic float32 arithmetic uses integer GPU
significands for ties-to-even rounding and gradual underflow; arithmetic NaNs
are canonical quiet NaNs. Software Vulkan CPU devices are excluded from
GPU discovery. Numeric sum/max/mean/min/argmax preserve CPU order and execute
serially per output, with mean restricted to float32. Matmul supports
auto/custom, separate multiply/add rounding and broadcast batches.
Log/sqrt/abs/clip/topk also execute on-device; sqrt uses integer root rounding,
log uses a bounded-error series, and clip/topk preserve selection bits.
Float32 activations/norms and inference now execute on-device, including
GPU-validated embedding indices and masked/causal attention. Host/GPU timing,
owned-allocation diagnostics and bounded reuse
are available. Explicit guarded MLIR/SPIR-V kernels
support the bounded float32 subset with private outputs and persistent code
caching. Explicit shared CUDA/Vulkan allocations are available in a combined
build; direct Vulkan DLPack remains outside this slice. See Vulkan setup and limits.
Compiler boundary
Section titled “Compiler boundary”MLIR is an optional compiler path, not a hardware backend. Its current CPU/CUDA/Vulkan runtime integration handles a guarded float32 elementwise subset. See experimental kernels for requirements and non-goals.
ROCm, float16/bfloat16, asynchronous streams and multi-device execution are not implemented. Additional GPU/platform acceptance and broader generated-kernel subsets remain open.
Separate execution experiments
Section titled “Separate execution experiments”NVIDIA OpenCL, PoCL CPU and Mesa lavapipe have passed bounded standalone probes. OpenCL is not a registered runtime backend, and software CPU execution does not extend the GPU support matrix. See the execution experiments for results, reproduction instructions and limits.
Engineering evidence
Section titled “Engineering evidence”Detailed, versioned records stay with the source: