Skip to content

Backends & support

This matrix describes the implemented pre-alpha runtime, not a promise of production support. All operations are synchronous; tensors are contiguous and CPU/Metal/CUDA expose index 0; Vulkan has indexed routes with one physical GPU validated.

Capability CPU Metal CUDA Vulkan (optional)
Float32/int32/bool tensor copy Yes Yes Yes Yes
Explicit .to() placement / forced same-device copy Yes Yes Yes Yes
Float32 fill, add, multiply Yes Yes Yes Yes
Int32 fill, add, multiply Yes Yes Yes Yes
Float32 subtract, divide, negate and scalar arithmetic Yes Yes Yes Yes
Int32 subtract, negate and scalar add/subtract/multiply Yes Yes Yes Yes
Contiguous reshape (float32/int32/bool) Yes Yes Yes Yes
Transpose to contiguous copy (float32/int32/bool) Yes Yes Yes Yes
Squeeze / expand dims views (float32/int32/bool) Yes Yes Yes Yes
Basic indexing / slicing copies (float32/int32/bool) Yes Yes Yes Yes
Concat / stack / split copies (float32/int32/bool) Yes Yes Yes Yes
Explicit float32/int32/bool conversion Yes Yes Yes Yes
Comparisons and where (float32/int32/bool) Yes Yes Yes Yes
Bool fill, logic, any/all and mask selection Yes Yes Yes Yes
Binary arithmetic broadcasting Yes Yes Yes Yes
Float32 batched/vector matmul Yes Custom + optional optimized path Custom path Custom path
Float32 sum, max, mean Yes Yes Yes Yes
Int32 sum, max Yes Yes Yes Yes
Sum/max/mean keepdims Yes Yes Yes Yes
All-axis / multi-axis sum, max, mean Yes Yes Yes Yes
Float32 log, sqrt Yes Yes Yes Yes
Float32/int32 min, argmax Yes Yes Yes Yes
Float32/int32 abs, clip, topk Yes Yes Yes Yes
Float32 linear, affine norms, embedding, attention Yes Yes Yes Yes
Float32 exp, GELU, SiLU Yes Yes Yes Yes
Float32 softmax, RMSNorm, LayerNorm Yes Yes Yes Yes
Generated kernels Optional MLIR subset Experimental MSL subset Optional MLIR subset Optional guarded MLIR/SPIR-V subset
empty allocation CPU storage Existing CPU staging/copy Direct device allocation Direct device allocation
DLPack exchange (contiguous float32/int32/bool) Yes Not implemented Device 0 Not implemented
Explicit shared allocation Same-device aliases Same-device aliases Matching Vulkan GPU Matching CUDA device 0
Installation and operation/dtype diagnostics Yes Availability/capabilities Yes Yes
Memory snapshots System physical memory Not implemented Device-wide free/total Heap totals + separate owned-allocation diagnostics
Native host profiling Bounded spans Not instrumented Bounded spans Bounded spans
Optional GPU timing No Not instrumented Kernel events Copy/compute timestamps
Persistent compiled-code cache Optional MLIR Experimental MSL Optional MLIR Optional MLIR/SPIR-V
Bounded buffer pool Not implemented Not implemented Explicit opt-in scope Explicit opt-in scope + command reuse

empty has unspecified contents on every backend. CUDA skips CPU payload allocation/upload and can reuse idle storage in an explicit pool. See memory reuse and profiling and the feature overview for the remaining scope.

Explicit accelerator-to-accelerator placement uses synchronous CPU staging; same-device copy=True allocates independent storage. See placement and storage sharing. The separate shared-allocation API creates native CUDA/Vulkan aliases on compatible matching Linux hardware, in either allocation direction. This does not establish actual two-GPU execution.

The mandatory reference backend. It runs without a GPU and is the baseline for device comparison tests. Start here for the simplest installation and debugging workflow. CPU-only CI covers Python 3.11, 3.12, and 3.13.

Apple Silicon is the primary Metal target. Building requires Xcode’s metal and metallib tools. Static kernels implement tensor operations; matmul can use an optional Apple optimized path or the custom correctness-first kernel.

cx.matmul(a, b, backend="custom") selects the custom implementation. backend="optimized" requires the enabled Apple optimized path. macOS CI has configurations with that path enabled and disabled. A job that skips device tests does not establish hardware execution.

CUDA is an opt-in backend. Its recorded real-device validation is limited to a GTX 980 Ti (sm_52) on Linux x86_64 with CUDA 12.4 and GCC 13. Tests compare outputs with CPU references and include native contracts and GPU sanitizer runs.

Additional NVIDIA architectures and toolchain combinations still need their own validation. The hosted CUDA build job tests compilation and no-device behavior; it does not run on an NVIDIA GPU. A local push gate exercises the validated GPU. Repository-wide GPU acceptance still needs an isolated runner.

The CUDA backend supports int32 copies/casts, empty/zeros/ones, wrapping add/subtract/multiply/negate, integer scalars and sum/max reductions. Division and mean require float32. The math extension also supports int32 abs, min, argmax, clip and topk. Float32 matmul supports auto and custom, not optimized.

The optional Linux backend supports GPU copies/factories/views, casts, transpose, basic indexing/joins, comparisons/masks, boolean/numeric reductions and broadcast float32/int32 arithmetic and float32 vector/batched matmul. It has real GTX 980 Ti validation, with no claim of other-GPU acceptance. Basic float32 arithmetic uses integer GPU significands for ties-to-even rounding and gradual underflow; arithmetic NaNs are canonical quiet NaNs. Software Vulkan CPU devices are excluded from GPU discovery. Numeric sum/max/mean/min/argmax preserve CPU order and execute serially per output, with mean restricted to float32. Matmul supports auto/custom, separate multiply/add rounding and broadcast batches. Log/sqrt/abs/clip/topk also execute on-device; sqrt uses integer root rounding, log uses a bounded-error series, and clip/topk preserve selection bits. Float32 activations/norms and inference now execute on-device, including GPU-validated embedding indices and masked/causal attention. Host/GPU timing, owned-allocation diagnostics and bounded reuse are available. Explicit guarded MLIR/SPIR-V kernels support the bounded float32 subset with private outputs and persistent code caching. Explicit shared CUDA/Vulkan allocations are available in a combined build; direct Vulkan DLPack remains outside this slice. See Vulkan setup and limits.

MLIR is an optional compiler path, not a hardware backend. Its current CPU/CUDA/Vulkan runtime integration handles a guarded float32 elementwise subset. See experimental kernels for requirements and non-goals.

ROCm, float16/bfloat16, asynchronous streams and multi-device execution are not implemented. Additional GPU/platform acceptance and broader generated-kernel subsets remain open.

NVIDIA OpenCL, PoCL CPU and Mesa lavapipe have passed bounded standalone probes. OpenCL is not a registered runtime backend, and software CPU execution does not extend the GPU support matrix. See the execution experiments for results, reproduction instructions and limits.

Detailed, versioned records stay with the source: