Tensor operations
This guide describes the shared tensor API on CPU, Metal, CUDA and optional
Vulkan. Inspect cx.capabilities(device) and the
support matrix for operation/dtype placement. Storage exchange
and generated kernels have separate limits.
Start with import tensorcx as cx. The following example works on CPU:
import tensorcx as cx
a = cx.tensor([[1.0, 2.0], [3.0, 4.0]], device="cpu")b = cx.ones((2, 2), dtype=cx.float32, device="cpu")
product = cx.matmul(a, b)print(product.numpy())# [[3. 3.]# [7. 7.]]
print(cx.sum(a, axis=1).numpy())# [3. 7.]
print(cx.softmax(a, axis=-1).shape)# (2, 2)
print(((a - 1) / 2).reshape((4,)).numpy())# [0. 0.5 1. 1.5]
print(a.sum(axis=1, keepdims=True).shape)# (2, 1)
values = cx.tensor([[1, 2, 3], [4, 5, 6]])bias = cx.tensor([0.25, 0.5, 0.75])result = values.astype(cx.float32) + biasprint((result - result.mean(axis=-1, keepdims=True)).numpy())# [[-1.25 0. 1.25]# [-1.25 0. 1.25]]
print(values.T.numpy())# [[1 4]# [2 5]# [3 6]]
expanded = values.expand_dims((0, -1))print(expanded.shape) # (1, 2, 3, 1)print(expanded.squeeze().shape) # (2, 3)
print(a.sum().numpy()) # 10.0print(a.mean(axis=(0, 1), keepdims=True).numpy()) # [[2.5]]
print(values[:, ::-1].numpy())# [[3 2 1]# [6 5 4]]print(cx.concat([values, values], axis=0).shape) # (4, 3)print(cx.stack([values, values], axis=1).shape) # (2, 2, 3)parts = values.split([1], axis=1)print([part.shape for part in parts]) # [(2, 1), (2, 2)]
mask = values > 3print(mask.dtype) # boolprint(values[mask].numpy()) # [4 5 6]print(cx.where(mask, values, 0).numpy())# [[0 0 0]# [4 5 6]]print(mask.any(axis=1).numpy()) # [False True]
# Leading batch dimensions broadcast; the last two axes form each matrix.batch = cx.ones((2, 1, 3, 4))weights = cx.ones((1, 5, 4, 6))print((batch @ weights).shape) # (2, 5, 3, 6)print((cx.tensor([1., 2., 3.]) @ cx.tensor([4., 5., 6.])).numpy()) # 32.0
scores = cx.tensor([[1., 4., 9.], [16., 25., 36.]])print(scores.sqrt().numpy()) # [[1. 2. 3.], [4. 5. 6.]]print(scores.log().shape) # (2, 3)print(cx.abs(-scores).min().numpy()) # 1.0print(scores.argmax(axis=1).numpy()) # [2 2]print(scores.clip(4, 25).numpy()) # [[4. 4. 9.], [16. 25. 25.]]top_values, top_indices = scores.topk(2)print(top_values.numpy()) # [[9. 4.], [36. 25.]]print(top_indices.numpy()) # [[2 1], [2 1]]API overview
Section titled “API overview”| Task | API | Contract |
|---|---|---|
| Construct | tensor, empty, zeros, ones, randn |
Explicit shape, dtype, and device; empty contents are unspecified |
| Inspect | Tensor.shape, .dtype, .device |
Metadata for a contiguous row-major tensor |
| Transfer | .to(device), .cpu(), .numpy() |
Explicit device transfers; NumPy export copies to host |
| Elementwise | a + b, a - b, a * b, a / b, -a |
Broadcast-compatible shapes, same dtype/device; real scalars work on either side; division requires float32 |
| Convert dtype | x.astype(dtype), cx.astype(x, dtype) |
Explicit float32/int32/bool conversion on the same device; copies by default |
| Reshape | x.reshape(shape), cx.reshape(x, shape) |
Shares storage; same element count; one inferred -1 allowed |
| Transpose | x.transpose(axes=None), cx.transpose(x, axes=None), x.T |
Reverses axes by default; always copies into a contiguous buffer on the same device |
| Singleton axes | x.squeeze(axis=None), x.expand_dims(axis); also cx.squeeze, cx.expand_dims |
Remove/insert size-one axes as views sharing storage |
| Index / slice | x[key] |
Integers, slices, negative steps, ellipsis and new axes; independent contiguous copies |
| Join / split | cx.concat, cx.stack, cx.split, x.split |
Same dtype/device; split uses equal sections or NumPy-style cut indices; all outputs copy |
| Compare | ==, !=, <, <=, >, >=; cx.equal, etc. |
Same dtype/device; broadcasting and matching scalars; bool output |
| Boolean logic | &, |, ^, ~; cx.logical_and, etc. |
Bool tensors/scalars; broadcasting |
| Conditional selection | cx.where(condition, x, y) |
Bool condition; same-dtype branches; three-way broadcasting |
| Boolean reduction | cx.any, cx.all; x.any, x.all |
Bool inputs; axes and keepdims like numeric reductions |
| Mask selection | x[mask], cx.masked_select(x, mask) |
Bool mask matches leading dimensions; stable contiguous copy |
| Matrix multiply | cx.matmul(a, b), a @ b |
Float32 vectors/matrices/batches; matching contracting dimensions; batch broadcasting |
| Reduce | cx.sum(x, axis=None, keepdims=False), max, mean |
All axes by default; integer or axis sequence; keepdims=True retains selected axes with size 1 |
| Math | log, sqrt, abs |
Float32 log/sqrt; float32/int32 abs |
| Select extrema | min, argmax, topk |
Min supports multiple axes; argmax/topk return int32 indices |
| Clip | cx.clip(x, lower=None, upper=None) |
Float32/int32; broadcast scalar/Tensor bounds |
| Activate | cx.exp(x), gelu, silu |
Float32 |
| Normalize | softmax, rmsnorm, layernorm |
Float32, explicit axis; preserves shape |
CUDA and Vulkan empty allocate directly on the device, without CPU staging, an upload
or a fill kernel. Their contents remain unspecified, including reused pool
storage: write all elements before reading, or use zeros/ones. CPU and Metal
keep their existing allocation paths. See the
CUDA factory contract and validation
and Vulkan allocation scope.
Support varies by backend, particularly for int32. See the
support matrix.
Numerical behavior
Section titled “Numerical behavior”- Float inputs are narrowed to
float32; precision loss and overflow are possible. - Broadcasting aligns axes from the right; dimensions must match or be 1.
(2, 3) + (3,)returns(2, 3). A dimension of 0 paired with 1 stays 0. No dtype promotion or device transfer occurs; the result is contiguous. astypetruncates float-to-int toward zero and rejects NaN, infinity and out-of-range values. Int-to-float can lose precision.copy=Falsereturns the input only when the dtype is unchanged; conversion still allocates.- CPU, Metal, CUDA and Vulkan
int32add/subtract/multiply/negate and sum wrap on overflow. Max compares signed integers. Division and mean require float32. - Scalars preserve the tensor dtype. Int32 requires an integer scalar within int32 bounds; float32 narrows real scalars before arithmetic. Booleans and complex scalars are rejected. Float division follows IEEE infinity/NaN rules.
- Reshape keeps the underlying buffer alive. Experimental Metal kernels writing
through a view also change other views sharing that storage. MLIR CPU/CUDA/Vulkan
launches preserve caller tensors and return private output storage. NumPy export still copies.
Export preserves shape within the installed NumPy’s rank limit; a larger rank
raises
ValueError. Internal tensors may have higher rank: usex.reshape((-1,)).numpy()to export a flattened copy explicitly. Empty shape inference such as(0, -1)is ambiguous and rejected. - Reductions support negative and multiple axes, such as
(0, -1); duplicates are errors.axis=Nonereduces all axes, whileaxis=()returns an independent same-shape copy under the same dtype restrictions. Multi-axis mean sums the selected group then divides once; no intermediate means are taken. Reducing a zero-size axis produces zeros for sum, NaNs for mean, and an error for max. NaNs propagate and float32 accumulation can overflow or lose precision. - Transpose accepts a full axis permutation, with negative indices; even an
identity or scalar transpose owns a new buffer. Squeeze removes all size-one
axes by default, or selected axes. Expand dims accepts one or more positions
in the final output shape:
(2, 3)at(0, -1)becomes(1, 2, 3, 1). Repeated/out-of-range axes and squeezing a non-singleton axis are errors. Both singleton operations share storage using the reshape ownership rules. - Softmax subtracts the maximum before exponentiation. NaNs propagate; slices containing positive infinity or only negative infinities produce NaNs.
- RMSNorm accepts keyword
weight; LayerNorm acceptsweightandbias. Parameters are vectors matching the selected axis (scalars for scalar input). Theirepsmust be finite, non-negative, and representable as float32. - Floating-point accumulation order can differ across backends. Compare with numerical tolerances, not bitwise equality.
Basic slicing supports negative indices/steps and clips slice bounds. Integer selection returns a tensor even for a scalar. Concat joins an existing axis; stack inserts a new one. Split returns a tuple of independent copies. Advanced integer indexing, mixed mask/index tuples, indexed assignment, autograd and general strided views are not implemented. Normalizations require one explicit axis. The experimental kernel subset has its own arithmetic rules and does not extend the ordinary tensor operators.
Boolean masks
Section titled “Boolean masks”Boolean input infers cx.bool; mixed boolean/numeric input needs an explicit
dtype. Numeric casts to bool use nonzero truth, including NaN and infinity;
bool casts back to numbers as 0/1. Arithmetic on bool tensors is rejected.
Comparisons use IEEE rules: NaN compares unequal to every value, including itself.
where selects already-computed branches; it is not lazy control flow. All tensor
arguments must share a device. Mask indexing does not broadcast: a (2,) mask
on (2, 3) selects rows, while a (2, 3) mask produces a flat vector. A scalar
bool Tensor mask adds a leading axis of size 0 or 1. Selection copies preserve
order and stored bits. GPU selection reads back only the selected count to size
its output; it does not export the tensor to NumPy.
any/all accept bool inputs, all/single/multiple axes and keepdims; empty
reductions return false/true respectively. Python bool(x) requires exactly one
element and reads that value to the host; use x.any() or x.all() for a mask.
Batched matrix multiplication
Section titled “Batched matrix multiplication”matmul broadcasts leading batch dimensions and multiplies the last two axes.
For example, (2, 1, 3, 4) @ (1, 5, 4, 6) produces (2, 5, 3, 6).
Rank-one inputs are promoted temporarily: (k,) @ (k,) returns a scalar Tensor,
(m, k) @ (k,) returns (m,), and (k,) @ (..., k, n) returns (..., n).
Scalar operands are rejected. Float32 dtype and device must match.
Empty batch/row/column dimensions yield empty outputs; a zero contracting dimension produces zeros. Outputs own independent contiguous storage. Neither broadcasting nor matmul exports input tensors to the CPU.
CPU accepts auto, cpu, reference; CUDA and Vulkan accept auto, custom. Metal
accepts custom and, when enabled, optimized (MPSGraph); auto selects the
available path. MPSGraph drops common singleton batch axes, then requires at
most 16 graph dimensions including the two matrix axes. Metal auto selects
the custom kernel for input ranks above 16; the custom path has no rank cap.
Empty products and zero contracting dimensions also work with optimized.
Math and selection
Section titled “Math and selection”log and sqrt accept float32, with IEEE NaN/infinity domain results. abs
also accepts int32; its minimum value wraps to itself. min, argmax, clip
and topk accept float32/int32 on CPU, Metal, CUDA and Vulkan. Bool inputs require an
explicit conversion. All seven functions also have Tensor methods.
min accepts the same axes/keepdims options as sum/max. argmax accepts one
axis, or None to search the flattened input; its int32 result identifies the
first maximum or first NaN. Both reject empty reduced dimensions. NaNs propagate
through min and clip; extrema ties treat signed zeros as equal.
Clip requires at least one bound. Bounds may be real scalars or matching dtype/device tensors, and all inputs broadcast. None leaves that side unbounded; a reversed pair of bounds selects the upper bound.
values, indices = cx.topk(x, k, axis=-1, largest=True, sorted=True) replaces
one axis with k. Values retain dtype; indices are int32. k may be zero. NaNs
rank above all numbers, and equal values use the smaller input index first.
sorted=False returns selected entries in original index order. Index spaces
are limited to INT32_MAX. GPU topk is a synchronous correctness-first selection
algorithm; large axes/k can be expensive. See the
full contract.
Inference
Section titled “Inference”cx.linear(x, weight, bias=None) computes x @ weight.T + bias: input ends
in in_features, weight has shape (out_features, in_features), and optional
bias has shape (out_features,). Values and parameters are float32 on the same
device. Affine RMSNorm/LayerNorm keep their existing explicit-axis API.
cx.embedding(indices, weight) gathers a float32 table using int32 indices.
The result has shape indices.shape + (weight.shape[1],). Negative and
out-of-range indices are errors, including for zero-width tables.
cx.attention(q, k, v, attn_mask=None, is_causal=False, scale=None) is an alias
for cx.scaled_dot_product_attention. Shapes are (..., L, E), (..., S, E)
and (..., S, Ev); leading dimensions broadcast and the output is (..., L, Ev).
E must be positive. Boolean masks permit True entries; float32 masks add score
bias, with negative infinity excluding entries. Masks broadcast to the score
shape. Causal masking permits key positions up to the query position, with
upper-left alignment when lengths differ; it can combine with an explicit mask.
is_causal and scale are keyword-only. Scale defaults to 1/sqrt(E) and accepts
finite float32-representable real values.
Fully masked rows have zero probabilities and zero output for finite values. Unmasked NaNs and positive-infinity scores can produce NaNs; nonfinite values can propagate through the final matmul even when their probability is zero. Empty key sequences produce zero output. All arguments are checked for empty inputs as well.
These APIs execute natively on CPU, Metal, CUDA and Vulkan. Attention materializes score and probability buffers and runs synchronously; it is not FlashAttention. There is no dropout, autograd, grouped-query head replication or KV cache. The API contract contains the full shape, numerical and ownership rules.