CUDA Tile is a new programming layer from Nvidia to make GPU programming easier.
CUDA Tile makes it possible to write algorithms at a higher level of abstraction, independent of the hardware such as tensor cores. NVIDIA is launching the technology as part of CUDA 13.1.
New programming model for GPUs
CUDA Tile is a new programming layer in Nvidia’s existing CUDA platform. This layer introduces a model in which developers write algorithms based on so-called ’tiles’, or data blocks. The compiler and runtime then take care of the execution on the underlying hardware, such as tensor cores. This should make it easier to develop code that performs well on different GPU architectures, without the developer having to worry about hardware details.
CUDA Tile is an alternative to the existing SIMT model (single-instruction, multiple-thread), which offers fine-grained control over GPU parallelism, but also involves greater complexity. Instead of assigning data to threads and blocks themselves, the developer divides data into larger blocks, after which the compiler assigns them further. The idea is similar to how Python libraries such as NumPy work with matrices.
Underlying layer and integration with existing tools
The technology is built around CUDA Tile IR (intermediate representation), a virtual instruction set that supports tile operations. This new layer works alongside PTX that supports current SIMT programs. CUDA Tile IR is intended for developers of compilers, libraries or domain-specific languages. For most programmers, use will be via higher software layers, such as Nvidia’s cuTile Python.
CUDA Tile is compatible with existing and future generations of Nvidia hardware with tensor cores. The technology is not a replacement for SIMT, but an additional programming option. Developers can combine both models, depending on the nature of their application.
CUDA Tile is available from CUDA 13.1. Documentation, sample code and software are available via Nvidia’s official CUDA Tile page.