NVIDIA adds Python bindings for DLA inference on Jetson AGX Orin
NVIDIA has detailed cuda.bindings.cudla, a Python layer over the cuDLA runtime API that ships in cuda-bindings 13.3.0 as part of the CUDA Python project. The module exposes thirteen functions, nine data types and eight enums, letting developers create a DLA device, load a compiled network from memory, inspect its tensor descriptors, register GPU buffers and submit inference on a CUDA stream without writing C++. On Jetson AGX Orin the cuDLA runtime arrives with JetPack, so no extra SDK is required; models are compiled into loadables using TensorRT's trtexec. Hybrid mode, which sequences GPU preprocessing and postprocessing around DLA execution, is supported, while standalone mode is not. NVIDIA says the identical API works on RTX Spark.
Why it matters
Developers can now orchestrate DLA offload entirely in Python, letting edge applications shift supported inference work off the GPU to save power and leave GPU capacity for other tasks without maintaining C++ runtime code.
Sources
- PrimaryNVIDIA Developer
Written by AIToolMaps from the source above and checked against it sentence by sentence.