Setting Up and Using ONNX Runtime for C++ in Linux

If you want to run machine learning models in a native C++ application on Linux, ONNX Runtime is one of the most practical tools available. It gives you a fast inference engine for models stored in ONNX format (.onnx file), which means you can train or export models elsewhere and then deploy them in a lightweight C++ program without having to bring along an entire Python environment. That combination is especially useful when your application is already written in C++, whether that means a backend service, a robotics stack, a desktop application, or an embedded system. In those settings, Linux and CMake are likely already part of the workflow, so ONNX Runtime fits naturally into the existing build process. In this post, I’ll walk through how to set up ONNX Runtime for C++ using CMake, and then show a simple image classification example to prove the setup works. Setup This is the project structure we'll follow: onnx-classifier ├── CMakeLists.txt ├── external │ └── onnxruntime/ ├── model