Migrating a model
Machine Learning Frameworks
To run your model on the Hailo device you have to first migrate it to the Hailo environment.
The model build process is now made easy through the support of a wide variety of machine learning frameworks, including: Keras, TensorFlow, Pytorch, TensorFlow-Lite and ONNX.
Bring your own model, either built on If your model is built on one of the supported machine learning frameworks as listed above or built on any other network. In that case, we recommend converting it to ONNX or TF-Lite using the Dataflow Compiler.
You can always use our Hailo Model Zoo to download ready-to-use models.
The Hailo Dataflow Compiler can receive models in 32bit float precision.
The Hailo parser which is the first step of the Dataflow Compiler process will translate the original model into a Hailo internal format.
The second step will run the optimizer which will quantize the weights into a 4/8/16-bit integer format (8-bit as default).
Hailo Model Zoo
The Hailo Model Zoo provides deep learning models for various computer vision tasks. The pre-trained models can be used to create fast prototypes on Hailo devices. Main features include:
- A variety of common and state-of-the-art pre-trained models and tasks in TensorFlow and ONNX
- Model details, including full precision accuracy vs. quantized model accuracy measured on Hailo-8™
- Each model also includes a binary HEF (Hailo Executable Format) file that is fully supported in the Hailo toolchain and Application suite (for registered users only)
- Retraining Dockers (for selected models)
- Ability to go through the whole process – parsing, model optimizer, compilation, inference, accuracy evaluation – either on the pretrained model or on a retrained model
Additionally, the Hailo Model Zoo Github repository provides users with the capability to quickly and easily reproduce the Hailo-8’s published performance on the common models and architectures included in our Model Zoo.Read more
Hailo Dataflow Compiler
The Hailo Dataflow Compiler is responsible for converting the model from the original format all the way to a binary format that can run inference on the Hailo device.
There are three major steps in the compilation process:
- Model Parser - translating the original model into Hailo format
- Model Optimizer - quantizing the model and optimizing the accuracy performance
- Compiler - allocating the needed resources and compiling it into a binary format that can run on the Hailo device.
On top of these mandatory steps in compilation there are two very important tools to help with the process:
- Profiler - profiling the expected model behavior while running on the Hailo hardware and predicting how many hardware resources are required to run the model
- Emulator - emulating the accuracy of the model and supporting in running the optimization process.
After migrating your model to the Hailo environment, you will run inference on the Hailo-8™ device using the different APIs Hailo provides in the HailoRT library.
The most basic ones will be used in order to:
- Load your model
- Send data; and
- Receive data.
The HailoRT includes many more APIs and functions to help you easily integrate your application with the Hailo device.
Use the TAPPAS (Template APPlications And Solutions) demo suite to see full demo applications with a single command line.
The TAPPAS demonstrates a variety of applications ranging from classification, detection, and segmentation, all the way up to more complicated pipelines including several models such as LPR and VMS applications.
The TAPPAS code is open and can be viewed to learn how to use the different APIs of the HailoRT for your application.
Other runtime frameworks can be supported and used such as ONNX Runtime, GStreamer and others.
OS IP Stack
Hailo supports PCIe interface as well as Ethernet.
For the PCIe, a driver needs to be installed on your system - either pre-compiled or specifically compiled for your platform.
For the Ethernet interface you have to use the OS IP Stack on your device.
The HailoRT library is your means to communicate with the Hailo device.
From the basic operations like loading models, sending and receiving data, to more complex pipeline management such as scheduling tasks, automatically postprocessing functionality such as NMS support on and off the Hailo device, and more.
The HailoRT supports Python, C and C++ APIs.
Hailo Software Suite
The Hailo Software Suite combines the Compiler and the run time environment into one easy-to-use environment.
This is the most recommended way to get started with Hailo, offering a one-stop-shop for all the Hailo tools in one docker/environment.
Take me to the tutorial
- Hailo-8™ M.2 starter kit Product Brief
- What’s new in SW Suite 2023-01 LTS?
- Hailo Software Suite – Docker 2023-01
- HailoRT – PCIe driver Ubuntu package (deb) V4.12.0
- HailoRT Windows Installer v4.12.0
- TAPPAS – Docker aarch64 V3.23.0
- TAPPAS – Docker x86_64 Ubuntu 20 V3.23.0
- TAPPAS – Docker x86_64 Ubuntu 22 V3.23.0
- TAPPAS – Linux installer V3.23.0
- Hailo Model Zoo – Python package (whl) V2.6.0
- HailoRT – Log Collector tool (bash) v1.0.3
- HailoRT – Log collector tool (whl) V1.0.3
- Hailo Integration tool v1.12.0 X86
- Hailo Integration tool v1.12.0
- Hailo Integration tool v1.12.0 ARM
- HailoRT – Python package (whl) for Python 3.8, x86_64 V4.12.0
- HailoRT – Python package (whl) for Python 3.9, x86_64 V4.12.0
- HailoRT – Python package (whl) for Python 3.10, x86_64 V4.12.0
- HailoRT – Python package (whl) for Python 3.9, aarch64 V4.12.0
- HailoRT – Python package (whl) for Python 3.10, aarch64 V4.12.0