Where do you see high-performance Edge AI making the most impact?

Automotive autonomy and driver assistance (ADAS) immediately comes to mind. To make the roads safer we increasingly let technology take the wheel – whether it’s supporting drivers or replacing them altogether. This takes a lot of AI (Artificial Intelligence), which translates into a lot of AI processing power that needs to be very efficient and be spread throughout the entire vehicle, including the vehicle edge. We saw a natural fit for our architecture here, so we designed an automotive-grade chip. Euro NCAP Vision Zero is something we feel strongly about, and our technology helps implement this vision in everyday cars.

Security is another field where AI is creating enormous value. It might not be a revolutionary leap like in Automotive, but AI is penetrating in a big way. AI for surveillance and security operations – private, commercial and governmental – are all looking for better quality and efficiency gains. You don’t want to miss a single intruder, tailgater or emergency situation, and you certainly don’t want to spend your whole day going through false alarm notifications. You want to be able to react and even prevent bad things from happening. That is where AI comes in and people’s appetite for it is just getting bigger. Nowadays, they don’t just want the camera and the neural network to be there, they have an expectation of how they should perform: real-time, high-accuracy and for a wide coverage area. They require significant AI processing capacity in their devices and thanks to the emergence of purpose-built processors like the Hailo-8, this is accessible – the barriers to implementation are low.

What are the major pain points customers experience with designing Edge AI projects?

One major pain point we keep hearing about is power limitations. Everyone wants as much performance and as many features as they can pack into their camera, NVR, IPC, etc., and one of the major restraints on that is power consumption – both in terms of how much power you can deliver to the entire device (which is crucial for battery-powered devices) and in terms of the heat generated by the processor and other elements in the device, where you are limited in how much you can dissipate.

To take intelligent cameras as an example, in our experience, manufacturers need an AI processor to fit into a 2-3W envelope. It might be because it’s a system that is constrained by its power supply, or a system that must be fanless because it is intended to work in environments that prohibit fan cooling due to dust or vibrations or because of various standards. You can read more about this in our Practical Guide to Power Efficiency. The reason this is a particular pain point is that at such low a power, you usually get very little performance using most of the processors on the market due to their poor efficiency. This is usually where manufacturers find us and are very impressed with how much we are actually able to do at a few Watts. It’s always an amazing moment for us, too, when they see it happening on a real chip running inside their system.

The problem of power is amplified in the Automotive world. As we mentioned earlier, what the industry is trying to do requires more and more AI TOPS within the car. Imagine the heat that this kind of system needs to dissipate! And, of course, power supply is also an issue, not only when it comes to electric vehicle batteries but gas milage, too. In this kind of scenario, every component, small and large alike, needs to pull its weight in terms of efficiency.

What was your vision when you founded Hailo?

We founded Hailo in 2017 because we saw an opportunity for technological innovation that could help solve real-world problems. Edge AI is succeeding because it is an answer to existing problems, as opposed to a technology looking for problems to solve.

The first waves of the modern Deep Learning revolution demonstrated the technological capabilities and did a lot to convince of its utility. The next wave – the one we are part of – does not need to convince anyone of its “why” – the usefulness and versatility are clear. All we needed to do was bring that technology to the small, low-power form factor in a meaningful way and make it work well there.

One of the beautiful things about what we do is how practical it is. You can see the effect and understand the value easily. When your system is able to automatically recognize that a car is going too fast or in the wrong direction, that someone is not wearing a facemask or a hardhat or that the item off the production line is defective – you can see it. You can also understand what that capability means to customers and what implications speed, accuracy and reliability have.

Solving human problems is a continuous effort. No matter how wide the variety of use cases we think we know and are working to address, new ones continue to appear. There is always that walk-in customer who’s heard about your solution and thinks it’s a great fit for his product, which is different from anything you have ever done or thought of doing.

What is Hailo doing differently?

First, we took a different approach to designing an AI processor. We started designing our hardware from designing software. For about the first two years we only had software engineers. This period was crucial to conceptualizing and planning our patented architecture. Where many others would go for high-level languages, we started practically from scratch. We wanted to speak the language of machine learning. The basic rules and properties of deep of neural networks dictated what our software would look like, which, in turn, structured our hardware. One obvious example of that is that we structured everything around the parameters, operations and pipeline of the neural network. The resources of memory, compute and control are the building blocks of the hardware and the software is built around allocating these resources for a given NN or task.

Secondly, we don’t just want to sell a product. We have seen that we become our customers’ trusted advisors on everything to do with edge implementation of machine learning. Many customers have told us how significant that is for them, for their decisions about the right choices and directions. We see a lot of value in that, which is we have invested in developing a strong ML expertise in-house and edge-specific practical know-how. This is also why our ML and hardware engineers publish and present research in leading industry publications and forums (check out this impressive recent example) – that’s not a trivial thing to do as a startup, but we believe it benefits us greatly.

And thirdly, I would say that our insistence on openness and transparency is not an obvious one. Many vendors try to take you to a closed garden. In some cases, they might not be very transparent from the start, and it is only later that the customer realizes he is captive and that the capabilities and differentiations are not quite what he was sold on. This is not what Hailo is about. We pride ourselves in our ability to attract and retain customers on the merit of our solution and the strength of our partnership with them. As I mentioned before, our goal is to be trusted advisors, not just to sell chips.

Having come such a long way, where do you see Hailo going next?

As AI processing becomes ubiquitous and a lot of it becomes sensor-adjacent, we want Hailo to be the most common solution for edge AI processing. We see this happening across industry verticals – from smart security for Smart City and Smart Home, to Intelligent Analytics for Smart Retail and Smart Factory. As our phones and personal compute devices have become multi-sensor and AI-enabled, so will so many other devices we use. Intelligent vision and voice are the future of our communication with machines and with each other, and Hailo’s AI processing technology is a major enabler of those. We are on our way to that, empowering new use cases and capabilities, partnering with manufacturers and solutions providers who are at the forefront of this technology.

5 Questions on the Edge (AI) is series where Hailo thought leaders talk about market and customer trends, technological challenges and what the future holds for Edge AI and Hailo.