Powerful Edge AI to Boost Intelligent Traffic Monitoring

Our cities are growing smarter as local and federal, private and public actors introduce AI and IoT technologies to better manage and plan them. Alongside energy efficiency, pollution reduction, and water and waste management, managing traffic and urban mobility are central pillars in what makes a Smart City “smart” (1).

Traffic congestion plagues metropolitan areas across the world, impacting the environment and quality of life in these areas, as well as local and national economies. According to recent research by INRIX, time lost in traffic has cost the US economy $53 billion in 2021, while on the other side of the pond the UK and German economies lost £8 billion and €3.5 billion, respectively (2).

As with many complex problems, AI is increasingly applied to solve traffic, mobility, and transportation management issues (3). Traffic congestion is a complex problem that requires a multi-faceted approach where data collection and real-time monitoring are important tools. The former informs strategy and planning, while the latter provides the tactical knowledge that enables to take timely action on the ground. This is where edge-based Intelligent Video Analytics (IVA) comes in.

What Smart Cameras and Intelligent Video Analytics Are Used For

Urban areas are characterized by narrower spaces and a higher density of vehicles. While street speeds are relatively low (typically under 50 km/h), road occupancy is very high in specific bottleneck areas at specific times. Points of entry and exit of the city are such spaces, where major intercity arteries drain into narrower streets with junctions and red lights.

To improve traffic management, ITS (Intelligent Transportation Systems) solutions are being deployed to monitor intersections and road segments, especially in the identified bottleneck areas. Real-time monitoring is used for smart red-light optimization that reacts to the state of traffic in the intersection and the roads feeding it. When deployed in multiple intersections and optimized across the network of monitored locations, it can provide meaningful congestion relief, whether as a standalone measure or as part of a larger strategy. It can also be used to improve public transport services – a dominant strategy that seeks to improve public alternatives with the goal of minimizing the number of vehicles on the road. This may be done by monitoring for enforcement of public transportation lanes and prioritizing public or communal vehicle traffic (buses, taxis, light rail, ride-sharing, etc) across the urban space.

Monitoring also makes possible wider traffic law enforcement that is necessary for road safety but also for traffic regulation. Deterring drivers from running red lights, entering busy intersections unlawfully to disrupt crossing traffic, parking or stopping outside of designated areas and so on, go a long way to minimize ad hoc disruptions that create or contribute to congestion.

Enforcing safety and traffic laws are also important in inter-city space. Inter-urban areas are usually multi-lane motorways with higher-speed but lower vehicle density. Here, traffic accidents are often fatal and road obstacles are potentially more disruptive and dangerous. Monitoring traffic for speed limit enforcement, rules of the road compliance, helmet and seatbelt use and so on, not only saves lives and safeguards people’s health but also minimized disruptions to the regular traffic flow.

With effective “eyes” on the road, authorities can identify irregular events and act effectively, be it reacting to an emergency in real-time or implementing policies and plans for longer-term prevention and readiness.

ITS contribution to de-congestion is also in its ability to automate tolling and make it more efficient. Cameras that identify vehicles, read their license plates (a function known as LPR: License Plate Recognition or ANPR: Automatic Number Plate Recognition) and charge the owner/subscriber automatically mean a shorter slow-down or stop at a toll checkpoint or even no stop at all on some highways that use free-flow tolling solutions.

Traffic Monitoring Challenges – Where More Powerful Edge AI Can Help

Whether in the city or outside of it, current traffic monitoring solutions have significant challenges in dealing with increasingly common complex traffic scenarios and more robust AI functionality requirements. These call for more powerful and efficient dedicated AI processing.

First, congested areas mean many vehicles and possibly other objects in a wide field of view (FOV, an intersection, overpass, or a multi-lane stretch of road). This multitude of objects that need to be detected, identified and tracked by the system typically requires more processing power, be it on-camera or on another local device.

Secondly, more often than not, adequate coverage of the FOV will need multiple cameras (to cover different parts and angles of an intersection or both directions of 2-3 lane road, for example). Moreover, if we need to do more than just detect the vehicle – if we need to read its license plate or see if the driver inside has fastened his seatbelt and is not using his mobile phone – those cameras, depending on their type and position, may need to be high resolution. More sensors and higher resolutions pose steeper requirements for computing throughput (you can read more about that in this dedicated Hailo blog post).

Another key parameter is how much the ITS device is expected to do. Multiple system functionalities and more robust metadata require more processing power, especially if we don’t want to compromise on the quality of our deep learning networks. AI processing requirements will vary substantially between a system that merely counts the number of vehicles in its FOV and a system that monitors for speed, lane and red light violations, combined with License Plate Recognition (LPR) to identify and punish offenders. To do a good, accurate job, the first example may employ 1-2 relatively basic Neural Network models, while the second may require object detection, a model for multiple objects tracking and a 3-stage LPR functionality. Some of those may need to be higher-end, more computationally intensive models to achieve higher accuracy.

A challenge that is more characteristic to inter-urban environments is the need to capture vehicles traveling at high speeds. Different highways and road segments have varying speed limits – from minimal speeds of 70-80 km/h, through typical 100-130 km/h speed limits and up to unlimited speeds on sections of high-quality roads (like the famous German autobahns). It takes a dedicated, often high-end, intelligent camera and software to perform accurate LPR even at bottom of this range, not to mention vehicles that exceed the speed limits. From an AI processing perspective, good accuracy (low false negative and false positive rates) requires a real-time (30 and even 60 FPS) processing output rate.

This is especially important for speed limit enforcement as well as free-flow tolling on such roads, where the camera has mere seconds to identify the vehicle, find its license plate and read it to be able to charge or fine the driver before the vehicle is out of its FOV. Occlusions such as vehicles/trucks in-between or other obstacles may further reduce the time the camera has to identify the vehicle.

On highways, these speed-related complexities are added to the same multi-sensor, multi-function and high-resolution FOV coverage requirements that exist in cities. Many inter-urban roads have more than 2 lanes in each direction, requiring more than one camera for optimal coverage. The ITS device may need to capture as many as 6 or 8 lanes, with periodic congestion, and also be required to produce rich metadata about the tracked vehicles and their movement.

The biggest challenge by far, is meeting all the above requirements in a fanless, ruggedized, and/or weatherproof, low-power system. This is true whether we use a smart camera or a dedicated analytics device like an Edge AI Box. AI being deployed on edge devices allows to reduce dependency on connectivity to cloud services for video analytics. This results in a lower costs, both by minimizing video transmission from large, geographically spread deployment of cameras or devices (not to mention mobile vehicle-mounted traffic monitoring systems), and by eliminating the high cost of cloud services for these applications.

These edge environment challenges are mitigated by the new generation of AI processors, which are optimized for AI and thus offer data center-level throughput at unprecedented power-efficiency. This kind of processing makes it possible to accommodate multi-sensor and robust analytics functionality on surveillance cameras and devices for traffic monitoring and many other applications in Smart City and ITS, including parking management and enforcement, public transportation management (monitoring stations or busses and trains), public spaces monitoring and access control for vehicles (4).

This post is part of a two-part series all about Powerful Edge AI for Intelligent Transportation Systems. Subscribe to stay tuned for part 2!

Sources

1. European Parliament, AIDA committee briefing: “Artificial Intelligence in smart cities and urban mobility”. https://www.europarl.europa.eu/RegData/etudes/BRIE/2021/662937/IPOL_BRI(2021)662937_EN.pdf
2. 2021 INRIX Global Traffic Scorecard, December 2021. https://inrix.com/scorecard/
3. Lakshmi Shankar Iyer. “AI enabled applications towards intelligent transportation.” Transportation Engineering, Volume 5, 2021. https://www.sciencedirect.com/science/article/pii/S2666691X21000397
4. Omdia: “Video Analytics Re-imagined: The Impact of Powerful Edge AI”. November 2021. https://ai.hailo.ai/video-analytics-whitepaper 

US DoT Federal Highway Administration. Traffic Congestion and Reliability: Trends and Advanced Strategies for Congestion Mitigation. March 2020. https://ops.fhwa.dot.gov/congestion_report/chapter2.htm