Powerful Video Analytics At Scale

Counting the number of people in a specific area is a common task, especially in places where occupancy is limited and has safety implications. Counting could be used, for example, to manage or control load in closed areas like an elevator or a concert hall, to optimize energy consumption in a large building, or to verify there is no living soul in a building during a fire or other emergency.
Machine learning is used for this task to achieve more accurate outcome in challenging cases like partially hidden objects, occlusions etc.
Furthermore, AI-driven scene understanding enables a ‘scene-aware’ counting, i.e., counting in relevant semantic areas within a given frame (e.g., in the illustrated scene – counting people on the walkway and on either side of it). Common neural network models used for such tasks are usually object detection or instance segmentation.

Identification of specific attributes of people’s face, people’s body, vehicles or other objects. These attributes can assist in unique identification of a person or an object in a specific scene, or re-identification of the same unique entity in the same scene over time, or in multiple locations. This data can be used to trigger specific events, or over time to analyze behavior and provide statistics.
Instance segmentation is usually required for this task in order to provide both classification and a unique ID per entity, alongside other models specifically designed for attribute recognition and facial landmark extraction.
A very important aspect of this task, that is very likely to be applied, is the anonymization of Personally Identifiable Information (PII) which is carried out to protect private or sensitive information that connects specific individuals to the stored data, thereby preventing disclosure of raw visual reference of the objects or other sensitive data in the scene.

Calculating the distance between two or more people or objects, based on the accurate 3D location of each entity in a defined space. Vast experience in this type of applications was gained from Advanced Driver Assistance Systems (ADAS), used to calculate the distance between adjacent cars to maintain safe driving distance and provide forward collision warning. This application relies mainly on 3D object detection AI models

Face and person attributes as well as gesture estimation and distance measurement, can all serve to secure and protect an area with restricted access or an unsafe zone. For example, entrance permission or denial can be granted based on facial attributes, without the collection and storage of personally identifiable information. A tripwire application can be used to alert security personnel when people or vehicles cross a pre-defined border and enter a restricted or dangerous area. This can be done in a direction sensitive manner so that one-directional or bi-directional restriction can be set. The advantage of using advanced machine learning algorithms is that the accuracy level is high, and no stray cat or dog will set false alarms. Another benefit of AI is the ability to dynamically set the border and adapt to changing physical conditions, rather than manually sketching a border line which does not accurately fit the contour of the restricted or monitored area.

Another application that can be very helpful for security personnel is searching and tracking lost people based on property indexing and gesture estimation. A quick search for certain keywords can help parents track a lost child in the mall or at the airport, or alternatively can help security locate a suspect in a crowd or large building. The same rational can be applied to locate unattended objects left behind by mistake or in more extreme cases, with intention to cause harm or damage.

LPR can be used for access control and billing in parking lots or garage. In these cases, a camera will usually handle one car at a time. In other cases, such as free-flow tolling, this task needs to be carried out in high-speed and high-resolution, even in poor lighting conditions such as night-time, rain or fog. In this case, a single camera should be able to capture in a matter of seconds the license plate number of multiple vehicles before they move out of the frame. To this end high performance AI-based video analytics are required. For additional information on LPR and Intelligent Transportation Systems, refer to our ITS eBook.

Summarization is another important feature of AI at the interrogation level, creating an edited and concise version of the camera input, which only includes the significant events and insights, while cropping out “dead footage”. For example, an intersection camera is recording vehicles and people crossing throughout the day, but the summary will only present instances in which people and vehicles got too close to each other, based on distance measurement applications, therefor storing only the events of accidents, or near-accidents.