IBM’s Uncertainty Qualification 360 is an open source library of Python algorithms for quantifying, estimating, and communicating the uncertainty of machine learning models.

IBM has created an open source Python library, called Uncertainty Qualification 360 or UQ360, that provides developers and data scientists with algorithms to quantify the uncertainty of machine learning predictions, with the goal of improving the transparency of machine learning models and trust in AI.

Available from IBM Research, UQ360 aims to address problems that result when AI systems based on deep learning make overconfident predictions. With the Python toolkit, users are provided algorithms to streamline the process of quantifying, evaluating, improving, and communicating the uncertainty of predictive models. Currently, the UQ360 toolkit provides 11 algorithms to estimate different types of uncertainties, collected behind a common interface. IBM also provides guidance on choosing UQ algorithms and metrics.

IBM stressed that overconfident predictions of AI systems can have serious consequences. Examples cited included a chatbot being unsure of when a pharmacy closes, resulting in a patient not getting needed medication, and the life-or-death importance of reliable uncertainy estimates in the detection of sepsis. UQ exposes the limits and potential failure points of predictive models, enabling AI to express that it is unsure and increasing the safety of deployment.

Previous IBM efforts to advance trust in AI have included the AI Fairness 360 toolkit, which mitigates bias in machine learning models; the Adversarial Robustness Toolbox, which is a Python library for machine learning security; and the AI Explainability 360 toolkit, which helps users comprehend how machine learning models predict labels.

The post IBM Python toolkit measures AI uncertainty appeared first on Artificial Intelligence.

Filter out the noise…

As computer vision and machine learning get more powerful, businesses are using live video streaming analytics to add value to their operations — from assessing flaws on production lines, to conducting dynamic crowd analysis, or more prosaically, digital doorbell firms sending alerts for package delivery.

Analysing video footage in real time is challenging, however; not least because it requires keeping up with video frame rates of up to 60 frames per second. Compute is not cheap – particularly with streams being run through neural networks for analysis – and businesses can rack up costly bills fast.

A new tool open sourced by Microsoft, Project Rocket, aims to help tackle that problem by rethinking how live video streams are analysed.

Project Rocket: A Video Streaming Analytics Filter

Project Rocket, open sourced this week by Microsoft, aims to tackle the problem by helping users reserve resource-intensive operations for when they’re really necessary. While computer vision and deep neural networks (DNNs) are becoming hugely more powerful, they still need the individual frames to work on: a costly, time-consuming bottleneck.

Project Rocket lets users build a video pipeline that includes a cascade of DNNs in which a decoded frame is first passed through a relatively inexpensive “light” DNN like ResNet-18 or Tiny YOLO with a “heavy” DNN such as ResNet-152 or YOLOv3 invoked only when required for more robust analysis. The platform can be plugged into any TensorFlow or Darknet DNN model.

Users can also augment the above pipeline with a simpler motion filter based on OpenCV background subtraction, as shown in the figure below.

In a blog by principle researcher Ganesh Ananthanarayanan and team, Microsot’s Ananthanarayanan said the platform has been tested on smart city applications, including via a partnership with the city of Bellevue, Washington.

This was used in the city for aggregate car and bicycle counts provided by a system built on the framework that let city policy makers accurately assess the value of adding a bike lane to its downtown area.

The Rocket platform is written in .NET Core, which is compatible on Windows, as well as Linux, comes with instructions to create Docker containers, and –nof course – comes with code to help invoke customised machine learning models in Microsoft’s Azure cloud. For enterprises making use of video analytics and who haven’t coded together something passingly similar already, the toolkit looks well worth a look. Project Rocket’s code is here.

The post Microsoft Open Sources Live Video Analytics Toolkit appeared first on Artificial Intelligence.