Security robots are slowly becoming a more common sight in malls, offices, and public spaces. But while these bots are often presented as replacements for human security guards — friendly robots on patrol — they’re collecting far more data than humans could, suggesting they’re more like mobile surveillance machines than conventional guards.

A new report from OneZerosheds some light on the scope of the data collection, featuring marketing material and contracts between Knightscope and various city councils. Both show that the main purpose of these robots is gathering data, including license plates, facial recognition scans, and the presence of nearby mobile devices. It’s the sort of constant low-level surveillance that only a machine can perform.

Exactly what each robot collects differs, as Knightscope leases its bots rather than selling them outright, tailoring each contract to customers’ needs. But it’s a fair bet that if you’ve seen one of these machines in person, it’s recorded your presence in one way or another.

As an internal report by California’s Huntington Park Police Department (HPPD) published by MuckRockbackin August noted, “Knightscope’s secret to the K5 robot is simply sensors — lots of them.”

HPPD started leasing a Knightscope K5 robot to patrol parks and buildings this June, and the robot soon made headlines when a passerby pressed its emergency button to report a nearby fight, to no effect. According to NBC News, the bot ignored the woman and continued moving down its preprogrammed path “humming an intergalactic tune” and pausing to tell visitors to “please keep the park clean.”

Stories like this suggest that, as a replacement for human security guards (people who can respond intelligently and spontaneously to emergency situations), Knightscope’s machines are lacking. But as surveillance devices, they have a lot of potential.

The report from the HPPD notes that the robots can identify nearby smartphones over an unknown range, recording their MAC and IP addresses. In Knightscope marketing material published by OneZero, this is a central part of the company’s sales pitch, with one slide telling customers: “90%+ of Adults Have Smartphones And Use WiFi When Available.” Recording the presence of cellphones is a subtle form of surveillance

Scanning phones is a subtle form of surveillance with a far-reaching impact. It’s not as invasive as identifying someone by name, but it can be a rich source of information, telling you a lot about someone’s daily routine, like how often they visit a certain area and how long they stay there. As Knightscope says, it can also be used as a proxy to keep out unwanted individuals: just create a whitelist of approved devices, and scan for unfamiliar ones.

It’s a job these robots are well-suited to. They’re dogged and consistent, with the patience of a machine. They can run 24 hours a day, have infrared cameras to see in the dark, and are, in a way, are less conspicuous than humans performing similar surveillance duties. A robot might be a novelty the first few times you see it, but machines become invisible, blending into the background while continuing to scoop up data.

Knightscope’s robots certainly aren’t physically capable enough to apprehend wrongdoers. They can’t run down criminals or even navigate stairs. And when they’ve made headlines in the past, it’s usually for some sort of pratfall, like when one of their bots drowned itself in a fountain or when another knocked down a toddler in a mall.

So what are they good for? Knightscope maintains that its robots are essentially supplementary devices, meant to compensate for a lack of personnel, to spot trouble and call the police. But in an age when automated systems are replacing humans in more and more fields (think: algorithms making decisions in areas like hiring and benefits), it’s likely they’ll gradually take on a more prominent role, leaning on their surveillance skills.

As roaming security cameras, they’ll continue to make an impact. As John Santagate, an analyst at IDC, told Recode last year, these robots can’t respond to emergencies, but they can intimidate people. “I use the analogy of the police car parked at the corner,” said Santagate. “Even when no one is in it, people around the car adjust their behavior.”

The post Security robots are mobile surveillance devices, not human replacements appeared first on Artificial Intelligence.

Researchers from the Universitat Autonoma de Barcelona and Carnegie Mellon University have developed a technique that could allow deep learning algorithms to learn the visual features of images in a self-supervised fashion, without the need for annotations by human researchers.

To achieve remarkable results in computer vision tasks, deep learning algorithms need to be trained on large-scale annotated datasets that include extensive information about every image. However, collecting and manually annotating these images requires huge amounts of time, resources, and human effort.

“We aim to give computers the capability to read and understand textual information in any type of image in the real-world,” says Dimosthenis Karatzas, one of the researchers who carried out the study, in an interview with Tech Xplore.

Humans use textual information to interpret all situations presented to them, as well as to describe what is happening around them or in a particular image. Researchers are now trying to give similar capabilities to machines, as this would vastly reduce the amount of resources spent on annotating large datasets.

In their study, Karatzas and his colleagues designed computational models that join textual information about images with the visual information contained within them, using data from Wikipedia or other online platforms. They then used these models to train deep-learning algorithmson how to select good visual features that semantically describe images.

As in other models based on convolutional neural networks (CNNs), features are learned end-to-end, with different layers automatically learning to focus on different things, ranging from pixel level details in the first layers to more abstract features in the last ones.

The model developed by Karatzas and his colleagues, however, does not require specific annotations for each image. Instead, the textual context where the image is found (e.g. a Wikipedia article) acts as the supervisory signal.

In other words, the new technique created by this team of researchers provides an alternative to fully unsupervised algorithms, which uses non-visual elements in correlation with the images, acting as a source for self-supervised training.

“This turns to be a very efficient way to learn how to represent images in a computer, without requiring any explicit annotations – labels about the content of the images – which take a lot of time and manual effort to generate,” explains Karatzas. “These new image representations, learnt in a self-supervised way, are discriminatory enough to be used in a range of typical computer vision tasks, such as image classification and object detection.”

The methodology developed by the researchers allows the use of text as the supervisory signal to learn useful image features. This could open up new possibilities for deep learning, allowing algorithms to learn good quality image features without the need for annotations, simply by analysing textual and visual sources that are readily available online.

By training their algorithms using images from the internet, the researchers highlighted the value of content that is readily available online.

“Our study demonstrated that the Web can be exploited as a pool of noisy data to learn useful representations about image content,” says Karatzas. “We are not the first, nor the only ones that hinted towards this direction, but our work has demonstrated a specific way to do so, making use of Wikipedia articles as the data to learn from.”

In future studies, Karatzas and his colleagues will try to identify the best ways to use image-embedded textual information to automatically describe and answer questions about image content.

“We will continue our work on the joint-embedding of textual and visual information, looking for novel ways to perform semantic retrieval by tapping on noisy information available in the Web and Social Media,” adds Karatzas.

The post A new machine learning strategy that could enhance computer vision appeared first on Artificial Intelligence.