In a study published on the preprint server Arxiv.org, DeepMind researchers describe a reinforcement learning algorithm-generating technique that discovers what to predict and how to learn it by interacting with environments. They claim the generated algorithms perform well on a range of challenging Atari video games, achieving “non-trivial” performance indicative of the technique’s generalizability.

Reinforcement learning algorithms — algorithms that enable software agents to learn in environments by trial and error using feedback — update an agent’s parameters according to one of several rules. These rules are usually discovered through years of research, and automating their discovery from data could lead to more efficient algorithms, or algorithms better adapted to specific environments.

DeepMind’s solution is a meta-learning framework that jointly discovers what a particular agent should predict and how to use the predictions for policy improvement. (In reinforcement learning, a “policy” defines the learning agent’s way of behaving at a given time.) Their architecture — learned policy gradient (LGP) — allows the update rule (that is, the meta-learner) to decide what the agent’s outputs should be predicting while the framework discovers rules via multiple learning agents, each of which interacts with a different environment.

In experiments, the researchers evaluated the LPG directly on complex Atari games including Tutankham, Breakout, and Yars’ Revenge. They found that it generalized to the games “reasonably well” when compared with existing algorithms, despite the fact the training environments consisted of environments with basic tasks much simpler than Atari games. Moreover, the agents trained with the LPG managed to achieve “superhuman” performance on 14 games without relying on hand-designed reinforcement learning components.

The coauthors noted that LPG still lags behind some advanced reinforcement learning algorithms. But during the experiments, its generalization performance improved quickly as the number of training environments grew, suggesting it might be feasible to discover a general-purpose reinforcement learning algorithm once a larger set of environments are available for meta-training.

“The proposed approach has the potential to dramatically accelerate the process of discovering new reinforcement learning algorithms by automating the process of discovery in a data-driven way. If the proposed research direction succeeds, this could shift the research paradigm from manually developing reinforcement learning algorithms to building a proper set of environments so that the resulting algorithm is efficient,” the researchers wrote. “Additionally, the proposed approach may also serve as a tool to assist reinforcement learning researchers in developing and improving their hand-designed algorithms. In this case, the proposed approach can be used to provide insights about what a good update rule looks like depending on the architecture that researchers provide as input, which could speed up the manual discovery of reinforcement learning algorithms.”

The post DeepMind’s AI automatically generates reinforcement learning algorithms appeared first on Artificial Intelligence.

Artificial intelligence (AI) advancements are showing no signs of slowing down. With new developments coming on the market daily, there’s now a decent amount of options for businesses to get a helping hand with many tedious and administrative tasks.

From the marketing perspective, the area with the largest room for productivity gains come from the creative side. Working with hard-to-manage visual assets, which have very little “data” associated with it comes with a wide set of challenges. But, as AI progresses, I anticipate we’ll see major advancements in how AI can help marketing teams make the most of their visuals.

Here are 4 predictions on how AI will enhance visual marketing.

Find-and-Replace Capabilities

For many organizations, change is a constant — branding updates, acquisitions, mascot updates, product launches, etc., lead to small, but continuous design changes. This can be a major pain for marketing and creative teams, who are responsible for making those updates across all pieces of content — website pages, PDFs, print materials. Not only is this incredibly tedious, it also leaves the organization susceptible to human error that can lead to market confusion and brand deterioration.

As AI begins to understand design files, like InDesign and Photoshop, AI models will be able to learn to make these kinds of updates instantaneously. Similar to the find-and-replace feature available for Word documents, AI will be able to recognize a new version of an asset — for example, a logo — and replace the previous version across all design files. These revisions could then automatically be replaced in the organization’s digital asset management system as a new version.

Outdated Collateral Identification

Managing discontinued products is another thing that can take up a significant amount of time for marketers. While this is often most associated with the retail industry, it’s also applicable across many more industries. The hospitality, technology and manufacturing industry, for example, all handle their own form of closures, discontinuations or “sunsets” of products. When a product is discontinued, so begins the painful task of hunting down and removing the product from each and every piece of collateral. It can be tireless and thankless work, but very necessary.

In the not too distant future, it’s likely that AI models will be trained to identify and flag pieces of collateral that contain discontinued items. Using the advancements already being made with customized AI capabilities, AI models would understand how to recognize the product by its physical characteristics, as well as associated SKU number, name and price. When a product is labeled as discontinued, the model could then quickly scan all advertisements, catalogues, pricing sheets, etc., and notify the marketing team of which pieces need to be updated.

Image Personalization

Content personalization is already a well-known application of artificial intelligence — one that’s being taken advantage of by many marketers today. Using customer data, history and preferences, AI models can predict which pieces of content will resonate with each individual customer and serve up that content. While this is applicable for blog posts, videos, ebooks and other text-based content, AI is still yet to offer content personalization for images.

This will likely change in the coming years. Artificial intelligence has had a huge impact on images as of late, enabling the auto-tagging of images with critical information like object, colour and text recognition. Using these additional pieces of information, there’s no reason that an AI model couldn’t surface images in each customer’s preferred style, colors, etc. When connected to a digital asset management system, it would also open up the AI model’s ability to select from each and every image available in the organization.

Asset Suggestions

Suparman Widjaja, technology manager for Verizon’s creative marketing group, shared his insights on how artificial intelligence could help optimize campaign success.

“There are so many opportunities to use artificial intelligence in our industry — specifically, to expand into more marketing and creative use cases. For example, AI could help our team predict the success of certain assets based on campaign goals, and identify assets that would be more successful based on campaign attributes, target segments, season, etc. As we measure the outcomes of these campaigns, we could feed the output back into the AI engine to continually improve its accuracy.”

He continued, “I’d like to be able to train models to analyze our copy and assets to help identify a strategy for each campaign. These kinds of output, while unable to replace human creativity, can significantly reduce the time we spend deciding on the focus for our creative work.”

Artificial Intelligence for Digital Marketing

While artificial intelligence still has a long way to go, there are plenty of ways to infuse AI into your digital marketing strategy today. Download your copy of 8 Ways AI Can Power Your Digital Strategy to see how you can get started.

The post 4 Predictions for the Future of AI in Marketing appeared first on Artificial Intelligence.

Turning data into insights doesn’t happen magically. You must first understand your data and use it to create reports that drive actions. If your competitors are using machine learning and artificial intelligence to automatically drive actions and you aren’t, you are at a disadvantage.

Getting your data ready for ML and AI involves combining structured and semistructured data sets in order to clean and standardize data into a format ready for machine learning or integration with BI and data visualization tools. When you prepare your data correctly, you benefit from insights that can be processed quickly and easily, resulting in faster time to value.

Data transformation and standardization help you build powerful models, reporting, and ad hoc analysis that all share a single source of truth. In fact, not only does data prep help you with AI models, you can use AI in your ETL process to prepare data for the data warehouse itself. For example, you can use AI to extract valuable sentiment data from customer comments without having to read them all. Either way, at the beginning of a data journey, a company’s problem is not analytics or model-fitting, it is data ingestion and transformation.

Based on our customers’ experiences, there are common data transformations required before data is ready for use in machine learning models.

Remove unused and repeated columns: Handpicking the data that you specifically need will improve the speed at which your model trains and unclutters your analysis.

Change data types: Using the correct data types reduces memory resources. It can also be a requirement — for example, making numerical data an integer in order to perform calculations or to enable a model to recognize what algorithms are best suited to the data.

Handle missing data: At some point you’ll come across incomplete data. Tactics for resolving the problem can vary depending on the data set. For example, if the missing value doesn’t render its associated data useless, you may want to consider imputation — the process of replacing the missing value with a simple placeholder or another value, based on an assumption. Otherwise, if your data set is large enough, it is likely that you can remove the data without incurring substantial loss to your statistical power. Proceed with caution. On the one hand, you may inadvertently create a bias in your model; on the other hand, not dealing with the missing data can skew your results.

Remove string formatting and non-alphanumeric characters: You will want to remove characters such as line breaks, carriage returns, and white spaces at the beginning and the end of values, currency symbols, and other characters. You may also want to consider word-stemming as part of this process. Although removing formatting and other characters makes the sentence less readable for humans, this approach helps the algorithm better digest the data.

Convert categorical data to numerical: Although not always necessary, many machine learning models require categorical data to be in a numerical format. This means converting values such as yes and no into 1 and 0. However, be cautious not to accidentally create order to unordered categories, for example, converting Mr., Miss, and Mrs. into 1, 2, and 3.

Convert timestamps: You may encounter timestamps in all types of formats. It’s a good idea to define a specific date/time format and consistently convert all timestamps to this format. It’s often useful to “explode” a timestamp (using a data warehouse date dimension) into its constituent parts — separate year, month, day-of-week, and hour-of-day fields all have more predictive power than milliseconds since 1960.

Getting Started

This list is not exhaustive and is offered as a simple guideline to get you started. There are other factors you may want to consider such as how to handle outliers. You may want to remove them from your data set depending on the training model you use. Retaining outliers may skew your training results, or you might need to include outlier data for an anomaly detection algorithm.

To get the most from data analytics and visualization tools, have your data ready and available for analytics by bringing all the relevant data together in a clean and standardized format to ensure that the data is high-quality and can be trusted. Preparing this as a pipeline of operations within a cloud ETL tool means that when you need to bring more data up to date, potentially from many different external sources, you can just press “Run” again and all data is refreshed.

The post Preparing Data for Machine Learning appeared first on Artificial Intelligence.

PyTorch has become popular within organisations to develop superior deep learning products. But building, scaling, securing, and managing models in production due to lack of PyTorch’s model server was keeping companies from going all in. The robust model server allows loading one or more models and automatically generating prediction API, backed by a scalable web server. Besides, it also offers production-critical features like logging, monitoring, and security.

Until now, TensorFlow Serving and Multi-Model Server catered to the needs of developers in production, but the lack of a model server that could effectively manage the workflows with PyTorch was causing hindrance among users. Consequently, to simplify the model development process, Facebook and Amazon collaborated to bring TorchServe, a PyTorch model serving library, that assists in deploying trained PyTorch models at scale without having to write custom code.

TorchServe & TorchElastic

Motivated by the request from Alex Wong on GitHub, Facebook and AWS released the much-needed service for PyTorch enthusiasts. TorchServe will be available as part of the PyTorch open-source project. Users can not only bring their models to production quicker for low latency prediction API, but also embed default handlers for the most common applications, such as object detection and text classification.

TorchServe also includes multi-model serving, model versioning for A/B testing, monitoring metrics, and RESTful endpoints for application integration. Developers can leverage the model server on various machine learning environments, including Amazon SageMaker, container services, and EC2 (Amazon Elastic Computer Cloud).

The post THIS LATEST MODEL SERVING LIBRARY HELPS DEPLOY PYTORCH MODELS AT SCALE appeared first on Artificial Intelligence.

Artificial intelligence (AI) – understood as intelligence demonstrated by machines – and machine learning (ML) – a subfield of AI that focuses on machines’ ability to automatically learn from experience without being explicitly programmed – are two of the computer science technologies that will arguably have the biggest impact on our lives in the coming decades. AI and ML are not exactly new – both terms date back to the mid-20th century – but it is mainly in recent years that their huge potential has become evident.

The technologies are now being used more and more extensively across a whole range of fields. Artificial intelligence can be found in autonomous vehicles, search engines, online assistants and spam filtering programs, to name just a few of its numerous common applications. Applications of machine learning include computer vision, DNA sequence classification, financial market analysis, Internet fraud detection, medical diagnosis and speech recognition. Both AI and ML are currently transforming whole sectors and will continue to do so in the future.

Pushing the boundaries

Of course, behind that transformation are concrete people, outstanding engineers and scientists who continue to push the boundaries of how theoretical concepts related to computer science and other similar fields can be turned into practical use. Curious folks, always on the lookout for new possibilities. Consider Pierre-Habté Nouvellon, the CTO and Head of Machine Learning at Snipfeed, a California-based startup that has come up with an eponymous AI-based news and information recommendation engine, mainly targeted at young (Generation Z) users.

Snipfeed, which he co-founded, provides daily, highly personalized pieces of news and information (snippets) that are filtered and recommended to its users depending on their needs. The technology used by the engine ensures that Snipfeed avoids recommending articles which do not look credible. An internal tool called Fakebuster detects fake news articles by verifying content, assessing the reliability of the facts in it and the quality of the writing style and the headline.

Falling for ML

Nouvellon discovered machine learning while he was doing a Master’s degree in aerospace engineering in France. He was working on a research project on theoretical supply chain problems and his supervisor advised him to see if reinforcement learning, a subfield of machine learning, could help him complete his task. “I took the well-known Andre Ng course in machine learning and fell in love with it. It was the perfect mix of statistics, algebra and computer science,” he recalled.

After that initial experience, Nouvellon realized that he wanted to study machine learning more thoroughly. He applied for a Master’s program in computer science at the University of California, Berkeley, and was admitted. There he met some of the leading professors in the field. Once in Berkeley, Nouvellon and two of his friends started working on an AI-based tutor called Jenyai, a conversational bot on messenger that was able to answer students’ questions in math, history and science.

One of the application’s features was a selection of news stories meant to help students better understand the discussed topics. This feature proved to be very popular with Jenyai’s users, which inspired Nouvellon and his partners to create Snipfeed. Nouvellon also had a chance to apply his knowledge of machine learning in a completely different field – healthcare. He was involved in a medical research project done by the University of California, Berkeley, and the University of California, San Francisco.

Driven by curiosity

The project’s goal was to apply machine learning algorithms to predict the occurrence of a certain disease among a major pool of patients by analyzing those patients’ electronic medical records and genotype data, provided by a large medical insurance group. The application of ML technologies in medicine and healthcare is set to become more and more commonplace in the near future, but such technologies will also be transforming many other sectors. The revolution will be driven by people like Nouvellon, who are always inspired by the new and unknown.

Becoming an inventor is something that he thought of from his early years. Nouvellon revealed that he had grown up in a family were science and creativity played a central role. Already when he was a child, he liked innovating, playing with such ideas as making up a new language and building a complex electric circuit. He would dream of becoming an astronaut, a chemist or a mathematician. “Inventing is doing something that has not been done before and thus there is an adventurous and exciting aspect to it that I have always loved,” Nouvellon admitted.

The post “The perfect mix.” The story of an ML engineer appeared first on Artificial Intelligence.

Taking good-quality photographs can be a challenging task, as it typically requires finding ideal locations, angles and lighting conditions. Although artistic pictures have so far primarily been taken by human photographers, in recent years, some researchers have started investigating the possibility of taking pictures automatically using robots.

To this end, a team of researchers at Purdue University and Adobe Research has recently developed a new framework that allows a robot to automatically capture photographs of humans, specifically portraits. This framework, presented in a paper prepublished on arXiv, makes robots follow a human user to an ideal spot and then take a picture of them.

LeRop, the framework developed by the researchers, was specifically designed to take indoor portraits of human subjects. First, the framework guides a robot towards a favorable or desired location for taking pictures and then it uses a photo evaluation model to propose the best views, as well as deep reinforcement learning (DRL) model to adjust the robot’s position and orientation to ensure the best lighting conditions.

“When composing is activated, the robot attempts to adjust its position to form the view that can best match the given template image and finally takes a photograph,” the researchers wrote in their paper. “A template image can be predicted dynamically using an off-the-shelf photo evaluation model by the framework, or selected manually from a predefined set by the user.”

LeRop is an interactive framework, as users can program it to follow one target to the spot where she wishes to capture a photograph. Once the user reaches that spot, the robot starts searching for the best view to capture. LeRop’s DRL component is what ultimately allows it to adjust its viewpoint based on how it matches template images.

The researchers decided to equip the robot with a 360-degree camera and a high-quality main camera, as this allows it to have a full view of its surroundings at all times without continuously rotating or switching to different viewpoints. Interestingly, the framework has a modular structure, which means that all its models can be replaced or adapted based on a user’s needs.

The team evaluated LeRop in a variety of trial tests on three indoor scenes, integrating it within a simple robot called Turtlebot. The robot took 20 photographs of a human user in each of these three settings, 10 with a predefined template and 10 using templates that it generated dynamically.

The average number of adjustments made by the robot was 11.20 for predefined templates and 12.76 for dynamically generated ones. In addition, using the researchers’ framework, the robot could take pictures in an average time of 22.11 seconds using predefined templates and 22.40 seconds using dynamically generated ones.

The LeRop framework could be a first step towards the creation of an efficient robot photography system that can take high-quality portraits quickly and automatically. However, the system developed by the researchers still has a number of limitations that could prevent it from being adopted on a large scale. For instance, so far, it has limited on-board computation power, and thus only works on a powerful remote computer.

In addition, the researchers have so far only integrated it within Turtlebot, which is a simple robot with very few degrees of freedom. In their next works, they would like to test their system on a more complex robot with more degrees of freedom.

“Our system currently only supports a single-person portrait,” the researchers wrote. “New policies would need to be re-trained to get better support on taking group photos. In future work, we also would like to test different photo evaluation aesthetic models and extend the work to outdoor scenes.”

The post LeRop: A deep learning-based model to automatically capture human portraits appeared first on Artificial Intelligence.