Have you ever thought about an AI-based machine playing poker with you? If your imagination has gone that wild then Facebook is all set to make it a reality with its new general AI framework called Recursive Belief-based Learning (ReBeL) that can even perform better than humans in poker and with little domain knowledge as compared to the previous poker setups made with AI.

With ReBel, Facebook is also going for multi-agent interactions – which means that the general algorithms will soon have the capacity to be deployed on a large scale and for multi-agent settings as well. The potential applications include workings like auction, negotiations, and cybersecurity or the operation of self-driving cars and trucks.

Facebook’s plan of combining reinforcement learning with search for AI model training can lead to some remarkable advancements. This is because Reinforcement Learning is based on agents learning to achieve goals in order to maximize rewards whereas search is basically defined as a process that starts from the plan to the stage of setting the goal.

One such example is of Deepmind’s Alpha Zero that is based on a similar program to deliver state-of-the-art performance in board games like chess, shogi, and Go. However, the combination falls short when it is being applied for games like poker because of imperfect information that can arise as a result of how the situation in the game changes. Actions then take help from probability or the playing strategy.

Hence, proposing a solution to the problem in the form of ReBel, Facebook researchers have now expanded the notion of “game state” while including the agent’s belief which relies on the state they are in while playing – counting the common knowledge and policies of other players as well.

When working, ReBel trains two AI models; one is of a value network and the other is of policy network. There is reinforcement learning happening with search during the self-play which eventually has resulted into a flexible algorithm that now holds the potential to beat human players.

For a high level, ReBel operates with public belief states rather than going for world states. If that has surprised you then public belief states are there to generalize the notion of “state value” in games with imperfect information like Poker. PBS is also more often regarded as a common-knowledge probability distribution over a limited arrangement of possible actions and states, which we sometimes call history as well.

Now in perfect-information games, PBS can be distilled down to histories just like the way it distills down to world states in two-player zero-sum games. Not to forget that a PBS is actually the decisions that a player can and also the outcomes of the possibilities on one hand.

As soon as ReBel starts to work for every new game, it creates a “subgame” in the beginning which is very much similar to the original one, except for the fact that its roots go back to the initial PBS. The algorithm actually wins by repeating the runtime of “equilibrium-finding” algorithm and then take advantage of the trained value network to create estimates on every stage of the iteration. Furthermore, with enforcement learning, the values come out easily and then added back to the network as training examples. The policies in the “subgame” are also added as examples. The process continues to repeat itself until PBS becomes the new subgame root and completes a certain accuracy threshold.

The researchers also benchmarked ReBel, as a part of the experiment, for games of heads-up no-limit Texas hold’em poker, Liar’s Dice, and turn endgame hold’em. They used 128 PCs with eight graphic cards only to generate the stimulated game data and of course place random bets and stack sizes (ranging from 5000 to 25000 chips) to test its abilities.

ReBel was also trained on a game with one of the best heads up poker players in the world Don Kim and the results turned out to be ReBel playing faster than two seconds per hand across 7,500 hands and how it didn’t take more than 5 seconds for any decision. Overall ReBel scored 165 thousandths – which is a pretty good result when compared to the previous poker playing system by the social media giant Libratus that resulted in 147 thousandths.

To prevent cheating, Facebook has decided that they will not release ReBel’s codebase for Poker. The company only open-sourced Liar Dice’s implementation, which according to researchers is easier to understand and adjust.

The post Facebook’s New Algorithm Can Play Poker And Beat Humans At It appeared first on Artificial Intelligence.

his might seem a particularly bad time to be investing in big tech.

President Trump said Tuesday morning that his administration would look into accusations that Google has been secretly working with the Chinese military. The charge came from Peter Thiel, a PayPal (PYPL – Get Report) co-founder and strong supporter of the president.

On the other hand, Bloomberg reported Tuesday that DeepMind, the artificial-intelligence arm of Alphabet,  (GOOGL – Get Report)  might be on the cusp of a major breakthrough in the way new drugs are discovered.

It’s an important innovation. It’s hiding inside the search giant. And it couldn’t come at a better time.

This business is on to something really big. Using data, machine learning and AI, Alphabet managers are incubating vibrant new businesses with innovative business models. One or more of these will become exciting stand-alone businesses.

Some analysts are already doing sum-of-the-parts analyses and they like what they see.

A Jefferies analyst pegged the value of Waymo, Alphabet’s self-driving-car business, at $250 billion in December 2018, according to a story at Business Insider.

Alphabet’s market capitalization is $798 billion, with units including YouTube, Google Search, Google Cloud, Android, the Nest security camera and peripheral businesses, Google Capital, and Stadia, its new video game streaming service set to launch in November.

Together, these parts are probably worth well over $1 trillion.

Until now, the business opportunity for DeepMind was not even on investors’ radar.

The subsidiary has its roots in DeepMind Technologies, a British AI startup that was making progress teaching computers the quirks of human short-term memory. Alphabet acquired the business in 2014.

Two years later, its custom AlphaGo code was so advanced that it became the first computer program to defeat a human in a match of Go, the ancient Chinese strategy game. That human happened to be Lee Sedol, the 18-time world champion.

At the CASP13 meeting in Mexico in December 2018, DeepMind was at it again. This time its human challengers were the brightest minds in biology. The task was predicting the shapes of proteins.

Understanding these structures is important because they govern how diseases form. The problem is there are more possible protein shapes than there are atoms in the universe, Bloomberg notes.

The math has vexed computational biologists for the past 25 years. They have been trying to build more predictive software models for protein folding, the process that leads to proteins taking three-dimensional shapes.

Despite its limited experience with folding, AlphaFold, DeepMind’s entrant, predicted the most accurate structure for 25 out of 43 proteins, taking the top spot over 98 participating teams, according to a report in the Guardian.

For perspective, the second-place team accurately predicted only three of the 43 proteins.

This does not mean Alphabet has an inside track to the next big drug discovery. It doesn’t work that way. Developing new drugs is both expensive and fraught with regulatory hurdles, patient trials and marketing expenses.

Even then, a 2013 study published by Nature Review Drug Discovery found that only 10% of medicines in development ever reach patients.

The business opportunity is increasing those odds.

In The Future Awakens, a November 2017 research study by Deloitte Center for Health Solutions, analysts posit that by 2022 medicine will be predictive, preventative (based on risk), personalized and participatory.

Computational biologists in hoodies and jeans will build personalized drug treatments based on what they know about a patient’s individual genomic makeup. Behind the scenes, data scientists using A, will comb through algorithmic models, looking for previously unseen biomarkers.

DeepMind has come out of nowhere to be a major player in that ecosystem, and it is hiding inside Alphabet shares, practically for free.

The parent’s stock trades at 21 times forward earnings and 5.6 times sales. These metrics reflect the consensus view that Alphabet is an advertising business, subject to regulatory attacks.

The regulation is coming. That’s true.

But the story of the stock is its valuable pieces. Investors are fretting about a potential breakup of Alphabet. They should be embracing that possibility. It will lead to much higher stock prices as the value of its businesses comes to light.

Growth investors should consider buying Alphabet shares into any material weakness.

The post Google AI on Track to Revolutionize Medicine appeared first on Artificial Intelligence.