Machine learning algorithms that can predict yeast metabolism from its protein content have been developed by scientists at the Francis Crick Institute. The findings could provide a basis for brewers to have greater control over the flavour of their beer, and scientists to personalise treatments for metabolic disorder patients, in the future.

Metabolism is the process by which organisms convert nutrients into energy and essential molecules, via a series of chemical reactions. When yeast metabolises sugar in the absence of oxygen, it ‘ferments’ to produce alcohol, acids and gases, including flavour compounds, that make bread, wine and beer taste good.

Within a cell, metabolism produces hundreds of small molecules, called metabolites. Although yeast is evolutionarily very distant to humans, many of these metabolites are identical, and are made in a similar way. Until now, however, the mechanisms controlling metabolism have not been fully understood.

The latest study, published in Cell Systems, shows that to a large extent, the metabolism of brewer’s yeast (S. cerevisiae) is predictable by machine learning algorithms, if they are provided with large amounts of protein expression information.

“Thanks to machine learning, we now have a better understanding of what controls metabolism, which is good news for brewers looking to create the perfect pint, or for Biotechnologists that use yeast to produce vaccines and other proteins that are medically important ” says Aleksej Zelezniak, first author of the paper and researcher at the Crick, and has recently moved to Sweden to establish his independent research group at the Chalmers University of Technology.

Linking proteins and metabolites

Until now, scientists have been divided over whether metabolism is self-regulating or controlled by gene expression changes; partly because existing methods have failed to detect any strong correlation between the read-out of genes—proteins—and metabolites.

In this study, scientists quantified enzyme expression in 97 different strains of S. cerevisiae, known to show differences in metabolism, linking it to changes in metabolite concentrations measured.

They developed machine learning algorithms that could pick up complex relationships between changes in gene expression and metabolites produced. They found that metabolism was controlled by lots of enzymes acting in concert—with no single enzyme having a major effect by itself.

“The relationship between enzyme expression and metabolism in yeast is so complex that previous models have failed to detect it,” says Markus Ralser, group leader at the Crick and senior author of the paper. “Changes in cellular metabolism are tightly bound to disorders that increase with age, including diabetes, various types of cancer, and neurodegenerative diseases. The fact that one can start to predict metabolism in simple cells like yeast cells, is a milestone for the effort to soon be able to predict metabolism also in human tissues.

“Similar Computational tools are used by tech giants like Amazon and Facebook all the time. But instead of using them to tailor advertisements or recommend friends, we’ve harnessed their power to predict a yeast cell’s metabolism. These insights not only inform our understanding of the basis of beer flavouring, but also some human disorders of metabolism.”

From beer to personalised medicine

The team is hoping to transfer their findings in yeast cells to the clinic in the next few years to help patients with metabolic diseases.

“For non-biologists it might seem strange that one can transfer our knowledge of yeast to humans, but in reality, many fundamental principles of what we know about human biology came from yeast research,” says Aleksej.

“We currently expand our algorithms, so that they will provide us with information also about a person’s metabolism, based on which proteins are present in their blood. This information could help doctors decide which treatment option is best for an individual patient.”

The post Machine learning predicts metabolism, helping drug developers and brewers appeared first on Artificial Intelligence.

Doomsday theorists say Artificial Intelligence and the machines that use it will destroy mankind. While that may be a stretch, there is some merit in the argument that AI will surely challenge human thinking and behaviour – to what outcome, that remains to be seen.

Take the example of a ride-hailing app – Uber, Ola or any other. You, the passenger, and the driver may know the route to a destination, but both are compelled to follow the one calculated by the AI engine or else the driver is penalised. Human intelligence such as experience with traffic patterns, temporary blockages due to repairs and construction, or even shorter routes off the main road are not taken into consideration.

Seen at a macro level, would this create a loss of identity where human intelligence and experience of both the driver and the customer is overridden by a machine? In simple terms, here, a human has been displaced by AI. We may not accept it, but this is, in essence, a machine planning what is best for a human.

Man’s inventions have sought to reduce human effort and this is apparent now more than ever. Every time electronic mediums evolve, they tend to make services obsolete, and now, they are making human decision-making obsolete too.

What is the basis that machines take decisions on? The answer here is data – hundreds and thousands of gigabytes of data is collected across the world – its collected when you walk into a store and buy a toothpaste, it is collected when you pay for a ticket online, its collected when you visit your doctor – the list is endless.

“Data helps you react fast to consumer needs and helps companies address them faster,” says Partha De Sarkar, CEO of Hinduja Global Solutions. He says statistical modelling, thanks to modern data libraries and computing power, has combined with AI and Machine Learning algorithms to throw up insights about a customer like never before.

AI is the beginning of a human-machine partnership but this partnership should start off with the coming together of many minds – sociologists, scientists, and engineers – who must deliberate on the effects of AI on communities and individuals.

“In the end, it is the treatment of the data where biases creep in,” says Varun Mayya, co-founder of Avalon Labs. He says every founder must be responsible for the AI platforms they up even before they get consumers and clients to use them.

Experiences make each person different, but that is not exactly how AI works. Its algorithms bucket humans in to different date types, disregarding cultures, and preferences.

The algorithm bias

The cognitive revolution, touted as the next best thing in AI, thus falls flat when engineers use data to typecast individuals in a data set. “It is important for those claiming to use AI for consumer services to work with psychologists and sociologists before claiming their systems are representative of all communities and races,” says Nischith Rastogi, co-founder of Locus, a logistics tech company.

One such example where machine learning models erred with biases is the underwriting of loans. The machine set higher rates for individuals who it thought came from certain communities, income brackets and geography, not taking into account individuals who had the ability to service a loan.

“Biases creep into AI fast. It is something that startups and corporates should be cognisant of,” says Nischith.

The question then is, why is data biased? It starts from the collection of this data and medium it is captured from – the smartphone.

Smartphones create billions of data points about our food habits, fitness regimens, conversations, shopping lists, and payments. Here are a few biases thrown in by AI –

Entertainment: When several members of a family together watch an online streaming service, recommendations are based on past selections. Now, these may be of a particular individual and not necessarily what would serve a common interest.

According to a blog by PWC there’s a need to understand the bias in data, the strengths of the algorithms used, and “generalisability” of unseen data.

The blog adds that while the governance structure used for standard statistical models can be used for machine learning, there are a number of additional elements of software development that must be considered. PWC continues to warn that the tests machine learning models “go through” need to be significantly more robust, and a machine learning governance quality assurance framework will make developers more aware of statistical and software engineering constructs that the model operates within.

According to IBM, AI systems are only as good as the data we put into them. Poor data can contain racial, gender, or ideological biases and many AI systems continue to be trained using bad data, making it an on-going problem. “But we believe that bias can be tamed and that the AI systems that will tackle bias will be the most successful,” says IBM in its blog.

Retail and fashion: If you shop for fashion or beauty products, there is not only peer pressure to contend with now, but that from AI recommendations as well. AI today tends to attack you with a plethora of choices. And with fashion comes its ugly cousin – body shaming!

Earlier, the written word, in the form of fashion magazines, carried bias with pictures, and now, the same biases are carried over when building AI recommendations. With younger individuals taking to smartphones, these ‘recommendations’ may lead to unreasonable expectations from oneself.

Food and life:  Everyone wants to live healthy and it is widely understood that cultural moorings play a big role in what one can and cannot eat. The world of food and nutrition apps, however, tends to standardise profiles into broad strokes, and fitting people in broad data buckets.

“We are training our data models to be as robust as possible when it comes to recommendations. The algorithms learn only if developers ask the right questions,” says Tushar Vashist, co-founder of Healthifyme.

No wonder then that governments are beginning to sit up and take notice, and action. The UK parliament has commissioned a study on AI and the ethics surrounding its applications. The House of Lords-appointed Committee to “consider the economic, ethical and social implications of advances in artificial intelligence”, was set up on June 29, 2017, and will seek answers to five key questions:

• How does AI affect people in their everyday lives, and how is this likely to change?
• What are the potential opportunities presented by Artificial Intelligence for the UK? How can these be realised?
• What are the possible risks and implications of Artificial Intelligence?  How can these be avoided?
• How should the public be engaged with in a responsible manner about AI?
• What are the ethical issues presented by the development and use of Artificial Intelligence?

There are also strong voices around the world on the need for regulatory bodies for Artificial Intelligence to study the ethics of AI.

Back home, what do Indian policymakers have to say about this? Nothing much, is the simple answer.

The Niti Aayog, which creates broad policy frameworks, is keen on creating opportunities for Indians to invest in AI, but is mum on the moral and ethical frameworks of the technology.

“We absolutely need auditability and explainability,” says K M Madhusudan, CTO of Mindtree. He adds there are two aspects to this – one is for serious enterprise-level AI adoption, for which technologists must ensure AI can explain why it made a particular decision. The second is to ensure it is not biased.

The list of biases can be endless. But, it’s time for startups using AI to wake up and smell reality. It is in their interest to do so because they will hopefully soon be liable for the instructions or recommendations made by the AI. To avoid this, one must venture into creating reams of data before providing choices to individuals. In the end we are just one big data set.

The post Will Artificial Intelligence override human intelligence and experiences? appeared first on Artificial Intelligence.

If you want build trust in machine learning, try treating it like a human, asking it the same type of questions.

During the 2008 financial crisis, the banking industry realized that their machine learning algorithms were based on flawed assumptions. So financial system regulators decided that additional controls were needed, and regulatory requirements for “model risk” management on banks and insurers were introduced.

Banks also had to prove that they understood the models they were using, so, regrettably but understandably, they deliberately limited the complexity of their technology, resorting to generalized linear models that offered simplicity and interpretability above all else.

In the past several years, machine learning and AI have made enormous strides in accuracy. Yet regulated industries (like banking) remain hesitant, often prioritizing regulatory compliance and algorithm interpretability over accuracy and efficiency. Some businesses even consider the technology untrustworthy, or dangerous.

In order to trust the recommendations AI and machine learning provide, businesses from all industries need to work to better understand it. Data scientists and PhDs shouldn’t be the only ones capable of clearly explaining machine learning models, because as AI theorist Eliezer Yudkowsky states, “By far, the greatest danger of AI is that people conclude too early that they understand it.”

Trust requires a human approach

When data scientists are asked how a machine learning model makes decisions, they tend to rattle off complex mathematical equations, leaving laymen dumbfounded and the question of how one can trust the model unanswered. Wouldn’t it be more productive to approach machine learning decision-making in the same way one would approach human decision-making? As Udacity co-founder Sebastian Thrun once said, “…artificial intelligence is almost a humanities discipline. It’s really an attempt to understand human intelligence and human cognition.”

So, rather than using complex mathematical equations to determine how, say, a human loan officer makes their decisions, one would simply ask, “Which information on the loan application form is the most important to your decision?” Or, “What values indicate good or bad risks, and how did you decide to accept or reject some specific examples of loan applications?”

An equally human approach is possible in determining how algorithms make similar decisions. For instance, by using a machine learning technique called feature impact, one could determine that the revolving utility balance, the applicant’s income, and the purpose of the loan are the top three most important pieces of information to the loan officer’s algorithm.

By using a capability called reason codes, one could see the most important factors in the estimate of each loan applicant’s details, and by leveraging a technique called partial dependence, one could see that the algorithm scores higher income loan applications as lower risk.

The value of objectivity, scalability, and predictability

In addition to better understanding AI and machine learning by analyzing its decision making as a human would, trust can be obtained by recognizing the unique abilities the technology has to offer, including:

● Solving the problem of credibility and data outliers: Traditional statistical models typically require assumptions about how the data was created, the processes underlying that data, and the credibility of that data. Machine learning, however, removes these restrictive assumptions by using highly flexible algorithms that don’t give more credibility than it deserves.

● Supporting modern computers and massive data sets: Unlike manual processes, machine learning doesn’t assume that the world is full of straight lines. Instead, it adjusts equations automatically to pinpoint the best patterns and test which algorithms and patterns work best against independent validation data (rather than testing only the data it was trained on).

● Leveraging missing values to predict the future: Rather than requiring hours of data cleansing, advanced machine learning can build a blueprint that optimizes the data for that specific algorithm, automatically detecting missing values, determining which algorithms don’t work with missing values, finding the optimal value to substitute for missing values, and using the presence of missing values to predict different outcomes.

Instead of doubting AI or machine learning recommendations, let’s work to better understand them by asking the same reasoning questions we’d ask a human. Let’s recognize the technology’s objective power in reducing data outlier credibility and its ability to provide scalable flexibility for the massive amounts of data available today.

Perhaps most importantly, let’s acknowledge AI and machine learning’s capability to better predict future outcomes by leveraging absent information. Because while the technology is certainly powerful enough to warrant vigilance and formal regulation, consumers and businesses alike only stand to benefit if a proper understanding and level of trust can be established.

The post How Can We Trust Machine Learning and AI? appeared first on Artificial Intelligence.

IT service security has many layers. The IT security layer; firewalls, intrusion detection and access controls. The infrastructure layer; power, network, server health and cooling. And, most important, the people layer. The right people with the right processes, tools and measures to ensure everything else is in working order. Artificial intelligence (AI) will by far have the biggest impact on the tools and measures that people use by amplifying capabilities, streamlining processes and increasing efficiencies.

AI and deep learning will become a necessity in parsing and analyzing the mountain of data generated within a data center to more effectively manage service delivery while mitigating risks like outages. This stems from the recent transformation in how we deliver application workloads.

Too much data?

In the last 10 years, we’ve moved from mostly single server single applications to distributed applications that run in containers. These are now being delivered by micro-services running on-premise and in the cloud–all managed by automation tools. Infrastructure has become part of the application, while other applications have become part of the infrastructure. If you are using a platform like Amazon S3 or Google Maps as an integral component of your service delivery, then you are experiencing this transformation first-hand.

The resulting impact on data center management is significant with power and cooling becoming just a fraction of what needs regular attention. Environmental controls, physical devices, virtual machines and public clouds all need to be monitored and managed round-the-clock to achieve efficiencies in cost and performance. Understanding where and when to move specific workloads becomes paramount.

The amount of data an enterprise collects, monitors and analyzes today to ensure business continuity has exploded. Consider the data generated just from sensors, applications, access control systems, power distribution units, UPS, generators, and solar panels. Add to that external data sources like application vulnerability information, power rates and weather forecasts. Robust data center infrastructure management (DCIM) tools are needed to store all of this data, analyze it and turn it into actionable intelligence. You can try to compartmentalize some of this, but it is becoming increasingly difficult.

AI and deep learning are becoming integral in data center and critical infrastructure management. Here are some of the more notable areas:

• Situational awareness
  Active dashboards with trends, correlations analysis and recommended actions.
• Preventive maintenance
  Deep learning used to identify and correlate data that predicts a failure in power, storage or network connection. This allow operators to mobilize and pro-actively move workloads to safer zones, while maintenance is being performed.
• Root cause analysis
  Machine learning used to trace the failure of several services to a root cause. This becomes learned and used for future preventive maintenance.
• Network security and intrusion detection
  Machine learning and deep neural networks used to spot unusual patterns in application sensors, access control systems and network systems–and provide better signal-to-noise and pro-active mitigations. Learning neural networks are used to continuously improve the enterprise’s security posture and ability to manage related issues.
• Automation
  A “Narrow AI” equipped with various automated mitigation techniques and resulting actions similar to a car applying the brakes if it sees an imminent collision.

Deep neural networks and machine learning algorithms will improve over time, allowing for higher efficiency and performance to match fast growing application workloads. With all of this on the horizon, there’s little doubt that AI will have a massive impact on how enterprises manage their data center.

The post How AI can enhance data center security appeared first on Artificial Intelligence.

Deep learning, the branch of AI that uses artificial neural networks to build prediction and pattern matching models from large datasets relevant to a particular application, is having a sizable impact on both consumer and enterprise software.

Whether for enabling home appliances to understand and respond to vocal commands or identifying hidden patterns endemic to all malware, deep learning algorithms allow machines to mimic and even improve upon human cognition in ways that are impossible with imperative or declarative programming.

Unfortunately, developing deep learning software isn’t easy since the models are customized for a particular use. Indeed, developing models is more like making a custom-fitted suit, not off-the-rack clothing in standard sizes.

Deep learning encompasses a large category of software, not a general-purpose solution, and describes a broad range of algorithms and network types, each better suited to particular types of problems and data than others. For example, convolutional neural networks (CNNs) based on the synaptic connections between neurons in the brain, are extremely effective at image recognition, exceeding 90% accuracy at identifying objects in a standard image dataset used by most developers.

However, the same type of model would be ineffective at learning to make the decisions required to play a strategy game like chess or go. Instead, models based on reinforcement learning that reward correct or good choices and penalizes bad choices are better at completing complicated tasks.

The most advanced applications of AI, such as robots used for multi-purpose manufacturing or logistics tasks will require several deep learning applications, such as a CNN for vision and object identification, recurrent neural networks (RNNs) for speech recognition and reinforcement learning for task completion.

The diversity of models, each requiring specialized AI development knowledge and domain expertise, limits the use of deep learning to organizations with the R&D budget and time horizon needed to produce software tailored to a particular problem.

We are still decades from Star Trek-style artificial general intelligence that could pass the Turing test or outperform humans on a gamut of unrelated cognitive tasks.

In the meantime, a promising compromise would be the ability to automate model selection and tuning based on the problem and available data, and then select the best options from a portfolio of deep learning software each designed for different applications.

That’s the promise of a new category of AI orchestration software like Conductor from Veritone, self-optimizing AI engines such as SigOpt (which I discussed in this column) and so-called AutoML systems such as those that competed in ChaLearn Automatic Machine Learning Challenge.

Meta machine learning: using AI to select AI algorithms

Simply put, “machine learning remains a relatively ‘hard’ problem,” writes Stanford AI researcher S. Zayd Enam. As he points out (emphasis added),

An aspect of this difficulty involves building an intuition for what tool should be leveraged to solve a problem. This requires being aware of available algorithms and models and the trade-offs and constraints of each one. By itself this skill is learned through exposure to these models (classes, textbooks and papers) but even more so by attempting to implement and test out these models yourself.

Enam also notes that debugging and optimizing ML is “exponentially harder” than conventional software both in the difficulty of figuring out what went wrong and the time required to train and execute the models.

Here, we would add that the black-box nature of deep learning networks, which makes them impossible to reverse engineer to understand how they arrived at a decision, only compounds the difficulty.

The emerging field of AutoML, which broadly consists of algorithm selection, hyperparameter tuning, iterative modeling, and model assessment, builds a meta-layer of abstraction on top of ML that can be used to automate model development and optimization.

Facebook’s FBLearner Flow is one such attempt at building a general-purpose platform that can automatically improve ML accuracy. As described on the Facebook Code blog,

Many machine learning algorithms have numerous hyperparameters that can be optimized. At Facebook’s scale, a 1 percent improvement in accuracy for many models can have a meaningful impact on people’s experiences. So with Flow, we built support for large-scale parameter sweeps and other AutoML features that leverage idle cycles to further improve these models. We are investing further in this area to help scale Facebook’s AI/ML experts across many products in Facebook.

Automated algorithm selection using an ecosystem of AI models

Chad Steelberg, a serial entrepreneur whose latest startup is Veritone, saw deep learning algorithm selection as a software opportunity when he realized that the types of problems he was tackling in audio/video content categorization couldn’t be solved with a single AI model.

According to Tyler Schulze, head of Veritone’s budding partner ecosystem, there are already over 5,000 commercial machine-learning algorithms targeting increasingly narrow niches. As he points out in this blog,

The transcription segment includes general-purpose solutions for converting speech-to-text, alongside algorithms that are designed for much more narrow uses, such as taking dictation of Spanish phrases or medical terms. All these engines get stamped with the transcription moniker, despite their radical variances in capabilities.

The premise behind Veritone is that the accuracy and efficacy of deep learning can be significantly improved by mixing and matching various algorithms for a particular problem.

As Steelberg explained in an interview, imagine that you are developing a general-purpose transcription engine that handles ordinary conversation quite well, but stumbles on specialized jargon such as medical or legal terminology. Suppose the system could call for help when it ran into words it couldn’t translate but could identify as belonging to a particular class of knowledge; medicine, pharmacology, astrophysics, corporate finance, whatever. By using an ensemble of models, the system’s overall accuracy would be significantly better. That’s the theory behind Veritone Conductor.

According to Steelberg, “Conductor chooses the best engines for each job, combining them where needed.” In tests on natural language processing, he says the best general purpose engine on its platform achieves 75% accuracy.

By combining multiple language engines and automatically selecting the ‘best’ one based on the input data, Conductor improves the overall system accuracy by 7 points, a significant achievement in a field where, as Facebook notes, even a 1% increase is meaningful.

Veritone currently has about 70 ML engines in its portfolio covering 7 categories targeting five industries or problem areas: media/advertising, politics, legal, law enforcement and government agencies/intelligence.

Schulze’s job is to expand the ecosystem by encouraging developers to integrate their models using Veritone’s APIs, contributing the required metadata describing model function and required data and building easily-deployed container images that can be used by the Conductor platform.

My take

As the number of ML and, particularly deep learning models explodes, automation systems will be required to expand usage beyond the relatively small number of organizations with the requisite AI and data science expertise to create and tune them.

Moving AI from an artisanal phase of handcrafted models to that of an automated production line with reusable ML ‘widgets’ will enable enterprises of all sizes and industries to exploit the power of AI to improve their products, services and business processes.

Whereas cloud services like Azure Cognitive services, Google Cloud ML Engine and similar offerings from AWS and IBM are democratizing infrastructure for AI specialists, these do little for the typical business application developer and systems analyst.

Instead, they need access to packaged AI algorithms and models that can be consumed, combined and optimized like programmable SaaS applications. The emerging field of AutoML along with systems like Veritone Conductor are promising steps towards broadening the use and effectiveness of ML and deep learning software.

The post Automated deep learning – finding the right model is half the battle appeared first on Artificial Intelligence.

The cost of fraud is rising.  According to CNBC.com, fraud and identify theft cost consumers more than $16 billion in 2016 – nearly $1 billion more than in 2015.

LexisNexis research cited on chargebacks911.com finds that for each dollar lost to fraud, “online merchants can ultimately expect to lose $2.40 in revenue due to the associated fees, lost merchandise, sales potential, and more.”

Machine learning promises to become our best weapon in the war on fraud.  But, why, all of a sudden, is it the panacea to the world’s fraud epidemic, and just how will machine learning affect your own security strategy?

The Ultimate Supercomputer

New advances in technology and science have dramatically enhanced the performance of machine learning algorithms, making them much more capable than they were just a few years ago.

Today’s machine learning systems also vastly outperform the modern neural network, pulling in and distilling far greater amounts of data by comparison.

Plus, they streamline and automate fraud detection in unprecedented ways. This is because machine learning systems evolve and improve their performance over time, without explicit programming.

Not only do they “learn as they go,” but they also learn at a mind-boggling pace.  Machine learning platforms today can identify even the most obscure threats in real time, catching and blocking new instances of fraud as they occur.

Addressing Big Data Challenges

According to the Nilson Report, credit card transactions rose 48 percent, debit card transactions 46 percent, and electronic transactions 45 percent between 2010 and 2015, for a collective increase of 34.2 billion transactions annually.  The proliferation of data can weigh heavily on traditional fraud detection resources.

Transaction data typically spans disparate systems and applications as well, which further complicates fraud detection – especially with new mobile wallet, IoT, P2P and digital banking technologies hitting the market daily.  As LexisNexis reports, “fraud through remote channels is up to 7 times as difficult to prevent as in-person fraud.”

Fighting Fire with Fire

But the most compelling reason to embrace machine learning now is this: Fraudsters are constantly evolving their tactics, and they are starting to use the technology themselves.  

This means that when your credit union creates a new rule going forward, tech-savvy fraudsters will find it much easier to get around it.  Machine learning can deliver the speed and flexibility needed to stay ahead of their advancements.

CO-OP’s Vision

To protect credit unions and their members in this new era of fraud, our team at CO-OP is developing a machine learning platform that unifies transaction data across all our systems and applications.

Initially, the platform will work side by side with advanced neural network technology. Over time, we may switch to machine learning entirely or keep both systems in place as the ultimate safeguard.  In the near term, we expect to have the platform in place on the account side by the end of this year, with credit and debit systems to follow.

The Importance of Scale

Achieving scale is critical to the success of any machine learning implementation; the more data these systems can access, the better they perform.

This year, CO-OP is on track to process more than 4 billion transactions. While we won’t share real data across credit unions, we will aggregate it for modeling. This means that whether you’re a $3 billion credit union or a $300 million credit union, you’ll receive all the benefits of our new machine learning technology.

AI and Digital Transformation

Fraud remains a hot topic.  Because the cost of fraud is so high, our investment in machine learning will reap dividends for our organization and client credit union community for years to come.

The initial process of aggregating data alone brings with it far-reaching benefits.  An important step in our own digital transformation journey, data integration at CO-OP will enable, for example, advanced predictive analytics and other forms of AI.

Ultimately, machine learning is probably the most important technology to emerge in the past five years. The fact that credit unions will soon be able to put it to work full force – first against fraud, then to improve marketing and the member experience – is big news for the industry.

The post This is why everyone is talking about machine learning appeared first on Artificial Intelligence.

Deep learning is a new method of artificial intelligence that is an active, fast-moving area of research where we can expect advances to become market-ready over the next several years. Unfortunately, market hype has turned deep learning into a buzzword that can contribute to the misconception that other approaches to AI are not relevant. After all, if you are not doing deep learning, surely you must be doing shallow learning, right?

In cybersecurity, we use various techniques, such as statistics, probability theory, and multiple machine learning algorithms (of which deep learning is one example), to look at use cases and the data available, selecting the best math or algorithm for the job. We take data from various sources — application logs, source code, etc. — choosing the right algorithms based on our understanding of the dataset and use case. This process is fairly artisanal because we are working with a relatively small dataset and the behaviors we are detecting are often very subtle, such as detecting insider threat from source code audit logs. Deep learning is just another specific technique within AI.

Simply described, deep learning is a class of machine learning algorithms that learns by using a large, many-layered collection of connected processes and exposing these processors to a vast set of examples. Deep learning processing is becoming possible across various industries because we have access to large amounts of compute power and processing units, such as with technologies like cloud and GPUs. With this large dataset at our disposal, research in deep learning techniques is fast and furious. Malware detection is one great example, the focus of several security startups attempting to leverage the large set of malware examples accumulated over many years. Other approaches are applying deep learning to smaller datasets; for example, one area of research involves looking at how much data is needed to train a medical image deep learning system.

Detecting malware using deep learning makes sense because we already have a large dataset that characterizes malware. The same cannot be said for insider threats. We just don’t have access yet to enough information from when companies experience these types of attacks. We have anecdotes and sometimes simulated data based on actual events, but anecdotes cannot be used by deep learning networks, and actual log files that correspond to true insider threats are few and far between, although this may change over time. Without large volumes of data on which to base our features, deep learning is simply overkill (or worse, ineffective) for insider threat – at least today.

In the future, the ability for deep processing of security networks to automatically adjust and tune connections with increasing volumes of data will improve the process of learning. In particular, this will allow us to automate and use networks to specialize in certain areas. The networks will learn which portions of the data are more predictable than others, in a way that reduces the dependency on human data scientists to guide the learning process. This “automatic feature learning” is potentially a very big deal for security. With deep learning, the security system can automatically learn by trying billions of combinations and making millions of observations. The potential for getting more accurate results is leading to the excitement that we currently are experiencing as hype.

In the meantime, deep learning systems are very tricky to set up. They are complex and costly, and many so-called hyperparameters are difficult to determine in advance without a lot of experience or experimentation. Training a deep learning model can require several orders of magnitude more compute capacity and cost compared to other, more straightforward machine learning models. For example, a logistic regression model is simple enough to run on one machine for a small dataset, and it remains a very effective approach for many classification tasks today. Progress in hardware acceleration (via GPUs and more recently Google’s TPUs) promises to drive the cost per computational unit down. But today, deep learning systems remain the most expensive machine learning method by a wide margin, and that alone may price the approach beyond the range of most use cases.

Thus, deep learning is still just one of many machine learning methods. It is very promising when aimed at a specific class of problems, but it is not a silver bullet. Just because a technology uses deep learning doesn’t mean other traditional AI and machine learning approaches are not more valuable or practical. Artificial intelligence is multi-purpose technology we can put to work in security and other industries as well, learning, iterating, and improving as we go.

In security, we know that you don’t have to go deep to catch the bad guy. At the end of the day, as long as the good guys win and the bad guys lose, the actual weapon used doesn’t matter.

The post Why deep learning isn’t always the best AI solution appeared first on Artificial Intelligence.