he hype about machine learning (ML) is warranted. Machine learning is not just making things easier for the companies that are taking advantage of it. It’s also changing the way they do business. For example, machine learning is:

• Being used by financial institutions to quickly detect fraudulent activity
• Enabling healthcare practitioners to diagnose diseases and prescribe treatments more effectively
• Helping manufacturers monitor equipment so issues can be dealt with before they disrupt operations
• Allowing streaming services to identify customers at risk of taking their business elsewhere and helping determine what steps can be taken to retain them

With increasing data volumes, low-cost data storage, and less expensive, more powerful data processing, the potential applications of machine learning will grow exponentially.

So why are so many companies hesitant to jump on the machine learning bandwagon – and why is the success rate so low for those that do embark on these projects? Afterall, organizations such as Gartner note that up to 85% of machine learning projects ultimately fail to deliver on their intended promises to business.

More importantly, what can companies do to ensure a higher success rate so they can leverage the promise of machine learning?

Machine Learning is Different

To increase the chances of machine learning project success, the first step is to understand that these projects are not the same as typical application and software development projects. There are different processes, terminology, workflows, and tools involved.

There are also different staffing requirements. Among the most important are data scientists, who are especially critical when it comes to defining the success criteria, final deployment, and continuous monitoring of the machine learning model.

Data engineers, business intelligence specialists, DevOps, and application developers also play key roles. Few organizations have the internal resources to fill all these positions. Their options: hire them, which isn’t always easy given that machine learning is still a relatively new field with few experienced professionals, or outsource.

Even if an organization does have the staffing covered, it can be difficult to facilitate collaboration and communication between different teams. Traditional software and application development usually differs greatly from data science projects. Whereas software development tends to be more predictable and measurable, data science can entail multiple iterations and experimentation. Expectations are different. Deliverables are different.

The Issue of Data Quantity and Quality

According to a number of research initiatives (e.g., Hidden Debt in Machine Learning Systems) technical debt resides in areas common to many machine learning projects: Data Quality, Model Quality, Feature Versioning, Model Monitoring, Data Labeling, Model Explainability, and Fairness, Process Automation, Human Intervention (Review) in-place capabilities.

There’s also the matter of data quantity and quality. Machine learning projects use large datasets, since larger datasets facilitate better predictions. But as the size of the data increases, so do the challenges.

Data is usually merged from multiple sources. Often that data is not in sync, which can create confusion. In addition, data can get merged that wasn’t meant to be merged, resulting in data points with the same name but different meanings. Bad data can generate results that aren’t actionable or insightful, or that are misleading.

The lack of labeled data can also be an issue. Some teams may try to take on the laborious task of labeling and annotating training data themselves. Some may even try to create their own labeling and annotation automation technology. The problem is that a great deal of time and expertise is committed to the labeling process rather than machine learning model training.

Outsourcing can save both time and money but doesn’t work well if the labeling task requires specific domain knowledge. In those cases, organizations also must invest in formal and standardized training of annotators to ensure quality and consistency across datasets. The other option is to develop their own data labeling tool if the data to be labeled is extremely complex. However, this can require more engineering overhead than the machine learning task itself.

Yet another data-related issue is that the data required in a machine learning project often resides in different places with different security constraints and in different formats — structured, unstructured, video files, audio files, text, and images. Data preparation is required, a process that includes searching, cleaning, transforming, organizing, and collecting data. It’s a tedious activity that can require teams to spend up to 80% of their time converting raw data into high-quality, analysis-ready output.

For both the data labeling and data preparation, automation can help remedy the situation – but again, requires expertise that internal teams often lack.

Great Expectations

Machine learning projects aren’t cheap, so it’s not uncommon for organizations to have overly ambitious goals for them. There are often expectations that a project will completely transform the company or a product and generate an enormous return on investment. That creates a lot of pressure that can, in turn, lead to second-guessing on strategies and tactics.

Not surprisingly, these kinds of projects tend to drag out. As a result, both the project teams and management lose confidence and interest in the project, and budgets max out. Even the most expertly run projects are doomed to fail if the goals are unrealistic.

In other cases, machine learning projects kick-off without alignment on expectations, goals, and success criteria between the business and project teams. Without clearly defined success indicators, it’s difficult to determine whether a project is successful, what changes need to be made, if the model is effectively solving the intended business needs, or if other options should be considered.

Machine Learning Success Factors

While there are no specific guidelines for ensuring a successful machine learning project, there are ways to overcome many of the issues that can lead to project failure. Among them:

• An understanding of how machine learning works, how it differs from other project types, and what’s required to execute a project
• A properly scoped project with realistic goals, budget, and leadership support
• The resources to run a machine learning project, including experienced team members — whether in-house or outsourced — and a commitment to collaboration and open communication
• Large amounts of data, preferably labeled
• Capabilities for collecting, storing, labeling, cleaning, quickly accessing, and processing large volumes of data
• Advanced tools for machine learning models and data monitoring
• Capabilities for a human to review the machine learning system and inference at any place and point in time
• Software tools for executing machine learning algorithms
• A development platform, such as AWS, Baidu, Google, IBM, or Microsoft

Follow these tips and your machine learning project won’t derail before your organization can enjoy the many benefits that this modern technology provides.

The post Why Machine Learning Projects Fail – and How to Make Sure They Don’t appeared first on Artificial Intelligence.

Manufacturing Global speaks to executive leaders at EY, Infor and GE Digital to get to the bottom of this question

Whilst the drive to digitally transform the manufacturing industry has been a topic of conversation for the last decade, recent events have only increased the need for the agility, scalability and resilience that Industry 4.0, smart manufacturing capabilities can provide. Speaking with Cobus Van Heerden, Senior Digital Product Manager at GE Digital, Mark Powell, Partner, EY (UKI Consulting), and Phil Lewis, Vice President, Solution Consulting EMEA at Infor Manufacturing Global looks at how technologies that harness AI and Big Data can help manufacturers unlock real-time operational visibility to achieve improved process reliability and performance.

What are the current applications of artificial intelligence (AI) and Big Data in the manufacturing industry?

CVH: Industrial AI uses a combination of targeted AI technologies, data, physics, and deep domain knowledge to solve key industrial business challenges. Traditional AI mimics human intelligence, whereas industrial AI builds upon it to unlock insights and determine causal knowledge in high-stakes, dynamic, and variable industrial environments. In Manufacturing, Industrial AI can be used to detect and predict key process and asset problems to help companies optimize their operations including capacity, quality, and cost structures.

PL: Textbook definitions of AI or Big Data miss the point that industries differ and will have drastically different demands for the technology.  It is about the application of a given technology to a specific issue that a business may be experiencing.  This issue may be an ‘industry-standard’ one or something that arises in the configuration of the technology.  But there is the most value in the application of tools such as Big Data and AI to the critical 10% of a business that is truly idiosyncratic.  We classify this as a 60/30/10 split and it is how we look to apply these technologies to drive maximum value.

For manufacturers looking to adopt Industry 4.0, smart manufacturing capabilities, why should manufacturers use AI and Big Data to do so?

CVH: Smart manufacturing deploys industrial advanced analytics to predict future asset and process performance using real-time and historical data and optimizing in a closed loop. This involves the use of AI and machine learning to enable process engineers to combine data across industrial data sources and rapidly identify problems, discover root causes of issues in the plant, predict the future performance of assets, and automate actions employees can take to improve quality, productivity, and operations.

MP: Digitisation is forcing manufacturers to reimagine their supply chains. As an example, most companies use internal data to track demand-supply balances and it is challenging for them to foresee external events impacting their supply chains. Using AI techniques that understand unstructured external data sets, such as social media and other data on events, manufacturers can plan for supply chain disruptions much sooner.

In addition, manufacturers can use AI and Big Data to build digital replicas of their manufacturing operations and tap into transformative possibilities of reducing cycle time in production, adding manufacturing capacity and predicting unplanned maintenance activities etc.

PL: Some of the poster child statistics for AI and Big Data simply demand attention.  Recently, Siemens automated one of its factories in Germany, with 75% of the processes digitised or having increased automation. Productivity improved by 1,400%. That is game-changing for any business. This means many manufacturers are now looking at how they plug AI and Big Data into their plans for the future. 

What is the best strategy for manufacturers striving to realise the value of AI and Big Data in their operations?

CVH: Process engineers have exceptional domain expertise to put together process models – or Process Digital Twins – and be able to interpret the models. This is the foundation for improving competitive advantage and success with analytics. To drive analytics and improve processes, manufacturers should put together a strategy that can align domain expertise to five capabilities: Analysis – automatic root cause identification accelerates continuous improvement; Monitoring – early warnings reduce downtime and waste; Prediction – proactive actions improve quality, stability, and reliability; Simulation – what-if simulations accelerate accurate decisions at a lower cost; and Optimization – optimal process setpoints improve throughput at acceptable quality by up to 10 per cent. 

All process engineers can and need to develop capabilities in analytics and machine learning to remain competitive. Over time, engineers can go from small projects to pilots to multi-plant optimization with deep application of analytics. Their deep domain expertise provides a foundation for modelling processes and developing the analytics that are game changers in very specific applications. 

Most importantly, get started with analytics. “Trystorm” some projects; put your intuitive ideas to the test and put data and analytics behind them. Don’t wait to become a data science expert. That isn’t necessary. Leverage proven easy-to-use industrial analytics tools fueled with your domain expertise. That’s going to drive big improvements quickly.

PL: Businesses – including manufacturers – tend to assess digital projects with a focus on either customer, supply chain, internal efficiency or people – those are the four main drivers for any foray into digital.  These are often organic and arise from an ongoing ‘how can we do better’ attitude. This has been accelerated by concerns of competition as companies are now fearful of being left behind competition and disruptive entrants.  There is palpable fear around being digitally relevant and this is promoting a lot of investment.

However, it is worth noting that many manufacturers have already invested heavily in technology (even before COVID forced a move to digitalisation) so the first point of definition is to align AI and Big Data to existing technology.  When businesses assess their technology in use today, they need to bear in mind not only a short-term perspective of will the technology handle current processes but also does it provide a platform for the future?  This latter perspective is built on data.  Both elements are equally important but the second ‘platform perspective’ demands big data.  It is no longer enough to choose a platform that just supports/tweaks the ongoing processes – there has to be future capabilities built-in.

There is then the need to ensure that this technology is deployed in the best way possible. This necessitates an open, cloud-based application landscape so a business can seize new opportunities such as Big Data or AI without having to go through a cumbersome integration and bolt-on process.  This makes an organisation more agile, focusing on the creative application of the technology to the needs of the business, such as identifying new opportunities for revenue.

What are the challenges when it comes to adopting AI and Big Data analytics into manufacturing operations? 

CVH: Manufacturers are challenged with reducing waste, costs, and risk while meeting customer demand. The combination of AI and data provides acceleration of digitisation through analytics-based solutions that empower workers with data in context so that people, assets, and processes work together efficiently.

Another challenge for companies is just getting started. They want to learn more about how to use analytics in their operations but don’t see it as a job for their current workforce. Fortunately, Industrial AI solutions can help and not require process engineers to be data scientists.

MP: The key challenge in adopting AI will come down to manufacturers’ ability to establish alignment across the organisation on some of the high-value areas where AI will make an impact. For example, using machine learning and computer vision to predict and identify faults in equipment before they occur, thus reducing production downtime and decreasing maintenance costs. Another challenge is establishing a culture of infusing AI into their processes through a test-and-learn culture.

For too long, organisations have talked about becoming ‘data driven’ and this has generally not worked as well as it had been hoped. Manufacturers need to take a different approach that starts with understanding where value can be driven from new insights and then focus on the data needed to drive the insights that can then drive business value. Organisations need to become ‘insight-driven and data enabled’  and not simply ‘data driven’ – only then will they really leverage the power of AI and big data.

PL: It is all about how attitudes towards data have changed.  It was previously seen as a necessary evil but is now the number one asset in a business. Typically this drives an obsession with big data labels but it is what you do with the data that matters – using the likes of AI / BI / IoT etc to turn that data into a truly valuable asset. The automotive industry is the prime example – using and selling the data produced by a car. Interestingly, we now almost take ‘cloud’ for granted – had we answered this question 24 months ago, cloud would have been the first consideration, but it is now table stakes.  It is no longer if a business will go cloud but more a question of what type of cloud/cloud use? – We have moved far beyond the infrastructure conversation –the how and into the what – and into the why a business looks to embrace digital.

Is artificial intelligence (AI) and Big Data driving the fourth industrial revolution (Industry 4.0)?

CVH: The combination of Industrial AI and data produces what we call a Process Digital Twin which helps manufacturers to rapidly troubleshoot continuous, discrete, or batch manufacturing process performance by mining insight from available sensor and production data. This technology, which utilises predictive analytics, enables users to analyse operating scenarios, qualifying the impact that operational changes will have on key performance metrics and identifying causes for performance variation. Digital Twins inspire continuous improvement, a key goal of the future of the industry by looking back to historical data as well as real-time to move forward rapidly.

PL: We see daily increases in AI/ML uses – inventory optimisation, maintenance, faster finance processes are all key areas that we see arise many times. For this to continue, and return on investment to continue, AI needs to be plumbed in and ready to go with other systems, rather than a bolt-on, or businesses face a hefty, and costly integration project. In terms of the next specific technology, it really depends on the maturity of the individual company or project – businesses are only just reaching the point of a digital fabric rather than a bunch of digital projects.  Prescriptive working, driven by AI and fed by masses of sensor data, holds a huge amount of promise for the B2B / industrial markets and we see some very encouraging early shoots in asset maintenance and field service.

The post Why Should Manufacturers Adopt AI and Big Data? appeared first on Artificial Intelligence.

Organizations can gain more business value from advanced analytics by recognizing and overcoming five common obstacles.

More and more companies are embracing data science as a function and a capability. But many of them have not been able to consistently derive business value from their investments in big data, artificial intelligence, and machine learning.1 Moreover, evidence suggests that the gap is widening between organizations successfully gaining value from data science and those struggling to do so.2

To better understand the mistakes that companies make when implementing profitable data science projects, and discover how to avoid them, we conducted in-depth studies of the data science activities in three of India’s top 10 private-sector banks with well-established analytics departments. We identified five common mistakes, as exemplified by the following cases we encountered, and below we suggest corresponding solutions to address them.

Mistake 1: The Hammer in Search of a Nail

Hiren, a recently hired data scientist in one of the banks we studied, is the kind of analytics wizard that organizations covet.3 He is especially taken with the k-nearest neighbors algorithm, which is useful for identifying and classifying clusters of data. “I have applied k-nearest neighbors to several simulated data sets during my studies,” he told us, “and I can’t wait to apply it to the real data soon.”

Hiren did exactly that a few months later, when he used the k-nearest neighbors algorithm to identify especially profitable industry segments within the bank’s portfolio of business checking accounts. His recommendation to the business checking accounts team: Target two of the portfolio’s 33 industry segments.

This conclusion underwhelmed the business team members. They already knew about these segments and were able to ascertain segment profitability with simple back-of-the-envelope calculations. Using the k-nearest neighbors algorithm for this task was like using a guided missile when a pellet gun would have sufficed.

In this case and some others we examined in all three banks, the failure to achieve business value resulted from an infatuation with data science solutions. This failure can play out in several ways. In Hiren’s case, the problem did not require such an elaborate solution. In other situations, we saw the successful use of a data science solution in one arena become the justification for its use in another arena in which it wasn’t as appropriate or effective. In short, this mistake does not arise from the technical execution of the analytical technique; it arises from its misapplication.

After Hiren developed a deeper understanding of the business, he returned to the team with a new recommendation: Again, he proposed using the k-nearest neighbors algorithm, but this time at the customer level instead of the industry level. This proved to be a much better fit, and it resulted in new insights that allowed the team to target as-yet untapped customer segments. The same algorithm in a more appropriate context offered a much greater potential for realizing business value.

It’s not exactly rocket science to observe that analytical solutions are likely to work best when they are developed and applied in a way that is sensitive to the business context. But we found that data science does seem like rocket science to many managers. Dazzled by the high-tech aura of analytics, they can lose sight of context. This was more likely, we discovered, when managers saw a solution work well elsewhere, or when the solution was accompanied by an intriguing label, such as “AI” or “machine learning.” Data scientists, who were typically focused on the analytical methods, often could not or, at any rate, did not provide a more holistic perspective.

To combat this problem, senior managers at the banks in our study often turned to training. At one bank, data science recruits were required to take product training courses taught by domain experts alongside product relationship manager trainees. This bank also offered data science training tailored for business managers at all levels and taught by the head of the data science unit. The curriculum included basic analytics concepts, with an emphasis on questions to ask about specific solution techniques and where the techniques should or should not be used. In general, the training interventions designed to address this problem aimed to facilitate the cross-fertilization of knowledge among data scientists, business managers, and domain experts and help them develop a better understanding of one another’s jobs.

In related fieldwork, we have also seen process-based fixes for avoiding the mistake of jumping too quickly to a favored solution. One large U.S.-based aerospace company uses an approach it calls the Seven Ways, which requires that teams identify and compare at least seven possible solution approaches and then explicitly justify their final selection.

Mistake 2: Unrecognized Sources of Bias

Pranav, a data scientist with expertise in statistical modeling, was developing an algorithm aimed at producing a recommendation for the underwriters responsible for approving secured loans to small and medium-sized enterprises. Using the credit approval memos (CAMs) for all loan applications processed over the previous 10 years, he compared the borrowers’ financial health at the time of their application with their current financial status. Within a couple of months, Pranav had a software tool built around a highly accurate model, which the underwriting team implemented.

Unfortunately, after six months, it became clear that the delinquency rates on the loans were higher after the tool was implemented than before. Perplexed, senior managers assigned an experienced underwriter to work with Pranav to figure out what had gone wrong.

The epiphany came when the underwriter discovered that the input data came from CAMs. What the underwriter knew, but Pranav hadn’t, was that CAMs were prepared only for loans that had already been prescreened by experienced relationship managers and were very likely to be approved. Data from loan applications rejected at the prescreening stage was not used in the development of the model, which produced a huge selection bias. This bias led Pranav to miss the import of a critical decision parameter: bounced checks. Unsurprisingly, there were very few instances of bounced checks among the borrowers whom relationship managers had prescreened.

The technical fix in this case was easy: Pranav added data on loan applications rejected in prescreening, and the “bounced checks” parameter became an important element in his model. The tool began to work as intended.

The bigger problem for companies seeking to achieve business value from data science is how to discern such sources of bias upfront and ensure that they do not creep into models in the first place. This is challenging because laypeople — and sometimes analytics experts themselves — can’t easily tell how the “black box” of analytics generates output. And analytics experts who do understand the black box often do not recognize the biases embedded in the raw data they use.

The banks in our study avoid unrecognized bias by requiring that data scientists become more familiar with the sources of the data they use in their models. For instance, we saw one data scientist spend a month in a branch shadowing a relationship manager to identify the data needed to ensure that a model produced accurate results.

We also saw a project team composed of data scientists and business professionals use a formal bias-avoidance process, in which they identified potential predictor variables and their data sources and then scrutinized each for potential biases. The objective of this process was to question assumptions and otherwise “deodorize” the data — thus avoiding problems that can arise from the circumstances in which the data was created or gathered.4

Mistake 3: Right Solution, Wrong Time

Kartik, a data scientist with expertise in machine learning, spent a month developing a sophisticated model for analyzing savings account attrition, and he then spent three more months fine-tuning it to improve its accuracy. When he shared the final product with the savings account product team, they were impressed, but they could not sponsor its implementation because their annual budget had already been expended.

Eager to avoid the same result the following year, Kartik presented his model to the product team before the budgeting cycle began. But now the team’s mandate from senior management had shifted from account retention to account acquisition. Again, the team was unable to sponsor a project based on Kartik’s model.

In his third year of trying, Kartik finally got approval for the project, but he had little to celebrate. “Now they want to implement it,” he told us, with evident dismay, “but the model has decayed and I will need to build it again!”

The mistake that blocks banks from achieving value in cases like this is a lack of synchronization between data science and the priorities and processes of the business. To avoid it, better links between data science and the strategies and systems of the business are needed.

Senior executives can ensure the alignment of data science activities with organizational strategies and systems by more tightly integrating data science practices and data scientists with the business in physical, structural, and process terms. For example, one bank embedded data scientists in business teams on a project basis. In this way, the data scientists rubbed elbows with the business team day to day, becoming more aware of its priorities and deadlines — and in some cases actually anticipating unarticulated business needs. We have also seen data science teams colocated with business teams, as well as the use of process mandates, such as requiring that project activities be conducted at the business team’s location or that data scientists be included in business team meetings and activities.

Generally speaking, data scientists ought to be concentrating their efforts on the problems deemed most important by business leaders.5 But there is a caveat: Sometimes data science produces unexpected insights that should be brought to the attention of senior leaders, regardless of whether they align with current priorities.6 So, there is a line to be walked here. If an insight arises that does not fit current priorities and systems but nonetheless could deliver significant value to the company, it is incumbent upon data scientists to communicate this to management.

We found that to facilitate exploratory work, bank executives sometimes assigned additional data scientists to project teams. These data scientists did not colocate and were instructed not to concern themselves with team priorities. On the contrary, they were tasked with building alternative solutions related to the project. If these solutions turned out to be viable, the head of the data science unit pitched them to senior management. This dual approach recognizes the epistemic interdependence between the data science and business professionals — a scenario in which data science seeks to address today’s business needs as well as detect opportunities to innovate and transform current business practices.7 Both roles are important, if data science is to realize as much business value as possible.

Mistake 4: Right Tool, Wrong User

Sophia, a business analyst, worked with her team to develop a recommendation engine capable of offering accurately targeted new products and services to the bank’s customers. With assistance from the marketing team, the recommender was added to the bank’s mobile wallet app, internet banking site, and emails. But the anticipated new business never materialized: Customer uptake of the product suggestions was much lower than anticipated.

To discover why, the bank’s telemarketers surveyed a sample of customers who did not purchase the new products. The mystery was quickly solved: Many customers doubted the credibility of recommendations delivered through apps, websites, and emails.

Still looking for answers, Sophia visited several of the bank’s branches, where she was surprised to discover the high degree of trust customers appeared to place in the advice of relationship managers (RMs). A few informal experiments convinced her that customers would be much more likely to accept the recommendation engine’s suggestions when presented in the branch by an RM. Realizing that the problem wasn’t the recommender’s model but the delivery mode of the recommendations, Sophia met with the senior leaders in branch banking and proposed relaunching the recommendation engine as a tool to support product sales through the RMs. The redesigned initiative was a huge success.

The difficulties Sophia encountered highlight the need to pay attention to how the outputs of analytical tools are communicated and used. To generate full value for customers and the business, user experience analysis should be included in the data science design process. At the very least, user testing should be an explicit part of the data science project life cycle. Better yet, a data science practice could be positioned within a human-centered design frame. In addition to user testing, such a frame could mandate user research on the front end of the data science process.

While we did not see instances of data science embedded within design thinking or other human-centered design practices in this study, we did find that the shadowing procedures described above sometimes operated as a kind of user experience analysis. As data scientists shadowed other employees to understand the sources of data, they also gained an understanding of users and channels through which solutions could be delivered. In short, the use of shadowing in data science projects contributes to a better understanding of the processes that generate data, and of solution users and delivery channels.

Mistake 5: The Rocky Last Mile

The bank’s “win-back” initiative, which was aimed at recovering lost customers, had made no progress for months. And that day’s meeting between the data scientists and the product managers, which was supposed to get the initiative back on track, was not going well either.

Data scientists Dhara and Viral were focused on how to identify which lost customers were most likely to return to the bank, but product managers Anish and Jalpa wanted to discuss the details of the campaign to come and were pushing the data scientists to take responsibility for its implementation immediately. After the meeting adjourned without a breakthrough, Viral vented his frustration to Dhara: “If data scientists and analysts do everything, why does the bank need product managers? Our job is to develop an analytical solution; it’s their job to execute.”

By the next meeting, though, Viral seemed to have changed his mind. He made a determined effort to understand why the product managers kept insisting that the data scientists take responsibility for implementation. He discovered that on multiple occasions in the past, the information systems department had given the bank’s product managers lists of customers to target for win-back that had not resulted in a successful campaign. It turned out that using the lists had been extremely challenging, partly due to an inability to track customer contacts — so the product managers felt that being given another list of target customers was simply setting them up for another failure.

With this newfound understanding of the problem from the point of view of the product managers, Viral and Dhara added to their project plan the development of a front-end software application for the bank’s telemarketers, email management teams, branch banking staff, and assets teams. This provided them with a tool where they could feed information from their interactions with customers and make better use of the lists provided by the data science team. Finally, the project moved ahead.

Viral and Dhara’s actions required an unusual degree of empathy and initiative. They stepped out of their roles as data scientists and acted more like project leaders. But companies probably should not depend on data scientists in this way, and they may not want to — after all, the technical expertise of data scientists is a scarce and expensive resource.

Instead, companies can involve data scientists in the implementation of solutions. One bank in our study achieved this by adding estimates of the business value delivered by data scientists’ solutions to their performance evaluations. This motivated data scientists to ensure the successful implementation of their solutions. The bank’s executives acknowledged that this sometimes caused data scientists to operate too far outside their assigned responsibilities. However, they believed that ensuring value delivery justified the diversion of data science resources, and that it could be corrected on a case-by-case basis, if the negative impact on the core responsibilities of data scientists became excessive.

The mistakes we identified invariably occurred at the interfaces between the data science function and the business at large. This suggests that leaders should be adopting and promoting a broader conception of the role of data science within their companies — one that includes a higher degree of coordination between data scientists and employees responsible for problem diagnostics, process administration, and solution implementation. This tighter linkage can be achieved through a variety of means, including training, shadowing, colocating, and offering formal incentives. Its payoff will be fewer solution failures, shorter project cycle times, and, ultimately, the attainment of greater business value.

The post Why So Many Data Science Projects Fail to Deliver appeared first on Artificial Intelligence.

Most companies are struggling to develop working artificial intelligence strategies, according to a new survey by cloud services provider Rackspace Technology. The survey, which includes 1,870 organizations in a variety of industries, including manufacturing, finance, retail, government, and healthcare, shows that only 20 percent of companies have mature AI/machine learning initiatives. The rest are still trying to figure out how to make it work.

There’s no questioning the promises of machine learning in nearly every sector. Lower costs, improved precision, better customer experience, and new features are some of the benefits of applying machine learning models to real-world applications. But machine learning is not a magic wand. And as many organizations and companies are learning, before you can apply the power of machine learning to your business and operations, you must overcome several barriers.

Three key challenges companies face when integrating AI technologies into their operations are in the areas of skills, data, and strategy, and Rackspace’s survey paints a clear picture of why most machine learning strategies fail.

Machine learning is about data

Machine learning models live on compute resources and data. Thanks to a variety of cloud computing platforms, access to the hardware needed to train and run AI models has become much more accessible and affordable.

But data continues to remain a major hurdle in different stages of planning and adopting an AI strategy. Thirty-four percent of the respondents in the Rackspace survey stated poor data quality as the main reason for the failure of machine learning research and development, and another 31 percent said they lacked production-ready data.

This highlights one of the main hurdles when applying machine learning techniques to real-world problems. While the AI research community has access to many public datasets for training and testing their latest machine learning technologies, when it comes to applying those technologies to real applications, getting access to quality data is not easy. This is especially true in industrial, health, and government sectors, where data is often scarce or subject to strict regulations.

Data problems crop up again when machine learning initiatives move from the research to the production phase. Data quality remains the top barrier when it comes to using machine learning to extract valuable insights. Data engineering problems also pose a significant problem, such as data being siloed, lack of talent to connect disparate data sources, and not being fast enough to process data in a meaningful way.

Both startups and established companies suffer from data problems, though scale seems to be the key differentiator between the two, according to Jeff DeVerter, CTO of Rackspace Technology. “Startups tend to be constrained with not all the right resources to implement a quality data pipeline and consistently managing it over time,” DeVerter said to TechTalks in written comments. “Enterprises usually have scale on their side and with that comes the rigor that’s required.”

The best way companies can prepare for the data challenges of AI strategies is to do a full evaluation of their data infrastructure. Eliminating silos should be a key priority in every machine learning initiative. Companies should also have the right procedures for cleaning their data to improve the accuracy and performance of their machine learning models.

AI talent is still in high demand

The second area of struggle for most companies is access to machine learning and data science talent. According to Rackspace’s survey, lack of in-house expertise was the second biggest driver of failure in machine learning R&D initiatives. Lack of skill and difficulty in hiring was also a key barrier in adopting AI technologies.

With machine learning and deep learning having reached mainstream use in production environments only recently, many smaller companies don’t have data scientists and machine learning engineers who can develop AI models.

And the average salary of data scientists and machine learning engineers matches those of experienced software engineers, which makes it difficult for many companies to put together a talented team that can lead its AI initiative.

While the shortage of machine learning and data science talent is well known, one thing that has gone mostly unnoticed is the need for more data engineers, the people who set up, maintain, and update databases, data warehouses, and data lakes. Per Rackspace’s figures, many initiatives fail because companies don’t have the talent to adapt their data infrastructure for machine learning purposes. Breaking down silos, migrating to cloud, setting up Hadoop clusters, and creating hybrid systems that can leverage the power of different platforms are some areas where companies are sorely lacking. And these shortcomings prevent them from making company-wide deployments of machine learning initiatives.

With the development of new machine learning and data science tools, the talent problem has become less intense. Google, Microsoft, and Amazon have launched platforms that make it easier to develop machine learning models. An example is Microsoft’s Azure Machine Learning service, which provides a visual interface with drag-and-drop components and makes it easier to create ML models without coding. Another example is Google’s AutoML, which automates the tedious process of hyperparameter tuning. While these tools are not a replacement for machine learning talent, they lower the barrier for people who want to enter the field and will enable many companies to reskill their tech talent for these growing fields.

“Lack of in-house data science talent is not the barrier it once was now that more of these services are able to use their own ML to help in this regard as well consulting firms having these talents on-staff,” DeVerter said.

Other developments in the field are the evolution of cloud storage and analysis platforms, which have considerably reduced the complexity of creating the seamless data infrastructures needed to create and run AI systems. An example is Google’s BigQuery, a cloud-based data warehouse that can run queries across vast amounts of data stored in various sources with minimal effort.

We’re also seeing growing compatibility and integration capabilities in machine learning tools, which will make it much easier for organizations to integrate ML tools into their existing software and data ecosystem.

Before entering an AI initiative, every organization must make a full evaluation of in-house talent, available tools, and integration possibilities. Knowing how much you can rely on your own engineers and how much it will cost you to hire talent will be a defining factor in the success or failure of your machine learning initiatives. Also, consider whether re-skilling is a possible course of action. If you can upskill your engineers to take on data science and machine learning projects, you will be better off in the long run.

Outsourcing AI talent

Another trend that has seen growth in recent years is the outsourcing of AI initiatives. Only 38 percent of the Rackspace survey respondents relied on in-house talent to develop AI applications. The rest were either fully outsourcing their AI projects or employing a combination of in-house and outsourced talent.

There are now several companies that specialize in developing and implementing AI strategies. An example is C3.ai, an AI solutions provider that specializes in several industries. C3.ai provides AI tools on top of existing cloud providers such as Amazon, Microsoft, and Google. The company also provides AI consultancy and expertise to take customers step by step through the strategizing and implementation phases.

According to the Rackspace report: “A mature provider can bring everything from strategy to implementation to maintenance and support over time. Strategy can sidestep the areas where AI and machine learning efforts may lose momentum or get lost in complexity. Hands-on experts can also spare organizations from the messy work of cleanup and maintenance. Such expertise, taken together, can make all the difference in finally achieving success.”

It is worth noting, however, that fully turning over an organization’s AI strategy to outside providers can be a double-edged sword. A successful strategy requires close cooperation between AI specialists and subject matter experts from the company that is implementing the strategy.

“This is very similar to companies who move to a DevOps development methodology and attempt to outsource the entirety of the development. DevOps requires a close partnership between the developers, business analysts, and others in the business,” DeVerter said. “In the same way, AI projects require strategy and technical expertise — but also require a tight partnership with the business as well as leadership.”

Outsourcing AI talent must be done meticulously. While it can speed up the process of developing and implementing an AI strategy, you must make sure that your experts are fully involved in the process. Ideally, you should be able to develop your own in-house team of data scientists and machine learning engineers as you work with outside experts.

How do you evaluate your AI strategy?

Finally, another area that is causing much pain for companies embarking on an AI journey is forecasting the outcome and value of AI strategies. Given the application of machine learning being new to many areas, it’s hard to know in advance how long an AI strategy will take to plan and implement and what the return on investment will be. This in turn makes it difficult for innovators in organizations to get others on board when it comes to garnering support for AI initiatives.

Of the respondents of the Rackspace survey, 18 percent believed that a lack of clear business case was the main barrier to adopting AI strategies. Lack of commitment from executives was also among the top barriers. Lack of use cases and commitment from senior management show up again among the top challenges in the machine learning journey.

“AI often wanders around as a solution looking for a problem within organizations. I believe this is one of the greatest impediments to its wide-scale adoption within organizations,” DeVerter said. “As AI practitioners can demonstrate practical examples of how AI can benefit their specific company — leadership will further fund those activities. Like any business venture — leadership needs to know how it will either help them save or make money.”

Evaluating the outcome of AI initiatives is very difficult. According to the survey, the top-two key performance indicators (KPI) for measuring the success of AI initiatives were profit margins and revenue growth. Understandably, this focus on quick profits is partly due to the high costs of AI initiatives. According to the Rackspace survey, organizations spend a yearly average of $1.06 million on AI initiatives.

But while a good AI initiative should result in revenue growth and lower costs, in many cases, the long-term value of machine learning is the development of new use cases and products.

“Short-term financial gains can be myopic if they aren’t paired with a long-term strategy that can be funded by those short-term gains,” DeVerter said.

If you’re in charge of the AI initiative in your organization, make sure to clearly lay out the use cases, the costs, and the benefits of your AI strategy. Decision-makers should have a clear picture of what their company will be embarking on. They should understand the short-term benefits of investing in AI, but they should also know what they will gain in the long run.

Ben Dickson is a software engineer and the founder of TechTalks. He writes about technology, business, and politics. This post was originally published here as a series exploring the business of artificial intelligence.

The post Why machine learning strategies fail appeared first on Artificial Intelligence.

Python is considered to be the easiest language to learn, which makes it a perfect fit for kids. As a new coder, kids can get started with Python as it has a lot of applications. Python is used for various tasks, including data analysis, visualization, automation, machine learning, etc.  Hence learning Python can give kids a life-long advantage. There are many versions of Python for kids that are specially designed for young champs who want to get started with the multi-purpose language.

Python language has an extensive library that supports everything, which makes it an easy language to learn, especially for the kids who want to get an experience of an object-oriented programming language. Python is used for visualization, machine learning, automation, data analysis, etc., which makes it one of the best multi-purpose languages. It is also used to build mobile apps, desktop applications, etc., which again is one of the interesting reasons for kids to get started with Python.

If you want your kids to get started with Python, now is the best time! The demand for coders is continuously growing, and hence coding for kids has become a primary skill. Online learning platforms like Cuemath help kids get started with the language through online classes. Cuemath also encourages kids to explore the real-world applications of the language in building mobile applications, web applications, etc. This sparks creativity amongst kids.

In this blog, we are going to explore reasons which make Python a perfect language for kids. Let us begin.

Reasons Why Python is the Perfect Programming Language for Kids

Following are some of the reasons that make Python a perfect programming language for kids:

Kid-Friendly

Python can be introduced as the first language for kids as it is one of the simplest languages to learn. It operates on text-based coding, which suits the kids perfectly. It also comes with visual, block-based coding programs, thus allowing kids to transform their ideas into reality. It is one of the most engaging languages with a friendly user experience.

Python can also be used for creative projects, thus creating a fun and highly engaging experience for the kids. You can find many resources and books based on Python programming. Nowadays, many institutions also include Python language in their curriculum.

Setting up Python is Easy

Python is user-friendly, and it can be easily set up on any device. If you have windows, you can literally install Python in three simple steps. Even in other operating systems, setting up Python is no big deal, which makes it an appropriate choice for the kids to get started with programming. The easy setup of Python in computers and other devices helps kids get a hands-on learning experience without any hassle.

Increasing Demand

The demand for programmers who can create professional applications is increasing exponentially. In the future, more and more jobs will require coders who can create innovative applications that make life easier. Thus learning Python can be extremely beneficial for kids in the long run. Therefore kids should be encouraged to get started with Python programming at an early age.

Allows Experimenting

One of the most excellent benefits of learning Python is that it allows students to implement their ideas. It lets kids get creative and experiment with different ideas. This also encourages students to create authentic models of their ideas instead of just writing them on a piece of paper. Python encourages students to become natural problem solvers.

The post Why is Python language perfect for kids? appeared first on Artificial Intelligence.

Before we talk about how big data is changing the world, it’s important to understand exactly what it means. In simple words, ‘Big Data’ refers to huge volumes of data. It can pertain to business activities, financial information, customer data, or anything. 

By MIKE KHOREV

The data can be unstructured as well as structured. On its own, it may not mean much. It’s what you do with the data that matters. Successful analyses of such data can offer great insights that can help with strategic and smart business moves.

Traditional data software cannot process big data. However, with emerging lightning-fast computing technologies, it’s possible to collect, process, and analyze tons of data in no time. In this post by SPOPLI Web Development & Services, we look at how Big Data has impacted the world in huge ways.

The Impact of Big Data Across Different Sectors

Government Planning & Census

Governments take all their important decisions based on the data collected from population census. The data helps them understand what people need, their preferences, and their wants. 

When the general public fills out a census form, it helps the government understand income levels, how much, on an average, each family member earns, and how many people need financial support from the government. 

Entertainment

The entertainment industry is big on using Big Data analytics. For example, it can collect data of the previous song or movie release. Based on the conclusions derived, it can predict/determine whether or not the upcoming release will be a hit. 

The data can further help understand entertainment companies the number of downloads a song has received and how much they can earn from CD/DVD sales based on historical data.

Medicine

Medicine is another crucial field where big data holds immense importance as there is always a need for drug improvement. To make regular developments possible, the medical industry collects data of previous medicine. It then looks at parameters such as reaction time on a person and its effect on a certain virus. 

Medicines need meticulous research and detailing of huge data. Based on the mining of large sets of data and research, experts can develop better medicines and find cures for diseases.

Sports

In sports, Big Data helps to understand the strengths and weaknesses of each player. It helps to better strategize the gameplay. In big leagues, data helps decision-makers pick the best players during team selections.

Technology

Technology is always in need of data mining. It is used for myriad purposes such as advancing devices and understanding old algorithms to refine them. Big Data holds immense value for IoT that relies on the collection of real-time data. 

AI (artificial intelligence) that uses machine learning also processes big data gathered from users’ mobile devices. Every time you look for a product or perform a Google search, that data is gathered by companies. 

Those companies then process the data to better understand customer usage patterns and their buying habits. it enables them to serve customers better by recommending relevant search data and results.

Impact of Big Data on the Financial Sector

Customer Service – One thing that the financial sector lacks is an understanding of its customers better. Knowing what the customers want and their pain-points is the key to devising better advertising strategies. 

Big Data Analytics has made it possible for the finance sector to understand evolving customer expectations better. With this understanding, they can serve their customer better by enhancing the overall customer experience.

Fraud Detection and Risk Management – This is one of the most useful implications of big data. It helps financial companies identify the potential for fraud so they can implement security measures in advance. With the help of a data-driven setup, companies can predict and prevent fraud. 

Better employee engagement – By deploying data-driven analytics, companies better enhance their work performance and create better employee engagement programs. HR managers can identify the best work performances and create strategies that improve the success ratio of employees across the organization.

How Big Data is a Game Changer for Marketers?

Big Data has become a game-changer for businesses as it’s emerging as a tool of great importance across all industries. From a small firm to MNCs, companies are using big data to beat the competition and secure an edge over others.

Below are a few ways in which big data helps marketers and companies:

Confident decision making – Companies thrive on the ability to make quick decisions to act them out quickly. Responding to operational changes and trends is essential for businesses to grow. But, making quick and healthy decisions requires analytics. That’s where Big Data supports companies in a big way. By offering insight based on a huge collection of data, companies can make informed decisions.

Cost Reduction – Businesses are always looking for ways to cut down unnecessary costs. By evaluating things like the effectiveness of staff, their working patterns, and energy usage, companies can identify areas where they can make cost-savings without negatively impacting the other business operations.

Better customer engagement – Every time a customer makes an online search query or looks at a product catalog, all that information is stored with companies. By analyzing that data, companies can serve their customers better. 

Taking their buying preferences, habits, and tendencies into account, companies can create a more custom experience. They can better serve customers by showing them exactly what they come to expect. It also helps with better targeting.

Helps to recognize new streams of revenue – By taking customer trends and expectations into account, Big Data Analytics can tell companies which business avenues are most likely to yield revenue. In this way, businesses can expand and make smart decisions that can pave the way for a profitable future.

Bottom Line

As you can see, Big Data analytics has changed the face of the business world, finance, and many other aspects of the world in lots of ways. And, it’s going to continue doing that.

The post Why Big Data Has Become A Big Game Charger In Today’s World appeared first on Artificial Intelligence.

Today, businesses are increasingly reliant on artificial intelligence and machine learning to solve critical problems. However, dealing with immense data complexities along with the pressure of having to provide rapid results, could be crippling. Most companies find building an ML-savvy framework quite overwhelming. In an engaging session at MLDS 2021, Sayanti Bhattacharya, Senior Manager, and Ashwin Pai, Manager at Ugam, a Merkle Company, addressed how businesses can apply machine learning to drive results. 

Common Misconceptions

Machine learning has become such a fashion statement that, more often than not, businesses jump the gun by implementing ML in a hurry, defying logic. Bhattacharya stressed on the importance of focusing on the right way to chart a company’s ML journey. For instance, said Bhattacharya, we often see ML applications in our daily lives in the form of maps, digital ads, object detection technology, personalised notifications, etc. But when it comes to business applications, leaders are daunted by thoughts such as; the ML is complicated; involves coding; dicey on the value it brings; and worried about its overall compatibility with their business plan, etc.

Pai has neatly laid out the ML concept in simple terms:

Taxonomy: It is essential to understand that machine learning is a subset of AI and includes supervised learning, unsupervised learning and reinforcement learning. They are further divided into tree-based models, association rules, neural networks, regression, clustering, similarity algorithms, transfer learning, deep reinforcement learning and more. “There is a whole lot of length and depth associated with machine learning,” stressed Pai.

Common beliefs: Pai has also brought up the uncalled pressure companies face in implementing ML just because it’s trendy. More often than not, companies talk about adopting ML without realising the underlying need for it. They fall for ‘bigger the better’ trap and end up integrating complex algorithms. 

When Should I Use ML?

Bhattacharya said the companies need to do a reality check to assess if ML is critical to their operations. Machine learning can work well for tasks that entail sequential decision making or rule-based decision making. “Having said that, bigger is not always better,” she added.

Picking up from Bhattacharya left off, Pai said the key is to keep ML scalable but straightforward. For instance, in the earlier days, apriori algorithms were used for concepts such as product affinity, ARIMA for forecasting and logistic regression for classification, but are now replaced by more complex algorithms such as LSTM and deep neural networks, as data grew in volume over the years. While there are many options available to approach a problem, it is crucial to break down the problem and then apply ML, if necessary, said Pai.

Incremental Improvements

Bhattacharya pointed out that for machine learning to create the most value, it is essential to consider ML as a marathon and not a sprint. She said, rushing into incorporating ML into an organisation’s workflow may lead to challenges such as data fatigue, infrastructure fatigue, time fatigue and most importantly, expertise fatigue. “Dealing with data requires collecting, analysing and harmonising it — jumping into it will lead to a stressful journey,” she said. Therefore, ML requires building endurance rather than speed. For instance, she said, areas such as customer review and ratings, website search metadata, customer service enhancement etc can be done with text analysis. 

To build M, laying a strong foundation is essential. Detailing a use case, she said they used elementary application of text mining for a situation where they had to understand what customers were saying about the products on a website—implementing a solution as easy as this resulted in a 20% reduction in product return and identification of unauthorised users. 

Building on this foundation, they further classified the problem to understand customer’s complaints on order deliveries from customer call logs. The team introduced topic modelling to identify related words and classify them into topics leading to 30% lesser customer complaints, better order tracking and delivery experience. 

The next goal was to understand what is important for consumers in a category and what do they like and dislike in the current assortment. The team applied topic modelling and sentiment analysis to identify themes, and overlaid a sentiment analysis framework to generate actionable insights. This resulted in a 30% increase in analysing customer feedback and a 25% reduction in cost.  

Is layering the only approach to implement ML? “No,” says Bhattacharya. Ensembling various methods is another good option. Use of agglomerative clustering, cluster profiling, rule mining, price grouping and rule-based binning can result in grouping similar listings that point to the same product. “Starting with one element and building upon it is the key,” she stated.

Pai also pointed out that while endurance is great, technology plays an important role to keep up with these developments. “Improving technology, tech stack, data engineering capabilities will help in maximising the impact,” he said. 

Key Takeaways

• All problems do not need high-end ML solutioning
• More complex does not equal to better outcomes
• Think marathon, not sprint
• Make incremental improvements
• Technology/ infrastructure capabilities 
• A measurement framework along with a north-star metric is crucial to measure the value

The post Why ML For ML’s Sake Is A Bad Idea? appeared first on Artificial Intelligence.

Do you want to minimize risk and have your business succeed in increasingly uncertain times? If so, then studying data science could be your next best step.

Before you take any decision to study, it is vital to read as widely as possible around the subject and all the possibilities available. There are numerous and varied resources out there and as many buzzwords as there are bees. So, get as much information as you can and read it.

Here are a few introductory tips as to why data science may be the perfect study choice for entrepreneurs and wantrepreneurs such as yourself in 2021 and beyond.

Data Science Will Help You Understand Your Business

For improved enterprise management and the protection of profitability, you need to understand your business. What are the historical risks? What are the key features of the business over time? How can you avoid these in the future? In order to answer these questions, you will need to have a wealth of information at your disposal and be able to manipulate this data to create possible business planning scenarios. That process is encapsulated in data science. It’s a complex process made easy with the data science degree Merrimack college has to offer. Taking a qualification like this will enable you to learn crucial skills that can help solve real problems that your business could come to face.

Keep Your Business at the Cutting Edge

Current uncertainties in the market have increased all business risk, and it is now more pertinent than ever to use everything at your disposal to keep your enterprise profitable. A growing mainstream/ big business trend is the use of data science to improve risk management. In this instance, if data science is good for mainstream business, just maybe it’s good for us. Furthermore, studying data science will provide for one of those crucial entrepreneurial skills, allowing you to express your willingness to learn.

One of the Fastest Growing Professions

A qualification in data science currently offers you the security that many entrepreneurs can only dream of. As once you understand the importance of big data and data science for your business, not only will you improve risk management for your enterprise, but you have at your fingertips the skills for a top professional job or consultancy option.

You can be a data scientist, research scientist, or senior research analyst, just to mention a few of the numerous professional jobs a data science degree or course can get you.

Understanding Data Science Can Reduce Your Exposure to Business Risk

Using as much historical data as can be collected and having a keen understanding of current trends, coupled with data science, allows the building of sensible business scenarios that serve to plan around risk. A good understanding of data science will allow you to mitigate the risk, dealing with it using advanced planning and adjustments made to the operations.

What Now?

If you’re serious about data science as your way to improve your career or business, then your next step is to strategize as to the many possible scenarios that big data mining may provide your business. It seems a lengthy process but can be simplified with the right knowledge base. Studying data science is thus the most sensible route. To gain a genuine understanding of the algorithms used to collect and process big data in a scientific way to improve the manner in which a business deals with risk, a professional data science qualification may be for you.

The post Why Should you Study Data Science? appeared first on Artificial Intelligence.

How to bridge the production gap between data and IT

Data scientist has been one of the superstar IT roles of recent years, with the promise of spinning data into gold with the application of AI and machine learning.

Using cutting-edge technology to extract deep insights from reams of business data, data scientists aim to help guide their organisations into a more innovative, efficient and profitable future. But so far, return on investment hasn’t always been what companies might hope.

“One of the biggest mysteries in data science today actually has very little to do with data science: What is that last mile to AI ROI?” says Sivan Metzger, managing director MLOps and governance at DataRobot. “You build your machine learning, you find the data, you get it cleaned up, you build the models, you try 90 different iterations, you make a good and clean one and it’s ready to go. What happens then? Why are we not seeing value at scale from AI?”

Metzger credits these issues to a disconnect between the data team, IT operations and stakeholders on the business side (i.e. the potential consumers of data science insights). Data science and IT operations teams have very different considerations and goals – and machine learning is very different from running software. This disconnect is known as the ‘production gap’, and can prevent AI solutions from being properly executed.

Machine Learning Operations (MLOps) is a combination of processes, best practices and underpinning technologies which seeks to bridge this gap by increasing collaboration and communication between data scientists and operations staff – and ultimately ensuring that AI is properly deployed and can begin to deliver the ROI promised.

To learn more about how MLOps can improve your returns on AI, watch IT Pro and DataRobot’s webinar ‘The Last Mile to AI ROI’, in which Metzger and data scientist Rajiv Shah discuss topics including:

• How to eliminate AI-related risks by adopting MLOps best practices
• The inherent challenges of production model deployment and how to overcome them
• Model-monitoring best practices
• Production lifecycle management and why it matters

The post Why isn’t your AI delivering ROI? appeared first on Artificial Intelligence.