Hi dear, welcome to the fascinating way of Jira’s concept of learning. Today I will tell you about the creation of projects in Jira. This is the most common and important feature because if you are using Jira then it is very important for you to know how to create the project in Jira so that you can start your journey with Jira. You will see what is the agenda in this article? So, in this article, we will learn how to create a project, what the classic project is and how to work with that, and how to create boards so, we will start from the first one. You will see first how to create projects so, let’s go to the cloud instance. There will be the cloud instance that you have created. In this section, you will see there is the default page of a Jira. If you want to change your default page in Jira then you can do this I will explain the setting later in this course in the article

How to create a project?

We will, we are going to create the projects so, you will go into the project section and you will see, you will haven’t create any project yet so, it will show you currently have no projects so for creating the project you will click on the create project button and you will see, it gives you options to enter your project name and to choose the template right now it’s showing the scrum template but if you want to change your project template then you can simply click on the change template and you will get a load of options available. There, you can choose according to your requirements the Kanban, scrum, task tracking, and bug tracking templates and many more are available there but you will use the scrum template so. You will go with that scrum one and select that scrum one. You will mention the project name and project name is project a. Once you will click on the advanced option you will see the key of a project. The key of a project should be very descriptive and easy to use. What is the logic behind the project key? It takes the first letter of our project name and if you want to name and if you want to change the project key then you would recommend changing it at the time of the creation of the projects because it is very difficult to change the project T later once you will create the project and adding issues or your requirements in the project so I would recommend you to change your project T then please change at the time of the creations so you will go and click on create button it will land you in that page where you will see the backlog options in the backlog you can create your requirements in the form of a story in the form of the task and you can create the epics.

I will explain to you what are the effects stories and tasks you can see the other issues types are available story tasks bugs and there are the options of creating the epic so, what are these issues types? So, you will see, what are options are available in that project you will see when you will create the project in Jira then it creates automatically a bowl. You can say this is a PA board if you want to create your own board then you will click on the plus icon and you can create the board it could be a scrum or it could be a Kanban board it’s up to you but for now, you will cancel it and the second option is a backlog as I told you, you can collect your requirements in the backlog and the third one is the active sprint then you don’t have any experience available there.      

What is a classic project?

So, it will look like as there are the three columns are available to do in progress and then I will explain to you how you can change these columns and how can you add more columns in this active sprint board and there are report options are available. As you will see, there are a lot of reports are available burndown charts, burnup charts sprint reports velocity, and the reports which are related to the issue analysis forecast and management and the others. You will see, there are no available sprints for this board that’s why this is the reason it will not show any report there so once you will start your sprint then how can you start your sprint? You will be able to see the report there so, you will back to the other one. You will see relays this is the best option. With the help of these release options you can create the versions and how you will create diversion you can manage your releases there next option is issues and filters all issues that you will create in that project will be listed down there so, you can choose all issues and it will show you all the issues in a list. I will show you how but this time you don’t have any issues so, you cannot see it and you will see there the other options are available. You will see open issues your reported issues recently viewed issues or recently created resort or updated issues as well so, I shall show you once you will create issues in the project and the next project is pages. You can show any pages which are related to your project there and the next one is the components. You can create the components and you can categorize your issues as for the components that are the best feature in the Jira and you can add the other items may be.

If you will click on that then you can add the repose repository link pages Ning or any shortcut which are frequently used so you will go into the last one and which is a project setting in that option that contains a lot of features. So first, we will start from the details you will see that you can change the project name and the project key there but you will be able to see that option because you will be admin and have access to change that project key but you will haven’t admin access then you can’t change it you can mention any URL but you can’t change the project type. If you want to change the project type then you cannot do it there for this you will have to create a new project and move your all the issues on that particular project type and there you can select an image and you may be put your project low logo and any other related images. You can write their description and mention the project lead there and you can set the default assignee as well the next option is people. In this option, you can add the people which are going to work into your project. So simply, you can add the people by clicking that arrow options you can choose the role and type the name then it will show automatically has done. I will explain to you later how you will create the roles and how can you add more team members? So that you can add the people to your project. The third one is a summary where you will be able to see the complete summary of your project. 

Related videos:

The post How to create a new project in Jira? appeared first on Artificial Intelligence.

he hype about machine learning (ML) is warranted. Machine learning is not just making things easier for the companies that are taking advantage of it. It’s also changing the way they do business. For example, machine learning is:

• Being used by financial institutions to quickly detect fraudulent activity
• Enabling healthcare practitioners to diagnose diseases and prescribe treatments more effectively
• Helping manufacturers monitor equipment so issues can be dealt with before they disrupt operations
• Allowing streaming services to identify customers at risk of taking their business elsewhere and helping determine what steps can be taken to retain them

With increasing data volumes, low-cost data storage, and less expensive, more powerful data processing, the potential applications of machine learning will grow exponentially.

So why are so many companies hesitant to jump on the machine learning bandwagon – and why is the success rate so low for those that do embark on these projects? Afterall, organizations such as Gartner note that up to 85% of machine learning projects ultimately fail to deliver on their intended promises to business.

More importantly, what can companies do to ensure a higher success rate so they can leverage the promise of machine learning?

Machine Learning is Different

To increase the chances of machine learning project success, the first step is to understand that these projects are not the same as typical application and software development projects. There are different processes, terminology, workflows, and tools involved.

There are also different staffing requirements. Among the most important are data scientists, who are especially critical when it comes to defining the success criteria, final deployment, and continuous monitoring of the machine learning model.

Data engineers, business intelligence specialists, DevOps, and application developers also play key roles. Few organizations have the internal resources to fill all these positions. Their options: hire them, which isn’t always easy given that machine learning is still a relatively new field with few experienced professionals, or outsource.

Even if an organization does have the staffing covered, it can be difficult to facilitate collaboration and communication between different teams. Traditional software and application development usually differs greatly from data science projects. Whereas software development tends to be more predictable and measurable, data science can entail multiple iterations and experimentation. Expectations are different. Deliverables are different.

The Issue of Data Quantity and Quality

According to a number of research initiatives (e.g., Hidden Debt in Machine Learning Systems) technical debt resides in areas common to many machine learning projects: Data Quality, Model Quality, Feature Versioning, Model Monitoring, Data Labeling, Model Explainability, and Fairness, Process Automation, Human Intervention (Review) in-place capabilities.

There’s also the matter of data quantity and quality. Machine learning projects use large datasets, since larger datasets facilitate better predictions. But as the size of the data increases, so do the challenges.

Data is usually merged from multiple sources. Often that data is not in sync, which can create confusion. In addition, data can get merged that wasn’t meant to be merged, resulting in data points with the same name but different meanings. Bad data can generate results that aren’t actionable or insightful, or that are misleading.

The lack of labeled data can also be an issue. Some teams may try to take on the laborious task of labeling and annotating training data themselves. Some may even try to create their own labeling and annotation automation technology. The problem is that a great deal of time and expertise is committed to the labeling process rather than machine learning model training.

Outsourcing can save both time and money but doesn’t work well if the labeling task requires specific domain knowledge. In those cases, organizations also must invest in formal and standardized training of annotators to ensure quality and consistency across datasets. The other option is to develop their own data labeling tool if the data to be labeled is extremely complex. However, this can require more engineering overhead than the machine learning task itself.

Yet another data-related issue is that the data required in a machine learning project often resides in different places with different security constraints and in different formats — structured, unstructured, video files, audio files, text, and images. Data preparation is required, a process that includes searching, cleaning, transforming, organizing, and collecting data. It’s a tedious activity that can require teams to spend up to 80% of their time converting raw data into high-quality, analysis-ready output.

For both the data labeling and data preparation, automation can help remedy the situation – but again, requires expertise that internal teams often lack.

Great Expectations

Machine learning projects aren’t cheap, so it’s not uncommon for organizations to have overly ambitious goals for them. There are often expectations that a project will completely transform the company or a product and generate an enormous return on investment. That creates a lot of pressure that can, in turn, lead to second-guessing on strategies and tactics.

Not surprisingly, these kinds of projects tend to drag out. As a result, both the project teams and management lose confidence and interest in the project, and budgets max out. Even the most expertly run projects are doomed to fail if the goals are unrealistic.

In other cases, machine learning projects kick-off without alignment on expectations, goals, and success criteria between the business and project teams. Without clearly defined success indicators, it’s difficult to determine whether a project is successful, what changes need to be made, if the model is effectively solving the intended business needs, or if other options should be considered.

Machine Learning Success Factors

While there are no specific guidelines for ensuring a successful machine learning project, there are ways to overcome many of the issues that can lead to project failure. Among them:

• An understanding of how machine learning works, how it differs from other project types, and what’s required to execute a project
• A properly scoped project with realistic goals, budget, and leadership support
• The resources to run a machine learning project, including experienced team members — whether in-house or outsourced — and a commitment to collaboration and open communication
• Large amounts of data, preferably labeled
• Capabilities for collecting, storing, labeling, cleaning, quickly accessing, and processing large volumes of data
• Advanced tools for machine learning models and data monitoring
• Capabilities for a human to review the machine learning system and inference at any place and point in time
• Software tools for executing machine learning algorithms
• A development platform, such as AWS, Baidu, Google, IBM, or Microsoft

Follow these tips and your machine learning project won’t derail before your organization can enjoy the many benefits that this modern technology provides.

The post Why Machine Learning Projects Fail – and How to Make Sure They Don’t appeared first on Artificial Intelligence.

Organizations can gain more business value from advanced analytics by recognizing and overcoming five common obstacles.

More and more companies are embracing data science as a function and a capability. But many of them have not been able to consistently derive business value from their investments in big data, artificial intelligence, and machine learning.1 Moreover, evidence suggests that the gap is widening between organizations successfully gaining value from data science and those struggling to do so.2

To better understand the mistakes that companies make when implementing profitable data science projects, and discover how to avoid them, we conducted in-depth studies of the data science activities in three of India’s top 10 private-sector banks with well-established analytics departments. We identified five common mistakes, as exemplified by the following cases we encountered, and below we suggest corresponding solutions to address them.

Mistake 1: The Hammer in Search of a Nail

Hiren, a recently hired data scientist in one of the banks we studied, is the kind of analytics wizard that organizations covet.3 He is especially taken with the k-nearest neighbors algorithm, which is useful for identifying and classifying clusters of data. “I have applied k-nearest neighbors to several simulated data sets during my studies,” he told us, “and I can’t wait to apply it to the real data soon.”

Hiren did exactly that a few months later, when he used the k-nearest neighbors algorithm to identify especially profitable industry segments within the bank’s portfolio of business checking accounts. His recommendation to the business checking accounts team: Target two of the portfolio’s 33 industry segments.

This conclusion underwhelmed the business team members. They already knew about these segments and were able to ascertain segment profitability with simple back-of-the-envelope calculations. Using the k-nearest neighbors algorithm for this task was like using a guided missile when a pellet gun would have sufficed.

In this case and some others we examined in all three banks, the failure to achieve business value resulted from an infatuation with data science solutions. This failure can play out in several ways. In Hiren’s case, the problem did not require such an elaborate solution. In other situations, we saw the successful use of a data science solution in one arena become the justification for its use in another arena in which it wasn’t as appropriate or effective. In short, this mistake does not arise from the technical execution of the analytical technique; it arises from its misapplication.

After Hiren developed a deeper understanding of the business, he returned to the team with a new recommendation: Again, he proposed using the k-nearest neighbors algorithm, but this time at the customer level instead of the industry level. This proved to be a much better fit, and it resulted in new insights that allowed the team to target as-yet untapped customer segments. The same algorithm in a more appropriate context offered a much greater potential for realizing business value.

It’s not exactly rocket science to observe that analytical solutions are likely to work best when they are developed and applied in a way that is sensitive to the business context. But we found that data science does seem like rocket science to many managers. Dazzled by the high-tech aura of analytics, they can lose sight of context. This was more likely, we discovered, when managers saw a solution work well elsewhere, or when the solution was accompanied by an intriguing label, such as “AI” or “machine learning.” Data scientists, who were typically focused on the analytical methods, often could not or, at any rate, did not provide a more holistic perspective.

To combat this problem, senior managers at the banks in our study often turned to training. At one bank, data science recruits were required to take product training courses taught by domain experts alongside product relationship manager trainees. This bank also offered data science training tailored for business managers at all levels and taught by the head of the data science unit. The curriculum included basic analytics concepts, with an emphasis on questions to ask about specific solution techniques and where the techniques should or should not be used. In general, the training interventions designed to address this problem aimed to facilitate the cross-fertilization of knowledge among data scientists, business managers, and domain experts and help them develop a better understanding of one another’s jobs.

In related fieldwork, we have also seen process-based fixes for avoiding the mistake of jumping too quickly to a favored solution. One large U.S.-based aerospace company uses an approach it calls the Seven Ways, which requires that teams identify and compare at least seven possible solution approaches and then explicitly justify their final selection.

Mistake 2: Unrecognized Sources of Bias

Pranav, a data scientist with expertise in statistical modeling, was developing an algorithm aimed at producing a recommendation for the underwriters responsible for approving secured loans to small and medium-sized enterprises. Using the credit approval memos (CAMs) for all loan applications processed over the previous 10 years, he compared the borrowers’ financial health at the time of their application with their current financial status. Within a couple of months, Pranav had a software tool built around a highly accurate model, which the underwriting team implemented.

Unfortunately, after six months, it became clear that the delinquency rates on the loans were higher after the tool was implemented than before. Perplexed, senior managers assigned an experienced underwriter to work with Pranav to figure out what had gone wrong.

The epiphany came when the underwriter discovered that the input data came from CAMs. What the underwriter knew, but Pranav hadn’t, was that CAMs were prepared only for loans that had already been prescreened by experienced relationship managers and were very likely to be approved. Data from loan applications rejected at the prescreening stage was not used in the development of the model, which produced a huge selection bias. This bias led Pranav to miss the import of a critical decision parameter: bounced checks. Unsurprisingly, there were very few instances of bounced checks among the borrowers whom relationship managers had prescreened.

The technical fix in this case was easy: Pranav added data on loan applications rejected in prescreening, and the “bounced checks” parameter became an important element in his model. The tool began to work as intended.

The bigger problem for companies seeking to achieve business value from data science is how to discern such sources of bias upfront and ensure that they do not creep into models in the first place. This is challenging because laypeople — and sometimes analytics experts themselves — can’t easily tell how the “black box” of analytics generates output. And analytics experts who do understand the black box often do not recognize the biases embedded in the raw data they use.

The banks in our study avoid unrecognized bias by requiring that data scientists become more familiar with the sources of the data they use in their models. For instance, we saw one data scientist spend a month in a branch shadowing a relationship manager to identify the data needed to ensure that a model produced accurate results.

We also saw a project team composed of data scientists and business professionals use a formal bias-avoidance process, in which they identified potential predictor variables and their data sources and then scrutinized each for potential biases. The objective of this process was to question assumptions and otherwise “deodorize” the data — thus avoiding problems that can arise from the circumstances in which the data was created or gathered.4

Mistake 3: Right Solution, Wrong Time

Kartik, a data scientist with expertise in machine learning, spent a month developing a sophisticated model for analyzing savings account attrition, and he then spent three more months fine-tuning it to improve its accuracy. When he shared the final product with the savings account product team, they were impressed, but they could not sponsor its implementation because their annual budget had already been expended.

Eager to avoid the same result the following year, Kartik presented his model to the product team before the budgeting cycle began. But now the team’s mandate from senior management had shifted from account retention to account acquisition. Again, the team was unable to sponsor a project based on Kartik’s model.

In his third year of trying, Kartik finally got approval for the project, but he had little to celebrate. “Now they want to implement it,” he told us, with evident dismay, “but the model has decayed and I will need to build it again!”

The mistake that blocks banks from achieving value in cases like this is a lack of synchronization between data science and the priorities and processes of the business. To avoid it, better links between data science and the strategies and systems of the business are needed.

Senior executives can ensure the alignment of data science activities with organizational strategies and systems by more tightly integrating data science practices and data scientists with the business in physical, structural, and process terms. For example, one bank embedded data scientists in business teams on a project basis. In this way, the data scientists rubbed elbows with the business team day to day, becoming more aware of its priorities and deadlines — and in some cases actually anticipating unarticulated business needs. We have also seen data science teams colocated with business teams, as well as the use of process mandates, such as requiring that project activities be conducted at the business team’s location or that data scientists be included in business team meetings and activities.

Generally speaking, data scientists ought to be concentrating their efforts on the problems deemed most important by business leaders.5 But there is a caveat: Sometimes data science produces unexpected insights that should be brought to the attention of senior leaders, regardless of whether they align with current priorities.6 So, there is a line to be walked here. If an insight arises that does not fit current priorities and systems but nonetheless could deliver significant value to the company, it is incumbent upon data scientists to communicate this to management.

We found that to facilitate exploratory work, bank executives sometimes assigned additional data scientists to project teams. These data scientists did not colocate and were instructed not to concern themselves with team priorities. On the contrary, they were tasked with building alternative solutions related to the project. If these solutions turned out to be viable, the head of the data science unit pitched them to senior management. This dual approach recognizes the epistemic interdependence between the data science and business professionals — a scenario in which data science seeks to address today’s business needs as well as detect opportunities to innovate and transform current business practices.7 Both roles are important, if data science is to realize as much business value as possible.

Mistake 4: Right Tool, Wrong User

Sophia, a business analyst, worked with her team to develop a recommendation engine capable of offering accurately targeted new products and services to the bank’s customers. With assistance from the marketing team, the recommender was added to the bank’s mobile wallet app, internet banking site, and emails. But the anticipated new business never materialized: Customer uptake of the product suggestions was much lower than anticipated.

To discover why, the bank’s telemarketers surveyed a sample of customers who did not purchase the new products. The mystery was quickly solved: Many customers doubted the credibility of recommendations delivered through apps, websites, and emails.

Still looking for answers, Sophia visited several of the bank’s branches, where she was surprised to discover the high degree of trust customers appeared to place in the advice of relationship managers (RMs). A few informal experiments convinced her that customers would be much more likely to accept the recommendation engine’s suggestions when presented in the branch by an RM. Realizing that the problem wasn’t the recommender’s model but the delivery mode of the recommendations, Sophia met with the senior leaders in branch banking and proposed relaunching the recommendation engine as a tool to support product sales through the RMs. The redesigned initiative was a huge success.

The difficulties Sophia encountered highlight the need to pay attention to how the outputs of analytical tools are communicated and used. To generate full value for customers and the business, user experience analysis should be included in the data science design process. At the very least, user testing should be an explicit part of the data science project life cycle. Better yet, a data science practice could be positioned within a human-centered design frame. In addition to user testing, such a frame could mandate user research on the front end of the data science process.

While we did not see instances of data science embedded within design thinking or other human-centered design practices in this study, we did find that the shadowing procedures described above sometimes operated as a kind of user experience analysis. As data scientists shadowed other employees to understand the sources of data, they also gained an understanding of users and channels through which solutions could be delivered. In short, the use of shadowing in data science projects contributes to a better understanding of the processes that generate data, and of solution users and delivery channels.

Mistake 5: The Rocky Last Mile

The bank’s “win-back” initiative, which was aimed at recovering lost customers, had made no progress for months. And that day’s meeting between the data scientists and the product managers, which was supposed to get the initiative back on track, was not going well either.

Data scientists Dhara and Viral were focused on how to identify which lost customers were most likely to return to the bank, but product managers Anish and Jalpa wanted to discuss the details of the campaign to come and were pushing the data scientists to take responsibility for its implementation immediately. After the meeting adjourned without a breakthrough, Viral vented his frustration to Dhara: “If data scientists and analysts do everything, why does the bank need product managers? Our job is to develop an analytical solution; it’s their job to execute.”

By the next meeting, though, Viral seemed to have changed his mind. He made a determined effort to understand why the product managers kept insisting that the data scientists take responsibility for implementation. He discovered that on multiple occasions in the past, the information systems department had given the bank’s product managers lists of customers to target for win-back that had not resulted in a successful campaign. It turned out that using the lists had been extremely challenging, partly due to an inability to track customer contacts — so the product managers felt that being given another list of target customers was simply setting them up for another failure.

With this newfound understanding of the problem from the point of view of the product managers, Viral and Dhara added to their project plan the development of a front-end software application for the bank’s telemarketers, email management teams, branch banking staff, and assets teams. This provided them with a tool where they could feed information from their interactions with customers and make better use of the lists provided by the data science team. Finally, the project moved ahead.

Viral and Dhara’s actions required an unusual degree of empathy and initiative. They stepped out of their roles as data scientists and acted more like project leaders. But companies probably should not depend on data scientists in this way, and they may not want to — after all, the technical expertise of data scientists is a scarce and expensive resource.

Instead, companies can involve data scientists in the implementation of solutions. One bank in our study achieved this by adding estimates of the business value delivered by data scientists’ solutions to their performance evaluations. This motivated data scientists to ensure the successful implementation of their solutions. The bank’s executives acknowledged that this sometimes caused data scientists to operate too far outside their assigned responsibilities. However, they believed that ensuring value delivery justified the diversion of data science resources, and that it could be corrected on a case-by-case basis, if the negative impact on the core responsibilities of data scientists became excessive.

The mistakes we identified invariably occurred at the interfaces between the data science function and the business at large. This suggests that leaders should be adopting and promoting a broader conception of the role of data science within their companies — one that includes a higher degree of coordination between data scientists and employees responsible for problem diagnostics, process administration, and solution implementation. This tighter linkage can be achieved through a variety of means, including training, shadowing, colocating, and offering formal incentives. Its payoff will be fewer solution failures, shorter project cycle times, and, ultimately, the attainment of greater business value.

The post Why So Many Data Science Projects Fail to Deliver appeared first on Artificial Intelligence.

Iguazio is adding a production-ready feature store to its data science platform.  It will allow users to more quickly and easily develop, deploy and manage AI apps, according to execs.

Iguazio is adding a production-ready feature store to its data science platform. The offering aims to bring off-the-shelf technology to the growing area of data science while also significantly reduce the skills barrier. 

Ignazio’s integrated feature store is designed to lower the skills required to work on AI applications, so that even firms without professional data scientists can participate.

In specific, Iguazio’s approach lets users catalog, store and share features centrally, making it easier for teams to collaborate on development, deployment and management of AI apps – even across hybrid, cloud and multi-cloud environments, according to Asaf Somekh, Iguazio Co-Founder and CEO.  

“For companies that don’t have hundreds of data scientists and data engineers, building a feature store from scratch, in-house, is not feasible,” Somekh said, adding “We wanted to bring this functionality to our customers, and provide them with an off-the-shelf solution for feature engineering across training, serving and monitoring in hybrid environments.”

Architecturally, Iguazio’s feature store is built on its open source MLOps framework, MLRun, enabling contributors to add data sources and contribute additional functionality.

Iguazio’s feature store offers a “unified” approach, as it is integrated within its data science platform. This unique design means It plugs seamlessly into the data ingestion, model training, model serving, and model monitoring components. This reduces significant development and operations overhead while also boosting performance, Somekh added. 

Iguazio provides “next-level automation of model monitoring and drift detection,” Somekh added, to support model training at scale and to run continuous integration and continuous delivery (CI/CD) of machine learning (ML), he added.

Also notable, Iguazio’s “unified” feature store is available online and offline. 

In a recent post which also appeared on Medium, Ignazio’s VP Product Adi Hirschtein explained the need for a modern “unified” feature store:

In detail, Iguazio’s integrated feature store provides users these important advantages:

Ability To Build Features Once and Plug Them Anywhere, Seamlessly:  Because the Iguazio feature store is a centralized and versioned catalog, everyone can engineer and store features (along with metadata and statistics), as well as share and reuse them, and analyze impacts on existing models. Users can collect many independent features into vectors and use those from their jobs or real-time services. Iguazio’s high-performance engines automatically join and accurately compute all features.

Real-Time Features and Drift Detection: Iguazio can detect model drift and inaccuracies automatically. Upon such discoveries, Iguazio can alert the users or initiate automated re-training workflows.

Robust Data Transformation: Users can create complex feature engineering processes with Iguazio’s built-in robust data transformation service. This service includes feature aggregations with sliding windows, dozens of pre-built transformations, or support for custom logic in native Python code. With a simple API and SDK, data scientists can easily create features without requiring long data engineering cycles.

Feature Catalog: To let users share, search and collaborate on features, evaluate features with detailed statistics and analysis, and see how features correlate to both data sources and models with an easy-to-use user interface.

Integrated Data and Model Monitoring: Iguazio captures the feature statistics in real-time, enabling drift detection based on actual data drift. Thanks to the Iguazio feature store’s integration, capabilities such as concept drift monitoring and feature monitoring are available out-of-the-box.

Real-Time Feature Engineering:  Users develop features once. The feature transformation pipeline calculates features in real-time based on incoming events or streams and serves the results at millisecond-level latency or pushes them directly into a stream.

Data Governance: With strict governance, Iguazio users can also keep the data lineage of a feature, with the tracking information capturing how the feature was generated, critical for regulatory compliance.

Iguazio customers include Payoneer, Quadient, and Tulipan for various use cases such as fraud prediction and real-time recommendations. 

The post Iguazio Data Science Platform Simplifies AI App Projects; Adds Integrated Feature Store appeared first on Artificial Intelligence.

Seventy percent of companies have reported minimal or no impact from Artificial Intelligence projects, according to a survey by MIT and Boston Consulting Group.

There are a number of reasons for this including a lack of focus on cultural change and training within an organisation as it adapts to new working practices, but the most important factor is poor data. This encompasses everything from inadequate data architecture and discovery, to modelling, quality, and governance.

If using the analogy that artificial intelligence is the “icing on the cake”, then data is the cake itself.

At some point over the next 12 months, with the global recession constraining budgets for every organisation, Chief Information Officers and Chief Data Officers will need to demonstrate a clear return on investment for their AI projects and provide evidence of measurable results.

To date, AI projects have been as much about showcasing that its mere presence demonstrates innovation and a digital transformation at the cutting edge of technology.  This will not be the case moving forward with increased financial and operational scrutiny so CIOs/CDOs will therefore need to “get it right” or risk funding for AI projects being curtailed.

Below are five data-related areas of focus that organisations should seek specialist guidance on to “get data ready” and enable success for these artificial intelligence projects.

Data Architecture

There is no more valuable role right now in any organisation than a data architect, and the best architects will understand that the end output from their role is to unlock value from data.

Data Architects understand the organisation’s strategy and business problems to be solved, but have the technical insight to get their hands dirty in the data itself. They know what data is needed and how that data is to be integrated by various systems.  They then set out clear guidance for everything from governance to security. This role is not the same as IT architecture and often organisations make the mistake of leaving this to the CTO or a traditional IT transformation provider, rather than bringing in the specialist support required to achieve success.

Data Access

Once the architecture is sound from a business perspective (rather than just an IT perspective), there are a huge number of data sources to feed the system and these will need careful management. The greater the richness of data from various sources the better the output, yet without management, an organisation becomes overloaded with messy, inaccurate, or incomplete data of different types and quality.

These data sources include enterprise data silos (in whatever format from databases to spreadsheets on someone’s hard drive), open-source data such as social media, government data, or data from the Internet of Things sensors.

However, the real challenge is that these are all separate data silos that cannot be moved into one convenient data warehouse for data scientists to extract, transform, and load. The growth of Data Catalogs demonstrate that they are quickly becoming a “must-have” for CIOs/CDOs in any organisation to solve the problem of multiple data silos with the more advanced data catalogs offering a discovery capability so users find data that they didn’t even know existed.

Data Modelling

Data modelling is often seen as boring and therefore overlooked but if your organisation wants to really get value out of its data then this is a critical activity. Why wouldn’t this be the case where data is now such a valuable asset in any organisation?

Time invested in data modelling ensures consistency of structure, terminology and standards throughout the organisation. Even the process from moving from a conceptual model to a physical data model encourages collaboration and agreement between all data stakeholders.

The data modelling process brings together business processes and aligns with data and IT communities. It will define key components and relationships between various sources of data and business workflows and will save time and cost for the organisation as well as improve performance. Put simply, it will enable everyone to understand how data is to be used within the organisation and, more importantly, how that data is turned into information, which delivers insight to an end-user.

Data Quality

Like any asset, its value depends on how reliable it is to the organisation.  The same applies to data – often organisations cannot or do not want to measure the cost to their organisation that can be directly attributable to poor quality data or missing data. Data quality is the standard to which an organisation’s data is accurate, timely and complete, as well as consistent with business rules.

Without good data quality, data itself cannot be relied upon for analytics or AI applications. This becomes even more important as the volume of data will grow, as will the types of data from different sources of data so this is an ongoing process that needs to be proactively managed through business rules and accepted by the entire organisation.

Data Governance

Having focussed on the above four items, data governance consists of the rules, enforcement and management of an organisation’s data as an asset. This ensures that the above four areas are not one-off activities that fall by the wayside once completed.

Data only has a certain lifespan for it to be useful. Good data governance accompanies the change in culture through training, which is required to complement AI projects.  Data governance must be a whole-organisation proactive activity to maintain a level of data access, quality, security and management to provide the organisation with data at the right time, and of the right quality, to turn into information, and therefore create data-driven insight on which business decision can be made.

A strategy that addresses how an organisation manages the above five areas of data management is a great start to demonstrate that an organisation is thinking about its data as an asset.  Artificial intelligence projects can then be far more successful in releasing the value of that data.

The post Why artificial intelligence projects fail appeared first on Artificial Intelligence.

Where is the fun in a pinball machine that can play itself? How about pancakes that can cook themselves?

For students in Kennesaw State University’s Department of Mechatronic Engineering, the intrigue is less about the act and more about what goes on behind the scenes to make it possible. From a fully automated pinball machine to a pancake vending system that can cook without human intervention, several student teams recently took full advantage of their senior capstone coursework to prove that engineering can be creative while practical.

Since its inception, the mechatronics engineering department has emphasized building physical prototypes for its senior capstone coursework, allowing students to pitch projects of their own or select one from a pool of industry sponsors, said Kevin McFall, associate professor of mechatronics engineering and assistant dean for research in the Southern Polytechnic College of Engineering and Engineering Technology. However, what makes the mechatronics capstone process distinct from others across the college is that the students can select the minimum success criteria for their project. This means each team creates their own criteria for judgement, reports it to their professors and is solely judged on those self-made parameters.

In order to pass, McFall said students must demonstrate their ability to build something that achieves the three fundamental principles of mechatronics engineering: sense, think and act. All projects must have a mechanical design, a way to acquire data from a series of sensors, some sort of programmable device and a way to control an actual moving part.

Some students, like student Tyler Gragg, embrace the freedom to generate a unique project. A self-declared pinball machine aficionado, Gragg has always felt the desire to build a machine of his own. When he pitched the idea to teammates Kevin Kamperman, Cody Meier and Omar Salazar Lima, they conceived a design that would allow the pinball machine to play itself using a video camera to detect when the ball enters the “flipper zone,” which then triggers the flippers to move automatically and keep the ball in play.

Following the spread of the coronavirus, the team worked with staff members in the Department of Architecture to fabricate final pieces while they remained off campus. Since most teams completed most of their design work and built their components in advance, they were able to the see their projects to completion amidst the changes caused by COVID-19.

For Tim Ervin, inspiration for his senior capstone project came in the form of a YouTube video in which a team of engineers cooked a four-foot pancake using a robotic arm. Rather than make an impractical and gargantuan pancake, teammate Jay Strickland suggested that they make something that can cook several smaller pancakes. Along with fellow former students Brittney Smith and Ryan McHale, they built what they call a pancake vending machine, which can accept digital payment in exchange for one perfectly cooked pancake.

The self-contained machine is able to dispense the correct amount of batter onto a griddle, and then, with a spatula attached to a robotic arm, flip the pancake for an even cook on each side. The entire process can be done in just five minutes.

The post Mechatronics projects mesh creativity with engineering appeared first on Artificial Intelligence.

Today most large companies are looking at the potential of AI/ML, and despite significant investments, hiring data scientists and investing time and money, data science fails to take things to the next level.

One of the biggest challenges present in AI/ML is that a large majority of models are not deployed in production. A lot of people in the enterprises have realised that typically when you have any kind of machine learning or data science work, it goes from a few weeks to develop the model, takes far longer when we talk about placing the developed models into production, maybe more than a year till the model is put into production.

The production takes a long time compared to the development of an ML model. Sometimes when you start rearchitecting the whole ML pipeline keeping deployment in mind, the entire work can go in vain. Deployment pipelines, deployment assumptions and deployment way of doing modelling is quite different. Is data science enterprise-ready?

In a Gartner’s survey of more than 3000 AI aware C-level executives, only 20% reported having AI production, and 80% said they are developing, experimenting and contemplating the use of AI. In another report by Mckinsey, the firm found that out of 160 reviewed AI use cases, 88% did not progress beyond the experimental stage.

As the market for AI technologies and techniques matures and grows, companies need more and better access to innovative AI models, applications and platforms. Unless things are in production, there is no return on investment.

“Technology innovation leaders are keen to apply DevOps principles for AI/ML projects, but they often struggle with architecting a solution for end-to-end automation pipelines across data preparation, model building, deployment and production because of lack of process and tooling know-how,” says Gartner.

Management Problems 

The management across several companies may not be fit to learn or comprehend data science. You may have the best model in the world, but if the management doesn’t realise its value, it is probably not going into production. A lot of these times, business intelligence and software stack offer clearer value to an organisation than complex data science systems. With the high expenses of developing AI projects, many organisations are reluctant to invest in the required staff and software to deliver on the promise of AI.

A lot of the times in data science, models do not survive the PoC stage and get dumped due to various challenges, which boils down to a lack of fundamental data literacy at senior levels that leads to data science getting ignored often. 

Technical Challenges

For the majority part, the reason why models are not deployed comes down to resources is that technology is new, and most IT-led companies are merely unfamiliar with the tools and specialised hardware needed to deploy data science models successfully. 

One of the essential things in data science is choosing the right problem and chasing the right solution. But, due to complicated technical details, people get caught up on and find themselves a year later having added zero value. Often in data science, projects end up being more complicated in comparison to the business value they are meant to produce. 

Data Collection Issues

According to experts like Bill Inmon, the vast majority of data scientists spend most of their time as data collectors, consolidating disparate data sources together, and formatting and cleaning data. Data sourcing, understanding, organising, cleaning are the most difficult part of most AI projects. 

Most organisations have highly siloed data which makes it very difficult to put a model in production. Not just data, ML pipelines also take place in isolation and not in a connected manner. This leads to a lack of collaboration among the team members.

Collection of the required data is a challenging task. Data always exists in different formats, structured and unstructured, video files, text, and images, stored in various places with unique security and privacy issues, which makes implementing AI challenging, because the data needs to be consolidated and cleaned. Unstructured data or unformatted data which may take most of the time for data cleaning and can be a reason for losing motivation. Insufficient data which is available for the analysis can also be a factor for failed AI projects. 

Incompatibility With Enterprise Systems

Data scientists use languages like Python that may not be compatible with the programming languages used in production systems. To make the model work with the existing systems, it takes a lot of time before the model is recoded, fully retested and tested before deployment. This process may take months and by the time the model is set for production, it can become unnecessary.

If a data science team deployed a model in production, it might need them to work with an engineer to implement it in Java or some other programming language to make it work for the enterprise. Now, this needs constant iterative effort as the model can become useless otherwise with the addition of new data.

The post WHY MAJORITY OF DATA SCIENCE PROJECTS NEVER MAKE IT TO PRODUCTION appeared first on Artificial Intelligence.

Artificial Intelligence (AI) is opening up the next phase of technological advances. Riding the AI wave, Google has started six AI-based research projects in India. These projects would focus on addressing social, humanitarian and environmental challenges in sectors such as healthcare, education, disaster prevention and conversation.

Google Research India, based in Bengaluru, will provide funding and computational resources besides supporting the efforts with expertise in computer vision, natural language processing, and other deep learning techniques, says Manish Gupta, director of Google Research Team in India. The research team will focus on two pillars: First, advancing fundamental computer science and AI research by building a strong team and partnering with the research community across the country and secondly, applying this research to tackle big problems in fields such as healthcare, agriculture, and education while also using it to make apps and services more helpful.

“Each of the projects is a collaboration between leading academic AI researchers and a nonprofit organisation with expertise in the respective area, with support from Google researchers, engineers and program managers,” he adds.
The IIT Delhi and Singapore Management University and Swasti project is about improving health information for high HIV/AIDS risk communities by applying AI to identify influencers among marginalised communities at high risk of HIV/AIDS contraction.

Researchers from IIT Madras led by Balaraman Ravindran and non-profit Armman will use AI to predict the risk of expectant mothers dropping out of healthcare programmes, to improve targeted interventions and increase positive healthcare outcomes for mothers and their babies.

The Singapore Management University and Khushibaby aims to improve consistency of healthcare information input by applying AI. Another project by Singapore Management University and Wildlife Conservation Trust is about predicting human-wildlife conflict in Maharashtra using AI.

While the Nanyang Technology University and Ashoka Trust for Research in Ecology and the Environment will work to improving dam and barrage water release using AI to help build early warning systems that minimise risk of disasters; AI4Bharat and IIT Madras and Storyweaver is working to support publishing of underserved Indian language content by building open-source input tools for underserved Indian languages to accelerate publishing of openly licensed content.

“Whether it is forecasting floods or detecting diabetic eye disease, we are increasingly seeing people apply AI to address big challenges. We believe that some of the biggest issues of our time can be tackled with AI,” explains Milind Tambe, director – AI for Social Good, Google Research Team in India. “This is why we have made research in AI for Social Good one of the key focus areas of Google Research India, the AI lab we started in Bengaluru last September.”

The post Google backs six artificial intelligence-based research projects – Details appeared first on Artificial Intelligence.

Data is the foundation of any machine learning model. Indeed, there are similarities between the data required for machine learning and any other data-centric project. In all kinds of projects, senior executives need to undertake proper levels of diligence to ensure that the data is reliable, consistent, and comprehensive. However, some data concerns are specific to machine learning. When engaging with data in machine learning projects, it helps to consider:

1. How much data is needed?
2. Is there potential for cross-contamination of data?
3. Is there bias in the data?
4. How is non-numeric data treated?

How Much?

While every machine learning problem is unique, and the amount of data required depends on the complexity of the exercise and the quality of the data, the answer is often “less than you think.”

While the term “machine learning” is often paired with the term “big data,” in reality, machine learning can also apply to data sets numbering in the thousands or even hundreds.

To test this, we applied common supervised machine learning algorithms in which we weighted 30 separate inputs, so as not to favor one input over another. They were then randomly selected to generate an output. A human analyst would never accurately predict an outcome on this randomly weighted data set. However, many of the machine learning algorithms predicted the outcome with greater than 90% accuracy after 4,000 observations. Big data is not necessary for machine learning to be useful.

Potential for “Cross Contamination”

When training a machine learning model, data is divided into training and testing sets. The algorithm optimizes its predictions on the training set before using the testing set to determine its accuracy. It is essential to be careful that the data in one set doesn’t contaminate the other set.

Dividing the data based on random selection can create problems if the data set has multiple observations of the same input over time. For example, supposed a retail company wanted to build a store profitability predictor using monthly observations of profitability for all locations over the last five years. Randomly splitting the data would result in both the training and testing sets including observations of the same store.

In that scenario, even if we eliminated store IDs from the data, machine learning algorithms would still be able to identify which store was which and accurately predict profitability by store. The algorithm might begin predicting profitability based on what the store ID was and not the other factors on which we were hoping to gain insight. The test vs. train results would reflect artificially high accuracy due to the cross-contamination of data.

We can void this problem by ensuring that we explicitly bifurcate the training and testing sets. In the example above, we could randomly assign the stores to the training set or testing set with no overlap between the two, as opposed to randomly assigning the monthly observations. That would result in more reliable predictions providing insights on the factors in which we are interested.

Is There Bias?

A key benefit of machine learning algorithms is that they do not apply the heuristics and biases prevalent in human decision-making. Algorithms use only the data and features provided to develop an optimal method for making predictions. The flip side is that if there is bias in the data, the algorithms won’t be able to reverse or rectify it.

That fact was famously evident when an audit of a machine learning-driven résumé screening company found that “good job candidates” were most likely to (1) be named Jared and (2) have played high school lacrosse.

Those constructing the algorithm in question probably assumed that by omitting factors such as race, gender, or background, they were creating an unbiased model. However, the data that was used still contained implicit biases (all lacrosse-playing Jareds get selected to the exclusion of other good candidates), thus resulting in an inexcusably biased output. The ratings of prior candidates’ performances were biased because they were made by people of a specific race and background, which resulted in biased outcomes from the algorithm.

In this example, the candidates’ background (factors), including their ranking (outcome), were used to predict rankings for future candidates. When asking the algorithm to predict future rankings, you should consider whether the historic rankings in the data set are biased, as they were in this case. If outcomes are based on human bias, the machine will replicate that bias in its predictions. In this example, the client requested to see the features being weighted and noticed this bias. Note that the screening company didn’t catch this, but the experience of senior executives did.

Treatment of Non-numeric Data

When developing a supervised machine learning algorithm, the data must be numerical. For quantitative measures, like revenue or profit, this poses no problems.

However, most projects require interpretation of non-numeric data, and not carefully transforming the text or labels into numeric data can lead to potential pitfalls. For instance, analysts may convert company sectors to index numbers based on alphabetical order. This approach may be easy to implement, but it could, for example, place “consumer staples” right next to “energy,” which can result in algorithms often recognizing them as being similar.

There are several ways to convert non-numeric data, such as vectorizing text — transforming text labels and their frequencies into numbers a machine can understand — or by simply being more intentional about how lists to order lists. As a stakeholder, your team should consider appropriate options and explore how each one impacts the accuracy of results.

Machine learning models are only as good as the data underlying them. Given their experience and perspective, senior stakeholders can add value to data scientist teams, especially in identifying bias and contamination. Data scientists working alongside senior executives as they consider quantity, quality, bias, and contamination of data is the best practice for the successful implementation of machine learning models.

The post Selecting and Preparing Data for Machine Learning Projects appeared first on Artificial Intelligence.

Microsoft  today announced two new open-source projects: Dapr, a portable, event-driven runtime that takes some of the complexity out of building microservices, and the Open Application Model (OAM), a specification that allows developers to define the resources their applications need to run on Kubernetes clusters and which Microsoft developed in cooperation with Alibaba Cloud.

As Microsoft Azure CTO Mark Russinovich  told me ahead of today’s launch, OAM very much solves a problem that a lot of developers and ops teams are facing every day. “If you take a look just at the Kubernetes ecosystem, Kubernetes  has no concept of an application,” he explained. “It’s got the concept of a deployment and services, but nothing that coherently connects these things together into one unit and deployment lifecycle that a developer would understand in the way they look at their applications.” He argues that while Kubernetes has Helm charts, once an application is deployed, Kubernetes doesn’t know about the relationships between the objects that were represented in that chart. “We need a first-class application concept in a Kubernetes cluster.”

OAM is essentially a YAML file. It can be put in a service catalog or marketplace and deployed from there. But what’s maybe most important, says Russinovich, is that the developer can hand off the specification to the ops team and the ops team can then deploy it without having to talk to the developer. He also argues that Kubernetes itself is too complicated for enterprise developers. “At this point, it’s really infrastructure-focused,” he said. “You want a developer to focus on the app. What we saw when we talked to Kubernetes shops, they don’t let developers near Kubernetes.”

As for the cooperation with Alibaba Cloud on this specification, Russinovich noted that the two companies were already working on other projects together and that they both encountered the same problems when they talked to their customers and internal teams. Over time, they plan to bring the specification into an open-source foundation.

Alibaba Cloud will launch a managed service based on OAM, and chances are that Microsoft will do the same over time. “We’re looking to see what adoption looks like to decide,” Russinovich said.

While OAM solves an obvious problem for developers and ops team and fills a gap, Russinovich argues that Dapr may actually be quite revolutionary. “If you take a look at Dapr, it is really going to make microservices, cloud-native development, accessible to the enterprise.”

So what is Dapr? Microsoft describes it as “open source, portable, event-driven runtime that makes it easy for developers to build resilient, microservice stateless and stateful applications that run on the cloud and edge.”

That’s a mouthful, but the general idea here is to make it easier for developers to write distributed, microservice-based applications. “If you take a look at the list of problems they run into, they want to be event-driven, so they have to manage things like events and responding to triggers,” he said. “They want communication between these microservices, so they’ve got to do pub/sub. They’ve got to do service discovery — how do I get a service from one microservice to another one. They’ve got to do state management — how does my microservice store state and retrieve it.” And then, depending on whether it’s a stateless or stateful app, developers have to work with different SDKs and programming models. Dapr, on the other hand, doesn’t need an SDK because it delivers its services through a local HTTP or gRPC endpoint, keeping the application code separate from the Dapr code. Because of this, Dapr is also independent of the language you write in.

Dapr abstracts a lot of this away and provides the building blocks (which can be accessed by HTTP or gRPC APIs) that encode best practices for building these distributed services.

The post Microsoft launches new open-source projects around Kubernetes and microservices appeared first on Artificial Intelligence.