Demand for data science professionals is growing, and educational institutions such as the Nanyang Technology University (NTU) have started offering interdisciplinary offerings that incorporate data science with topics such as economics.

But can good data scientists be churned out by simply putting them through a curriculum or having them sit through some certification?

In a contributed opinion piece to The Business Times, David Hardoon, a senior advisor in data and artificial intelligence for UnionBank argues that the diversification and complexity of data science pose a real challenge to data science education.

Data science is a complex field

“[Data science] is a hybrid of statistics, machine learning, and data mining, and covers programming languages, technology frameworks, development platforms, as well as visualization tools,” writes Hardoon.

“How does one balance the multiplicity of areas, covering them with sufficient depth in order to produce a well-rounded data scientist who can provide impactful value to a future employer?”

Indeed, in an article earlier this year titled “Placing ‘Practice’ at the Center of Data Science Education”, Eric Kolaczyk, Haviland Wright, and Masanao Yajima described the downsides of typical postsecondary training approaches in core data science fields.

Practice is treated as something that is done only to capstone projects or as a finale, they wrote, and this results in “students, upon exiting academia, needing a nontrivial ramping-up period before they can truly have an impact with their first employers.”

Having helmed or served in advisor roles to a diverse range of organizations in Singapore such as the Corrupt Investigation Practices Bureau (CPIB), the Central Provident Fund (CPF), and the Monetary Authority of Singapore (MAS), Hardoon notes that this feedback is common and goes beyond the degree itself and into the practical applications of the coursework.

“The majority of data scientists, upon completion of their education, still require a hands-on education at their place of employment to refine their learned skills to practical necessities… a freshly minted, well-rounded data scientist is unlikely to hit the ground running at their first employ despite a presumption otherwise,” he wrote.

A call for change

To help data scientists close this gap and produce effective work quickly upon completion of training, organizations might want to consider operationalizing data science by identifying key data science roles within the organization, suggests Hardoon.

One solution is to segregate between key components of data science operations, with the creation of roles. Some examples: Data science developer (Focus on data science methodologies and techniques), data science engineer (Focus on data piping, data quality, and data ingestion), data science solution architect (Deep understanding of platforms and data enterprise architecture), and data science storyteller (Focus on data visualization with strong business acumen).

Alternatively, existing data science programs can be tailored to offer greater depth with a focus on niche areas or professional specialization to better reflect the maturity of the data science field today, Hardoon notes.

Don’t expect this to happen tomorrow, though. While we do see data science moving in the direction suggested by Hardoon, these changes will take time yet to fully materialize.

In the meantime, employers may just have to accept that a newly minted data scientist is unlikely to produce valuable, actionable insights from day one – or even the first few months. To help a new data science hire get started quicker, additional on-the-ground training is essential.

The post When a Data Science Education is Not Enough appeared first on Artificial Intelligence.

SAMEER S SOMALPABLO A RUZ SALMONES

BIG DATA has been on the tip of everyone’s tongue for the past several years now, and for good reason. As digital devices and touchpoints proliferate, so too does the amount of data we each create. This information can be used to help us better understand clients and customers, make more effective decisions, and improve our business operations – but only if we can make sense of it all.

By choosing the right Big Data sources and applications, we can put our organisations at a competitive advantage. But to do that, we need to understand Big Data’s definition, capabilities, and implications.

Big Data already has widespread applications. From Netflix recommendations to healthcare monitoring, it drives all types of predictive models that improve our daily lives. But the more we depend on it, the more we need to question how it shapes our lives and whether we should be relying on it so much.

While progress is inevitable and something to embrace, Big Data’s contribution should not be measured by how many companies apply it, but by how much better off it makes society as a whole.

Big data is more than just large datasets. It is defined by the three Vs of data management:

• Volume: Big data is often measured in terabytes.
• Variety: It can contain structurally different datasets, such as text, images, audio, and so on.
• Velocity: Big data must be processed quickly because of the increasing speed at which data is generated.

As the volume, variety, and velocity of data expand, they morph into Big Data and become too much for humans to handle without assistance. So we leverage artificial intelligence (AI) and machine learning to help parse it. While the terms Big Data and AI are often used interchangeably and the two go hand-in-hand, they are, in fact, distinct.

Simply put, Big Data powers AI with the fuel it needs to drive automation. But there are risks.

Realising Big Data’s business potential

Properly applied, Big Data helps companies make more informed – and therefore better – business decisions.

Melvin Greer, Intel’s chief data scientist says in an article: “A few examples include the hyper-personalisation of a retail experience, location sensors that help companies route shipments for greater efficiencies, more accurate and effective fraud detection, and even wearable technologies that provide detailed information about how workers are moving, lifting or their location to reduce injuries and increase safety.”

But this crucial competitive advantage is underused because so many companies struggle to sift through all the data and distinguish the signal from the noise.

Five principal challenges keep companies from realising big data’s full potential, according to Mr Greer:

• Resources: Not only are data scientists in short supply, the current pool also lacks diversity.
• Data aggregation: Data is constantly being created and it is a challenge to collect and sort it from all the disparate channels.
• Erroneous or missing data: Not all data is good or complete. Data scientists need to know how to separate the misleading from the accurate.
• Unfinished data: Cleaning data is time-consuming and can slow down processing. AI can help manage this.
• Truth seekers: We should not assume data analysis will yield a definitive answer. “Data science leads to the probability that something is correct,” Mr Greer writes. “It’s a subtle but important nuance.”

Addressing the first challenge is of paramount importance. The only way to solve the other issues is to first create the necessary human capital and provide them with the necessary tools.

The true promise of Big Data

Data is a wonderful instrument, but it is not a cure-all. Indeed, “too much of a good thing” is a real phenomenon.

Jacqueline Nolis, Saturn Cloud head of data science, writes in an article: “In my years working with many businesses, I have indeed seen some companies that fell into the situation of not using data enough. However, these occurrences paled in comparison to the number of times I have seen the reverse issue: companies with an over-reliance on data to the point that it was detrimental. The idea that data is needed to make a good decision is a destructive one.”

To illustrate her point, Ms Nolis describes Coca-Cola’s introduction of Cherry Sprite. What motivated the decision? Data. People were adding cherry-flavoured “shots” to Sprite at self-service soda dispensers. So score one for Big Data.

But as Ms Nolis points out, the very similar-tasting Cherry 7UP already existed – and had since the 1980s. So the data team might have come up with the new flavour more efficiently simply by perusing the soft drink aisle at the local grocery store. The lesson: Too heavy a reliance on data can be a barrier to common sense decision making.

Big Data applications: When and how

So how do we know when to put big data to work for our business? That decision needs to be made on a case-by-case basis according to the demands of each individual project.

The following guidelines can help determine whether it is the right course:

• Consider the desired outcome. If it’s to catch up with a competitor, investing in something the competitor has already done may not be a good use of resources. It might be better to let their example serve as guidance or inspiration and reserve Big Data analysis for more complicated projects.
• If disruption is the goal, Big Data can be applied to test new ideas and hypotheses and maybe reveal other possibilities. But we need to beware of the downsides: Data can kill creativity.
• If a business decision is urgent, the “data is still being analysed” is not an excuse to delay it.

Amid a public relations crisis, for example, we won’t have the time to mine the available data for insights or guidance.

We have to rely on our existing knowledge of the crisis and our customers and take immediate action.

Of course, sometimes Big Data is not just useful but essential.

Some scenarios call for Big Data applications:

• To determine if a strategy is working as planned, only the data will tell the story. But before we measure whether success has been achieved, we first have to establish our metrics and define the business rules that determine what success looks like.
• Mining Big Data may uncover sales or marketing performance anomalies that would not be otherwise diagnosable. Similarly, AI can help improve energy efficiency and offer insights into customer and employee behaviours.
• Big Data can help process and create models out of vast amounts of information. So as a general rule, the larger and more data-intense the project, the greater the likelihood Big Data could be helpful.

Big Data might be the trendy topic in technology today, but it is more than a buzzword.

Its potential to improve our businesses and our lives over the long term is real.

But that potential needs to be leveraged purposefully and in a targeted fashion. Big Data is not the business equivalent of a wonder drug.

We need to be mindful of where its applications can help and where they are superfluous or harmful.

Indeed, the full promise of Big Data can only be realised when it is guided by thoughtful human expertise.

• Sameer S Somal, CFA, is the CEO and cofounder of Blue Ocean Global Technology. Pablo A. Ruz Salmones is the co-founder and CEO of Grupo Ya Quedó, a software development and artificial intelligence (AI) company.

The post When to use Big Data and when not to: it should be guided by thoughtful human expertise appeared first on Artificial Intelligence.