What is it about “big data” that resists definition? Today we have myriad competing definitions that each attempt to circumscribe just what it is we mean when we talk about the idea of using “big data” to understand the world around us. The notion that the size, speed or modality of data warrants such a label falls apart when we recognize that every Google search involves analyzing a 100-petabyte archive using hundreds of query terms. Instead of referring to the size of our datasets, could “big data” refer to the way in which we utilize our data, regardless of its size?

The question of just what constitutes “big data” has become a perennial point of debate in the digital world. Typically, most definitions relate to the characteristics of the data being analyzed, but such definitions become increasingly strained when we recognize that the most mundane of internet tasks, from conducting a Google search to querying Twitter all involve processing enormous volumes of multimodal material that is growing rapidly.

Using the example of a Google search, it seems absurd to label every Web search a “big data analysis” merely because it examined 100 petabytes using hundreds of parameters.

Yet what differentiates a keyword Google search from an SQL query of a data warehouse of the sort that is routinely described as precisely such a big data analysis? Does a keyword search written as an SQL query count as big data where a keyword typed into a Web page does not? Does an SQL-computed histogram count or does it take at least a linear regression?

Does using an SQL query to count how many records there are in a ten petabyte database count as a big data analysis? What about a summation or field extraction?

Where do we draw the line between ordinary data and “big data?” Does that boundary depend on the industry in which we work? To the Google’s of the world, petabytes are passé. In the arts, humanities and social sciences, datasets of hundreds of megabytes are still often referred to in the literature as “big data” analyses and genuinely reflect in some fields datasets far larger than those ordinarily used.

Does it matter whether we are the ones storing or analyzing that data or whether it is outsourced? If an enterprise manages tens of petabytes of desktop backups in its own data centers, there are very real complexities to the management of large datasets. At the same time, today there are plenty of vendors that sell turnkey petascale storage systems complete with onsite representatives to manage and service the units. Does that count as big data management? What if a company simply ships their petabytes to the cloud and accesses them using a giant cloud-provided fileserver? Does that count as “big data” if they themselves are not actually doing any of the management?

Similarly, does it count as big data analytics if we use a point-and-click analysis tool that alleviates us of the need to write a single line of code? What about a tool like Google’s AutoML that leverages transfer learning and incredibly sophisticated model generation and tuning algorithms to quite literally allow the creation of state-of-the-art deep learning models with a few mouse clicks – no coding or AI experience necessary? Does using or deploying an AutoML model count as big data even if we didn’t have to write any code ourselves?

Perhaps the answer to what counts as “big data” lies in how we use all of that data.

Using Google to conduct a keyword search implies a human-directed task. A human being has a question, translates that question into a query, enters that query into a search box and peruses the results. Such a workflow hardly seems to justify the big data label.

What if instead, Google’s algorithms monitored the world’s information on our behalf, searching out insights and new developments it believes are of greatest relevance to us and providing us a real-time summarized digest of the top highlights most relevant to our needs at the moment.

The latter sounds far more like a “big data” application that the former, yet both involve the exact same dataset being searched.

In fact, in many ways the functional tasks each performs are the same. The difference lies in who performs that analysis – the machine or the human. When a human manually queries a dataset is it “big data” or does an analysis require some degree of creative or advanced machine assistance to be worthy of that moniker?

Using the example of an SQL query, if a human manually interrogates a dataset using simplistic queries like counting rows that match different criteria, it seems to strain credibility to call such tasks, which differ little from a keyword search under a different name, as “big data.” Alternatively, if a human interrogates that same dataset using more complex queries like applying machine learning algorithms or complex analytic models, the label would seem to more readily apply.

Putting this all together, perhaps instead of focusing on petabytes or exabytes or trillions of rows, the answer to what constitutes “big data” lies in what we do with all of that data. Simple keyword searches or SQL queries might interrogate exabytes, but it seems unreasonable to classify every Google search as a “big data analysis.” Instead, if we focus on how that data is used and in particular the use of machine creativity to analyze data proactively on our behalves or to surface patterns and trends we were not expecting or to perform complex queries on our behalves, perhaps that might yield a more satisfactory definition.

In the end, shifting our gaze from how much data we hoard to what we actually do with all of that data would go a long way towards moving the field from meaningless marketing buzzword towards genuine business insights.

The post Is ‘Big Data’ About What We Do With Our Data Not How Much Of It We Have? appeared first on Artificial Intelligence.

Human beings are an innovative species and digitalisation is proof of this. It is undeniable that the world is becoming more and more digital; today people rely on smartphones and tablets to order our shopping, research and find destinations, connect with friends, improve our health, control devices in our home, stream our entertainment and follow the developments in the world around us.

Data can be processed at a micro or a macro level to provide valuable insight around an individual or a society. Knowing this, government agencies and large enterprises are reshaping their approaches to better manage this data deluge. But today’s big data buzz is not new – for more than twenty years, big data has been evolving into the hot topic we see.

With so much data out there, organisations are feeling the pressure to implement big data technologies in order to stay ahead. However, the road ahead is riddled with speed bumps and potholes – namely, lack of budget and appropriate resources and skills, as well as legacy systems.

What’s so special about big data?

Big data provides a fuller picture. It allows organisations to understand the people they serve, choose better strategies, allocate their resources more effectively and operate smarter.

For instance, right now big data analytics is playing an important part in understanding cyber security trends. Using big data analytics, it is possible to detect vulnerabilities and identify breaches that are already happening.

Next generation technologies such as IoT are going to rapidly accelerate this by connecting larger numbers of things to the internet. With modern sensors, we are able to measure, observe and share data for an endless number of applications. By 2020 there will be three times as many devices as there are people in the world, growing the volume of data to petabytes, exabytes, and even zettabytes.

Plus, you can look at where the threats are coming from and combine it with cognitive computing to keep ‘unwanted’ people out. In this way, big data is now becoming a mission-critical asset applicable to all, and must be viewed as such.

Of course, many enterprises are also extending the use of big data to better serve their customers and to anticipate their requirements. Innovation, technical expertise and good technology leadership are seeing pioneering organisations redefine the way industries operate.

What’s holding things up?

As the old saying goes, good things take time, and widespread adoption of big data analytics is no exception.

However, a silver lining has appeared on the horizon in the form of the cloud. Cloud services allow access to big data tools and environments, bridging the gap in this modernisation push. As more enterprises adopt cloud-based solutions, they’ll have the benefits of big data analytics tools at their disposal, too.

Looking ahead, big data has limitless potential to advance businesses. The future of data analytics does not have to be daunting; instead this technology, working in sync with the cloud, is another piece of the IT modernisation puzzle.

Getting up to speed with big data

Big data offers many exciting opportunities, from increased efficiency to enhanced customer engagement, and now is the time for businesses to get involved. One challenge can be gathering the necessary skills together to equip the existing workforce with the technical knowhow needed to harness analytics and data for business benefits.

Carrying out transformation in-house will involve an investment of time, resources and money and the solution lies in finding partners to collaborate with, to become more competitive.

Partners who want to move into new technology areas can work with a distributor to help identify any skills gaps and find ways of filling these gaps, with that distributor’s own IT experts, with other partners, or through technical IT training and certification to develop their own in-house skills.

Once these gaps have been identified, partners can gain access to a comprehensive framework, like a Practice Builder, which provides enterprises with a clear approach to new technology areas, via workshops, technical and sales training and marketing programmes.

Getting access to the right technology is also critical; a good distributor will have contracts and established relationships in place with a wide range of larger, as well as born in the cloud, vendors, saving you time and set-up costs.

Digital transformation is not an IT decision, but a business decision, so having the skilled people who can explain to a board of directors the business reasons for investing in changing their business is important. A good distributor will support you to develop a credible, logical and persuasive narrative.

Digital transformation and beyond

The transformation process is a journey. If you embark on this path, your customers will look to you to guide them as the technology evolves and as their business evolves. Rather than being seen as a provider of IT, you become a business innovator and someone who can help them gain a competitive advantage. That is how you become a valued trusted advisor to your customers.

The post What are the real opportunities for big data in the digital world? appeared first on Artificial Intelligence.

Defined as the massive volume of structured and unstructured data that can’t possibly be processed using traditional software or database strategies, Big Data is affecting every corner of the business world. It’s no surprise, really, seeing that more data has been created in the past two years than in the entire history of the human race. By 2020, roughly 1.7 megabytes of new information will be created for every second for every human being and, at that point, the digital universe will be 44 zettabytes strong (up from a current 4.4 zettabytes).

As supply chain managers scramble to wrap their arms around the reams of information now at their fingertips, a growing number of software providers are making the task more manageable and useful. In other words, simply having the data at your avail isn’t enough; it’s about taking that information and transforming it into actionable insights that help drive operational efficiencies across the supply chain.

“Supply chains are more complex than ever, and with these complexities come many challenges,” says Shannon Vaillancourt, president at RateLinx. “Big Data allows companies to diagnose the issue so they truly understand what is causing it.” Of course, capturing the data and then using it to make good decisions are two entirely different things. To help fill that “gap,” Vaillancourt says software developers are focusing on the 5 Vs of Big Data: variety, velocity, veracity, volume and value.

Vaillancourt says the final “v” is extremely important and often overlooked. “Companies need to be looking for software that turns all of their data into value—or, actionable,” he points out. “Actionable data is created through analytics; it’s the analytics that tells the user what to do, and ultimately what action to take.”

Top 20 Supply Chain Managment Sofware Suppliers

SCM (SCE, SCP, Procurement) Total Software Revenue

Following are six big ways that Big Data is affecting the supply chain and helping companies take the right actions.

1. Get better diagnostic information.

To solve problems and circumvent future challenges, companies need good diagnostic data. Big Datagives them that, according to Vaillancourt, while also ensuring that their future strategies are based on solid historical information. “Big Data can help companies diagnose many issues, which will in turn allow them to develop strategies to solve the issues,” he says, “and then ultimately deploy the strategies successfully.”

For example, the organization that wants to leverage Big Data for track and trace of its products can do so by combining the purchase order (PO) details, shipment information and the carrier’s tracking information. Then, once that data is standardized and cleansed, analytics can be applied to it in a way that truly makes the information actionable. “If the analytics notifies the user about a late shipment before the carrier issues the notification,” Vaillancourt explains, “then that user can enact a contingency plan and get the product faster from an alternate source.”

2. Get a clearer “crystal ball” for the future.

Defined as the data mining, statistics, modeling, machine learning, and artificial intelligence used to analyze current data to make predictions about the future, predictive analytics is the modern-day supply chain manager’s crystal ball. “Predictive analytics makes it possible to analyze data and create assumptions as to what will happen to not only predict the future, but influence it as well,” says Marcell Vollmer, chief digital officer at SAP Ariba.

In Kansas City, for instance, a local police department is using data to stop crime before it happens by identifying “hot spots,” patrolling those areas more aggressively and then more closely monitoring the activities of recent parolees. In the business world, predictive analytics is allowing firms to more clearly understand customer needs and adapt their business to accommodate them. Take pricing, for example. Using predictive analytics, companies can predict equilibrium before releasing a new product, thus maximizing the revenue of the solution out of the gate while also understanding future demand. “Data is the new currency,” Vollmer adds, “and predictive analytics is the key to collecting the dividends it pays.”

3. Manage external factors that are beyond your control.

External factors can have a substantial impact on supply chains, yet in many cases these outside forces are hard to control and even detect. “From weather to oil prices to consumer demand, supply chain executives who can quantify and anticipate such impact can better plan their materials and inventory,” says Rich Wagner, CEO at Prevedere. He says retailers are particularly well positioned to leverage this advantage, namely because they’re operating in a dynamic environment where consumers expect quick, accurate deliveries. “If a product is unavailable, manufacturers and retailers alike risk not only losing a customer forever, but also a digital media backlash,” Wagner points out. How can Big Data help? By helping firms better predict demand, and therefore better plan their inventory to mitigate against shortages. The same benefits apply on a global scale, where both supply chains and operations are becoming more interconnected and, subsequently, more impacted by world events. “By coupling Big Data with predictive analytics,” Wagner says, “it’s quite possible to keep a handle on numerous economic and consumer behavior metrics to be better prepared for what’s coming next.”

4. Make more profitable supply chain demand forecasts.

Access to global data, combined with the power of Cloud computing, is giving technology more power to tackle even the toughest supply chain challenges. “With today’s advancements in machine learning, companies can use technology to constantly monitor those external forces,” says Wagner, “and get a real-time view of what’s ahead.” He sees this as a fundamental change in demand planning—compared to traditional forecasts built on past performance with the assumption of stable economic conditions. “Executives know that they can’t rely on precedence and they need insights to make decisions about the future with certainty,” says Wagner. “This desire to be immediately notified of shifts in momentum is now a reality.” For example, one global beverage manufacturer saved about $9 million by improving product distribution through predictive demand models. “The manufacturer realized that external factors (e.g., the architectural billings index) were leading indicators of performance,” says Wagner, “so it adapted its supply chain planning across 400 brands and 21 distributors.”

5. Reduce demand variability and cycle times.

Big Data is turning supply chain managers into “mind readers,” allowing them to predict and react to buyer behaviors in new ways. On the demand side, for instance, Big Data helps companies gain better understanding over consumer behaviors, foot traffic, buyer preferences and the actions that their competitors are taking. “This gives companies a solid offensive footing,” says Dennis Groseclose, president and CEO at TransVoyant, “and allows them to fuse external data and demand patterns to more effectively reduce demand variability.” Having actionable data also helps companies better manage lead times, variability and capacity. This, in turn, helps them better understand manufacturer and carrier behaviors. “With this information in hand, companies can squish planning cycle times down to one month vs. five months,” says Groseclose, “or to one week vs. five weeks.”

6. Prepare for the “SNEW” wave.

Here’s a buzzword you may not have heard of yet: SNEW, or social media, news, event and weather data, is the next acronym that’s either going to make supply chain managers sit up and take notice, or make them roll their eyes and groan. Either way, SNEW data promises to help improve supply chain capabilities and give companies even more data to sift through, digest and make sense of. An existing forecast, for example, can be adjusted accordingly when accurate weather predictions are factored into the equation.

Driven by the Internet of Things (IoT), SNEW uses a “combination of data feeds to determine the best routing, risk management, and other supply chain decisions,” according to Steve Banker, vice president, supply chain management at ARC Advisory Services, who sees SNEW as a potential player in the future of supply chain visibility and risk avoidance (or mitigation).

“This is a new solution to the market, and it’s being driven by the emergence of new technological capabilities,” Banker notes. The integration of social media, news, event and weather data into the manufacturing and distribution process is also getting a boost from the ongoing digitization of the supply chain.

“What we’re looking at is a series of technologies that are either rapidly emerging or already further along in terms of emergence,” says Banker, noting that while IoT is a bit further along in terms of maturity, concepts like SNEW and blockchain (i.e., a digital ledger where transactions made in bitcoin are recorded chronologically and publicly) are still nascent. “Over time,” he concludes, “these innovations will continue to generate Big Data that companies will use for decision making.”

The post 6 Ways Big Data is enhancing the global supply chain appeared first on Artificial Intelligence.