Machine learning is compute-intensive

Advances in innovation to capture and process a lot of data have left us suffocating in information. This makes it hard to extricate insights from data at the rate we get it. This is the place where machine learning offers some benefit to a digital business.

We need strategies to improve machine learning performance all the more effectively. Since, supposing that we put forth efforts in the wrong direction, we can’t get a lot of progress and burn through a lot of time. Then, we need to get a few expectations toward the path we picked, for instance, how much precision can be improved.

Articulate the issue

There are by and large two kinds of organizations that participate in machine learning: those that build applications with a trained ML model inside as their core business proposition and those that apply ML to upgrade existing business work processes. In the latter case, articulating the issue will be the underlying challenge. Diminishing the expense or increasing income should be limited to the moment that it gets solvable by gaining the right data.

For example, if you need to minimize the churn rate, data may assist you with detecting clients with a high “fly risk” by analyzing their activities on a website, a SaaS application, or even online media. In spite of the fact that you can depend on traditional metrics and make suppositions, the algorithm may unwind shrouded dependencies between the data in clients’ profiles and the probability to leave.

Resource Management

Resource management has become a significant part of a data scientist’s duties. For instance, it is a challenge having a GPU worker on-prem for a group of five data scientists. A lot of time is spent sorting out some way to share those GPU’s simply and effectively. Allocation of compute resources for machine learning can be a major agony, and takes time away from doing data science tasks.

Focus on Quality of Data

Data science is an expansive field of practices pointed toward removing significant insights from data in any structure. Furthermore, utilizing data science in decision-making is a better method to stay away from bias. Nonetheless, that might be trickier than you might suspect. Indeed, even Google has as of late fallen into a trap of indicating more esteemed jobs to men in their ads than to women. Clearly, it isn’t so much that Google data scientists are sexist, but instead the data that the algorithm utilizes is one-sided on the grounds that it was gathered from our interactions on the web.

Embrace Hybrid Cloud

Machine learning is compute-intensive. A scalable machine learning foundation should be compute agnostic. Joining public clouds, private clouds, and on-premise resources offers flexibility and agility as far as running AI workloads. Since the kinds of workloads shift significantly between AI workloads, companies that construct a hybrid cloud infrastructure can dispense assets all the more deftly in custom sizes. You can bring down CapEx expenditure with public cloud, and offer the scalability required for times of high compute demands. In companies with strict security demands, the expansion of private cloud is essential, and can bring down OpEx over the long-term. Hybrid cloud encourages you to accomplish the control and flexibility necessary to improve planning of resources.

Be prepared to Iterate

A large portion of the models are created on a static subset of information, and they capture the conditions of the time frame when the data was gathered. When you have a model or various them deployed, they become dated over time and give less exact expectations. Contingent upon how effectively the patterns in your business climate change, you should pretty much regularly replace models or retrain them

The post IMPROVE MACHINE LEARNING PERFORMANCE WITH THESE 5 STRATEGIES appeared first on Artificial Intelligence.

Australia’s data centres are concentrated in Sydney, Melbourne and Brisbane, but that is starting to change as storage and cloud providers and their customers see the advantages of situating them in regional Australia.
For example, Australian-owned cloud, data centre and connectivity provider iseek is set to open a data centre in Townsville shortly.

Founder and managing director Jason Gomersall says the decision to build in Townsville was prompted by a number of local organisations wanting to have a data centre nearby.

“In theory it doesn’t matter where it is but we’re finding government organisations in particular, and also corporates, are starting to care more and more about where their data is hosted and who it’s hosted with,” Gomersall says. “It’s just the security of knowing where it is and who’s got it.”
Regional data centres can also boost the resilience of Australia’s data by boosting the geographic area over which it is spread, he says.

“The internet as such was designed to distribute information globally and does that very efficiently and effectively. Then ironically we go and then concentrate all our data in one geographic location. It seems a little bit counterintuitive to me.

“From a policy perspective, I think the government should be looking at how they sort of spread the data centre load around the nation.”

Developing skills

Gomersall says the cost of connecting regional data centres to capital cities had been an impediment in the past, but this is now being addressed with better infrastructure.

Regional data centres can also play a significant role in local job creation and keeping and developing IT skills in the regions.

In the short term the Townsville data centre will create a handful of jobs but iseek is already starting to consider how it will expand that workforce. Data centres also create indirect jobs, such as the cloud services and hosted services within them.

“If we take a five to 10-year view on this – and when you build a data centre you’re taking a 20 year-plus view – I see significant job creation,” Gomersall says.

“We’re going to create the skills and do the training to build those skills in those areas.”

Joe Craparotta, vice-president secure power at Schneider Electric, says improvements in Australia’s telecommunications infrastructure outside the major carriers has made regional data centres more feasible while the rising cost of land in capital cities has made them more desirable.

Another advantage of housing them in cooler climates such as Toowoomba, where the $40 million Pulse Data Centre supported by Schneider Electric, Telstra and the Queensland government opened last year, is they draw on less power to keep cool.

“The running cost of the data centre can be a lot more efficient and a lot more optimised than a metro data centre,” Craparotta says. “There’s almost free cooling for a big chunk of the year.”

Close to power source

Additionally, they can often have access to good power, being close to the source of generation.

Craparotta says keeping IT skills in the regions will become more important as regional Australia becomes more digital and the use of the internet of things increases, creating further benefits for local industry.

“They have a facility locally they can rely on to help them digitise, either their farm – from paddock to plate – or the local university or school, or just the local businesses in general.”

While he is not suggesting that regional data centres will replace metro data centres, he does expect Australia will have a mix of both.

“I would expect it to be aligned with the population, so I would expect that there’s more regional data centres that are built over the next five years to accommodate the third of the population that is outside metro area,” Craparotta says.

“I’d predict in the next two to three years, we will see a much faster acceleration as the 35 per cent of Australia that lives outside the metro areas becomes more digital and more reliant on those services.”

The post Big data centres on regional areas appeared first on Artificial Intelligence.

The post 6 Design Principles for Artificial Intelligence in Digital Business appeared first on Artificial Intelligence.

Enterprises are adopting data science pipelines for artificial intelligence, machine learning and plain old statistics. A data science pipeline — a sequence of actions for processing data — will help companies be more competitive in a digital, fast-moving economy.

Before CIOs take this approach, however, it’s important to consider some of the key differences between data science development workflows and traditional application developmentworkflows.

Data science development pipelines used for building predictive and data science models are inherently experimental and don’t always pan out in the same way as other software development processes, such as Agile and DevOps. Because data science models break and lose accuracy in different ways than traditional IT apps do, a data science pipeline needs to be scrutinized to assure the model reflects what the business is hoping to achieve.

At the recent Rev Data Science Leaders Summit in San Francisco, leading experts explored some of these important distinctions, and elaborated on ways that IT leaders can responsibly implement a data science pipeline. Most significantly, data science development pipelines need accountability, transparency and auditability. In addition, CIOs need to implement mechanisms for addressing the degradation of a model over time, or “model drift.” Having the right teams in place in the data science pipeline is also critical: Data science generalists work best in the early stages, while specialists add value to more mature data science processes.

The post Building a data science pipeline: Benefits, cautions appeared first on Artificial Intelligence.

Artificial intelligence (AI) everywhere, transparently immersive experiences and digital platforms are the trends that will provide unrivaled intelligence, create profoundly new experiences and offer platforms that allow organizations to connect with new business ecosystems, according Gartner’s Hype Cycle for Emerging Technologies 2017.

“In addition to the potential impact on businesses, these trends provide a significant opportunity for enterprise architecture leaders to help senior business and IT leaders respond to digital business opportunities and threats by creating signature-ready actionable and diagnostic deliverables that guide investment decisions,” said Mike J. Walker, research director at Gartner.

Artificial intelligence technologies will be the most disruptive class of technologies over the next 10 years due to radical computational power, near-endless amounts of data, and unprecedented advances in deep neural networks; these will enable organizations with AI technologies to harness data in order to adapt to new situations and solve problems that no one has ever encountered previously.

Enterprises that are seeking leverage in this theme should consider the following technologies Deep Learning, Deep Reinforcement Learning, Artificial General Intelligence, Autonomous Vehicles, Cognitive Computing, Commercial UAVs (Drones), Conversational User Interfaces, Enterprise Taxonomy and Ontology Management, Machine Learning, Smart Dust, Smart Robots and Smart Workspace.

Also, technology will continue to become more human-centric to the point where it will introduce transparency between people, businesses and things. This relationship will become much more entwined as the evolution of technology becomes more adaptive, contextual and fluid within the workplace, at home, and in interacting with businesses and other people.

Critical technologies to be considered include 4D Printing, Augmented Reality (AR), Computer-Brain Interface, Connected Home, Human Augmentation, Nanotube Electronics, Virtual Reality (VR) and Volumetric Displays.

Further, emerging technologies require revolutionizing the enabling foundations that provide the volume of data needed, advanced compute power, and ubiquity-enabling ecosystems. The shift from compartmentalized technical infrastructure to ecosystem-enabling platforms is laying the foundations for entirely new business models that are forming the bridge between humans and technology.

Key platform-enabling technologies to track include 5G, Digital Twin, Edge Computing, Blockchain, IoT Platform, Neuromorphic Hardware, Quantum Computing, Serverless PaaS and Software-Defined Security.

The post 3 megatrends for digital business into the next decade appeared first on Artificial Intelligence.

When it comes to cybersecurity, the big data explosion represents both a liability and an asset. On the one hand, big collections of data represent a treasure trove that hackers would love to get their dirty little mitts on. But on the other hand, the capability to collect, store, and analyze huge reams of data gives the good guys a powerful tool to thwart the bad guys.

These are two ends of the same stick. If you’re building a data lake atop Hadoop, Amazon S3, MongoDB or any other big data platform, you sure as heck better be securing that data. After all, if that data is valuable to you – which is hopefully why you’re collecting it in the first place – then it will likely have value to some digital bottom dweller who steals for a living.

There are various open source and commercial tools designed to help you lock down access to that data. But there’s another element of big data security, and it involves using advanced analytics to better detect when bad guys (or bad software) are trying to do us harm.

Here’s how the good guys are using big data analytics to go after the black hats of the world.

Machine Learning De Rigueur

Machine learning techniques have been reducing spam in our email inboxes for years. Now that same technology is being applied by leaders in the cybersecurity world, including McAfee, which started using it widely with a major update to its Internet security software last fall.

“I do think it’s important to know that we have been shipping machine learning in our product at scale for many, many months,” says McAfee CTO Steve Grobman. “We did a massive launch last year in late October, November that was really retooling our detection technology to take full advantage of machine learning capabilities.”

That ML capability extends from the data lakes that McAfee uses to store examples of malware and infiltration techniques, all the way out to the millions of end-points that it protects.

“Our consumer product line, as well as our most current enterprise product, both have these technologies built into them,” Grobman tells Datanami. “We’re able to classify a much larger set of malicious scenarios using non-deterministic machine learning capabilities to find either new forms of malware or threat scenarios as compared to traditional techniques.”

Speed to Detection

While established security firms like McAfee play large roles in cybersecurity, organizations are also turning to big data analytic software companies to get a leg up on the detection of bad stuff going on in their networks.

One of those analytics firms seeing increased uptake for security use cases is Pentaho. Chuck Yarbrough, a vice president with the Hitachi subsidiary, says he’s seeing an uptick in customers using its big data software to bolster their company’s security.

“The reality is, you’re going to get intrusions,” Yarbrough says. “There are tools to help prevent intrusions….But oftentimes the challenge is to recognize that you’ve had an intrusion, and being able to figure that out as quickly as possible.”

On average it takes organizations 205 days to discover that they’ve had an intrusion into their internal systems, Yarbrough says. That means these ticks get nearly seven months to poke their nasty little heads into organizations’ digital crevices before the host figures out that something’s amiss. That’s just too long.

“The amount of time it takes to identify a problem is crazy,” Yarbrough says. “How can we bring that number down? Ultimately we want to get it down to hours or minutes, and part of that is blending multiple data sets together.”

The typical security engagement with Pentaho starts small with just a few data sources and analysts prepping the data. Then the customers will combine data sets, such as bringing employee social data or badge swipe data, to bear against log files recording network activity.

Once the data is landed, prepped, and mixed, the data scientists use machine learning algorithms, usually in R or Python, to discover anomalies that could indicate a breach. The Pentaho software helps to operationalize this entire data pipeline and run it repeatedly on Hadoop or other big data platforms.

“There are a lot of security systems out there that do parsing of data all the time, and they’re really good at it,” Yarbrough says. “But now we can parse that data in an appropriate way and blend in additional data sets to help it add that context. I build the data model on the fly so that either the data scientists a security analyst can do some level of interactive analytics against that to find the cause, or the potential problems, and then be able to tack action.”

Context Is Key

Finding context in the data is critical to stopping cybercrimes, such as fraud, says Poornima Ramaswamy, vice president of business analytics and insights at Cognizant Digital Business, a technology consultancy.

“We are basically trying to combine human science and data science to be able to create analytical solutions that are more human-based, deeply contextual, and can actually truly address the problems that we face,” Ramaswamy says.

For example, fraud conducted in-store is much different than fraud conduced over the Internet. In engagements with big box retailers, Cognizant collects data about the movement of individuals within the store, and combines that with other data to come up with a solution.

“We overlay the contextual data…on top of big data to look for patterns of human behavior,”Ramaswamy tells Datanami. “With this approach we can apply the contextual data on top of the data to get more human-based segmentation and then your analytics and insights are a lot more realistic.

A recent Cognizant project involved researching elements around fraud, and included interviews with 11 actual fraudsters to get more insight into their mindsets. What the research discovered is that cyber thieves don’t like to take unnecessary risks.

“The biggest characteristics for fraud are speed and liquidity and efficiency,” she says. “There’s very little [research] in looking at that end-to-end chain and looking at it as an opportunity to make it less attractive, where you’re not going to be able to liquidate the assets.”

Bad Guys Using It, Too

Big data analytics is helping the good guys, but the cybercriminals are getting hip to the techniques – and some are starting to use it themselves.

According to McAfee’s Grobman, hackers are starting to use machine learning “poisoning” techniques that basically involve throwing a lot of white noise at the good guys’ data receptors with the goal of confusing the model – and thereby throwing the good guys off their trail.

“They’re making it very difficult to tease out the signal within a very noisy set of data,” he says. “An attacker can craft things that look like an attack but are actually benign in order to intentionally create false positives that become very expensive things for the company to deal with, and are forced to lower the sensitivity.”

You can expect these evasion tactics to become more common in the years to come, as security professionals increasingly rely on machine learning and artificial intelligence to automate rote tasks.

“It’s absolutely something that’s beginning to be analyzed,” Grobman says. “We’re just now hitting the saturation point where there’s enough AI and ML in the industry for the counter measures” to work.

McAfee has also observed bad actors using big data technology and techniques to make their attacks more effective and produce a higher return on investment.

“If you look at what are some of these algorithms and technology are good at, one of the things is classification,” he says. “One of the things an attacker can do is evaluate many potential victims, then use a machine learning classifier to choose victims who have a high probability of ease of breach, or they have a high probability to extract high value data.”

Think of it as Netflix for digital ne’er-do-wells. “Just as Netflix wants to suggest to you the movie you’d like the most, an attacker wants to not waste their time on victims that are difficult to break or unlikely to yield high value,” Grobman says.

The rapid evolution of digital technology has benefitted people in many ways. The incredible power of personalization has resulted in consumers now expecting what Forrester analyst Mike Gualteiri adeptly calls “the celebrity experience.” And thanks to machine learning, we’re able to spot bad guys lurking in our networks faster than before.

The post Hacker Hunting: Combatting Cybercrooks with Big Data appeared first on Artificial Intelligence.