In the digital age, disruption is not an exception, but a norm. Everything already is, or is becoming digital. Data is everything and everywhere – how we communicate, learn, spend, work, and even elect governments.

The digital playing field will eventually even out as more businesses unlock newer benefits of digital business models and processes. As we enter the post-digital age, the game changer will be a new technology stack– DARQ.

The DARQ Future

While the SMAC (social, mobile, analytics and cloud) convergence was the foundation of the digital age, the post-digital age is DARQ. The next set of massive disruption changes comprises of Distributed ledger technology (DLT), Artificial intelligence (AI), Extended reality (XR), and Quantum computing (QC).

Together, DARQ promises to drive incredible opportunities to reimagine the human future – The way we play, interact and work could (and in all probability would), be massively different in the next decade or so. “Scotty, beam me up”, could be a (virtual) reality very soon.

DLT for trust

DLT is the backbone for technologies like cryptocurrency and blockchain. Its utility lies in making data validation and non-repudiation redundant. Since information is locked in an implicit chain of trust, DLT has significant applications. In a hyper-connected world (think smart cities, self-driven cars, smart power grids and buildings etc.) trust in the virtual world is key. Authentication, Authorization, and Non-repudiation become centre stage as people, devices, and machines interact with each other in a digital socio-economic tango.

DLT offers greater agency in information verification and control, data and privacy. On June 18,2019, Facebook announced its foray into DLT with the Libra Blockchain, a cryptographically authenticated database designed to support its cryptocurrency, Libra coin. With it, you can buy or cash out online or at local exchange points like grocery stores, and spend it using interoperable third-party wallet apps or Facebook’s own wallet to be built into WhatsApp, Messenger and its own app.

That’s one blip on the radar: today, DLT has embedded itself in sectors like healthcare, supply chain and logistics amongst others and is cementing trust into a variety of transactions.

Intelligence gain

Humanity has progressed far in its short time on the planet. Driven by a global network of goods and services, fuelled by Absolute Advantage (The Wealth of Nations, Adam Smith), we have specialized into our areas of strength. Strong advances in computing & storage power have enabled machines to learn these “specialized but narrow” skills and start to merge with us in our daily lives.

Artificial intelligence (AI) and its constellation of technologies are front and centre in producing technology-driven business and customer interactions more efficiently, faster, and at lower costs. Across sectors, AI is optimizing processes to improve decision-making from smart cars, diagnosing illnesses, participating in debate, writing realistic text, to running presidential campaigns.

AI and the Internet of Medical Things (IoMT) are already showing encouraging consumer health applications in real-time patient health monitoring, diagnostics, and emergency response systems. US caregiver, Symphony Post-Acute Network integrated machine learning (ML) with a cloud-based AI engine to predict and improve care for 80,000 patients. In Retail, consumers could see ML use purchase data to auto-fill shopping lists, and even perhaps, place orders based on purchase data and shelf-life.

Multiple realities
Faster and more intelligent computers, cheaper storage, and hyper-speed networks are enabling Augmented reality to move beyond aircraft simulators into everyday lives. Be it gaming, tele-medicine, or virtual meetings – XR is beginning to become omni-present and could disrupt the way we work and play. Would you still fly to Hawaii if XR replicated the same experience down to the last sensory experience in your living room?

In a recent event, Microsoft scanned Julia White, CVP Azure Marketing, and transformed her into an exact hologram replica. Both Julia and her virtual clone then appeared together on stage for the keynote. Microsoft used its Azure AI technologies and neural text-to-speech and the future is rapidly becoming our reality now. The challenge is to bring these capabilities to life in useful, relevant and progressive ways – to make life better, to help us all achieve more. Obviously, Extended reality (XR) promises to be a game-changer.

Together with AI, XR makes for serious business. They have already identified potential applications of XR—like virtual training, “hands-on” education, or even immersive online shopping. Currently, almost 50% of XR is currently used in higher education. In medicine, students rehearse surgeries in VR as a part of their training, while several enterprises have internal training components made immersive by XR. Aside from learning and education, XR is progressively being used in design, architecture and engineering. Similarly, automotive engineers use VR suites to sculpt concept cars to bring down real-world prototype costs.

For XR to shape and augment the world as we know it, businesses must invest in network connectivity, application development, and most importantly, processing power.

Quantum leap

When it comes to processing, Quantum computing (QC) is a holy grail in solving the toughest computational problems. Google says its colossal D-Wave 2X quantum computing machine has been figuring out algorithms at 100,000,000 times the speed of a traditional computer chip can.

Yet, Quantum computer is more than just its processor power, and is furthest from maturity. Nevertheless, it’s accelerating as companies begin exploring its potential. Several leaders in QC have released platforms that enable connections to their quantum computers via the cloud. Last year, Aliyun, Chinese giant Alibaba’s cloud service subsidiary, and the Chinese Academy of Sciences launched a joint 11-qubit quantum computing service encouraging researchers to conduct quantum experiments.

QC is necessary to power next- generation technologies, yet it is the costliest piece of the DARQ puzzle. All that computing power is of little use data back-up isn’t possible; quantum bits (qubits) take up vast storage spaces physically. To solve this, scientists are now looking at the natural world for the solution: as a 3D storage system, DNA offers an extra dimension in storing the vast quanta of data.

Quantum computers have the potential for pathbreaking applications in material design, logistics, AI and Machine learning, encryption, manufacturing, finance and even energy to name a few. Recently, Google announced ‘quantum supremacy’. Sycamore, Google’s quantum computer, calculated a task in 3 minutes, 20 seconds which otherwise, the world’s best supercomputer, Summit, would have taken 10,000 years to compute.

There’s still some time for QC to turn into an ROI-focused technology market and involves industrial-academic collaboration and public-private partnerships compelled to improve both digital and physical infrastructure.

What’s bright with DARQ

Enterprises must realise that only a strong digital foundation will help them pilot DARQ technologies to their advantages.

At the core of DARQ is innovation and this is critical for its foundational role in building the future. Only when all four technologies are viable at scale will their impact will grow significantly. Businesses must have the will to seize the opportunities and begin exploring possibilities and investments with a strategic focus—to leap ahead of competition in a brand-new competitive landscape.

The author is SVP & Head of APAC, GlobalLogic.

DISCLAIMER: The views expressed are solely of the author and ETCIO.com does not necessarily subscribe to it. ETCIO.com shall not be responsible for any damage caused to any person/organisation directly or indirectly.

The post The future is DARQ but bright appeared first on Artificial Intelligence.

By definition, Big Data is all about collecting large (or “Big”) volumes of structured and unstructured data. What makes Big Data useful is analysis of the collected information to find patterns and meaning that otherwise would be left undiscovered. Making sense of Big Data is the realm of Big Data analytics tools, which provide different capabilities for organization to derive competitive value.

What should you look for when selecting Big Data Analytics tools for your business?

• Analytic Capabilities. There are multiple types of analytics capabilities with different models for various types of analysis including:  predictive mining, decision trees, time series, neural networks, path analysis, market basket analysis, and link analysis.

• Integration. Often additional statistical tools and programming languages (such as R) are needed by organization to conduct other forms of custom analysis.

• Data Import and Export. Getting data in and out of various tools is a critical feature and understanding how difficult (or easy) it is connect the analytics tool to the big data repository is a key consideration.

• Vizualization. Seeing the numbers is one thing, but having data displayed in a graphical format, often makes the data more useable.

• Scalability. Big Data can be big to start with, and generally has a tendency to grow even bigger over time. Organizations need to consider and understand the scalability options for the analytics tools they choose.

• Collaboration. Analysis can sometimes be a solitary exercise, but more often than not it involves collaboration.

In this Datamation guide, we look at 8 of the top Big Data Analytics Tools that cover multiple aspects of the market.

• Cloudera
• Microsoft Power BI
• Oracle Analytics Cloud
• Pentaho Big Data Integration and Analytics
• SAS Institute
• Sisense
• Splunk
• Tableau

Cloudera

When it comes to the core of Big Data, few if any companies are as closely tied with the core Hadoop Big Data open source platform as Cloudera. After all, the founders of Hadoop itself started the company. Cloudera recently got  an even bigger foothold in the Hadoop ecosystem with the merger of Hortonworks which was its primary rival.

The key differentiator for Cloudera is the company’s deep understanding and core competence in Hadoop, which carries through its portfolio including the company’s Cloudera Enterprise platform. This is built on top of the open source CDH distribution.

Cloudera’s Big Data tools are a good fit for organizations that need a full stack that includes the core Hadoop technology for collecting and creating Big Data. With Cloudera Enterprise, organizations are able to create and process predictive analytics models, using a variety of integrated tools.

Microsoft Power BI

Microsoft’s Power BI has been a perennial favorite for analyst firms in the business intelligence space, based largely on the platform’s ease of use and accessibility.

In 2018, Microsoft expanded Power BI, extending the same ease of use to Big Data, enabling data ingest and transformation. The key differentiator for the platform is integration with the Azure Data Lake Storage Gen2 which supports HDFS (Hadoop Distributed File System) for advanced big data analytics.

Power BI is a good choice for organizations looking for an easy on-ramp into Big Data Analytics and is a particularly obvious choice for those that have already standardized on a Microsoft stack. Power BI provides cloud based business analytics and integrates what Microsoft calls “content packs” with pre-built dashboards and report for different types of analysis and data monitoring. The collaboration capabilities in the platform enables users to share data and dashboard, while also providing alerting capabilities.

Oracle Analytics Cloud

Oracle hasn’t always been known as a Big Data analytics provider, but it’s a space where the database giant has moved aggressively into in recent years. Self-service Big Data analytics on a consumption usage model is what the Oracle Analytics Cloud is all about.

Among the key differentiators of the Oracle Analytics Cloud that users comment on is the platform’s automation capabilities for different types of analytics and Big Data analysis use-cases. Organizations that are already used to using Oracle tools, including Oracle’s namesake database, will likely be the most attracted to the Analytics Cloud offering.

The ability to bring multiple data sources together is a core capability of the Oracle Analytics Cloud, with a strong infrastructure that including the Oracle Event Hub Cloud service to ingest data and the Oracle Big Data Cloud Service to store data.

Hitachi Vantara Pentaho

Hitachi is not a name that many would associate with Big Data, but ever since the company acquired Pentaho in 2015, it has been a solid player in the space.

Pentaho’s roots are with its open source analytics platform upon which the more expansive Enterprise edition is built. It’s the open source nature of the platform that is a key differentiator and has led to a broad community of users that is also often seen as a key strength by users.

Pentaho is a good choice for organizations with lots of different types of data and big data sources. The ability to rapidly ingest and blend data from different sources is another key benefit that users gain from the Pentaho Big Data Integration and Analytics platform. Pentaho’s platform enables multiple models including predictive analytics to help organizations guide toward specific outcomes.

SAS Visual Analytics

SAS Institute has a long history in the analytics market that predates the use of Big Data as both a term and a technology by decades. The company has deep domain expertise in analytics which is manifest across a number of different offerings that can help with Big Data Analytics, among them is the Visual Analytics solution that runs on the broader SAS platform for analytics.

Visual Analytics is for users and organizations that are looking for deep analytics tools, with drag and drop functionality for building advanced visualizations. Extensibility of the platform for different types of business intelligence and data reporting needs is a key differentiator for the platform.

Collaboration is a core component as well with the ability to share information and comments across multiple options including mobile devices, web browsers and even Microsoft Office applications. SAS Visual Analytics can be deployed on-premises or as a service in the cloud.

Sisense

Getting Big Data repositories in a state where they can be rapidly used for analytics is a non-trivial challenge, that Sisense aims to help solve with its platform

The promise of helping to make it easier to get Big Data ready for analysis is an area of strength and a key differentiator for Sisense, with its Big Data preparation capabilities that aim to make is easier for users to model data.

Sisense is a good choice for larger organizations that are looking for fast implementation time and solid customer support. The data visualization via the systems dashboard is often seen by users as being easy to use and as a time saver to get the required results. Accessing the dashboards and sharing data is another core strength of the platform, with mobile and web options as well as the ability to easily generate different types of reports.

Sisense offers both on-premises as well as cloud-based offering for its platform.

Splunk

Splunk started out as a log analysis platform and has found a loyal based of users and organizations that love the way the platform works and enables data manipulation and visualizations. For those organizations that are already using Splunk for log or other types of analysis, embracing Splunk Analytics for Hadoop is an easy step.

Splunk as a platform is known for its user-friendly web based log inspection and analytics capabilities, which can be extended to look at Big Data stores in Hadoop systems. The platform benefits from a proven collaboration component and enables users to create and share graphs and analytics dashboards.

Key differentiators for Splunk include the ability to integrate with other elements of the Splunk platform, including security controls and Splunk’s own search process language (SPL) which further provides strong benefits to users.

Tableau

The Tableau platform is a recognized leader in the analytics market and is a good option for non-data scientists working in enterprises, across any sector.

The VizQL data visualization technology at the core of Tableau is a key differentiator for the platform overall, creating data visualization without the need to first organize data. Connectivity to different types and backends of Big Data is also a core attribute of the Tableau platform.

A big benefit that users find from Tableau is the ability to reuse existing skills, in the Big Data context. Tableau makes use of a standardized SQL (Structured Query Language) to query and interface with Big Data systems, making it possible for organizations to make use of existing database and analyst skills sets to find the insights they are looking for, from a large data set. Tableau also integrates its own in-memory data engine called “Hyper” enabling fast data lookup and analysis.

The post 8 Top Big Data Analytics Tools appeared first on Artificial Intelligence.

The post Analytics market to keep growing with digital transformation appeared first on Artificial Intelligence.

At the core of any DevOps initiative is the judicious employment of containers and microservices, which dramatically speed up and simplifying the jobs of developers and operations teams alike. While many of the tried-and-true rules of IT management apply, containers and microservices also add new considerations, and new ways of doing things.

To explore many of the IT management concerns that accompany successful container and microservice deployments, we turn to the observations of two seasoned experts in the field, Ashesh Badani, VP and general manager of OpenShift for Red Hat and Marc Wilczek, a highly regarded industry thought leader.

Badani, writing at The Enterprisers Project, observes that the ultimate goal of containers and microservices — and the DevOps they enable — is agility. “Containers corral applications in a neat package, isolated from the host system on which they run. Developers can easily move them around during experimentation, which is a fundamental part of DevOps. Containers also prove helpful as you move quickly from development to production environments.”

Developers are especially enthusiastic about microservices enabled by containers for a couple of reasons, Wilczek explains in a recent post in CIO. “They enable developers to isolate functions eas­ily, which saves time and effort, and increases overall productivity. Unlike monoliths, where even the tiniest change involves building and deploying the whole application, each microservice deals with just one concern.”

As with any promising new technology or methodology, there are barriers to overcome, both organizational and technological. Badini and Wilczek offer sage advice on overcoming the major speed bumps that may flummox the move to a microservices and containerized architecture:

Organizational skills and readiness: Before any container of microservices effort can get underway, people across the enterprise need to be on board with it, and ready to adapt their own mindsets. “IT leaders driving cultural change need support from both the C-suite and evangelists in the smaller teams,” says Badini, warning that all too often, “the easiest thing to do is just do nothing.” But today’s hyper-competitive and hyper-fast economic environment demands the speed and agility containers and microservices make possible. The good news, Badani adds, is “you don’t need all the resources or skills of Facebook in order to make significant business change. Start experiments with smaller groups. As you succeed and become more comfortable, expand out in terms of technology and talent. Encourage people on your team to engage with their peers outside the company, to talk about technology and culture challenges.”

Platform. Choice of platform is key to a container and microservices efforts. “A platform addresses management, governance, and security concerns,” says Badani. “While there are plenty of open source container tools to experiment with, an enterprise-grade container platform typically comprises dozens of open source projects, including Kubernetes orchestration, security, networking, management, build automation and continuous integration and deployment capabilities out of the box.”

Capacity and lifecycle management: “Both containers and microservices can easily be replaced and therefore tend to have a relatively short lifespan” — often measured in days, says Wilczek. “The short lifespan combined with the enormous density lead to an unprecedented number of items that require monitoring.” In addition, containers need a lot of memory space. The challenge is that “with their own operating environment attached, images can easily reach a couple of hundred megabytes in size,” Wilczek says. He recommends ongoing lifecycle management practices — “especially retiring old images to free up shared resources and avoid capacity constraints.” An ability to quickly retire containers to free up memory space requires a comprehensive lifecycle management effort.

Network layer: Wilczek cautions that networks — or even virtualized network layers — may prove to be bottlenecks in the performance of microservices and containerized architectures, and thus require “close monitoring in terms of performance, load balancing, and seamless interaction.”

Balancing legacy and cloud-native apps: Trade-offs between existing infrastructure and new, cloud-borne applications may be a sticking point, but is normal, Badani explains. “Some CIOs still have COBOL apps to support. Grappling with both old and new technologies, and making tradeoffs, is normal. Some companies seek containers mostly to house cloud-native apps being created by application development teams, including new work and revamps of existing apps. These apps are often microservices-based. The goal is to break up an app into its underlying services, so teams can update the apps independently.”

Monitoring: “Many traditional IT monitoring tools don’t provide visibility into the containers that make up those microservices, leading to a gap somewhere between hosts and applications that is ultimately off the radar,” Wilczek warns. “Organizations need to put one common monitoring in place comprising both worlds and covering the entire IT stack – from the bottom to the top.”

Manageability. Organizations need to ensure that there are enough staff resources dedicated to container and microservice deployment and management. “All too often, developers are tempted to add new functionality by creating yet another microservice,” Wilczek says. “In no time, organizations find themselves attempting to manage an army of containers and countless microservices competing for the same IT infrastructure underneath.” He recommends employing “analytics tools that discover duplicative services, and detect patterns in container be­havior and consumption to prioritize access to systems resources.”

Security. “Because containers contain system specific libraries and dependencies, they’re more prone to be affected by newly discovered security vulnerabilities,” Badani says, who recommends the use of “trusted registries, image scanning, and management tools” that can help automatically identify and patch container images.

The post 8 speed bumps that may slow down the microservices and container express appeared first on Artificial Intelligence.