Oracle launched an observability and management platform that can conduct those deeds across virtually any infrastructure construct and tackle what has been an ongoing challenge for organizations in attempting to see what is exactly happening within their dispersed workloads.

The new Cloud Observability and Management Platform is available through Oracle Cloud Infrastructure (OCI). It allows users to monitor cloud-native and traditional software deployments across their multi-cloud infrastructure down to their on-premises locations.

Clay Magouyrk, EVP for cloud infrastructure engineering at Oracle, explained in a blog post that the platform begins by using a logging tool to scrape all of the necessary information into a single repository. That logging tool uses the open source Fluentd collector to pool the data in a Cloud Native Computing Foundation (CNCF) CloudEvents-compatible format for easier parsing and view of that data.

Users can also set their own rules for acting on that data, including using Oracle’s Streaming service to send logs to any destination. The Oracle logging tool also uses machine learning to detect any issues and provides possible fixes to those problems in real time.

Magouyrk said that the platform’s ability to log data from across different environments is what sets it apart from competing offers like Amazon Web Services (AWS) CloudWatch and Microsoft Azure Monitor.

The platform’s monitoring component provides what Magouyrk described as “real and synthetic end-user monitoring, server monitoring, and distributed tracing, compatible with the open source OpenTracing and OpenMetrics frameworks.” This allows a user to monitor down to the end-user experience for each interaction, which is common in a microservices-based environment.

Workload Spread

This ability to observe workloads across different environments is becoming more important for organizations that want to operate in a multi-cloud environment. Those moves are becoming even more dispersed as those distributed workloads also begin operating in smaller subsets of those clouds like virtual machines (VMs), containers, and serverless.

“The problem with all this is that IT is an increasingly hybrid, distributed-but-integrated, and complex endeavor, both on and off premises; with burgeoning scale, applications, data, devices … and vendors,” wrote Mark Peters, senior analyst at ESG in a blog post. “Modern applications frustrate most legacy monitoring solutions as they are ephemeral, with services being spun up and down in mere seconds, which makes it difficult for solutions that capture data at every 15, 5, or even 1-minute intervals.”

Peters added that the Oracle platform is a “pretty sexy answer to this long-term vexing issue. It gives organizations a level of ‘arms wrapped around everything’ control, based upon its single source of truth and comprehensive visibility.”

Outside of Oracle’s claimed performance advantage over cloud rivals AWS and Microsoft, a number of other vendors offer similar multi-cloud management tools. VMware, for instance, offers its CloudHealth product that it gained from acquiring CloudHealth in 2018.

The post Oracle Observability, Management Spreads Everywhere appeared first on Artificial Intelligence.

In June 2019, the announcement surprised . Oracle and Microsoft forged an unexpected alliance in the cloud, creating interconnections between their respective data centers. First in the United States, then in the United Kingdom in September .

Everything else ? If from a strategic point of view the alliance made sense, from a practical point of view it seemed less clear. Because even in the case of a classic architecture (on-site development with Microsoft technologies on an Oracle on-site base that we migrate to the cloud), the issues of multicloud , latency, version upgrades and Updates on different cycles on two different platforms remained pending.

Of course, Microsoft and Oracle have a lot of common customers. Certainly both are present in large accounts. And of course they have AWS as their common “enemy”. But the real use cases that led to this merger therefore deserved to be specified.

This is what Applications & Data (LeMagIT) did during the visit to Europe (more precisely in London) of Clive D’Souza , the very courteous “Head of Product Management & Business Strategy, VMware & Azure Interconnect at Oracle Cloud Infrastructure ”from Oracle.

Three use cases for a wedding

For Clive D’Souza three main use cases emerged from common customers.

Use case 1 – This is the most obvious, mentioned above. “One of the most popular use cases is an application developed in .NET on top of an Oracle database that a company wants to migrate to the cloud. It can now modernize it and port it while keeping the code on Azure and the data on Oracle Cloud, ”explains Clive D’Souza (see below).

Use case 2 – Companies may want to make new developments using Azure services (Cognitive or others) on data in Oracle Cloud DB.

As opposed to the previous one, “this case is essentially ‘cloud native’, with customers wanting to leverage what Azure really does well – like Power BI , analytics, and the dashboard. They want to develop on top of that. But if there is critical data – corporate data – they can separate it and put it in an Oracle database ”. A configuration that confirms the analysis of Holger Mueller from Constellation Research.

Why, in this configuration, not instead choose the analytical or cognitive services of Oracle Cloud  – which Oracle also extensively highlighted in London in February? “The choice to put such or such part of the application stack in such or such cloud depends a lot on the application and the proximity of the customer [with Oracle and Microsoft]”, answers Clive D’Souza.

Use case 3 – Companies would also have requested the possibility of having an Oracle development stack (Full Stack Oracle Enterprise Apps Development) – like JDE, Hyperion, EBS, etc. – on OCI, but with middleware triggering triggers and alerts, from and to a SQL Server database on Azure.

3 questions to Clive D’Souza (Oracle) and a “Cloud Center of Excellence”

These three use cases do not, however, make us forget that multicloud – even with direct interconnection between infrastructures – is not trivial. Clive D’Souza does not hide it in any way.

On the contrary, he speaks of a long-term transformation project, throughout the company, around a “Cloud Center of Excellence”.

A&D / LeMagIT: For use case # 1, a big problem is that the code of an application is particularly sensitive to the version of the underlying database. An ERP is clearly very sensitive to this. So when you change the base version, there is a good chance that you will also have to rework the code.

If I have my code on Azure and the database updates itself regularly – because that’s what your standalone database does on OCI – how do I handle this?

Clive D’Souza  : Between the Autonomous Database and the application, there is a management and orchestration layer (a “control plan” or a “data plan”). And in this layer, there are “triggers”. Whenever a change occurs – whether on the code side or the base side, with a patch for example – an alert is sent from both sides [Editor’s note: to developers and DBAs ].

But what you’re talking about is a broader discussion of how you come to a form of excellence – a “Cloud Center of Excellence” – and how you sustain that.

A&D / LeMagIT: But in conclusion, will I have to regularly touch the code?

Clive D’Souza  : Yes… but again: it’s not just a matter of database or application. We are entering a more modern cloud operating model (“modern cloud operating model”). This is the “Cloud Center of Excellence”.

When you are there, with this new cloud operating model, you step into the lands of DevOps and CI / CD . It is obligatory.

And when you reach these levels of deployments and this very extensive management of operations, it’s not to do a single application. It flows through the entire organization, in the form of microservices in a DevOps model (you can think of this as very fine piping).

A&D / LeMagIT: Are there many companies in Europe that have reached this level of maturity or are we still only at the very beginning?

Clive D’Souza  : Those on the move are some of the biggest names in the business. Unfortunately, I am not yet allowed to give them to you. But Suzanne Holliday [Editor’s note: present during interview] oversees my team’s efforts for Europe and you should soon have public examples.

But yes, there are different maturities of PoC and deployments.

That said, obviously, the interconnection in Amsterdam [Editor’s note: which Oracle announced at the start of the year] was a “must have” [to get the hybrid cloud between Azure and OCI off the ground] in Europe. And the more regions and interconnections there are, the more customers we will see doing this.

The post Oracle Cloud & Microsoft Azure: the use cases behind the historic agreement appeared first on Artificial Intelligence.

Oracle introduced new data services that expand the number of services available on its cloud platform.

The marquee new service from the software giant is the Oracle Cloud Infrastructure Data Science offering — an evolved version of the DataScience.com platform that Oracle acquired in 2018.

The Oracle Data Science service provides an automated workflow for machine learning and data analysis. Oracle is also launching a new data catalog service that helps users organize data for analysis. Another new capability is the Cloud SQL service that enables users query cloud data stores, while the Data Flow service enables users to run Apache Spark big data analysis as a service.

Oracle is playing to its strength in data with the new services, unveiled Feb. 12, according to Nucleus Research analyst Daniel Elman.

“Oracle made its name on database technology and remains to this day a preeminent leader in the space,” Elman said. “With these services, it’s leveraging this expertise with data management and offering its thousands of database customers a natural route to enabling data science initiatives without having to migrate data or learn new specialized tools.”

Oracle Data Science positioned for ease of use

Oracle is marketing the Data Science service as a way for teams of data scientists to work together collaboratively to generate machine learning models and then apply them to production applications.

The data science service has a project environment that sets up all the infrastructure and the networking needed to access data assets, as well as providing the tools needed for data science, explained Greg Pavlik, senior vice president of product development, data and AI services at Oracle. Among the tools is an automated machine learning feature that provides these capabilities for common data science tasks such as algorithm selection.

Oracle getting into the data catalog market
Alongside the Oracle Data Science service, the vendor launched a new data catalog to help organizations track all the data sets that come into a cloud deployment.

“Say you’re setting up a data warehouse, we can introspect the data warehouse model, and allow users — it could be data scientists, it could be data stewards, it can be analysts — to find out what data is available, who owns it and what it’s meant to be used for,” Pavlik said.

The Oracle data catalog also provides tagging capabilities that enable administrators to define taxonomies and start to organize data sets hierarchically.

Data Flow service enables Apache Spark Big Data
The new Data Flow service also helps meet a different need, enabling users to run Apache Spark jobs as service in the Oracle cloud. One of the challenges some organization face with running Spark analytics jobs is that they are often running on top of Hadoop clusters, which introduces additional complexity, Pavlik noted.

All that’s needed to run a big data workload in the Data Flow service is to upload the script, click on an application that is sort of the pointer to the script, and then specify how many CPUs the job should run on, Pavlik explained.

“We will synthesize the job on the fly in a totally serverless architecture, executing in tens of seconds,” he said. “We really think about this as a big generational leapfrog in terms of how to how to make big data workloads consumable by the enterprise.”

Oracle is also expanding the ability of users to query data in the cloud with the new

Oracle Cloud SQL offering. Users can use the SQL capability to query against cloud-based object stores.

“So you can reach out into a cloud-based data lake and apply the full semantic richness of the Oracle database,” Pavlik said.

Data integration service is coming

In addition to the Oracle Data Science services, the vendor has more data services in the works, among them a data integration service. Pavlik said that an upcoming data integration service will provide data preparation and ETL capabilities.

“It figures out where’s the most cost-effective way to run elements of the flow so that it’s filtering data and minimizing data movement,” Pavlik said. “It’s also filled with a data immersive view, so you can really drill down, understand your datasets and manipulate the data.”

The post Oracle Data Science efforts advance with new services appeared first on Artificial Intelligence.

Oracle Wednesday staked its claim in the data science platform space with the availability of the Oracle Cloud Data Science Platform.

The platform, built on the foundation of DataScience.com acquired by Oracle in 2018, is geared for teams of data scientists working collaboratively. Its capabilities include shared projects, model catalogs, team security policies, reproducibility, and auditability.

The platform has the Oracle Cloud Infrastructure Data Science service at its core. It provides users the ability to build, train, and manage machine learning algorithms on the Oracle Cloud using Python, TensorFlow, Keras, Jupyter and other popular data science tools. Six additional services round out the platform, including new machine learning capabilities integrated in Oracle Autonomous Database, the Oracle Cloud Infrastructure Data Catalog, Oracle Big Data Service, Oracle Cloud SQL, Oracle Cloud Infrastructure Data Flow, and Oracle Cloud Infrastructure Virtual Machines for Data Science.

“The service is really the first of its kind in terms of a native cloud service in that it’s really targeted for the enterprise,” says Greg Pavlik, senior vice president product development of Oracle Data and AI Services. “It is focused on providing an environment for collaboration and governance for data scientists.”

According to Pavlik, the offering targets the full lifecycle of machine learning within the enterprise, meaning that it’s not just about developing or training models, but also taking those models into production and maintaining them.

“As data changes, models become potentially less valid and users need to be able to continue to leverage them inside of applications or inside the analytic reports on the one hand. On the other hand, they have to have a high confidence that what they’re using is actually giving them good answers or correct responses,” Pavlik says.

Simplifying data science

With Oracle Cloud Infrastructure Data Science, Oracle is taking on platforms from competitors such as Alteryx, KNIME Analytics Platform, and RapidMiner with a focus on automating the data science workflow.

The platform leverages AutoML algorithm selection and tuning, using machine learning models to select the best-fit algorithm for a specific use case, and to help users choose algorithm inputs and tune the model, Pavlik says. The platform also simplifies feature engineering by automatically identifying key predictive features from larger data sets.

Oracle Cloud Infrastructure Data Science also aids in model evaluation by generating a suite of metrics and visualizations to help users measure model performance against new data and rank models over time.

To support regulatory compliance efforts and help data teams establish trust in the output of their algorithms, Oracle’s offering provides automated explanation of the weighting and importance of factors used to generate a prediction.

“We have advanced capabilities that we’ve developed in our Oracle Labs organization for model explainability,” Pavlik says. “That’s really understanding what is driving the model to its prediction, which is particularly important for regulatory situations where you have to be able to give an accounting of why: Why is the business making this decision? Why is the model telling us to do this?”

Shared projects

To support collaboration, Oracle has drawn inspiration from modern software development processes, adding capabilities that support shared projects, model catalogs, team-based security policies, and reproducibility and accountability.

“The big problem that we often see with teams is the data scientists are off downloading a bunch of stuff on their laptop and then they’re working in relative isolation,” Pavlik says. “You lose some of the sense of accountability, safety, some of the best practices you’d have from software development. So, we’re looking to help organizations solve that problem without taking anything away from the data scientist.”

The platform enables teams to leverage version control and share data and notebook sessions. Using model catalogs, teams can also share models and the artifacts necessary to modify and deploy them. Team-based security policies provide access controls to models, codes, and data, all integrated with Oracle Cloud Infrastructure Identity and Access Management. Enterprises can also track assets via the platform, thereby ensuring models can be reproduced and audited, even if team members leave.

Additional data and machine learning services

Oracle Cloud Infrastructure Data Science sits at the core of the new Oracle Cloud Data Science Platform, but Oracle also unveiled six other data and machine learning services to support the platform and integrate it with the company’s overall cloud offering.

“If you’re working in your notebook, you’re doing Python training, it allows you to transparently go out, use compute resources, do scale-out training jobs, without having to drop into an IT administrative type mode. You can, within the tool itself, leverage the elastic capabilities of the cloud as part of your model training and model experimentation process,” Pavlik says.

The additional six services include:

• New machine learning capabilities in Oracle Autonomous Database. Oracle has added support for Python and automated machine learning to Oracle Autonomous Database. Forthcoming integration with Oracle Cloud Infrastructure Data Science will give data scientists the ability to develop models using open source and scalable in-database algorithms.
• Oracle Cloud Infrastructure Data Catalog. The data catalog provides the ability to discover, find, organize, enrich and trace data assets. It features a built-in business glossary.
• Oracle Big Data Service. This service offers a full Cloudera Hadoop implementation, as well as machine learning for Spark.
• Oracle Cloud SQL. This service gives users the ability to run SQL queries on data in HDFS, Hive, Kafka, NoSQL, and Object Storage.
• Oracle Cloud Infrastructure Data Flow. This fully managed service lets users run Apache Spark applications without deploying or managing infrastructure.
• Oracle Cloud Infrastructure Virtual Machines for Data Science. This service offers preconfigured GPU-based environments for $30 a day.

The post Oracle unleashes cloud-based data science platform appeared first on Artificial Intelligence.

A year ago, we spoke of Oracle’s positioning of its cloud as a second generation, both literally and figuratively. It’s no coincidence that Oracle’s cloud unit is based in Seattle and counts many AWS and Azure alumni among its team. A year later, Oracle has an agreement to link up to Microsoft Azure data centers.

An example is how Oracle has extended elasticity from compute to storage. It is virtualizing its storage into block volumes that can be configured by performance or cost. While in other clouds, customers typically select specific tiers of storage, such as disk, SSD, or memory, in Oracle Cloud you can buy performance units on the fly that can increase or decrease IOPS. While currently this is a manual process, Oracle plans to add an automated option that would, in essence, extend autoscaling to storage.

It’s all part of the narrative that Oracle is spelling out, that the other folks did cloud first and they had a chance to learn the lessons of the first generation. With the Azure partnership, there is another part of that narrative: Oracle is not trying to be all things to all people. The tie-in with Azure reflects the fact that most Oracle customers already have Microsoft in the front office.

Growing the footprint

A year ago, we wrote that Oracle Cloud was preparing for global build-out. With the Gen2 infrastructure introduced at OpenWorld 2018, the template was established for expanding the footprint. A year ago, Oracle counted less than a half dozen global regions, today it is up to 20, with plans to almost double that by the end of the year.

For some perspective, Oracle uses the term “regions” differently from AWS or Azure – its regions are the equivalent of data centers or availability zones with the other clouds. And so, a region with AWS or Azure means that there are at least two separate data centers in it (and increasingly, three). So, most of Oracle’s early regions have only one data center, but this year, that’s going to change. About half the growth that Oracle is planning in 2020 will be adding those additional data centers in the geographies where it already sits. Today, Oracle Cloud has in-country disaster recovery in the US and Japan; by year’s end, it plans to have nine more countries with duplicate data centers that are based in different cities. A key driver in all this is responding to data sovereignty regulations requiring companies to keep their data within the country of origin.

As Oracle raises the profile of its cloud positioning, it is being realistic in not trying to be all things to all people. While it contends that its Infrastructure-as-a-Service offering is based on more current technology than rival clouds, the primary role of IaaS at Oracle Cloud is to serve as building block to the company’s enterprise application SaaS and autonomous database PaaS services, where it better differentiates with the usual suspects. The partnership with Microsoft, for building high-speed links and unified identity and access management between Oracle and the Azure clouds, reflects the fact that Microsoft, not Oracle, is the default front office.

THE AUTONOMOUS DATABASE TO BECOME MORE THAN DATABASE

The autonomous database is a key differentiator for the Oracle Cloud. As we noted last fall, with the autonomous database, Oracle now has at least a year or two track record with some of the earliest clients, with the common themes being superior performance at lower cost, and changing of the DBA role.

Going forward, Oracle’s Autonomous Data Warehouse will spread its functional footprint to encompass data transformation/ETL with a no-code drag and drop experience that will be useful for relatively simple data transformations (it won’t replace Oracle Data Integrator for the more complex work performed by data engineers). Additionally, it will have an “Auto Insights” capability for data discovery on incoming data, for detecting outliers and relationships with existing data. And there will be a capability for performing machine learning inside the database geared to “citizen data scientists.” It will automate the selection of algorithms, feature selection, and parameter tuning. Rounding it off is Oracle visual database application development language (APEX) and federated query capability via Cloud SQL (not to be confused with the Google Cloud service of the same name).

Oracle is hardly the only analytics provider to expand the role and definition of the cloud data warehouse to go beyond just being a database. Last fall, Microsoft unveiled Azure Synapse Analytics, which embeds the data pipelining capabilities of Azure Data Factory into the data warehouse. At the other end of the spectrum, SAP extended its HANA Data Warehouse Cloud, but with self-service analytics capabilities adapted from its Analytics Cloud. All of these signify an emerging trend among cloud SaaS and PaaS providers to extend the database into a one-stop shop for either the data engineer or business user.

It’s all part of the narrative that Oracle is spelling out, that the other folks did cloud first and they had a chance to learn the lessons of the first generation. With the Azure partnership, there is another part of that narrative: Oracle is not trying to be all things to all people. The tie-in with Azure reflects the fact that most Oracle customers already have Microsoft in the front office.

Growing the footprint

A year ago, we wrote that Oracle Cloud was preparing for global build-out. With the Gen2 infrastructure introduced at OpenWorld 2018, the template was established for expanding the footprint. A year ago, Oracle counted less than a half dozen global regions, today it is up to 20, with plans to almost double that by the end of the year.

For some perspective, Oracle uses the term “regions” differently from AWS or Azure – its regions are the equivalent of data centers or availability zones with the other clouds. And so, a region with AWS or Azure means that there are at least two separate data centers in it (and increasingly, three). So, most of Oracle’s early regions have only one data center, but this year, that’s going to change. About half the growth that Oracle is planning in 2020 will be adding those additional data centers in the geographies where it already sits. Today, Oracle Cloud has in-country disaster recovery in the US and Japan; by year’s end, it plans to have nine more countries with duplicate data centers that are based in different cities. A key driver in all this is responding to data sovereignty regulations requiring companies to keep their data within the country of origin.

As Oracle raises the profile of its cloud positioning, it is being realistic in not trying to be all things to all people. While it contends that its Infrastructure-as-a-Service offering is based on more current technology than rival clouds, the primary role of IaaS at Oracle Cloud is to serve as building block to the company’s enterprise application SaaS and autonomous database PaaS services, where it better differentiates with the usual suspects. The partnership with Microsoft, for building high-speed links and unified identity and access management between Oracle and the Azure clouds, reflects the fact that Microsoft, not Oracle, is the default front office.

THE AUTONOMOUS DATABASE TO BECOME MORE THAN DATABASE
The autonomous database is a key differentiator for the Oracle Cloud. As we noted last fall, with the autonomous database, Oracle now has at least a year or two track record with some of the earliest clients, with the common themes being superior performance at lower cost, and changing of the DBA role.

Going forward, Oracle’s Autonomous Data Warehouse will spread its functional footprint to encompass data transformation/ETL with a no-code drag and drop experience that will be useful for relatively simple data transformations (it won’t replace Oracle Data Integrator for the more complex work performed by data engineers). Additionally, it will have an “Auto Insights” capability for data discovery on incoming data, for detecting outliers and relationships with existing data. And there will be a capability for performing machine learning inside the database geared to “citizen data scientists.” It will automate the selection of algorithms, feature selection, and parameter tuning. Rounding it off is Oracle visual database application development language (APEX) and federated query capability via Cloud SQL (not to be confused with the Google Cloud service of the same name).

Oracle is hardly the only analytics provider to expand the role and definition of the cloud data warehouse to go beyond just being a database. Last fall, Microsoft unveiled Azure Synapse Analytics, which embeds the data pipelining capabilities of Azure Data Factory into the data warehouse. At the other end of the spectrum, SAP extended its HANA Data Warehouse Cloud, but with self-service analytics capabilities adapted from its Analytics Cloud. All of these signify an emerging trend among cloud SaaS and PaaS providers to extend the database into a one-stop shop for either the data engineer or business user.

ORACLE’S DATA SCIENCE CLOUD EMERGES

Following the 2018 acquisition of DataScience.com, Oracle has been refashioning the platform to become a native service in the Oracle cloud. The service is focused on the lifecycle management of data science and machine learning projects that are rooted in Jupyter notebooks. While not restricted to Oracle data sources, the new OCI (Oracle Cloud Infrastructure) Data Science service does come with connectors to the Autonomous Data Warehouse and cloud object storage. It will leverage the new OCI Data Catalog, which will be the metadata backbone of all Oracle analytic services (including the Oracle Analytics Cloud for BI and augmented self-service analytics) and bundled with them.

As a first release, Oracle still has some finishing up work to do on the new OCI Data Science service. Related services, such as the OCI Data Flow service (Oracle’s answer to Amazon EMR and Kinesis) are not yet a unified experience, as you still have to boot them up as separate services. Another critical enhancement not yet in the first release is support for distributed training; initially, out of the box, OCI Data Science will only run on a single node. We expect that Oracle will add these features over the course of the year.

MEANWHILE, BACK ON THE MOTHER SHIP

The cloud has sparked the debate on the fit-for-purpose vs. multi-model database. Amazon has led the charge for bespoke databases, with 15 database platforms now in its portfolio. Meanwhile, Oracle, Microsoft, and SAP have placed their bets on the database as a jack of all trades (or at least, data types). Oracle is now amplifying that message by terming its database as a “converged database.”

Oracle database has always supported multiple data types from text to XML, spatial, or Blobs, before it ever got to the cloud. But with some exceptions, those nonrelational data types were not first-class citizens. For instance, Oracle’s JSON support was through representing it as a variable character string, which can contain a JSON document, but is not very efficient to query. Oracle is now stepping up its game in representing non-relational data types along with variants of relational data.

For instance, it is introducing a binary JSON data type that will improve query and update performance. This is not about compatibility with MongoDB, but about improving performance and supporting a wider range of data types. So, Oracle, like most other databases (excluding IBM Db2), do not support the BSON binary JSON format of MongoDB. There is compatibility if developers drop back to using standard, but less performant, character JSON data types.

Then again, given MongoDB’s latest licensing changes, no database — relational or NoSQL — will be compatible with Mongo. The best that Amazon and others are doing is supporting MongoDB-compatible APIs providing access to core features, something we hope will be on Oracle’s roadmap.

While Oracle binary JSON won’t replace MongoDB, it will be useful for boundary cases where you have mixed relational and JSON data and want to take advantage of a mature SQL implementation.

Oracle is also introducing Blockchain tables that will still function as relational tables, but with the append-only and cryptographic support to make the tables immutable. And, it already includes a graph capability that will support RDF and property graphs, plus the ability to ingest IoT streaming data, and many built-in machine learning algorithms such as simple classification and complex neural networks.

The post Oracle Cloud update: Literally doubling down appeared first on Artificial Intelligence.

Algorithms, especially those that utilize deep learning in some manner, are notorious for being opaque. To be clear, this means that when you ask a deep learning algorithm to answer a question, the algorithm gives you an answer without any explanation of how it came to that conclusion. It does not show its work; you simply ask it a question and it generates an answer, like a mysterious oracle. 

As author Scott Fulton III points out, “We’ve created systems that draw mostly, though never entirely, correct inferences from ordinary data, by way of logic that is by no means obvious.” Should these systems be trained on faulty or incomplete data, they have the capacity to further entrench existing biases and perpetuate discriminatory behavior.

Bias isn’t inherent

I believe that there are ways that we can use deep learning to help eliminate these inequalities, but it requires organizations to interrogate their existing practices and data more deeply. We are beginning to see the biased and hurtful results of this in the advertising industry, as deep learning is increasingly used to decide what ads you see. Everyone needs to exhibit greater awareness about how their ads are perceived, as well as who is viewing them.

The problem that many marketers currently face is that they rely on third-party platforms to determine who their ad is shown to. For example, while advertising platforms like Facebook allow marketers to roughly sketch out their target audience, it is ultimately up to the algorithm itself to identify the exact users who will see the ad. To put it another way, a company might put out an ad that is not targeted to a specific age group, ethnicity, or gender, and still find that certain groups are more likely to see their ads than others.

How algorithms can perpetuate bias

Earlier this year, a team from Northeastern University decided to carry out a series of experiments designed to identify the extent to which Facebook’s ad delivery is skewed along demographic lines. While Facebook’s algorithm does allow advertisers to target certain demographics, age groups, and genders with precision, the researchers wanted to see whether giving the algorithm neutral targeting parameters for a series of ads would also result in a similar skew, despite not targeting any single specific group; in other words, whether people with particular demographic characteristics were more likely to see certain ads than others.

To test this hypothesis, the researchers ran a series of ads that were targeted to the exact same audience and had the same budget, but used different images, headlines, and copy. They found that ads with creative stereotypically associated with a specific group (ie. bodybuilding for men or cosmetics for female audiences) would overwhelmingly perform better amongst those groups, despite not being set up to target those audiences specifically.

Researchers also discovered that, of all the creative elements, the image was by far the most important in determining the ad’s audience, noting that “an ad whose headline and text would stereotypically be of the most interest to men with the image that would stereotypically be of the most interest to women delivers primarily to women at the same rate as when all three ad creative components are stereotypically of the most interest to women.” 

These stereotypes become much more harmful when advertising housing or job openings, to name a few sensitive areas. As the researchers from Northeastern discovered, job postings for secretaries and pre-school teachers were more likely to be shown to women, whereas listings for taxi drivers and janitors were shown to a higher percentage of minorities.

Why does this happen? Because Facebook’s algorithm optimizes ads based on a market objective (maximizing engagement, generating sales, garnering more views, etc.), and does not pay as much attention to minimizing bias. As a result, Karen Hao notes in the MIT Technology Review, “if the algorithm discovered that it could earn more engagement by showing more white users homes for purchase, it would end up discriminating against black users.”

There are other approaches

However, the algorithm does this because it has been taught to approach the issue from a purely economic perspective. If, on the other hand, it had been trained from the start to be aware of potential discrimination, and to guard against it, the algorithm could end up being much less biased than if marketers were left to their own devices. Brookings Institute suggests that developers of algorithms create what they term a “bias impact statement” beforehand, which is defined as “a template of questions that can be flexibly applied to guide them through the design, implementation, and monitoring phases,” and whose purpose is to “help probe and avert any potential biases that are baked into or are resultant from the algorithmic decision.”

Take, for instance, mortgage lending. Research has shown that minorities, especially African Americans and Latinos, are more likely to be denied mortgages even when taking into account income, the size of the loan, and other factors. An algorithm that relies solely on data from previous loans to determine who to give a mortgage to would only perpetuate those biases; however, one that is designed specifically to take those factors into account could end up creating a much fairer and more equitable system.

While there is clearly more work to be done to make sure that algorithms do not perpetuate existing biases (or create new ones), there is ample research out there to suggest a way forward — namely, by making marketers and developers aware of the prejudices inherent in the industry and having them take steps to mitigate those preferences throughout the design, data cleansing, and implementation process. A good algorithm is like wine; as it ages, it takes on nuance and depth, two qualities that the marketing industry sorely needs.

The post How deep learning can reduce bias in advertising appeared first on Artificial Intelligence.

Here’s the challenge: you need human intelligence in your SIEM cybersecurity for its optimal performance. 

Why? Unfortunately, while SIEM can perform many functions autonomously, it relies on human intelligence at least partially. Most next-generation SIEM solutions work to automate as many parts of the process as possible to mitigate this need.

For example, most SIEM log collection uses automation to collect the relevant security event information, normalize it, and scan it for potential breaches. However, only with human intelligence in SIEM can your enterprise conduct coordinated incident response efforts among departments. Additionally, only with human intelligence in SIEM can you change the correlation rules to fit with threat intelligence.

With the recurring cybersecurity staffing crisis still in full effect, finding cybersecurity human intelligence proves a major obstacle. Fortunately, SIEM capabilities have worked to reduce the need to rely on human intelligence in your cybersecurity. 

Here’s how:   

3 Ways to Reduce for Human Intelligence 

1. Managed Security Services

Managed security services work to alleviate the problem of human intelligence in SIEM due to missing security talent through third-party services. In fact, managed security services for enterprises operate through third-parties to conduct cybersecurity monitoring and management. 

Thus, it conducts incident detection and response, as well as incident containment. Importantly, these managed security services can operate twenty-four hours a day, seven days a week. If your IT security team tried to maintain that schedule, they would quickly suffer burnout.

Yet having around-the-clock monitoring proves essential for protecting your databases and servers from hackers. After all, hackers could strike at any hour and may plan their attacks to take advantage of lapses in monitoring. Moreover, active threat hunting could uncover dwelling threats lurking in your network. 

Human intelligence in SIEM can feel limited when you need to rely on your own team. So why not borrow another team to alleviate the burden?  

2. Artificial Intelligence

Artificial intelligence (AI) can’t replace your human intelligence in SIEM—at least not entirely. Unfortunately, machine learning just can’t match the power of human ingenuity, communication, and collection collaboration. 

However, there is also good news. AI in SIEM can optimize all of these once human-reliant processes. Through its predictive and automated capabilities, it can provide the groundwork to your IT security team. 

For example, it can perform automated threat hunting through your security correlation rules; AI can even identify false positives through the automatic application of contextualization on all alerts. Even in enterprises with limited human intelligence, AI in SIEM can speed up their response and detection times. 

Moreover, machine learning can actually halt processes it suspects as malicious. Not only can this help with investigations and threat remediation, but it also mitigates damage even before your incident response begins!

Hard to argue with that. 

3. Behavioral Analytics 

Behavioral analytics examines trends, patterns, and activities among your users and applications. It looks for habits and quirks in workflows and creates profiles for each user. For example, it can determine how times a day on average an employee accesses a particular database. With more next-generation technology, it also recognizes the endpoint they use to make these access requests. The behavioral analytics SIEM capability uses this information to establish a behavioral baseline.

Then, assume something happens. Maybe an employee tries to (incorrectly) log in to a database they never use—multiple times. Are they handling a special project? Or are they an imposter? In either case, your cybersecurity solution can put an injunction on the access requests and alert your security team to investigate. 

Human intelligence in SIEM can detect these kinds of attacks or security events. However, the problem comes with scale—trying to find all possible events in your enter enterprise is a tall order. Behavioral analytics can more than help you concentrate your human intelligence in SIEM where it needs to be: threat hunting and remediating. 

The post Global Data Wrangling Market Research Insights 2019 : IBM, Oracle, SAS, Trifacta, Datawatch appeared first on Artificial Intelligence.

NEW DELHI: Beating frontrunner Amazon, Microsoft on Friday won the lucrative $10 billion Cloud-computing contract from the US Department of Defense known as “JEDI”, the short-form for the Joint Enterprise Defense Infrastructure cloud project.

Exactly a year ago — on October 25, 2018 — the software giant’s CEO Satya Nadella addressed ethical concerns surrounding the supply of technology to the military.

While the size of the project evoked interest from the technology giants in the US — with Oracle, IBM and Google also joining Microsoft and Amazon in the bidding process spanning nearly two years — employees of some of these organisations raised objections to helping defence forces become more lethal.

In fact, Google dropped out of the bidding for JEDI in October last year. The tech giant last year decided not to renew a Pentagon project called Maven after thousands of its employees signed a petition demanding “a clear policy stating that neither Google nor its contractors will ever build warfare technology.”

Following this, Google launched ethical principles for Artificial Intelligence (AI) and claimed that these principles conflicted with the JEDI project.

After Google exited the project, an open letter claiming to be from an unspecified number of Microsoft employees urged the tech giant to also back out of the military project. But Microsoft stuck to its principle of supplying its technology to the US military.

In the Q&A session with employees last year, Nadella, who earned $42.9 million in total compensation for the fiscal year 2019 — 66 per cent raise from the prior fiscal year, was joined by Microsoft President Brad Smith.

The two top Microsoft executives said that the company decided to pursue the JEDI project, which aims to re-engineer the US Defense Department’s end-to-end IT infrastructure, from the Pentagon to field-level support of the country’s servicemen and women, given the company’s longstanding support for Pentagon.

“All of us who live in this country depend on its strong defence. The people who serve in our military work for an institution with a vital role and critical history,” Smith said in a blog post that detailed what transpired in the meeting.

“We believe in the strong defence of the US and we want the people who defend it to have access to the nation’s best technology, including from Microsoft,” Smith said.

“As is always the case, if our employees want to work on a different project or team — for whatever reason — we want them to know that we support talent mobility,” he added.

Microsoft has worked with the Department of Defense on a longstanding and reliable basis for four decades.

“You’ll find Microsoft technology throughout the American military, helping power its front office, field operations, bases, ships, aircraft and training facilities. We are proud of this relationship, as we are of the many military veterans we employ,” the Microsoft President wrote in the blog post.

The post How Microsoft defended its $10bn JEDI bid exactly a year ago appeared first on Artificial Intelligence.

Facebook seeks to placate lawmakers over Libra but Mnuchin raises “serious concerns”
Facebook has made known that it won’t launch its digital currency Libra until it receives regulatory approval especially after US Treasury Secretary Steve Mnuchin expresses very serious concerns that Libra could be misused by terrorists. Apart from being supervised by the Swiss Financial Markets Supervisory Authority it will also register in the US with FinCEN. Libra does not intend to compete with any sovereign currencies or intruding the monetary policy arena. It is designed to act like cash only to create economic empowerment.

Google’s new ‘data echoing’ technique speeds up AI training
Search engine giant, Google’s researchers think that the best-performing data echoing algorithms can match the baseline’s predictive performance using less upstream processing. The approach used by these researchers in data echoing inserts a stage in the pipeline that repeats the output data of the previous stage, recovering idle compute capacity. Data echoing is an alternative to optimizing the training pipeline or adding additional workers to perform upstream data processing which may not be possible.

Salesforce named a leader in data management platforms by independent research firm
CRM solutions provider, Salesforce has been recognised a leader by Forrester Research in its report namely ‘The Forrester Wave: Data Management Platforms,’ Q2 2019. Salesforce’s Data Management Platform (DMP), Audience Studio enables marketers to know, personalise and engage their customers across all verticals. It also provides a platform to capture customer data, segment audiences with Einstein AI for engagement. Salesforce Marketing Cloud helps marketers customise marketing with Einstein and analyse the impact to improve their campaign performance.

Synthio introduces Contact Data Maintenance for Salesforce
Synthio has revealed immediate availability of its Contact Data Management for Salesforce (CDM-SF) platform. This platform compares a customer’s Salesforce database against more than 160 million data records and automatically updates it with complete and correct information. Companies implementing CDM-SF are certain about the data they receive being correct and complete in order to build Account Based Marketing and demand generation programs, customer billing, and partner management.

IBM builds AI ladder with partnerships
Multinational information technology company, IBM has joined forces with data preparation technology provider Trifacta to improve the data preparation technology it provides in its data and artificial intelligence stack. This partnership has resulted in InfoSphere Advanced Data Preparation to enable organisations in formatting, structuring and enriching datasets while working with their existing environments including data lakes. Trifacta’s clients include GSK, the Centers for Disease Control, Google, and Pepsico.

IBM announces AutoAI For Watson Studio
IBM has announced AutoAI, a new set of capabilities for Watson Studio to automate complicated tasks associated with designing, optimising and governing AI in the enterprise. Watson Studio’s AI capabilities work efficiently with Watson Machine Learning (ML) to solve the challenges by automating and speeding steps in the AI lifecycle. The AutoAI capabilities are available on the IBM Cloud and they automate data prep and pre-processing including model development and feature engineering. They also contain a suite of powerful model types for enterprise data science and aim to scale ML deployment quickly.
Read More >>

IBM offers employees ‘master’s programs’ in AI and data science
IBM is now offering its employees the opportunities to take courses in data science and artificial intelligence with Simplilearn. The courses will be web-based including live virtual classrooms access to teaching assistants, self-paced video instruction courses and assessments. Upon completion, employees will be granted certificates from Simplilearn and IBM.

Oracle bets big on ‘autonomous offering’ for security as cyber criminals embrace AI, ML
Cloud services provider, Oracle aims to automate its database by creating a tool, Web Application Firewall, that eliminates manual errors in database tuning, security, backups, and updates. The cloud service provider believes that security should be embedded by default. This tool makes use of AI and ML to secure itself and has features that can identify good bots from bad bots by furthering whitelisting good bots letting more traffic in and blacklisting bad bots.

Amazon partners with the NHS to help UK patients access health advice
Amazon has partnered with UK’s NHS to help elderly people access their health advice online through Amazon’s AI-powered voice assistant Alexa. The Department of Health is looking forward to this partnership to empower patients take control of their health. The UK government has set up a new unit called NHSX to boost the use of digital technologies in the UK health service. Secretary of State for Health and Social Care, Matt Hancock, said: “We want to empower every patient to take better control of their healthcare and technology like this is a great example of how people can access reliable, world-leading NHS advice from the comfort of their home, reducing the pressure on our hardworking GPs and pharmacists.”

FCA to explore the ‘explainability’ of AI in consumer finance
The Financial Conduct Authority (FCA) is joining forces with the Alan Turing Institute to test the transparency and explainability of AI in the financial sector. The watchdog wants a practical understanding of machine learning and its functions in comparison to just debates. Christopher Woolard, executive director of strategy and competition at the FCA said: “AI and its application in financial services is causing us to ask big questions – and the answers we arrive at have the potential to fundamentally alter society and the established order.”

Samsung Pay integrates loan and credit card applications in India
Samsung Pay has tapped Paisabazaar to let people apply for things such as credit cards and loans through Samsung Pay. This service calls Paisabazaar’s ‘Chance of Approval’ feature that makes use of predictive algorithms to connect consumers with suitable lenders with a possibility of approving a loan. Sanjay Razdan, senior director, services, management, Samsung India, says: “Our partnership with Paisabazaar.com has helped us create a holistic platform that gives them the comfort of opting for various financial products besides using it as a payment platform.”

Lloyds partners with Blue Motor Finance to develop instant payments API
Lloyds Bank has partnered with Blue Motor Finance to provide motor dealerships with realtime loan payments through a commercial direct debit API collaboration. Co-developed by Lloyds Bank Commercial Banking API Lab, the programme taps the Faster Payment rails and aims to deliver instant loans to motor trader clients of Blue. Steve Everett, MD, payments and cash Management, Lloyds Bank Commercial Banking said: “We’re excited to partner with Blue Motor Finance to bring this innovative instant payment service to its motor trader customers and create real value for businesses in the car financing industry through new API-enabled solutions.”

The post Tech update including Amazon, IBM and Oracle appeared first on Artificial Intelligence.