Oracle Wednesday staked its claim in the data science platform space with the availability of the Oracle Cloud Data Science Platform.

The platform, built on the foundation of DataScience.com acquired by Oracle in 2018, is geared for teams of data scientists working collaboratively. Its capabilities include shared projects, model catalogs, team security policies, reproducibility, and auditability.

The platform has the Oracle Cloud Infrastructure Data Science service at its core. It provides users the ability to build, train, and manage machine learning algorithms on the Oracle Cloud using Python, TensorFlow, Keras, Jupyter and other popular data science tools. Six additional services round out the platform, including new machine learning capabilities integrated in Oracle Autonomous Database, the Oracle Cloud Infrastructure Data Catalog, Oracle Big Data Service, Oracle Cloud SQL, Oracle Cloud Infrastructure Data Flow, and Oracle Cloud Infrastructure Virtual Machines for Data Science.

“The service is really the first of its kind in terms of a native cloud service in that it’s really targeted for the enterprise,” says Greg Pavlik, senior vice president product development of Oracle Data and AI Services. “It is focused on providing an environment for collaboration and governance for data scientists.”

According to Pavlik, the offering targets the full lifecycle of machine learning within the enterprise, meaning that it’s not just about developing or training models, but also taking those models into production and maintaining them.

“As data changes, models become potentially less valid and users need to be able to continue to leverage them inside of applications or inside the analytic reports on the one hand. On the other hand, they have to have a high confidence that what they’re using is actually giving them good answers or correct responses,” Pavlik says.

Simplifying data science

With Oracle Cloud Infrastructure Data Science, Oracle is taking on platforms from competitors such as Alteryx, KNIME Analytics Platform, and RapidMiner with a focus on automating the data science workflow.

The platform leverages AutoML algorithm selection and tuning, using machine learning models to select the best-fit algorithm for a specific use case, and to help users choose algorithm inputs and tune the model, Pavlik says. The platform also simplifies feature engineering by automatically identifying key predictive features from larger data sets.

Oracle Cloud Infrastructure Data Science also aids in model evaluation by generating a suite of metrics and visualizations to help users measure model performance against new data and rank models over time.

To support regulatory compliance efforts and help data teams establish trust in the output of their algorithms, Oracle’s offering provides automated explanation of the weighting and importance of factors used to generate a prediction.

“We have advanced capabilities that we’ve developed in our Oracle Labs organization for model explainability,” Pavlik says. “That’s really understanding what is driving the model to its prediction, which is particularly important for regulatory situations where you have to be able to give an accounting of why: Why is the business making this decision? Why is the model telling us to do this?”

Shared projects

To support collaboration, Oracle has drawn inspiration from modern software development processes, adding capabilities that support shared projects, model catalogs, team-based security policies, and reproducibility and accountability.

“The big problem that we often see with teams is the data scientists are off downloading a bunch of stuff on their laptop and then they’re working in relative isolation,” Pavlik says. “You lose some of the sense of accountability, safety, some of the best practices you’d have from software development. So, we’re looking to help organizations solve that problem without taking anything away from the data scientist.”

The platform enables teams to leverage version control and share data and notebook sessions. Using model catalogs, teams can also share models and the artifacts necessary to modify and deploy them. Team-based security policies provide access controls to models, codes, and data, all integrated with Oracle Cloud Infrastructure Identity and Access Management. Enterprises can also track assets via the platform, thereby ensuring models can be reproduced and audited, even if team members leave.

Additional data and machine learning services

Oracle Cloud Infrastructure Data Science sits at the core of the new Oracle Cloud Data Science Platform, but Oracle also unveiled six other data and machine learning services to support the platform and integrate it with the company’s overall cloud offering.

“If you’re working in your notebook, you’re doing Python training, it allows you to transparently go out, use compute resources, do scale-out training jobs, without having to drop into an IT administrative type mode. You can, within the tool itself, leverage the elastic capabilities of the cloud as part of your model training and model experimentation process,” Pavlik says.

The additional six services include:

• New machine learning capabilities in Oracle Autonomous Database. Oracle has added support for Python and automated machine learning to Oracle Autonomous Database. Forthcoming integration with Oracle Cloud Infrastructure Data Science will give data scientists the ability to develop models using open source and scalable in-database algorithms.
• Oracle Cloud Infrastructure Data Catalog. The data catalog provides the ability to discover, find, organize, enrich and trace data assets. It features a built-in business glossary.
• Oracle Big Data Service. This service offers a full Cloudera Hadoop implementation, as well as machine learning for Spark.
• Oracle Cloud SQL. This service gives users the ability to run SQL queries on data in HDFS, Hive, Kafka, NoSQL, and Object Storage.
• Oracle Cloud Infrastructure Data Flow. This fully managed service lets users run Apache Spark applications without deploying or managing infrastructure.
• Oracle Cloud Infrastructure Virtual Machines for Data Science. This service offers preconfigured GPU-based environments for $30 a day.

The post Oracle unleashes cloud-based data science platform appeared first on Artificial Intelligence.

Big data analytics is the use of advanced analytic techniques against very large, diverse data sets that include structured, semi-structured and unstructured data, from different sources, and in different sizes from terabytes to zettabytes.

Big data is a term applied to data sets whose size or type is beyond the ability of traditional relational databases to capture, manage and process the data with low latency. Big data has one or more of the following characteristics: high volume, high velocity or high variety. Artificial intelligence (AI), mobile, social and the Internet of Things (IoT) are driving data complexity through new forms and sources of data. For example, big data comes from sensors, devices, video/audio, networks, log files, transactional applications, web, and social media — much of it generated in real time and at a very large scale.

Analysis of big data allows analysts, researchers and business users to make better and faster decisions using data that was previously inaccessible or unusable. Businesses can use advanced analytics techniques such as text analytics, machine learning, predictive analytics, data mining, statistics and natural language processing to gain new insights from previously untapped data sources independently or together with existing enterprise data.

The life of an enterprise architect is becoming busy and difficult. Before the era of big data, the enterprise architect “only” had to worry about the data and systems within their own data center. However, over the past decade there were revolutionary changes to the way information is used by businesses and how data management platforms support the information available from modern data sources.

Cloud broke down the boundaries of enterprise data centers, with applications housed and data created outside the “four walls” of an organization. This introduced a host of complexities for enterprise architects focused on security, privacy, and control. Mobile influences continued to push data outside the data center. Maintaining data flows to each of those data access points, often as tablets or mobile phones, introduced additional troubles. Incoming data from mobile devices brought new data formats and a flood of information to the enterprise architect.

These alterations in the formats and locations of systems and data created massive change for data-driven organizations who want to develop competitive advantage. That advantage may come in the form of new data sources such as device sensor logs, social media streams, and mobile device geolocation information; create new projects to take advantage of these new data sources; and establish environments with diverse data management platforms to support these efforts.]

New and Exciting Data Sources
The transformations over the past decade mandated and created a range of additional data sources, both inside and outside an organization, for enterprise architects to consider.

External data sources from third-party content, often via cloudbased providers, can change their data structure without notification to downstream organizations using that information. Event log and device sensor information also have variability based on their individual configuration and frequently come in the form of multistructured formats such as XML or JSON. Social data sources created by customers and the general public are based on textual formats and audio/video content. Both text and audio/visual are difficult to store and utilize due to the nature of the information.

Traditional relational data sources are also included in the wave of big data change, but they have their own challenges. Increasingly, the information is coming from outside the data center from third parties or cloud-based implementations of corporate data, which requires enterprise architects to seek out and learn how to utilize that information.

Business-oriented data consumers can manage their own pursuits, which can take the form of data discovery or exploratory activities to find new uses for big data sources. They can be analytical projects to align costs via cost-management activities, or advanced analytical projects to find the next set of attributes for high-revenue customers.

This is not a one-off implementation like a “spreadmart” or an unmanaged shadow IT project. Instead, it is a supervised and administered environment strategically provided to business stakeholders so they can meet their own needs while utilizing corporate assets. This may be in the form of new big data sources and platforms, and capturing valuable metadata that can be utilized across the organization.

Enterprise architects create self-service capabilities for big data. By designing and implementing application environments with configurable software components, empowering technologists through skills development, and actively sanctioning interaction between technology implementation teams and the various lines of business, enterprise architects provide environments where business stakeholders can have requirements met at the “speed of business” rather than the speed of an implementation backlog. The business can focus on how to best use new big data resources without waiting on tactical IT workflows while maintaining the implementation components for distribution and continued development across the organization.

The post Making Sense Of Big Data appeared first on Artificial Intelligence.