Why has data analytics become so critical in the current times, particularly for the payments industry?

Data analytics help tapping the power of data by using tools and technologies to find patterns that yield insights. It’s these insights that businesses and governments find truly actionable, especially in uncharted times when there is no rear view or past trends. Its problem-solving prowess becomes an essential navigational tool.

The payments industry generates lots of data that is helpful and important for governments and businesses to make informed decisions for fraud prevention, risk exposure assessment, improved customer service, better customer targeting and top channel performance. In the current data-driven era, analytics can support businesses optimise, streamline and grow, as well as deliver value to consumers during and post-Covid.

How is Mastercard all set to leverage big data analytics for business in a post-Covid world?

At Mastercard, our analytics platforms enable organisations globally to make faster and better business decisions based on real-time, anonymised and aggregated transaction data and proprietary analysis.

We generate insights through a comprehensive array of software platforms and services that provide solutions to the core challenges faced by the businesses today. We enable customers to innovate strategically and distil insights from this data through software platforms and industry expertise, driving strong consumer connections.

Today, we are working closely with various banks, governments and businesses by making insight-driven tools available to them to give a timely snapshot of economic performance and help them make informed decisions critical to the long-term success of companies and communities around the world.

To cite an example, when a quick-service restaurant chain in Asia Pacific was at a financial loss, data based insights allowed the restaurant chain to focus resources on those outlets that had a better chance of rebounding with aligned timings and menu options of work-from-home habits and tailored promotions for larger transaction volumes.

In another instance, we leveraged our data driven proprietary platforms to help an international airline that saw bookings drop 60 percent and continued to fall precipitously toward an overall drop of 90 percent across the entire travel and hospitality sector in the airline’s country of origin. With relevant insights on price reduction, trip duration, the immediacy of departure and fare type, the airline saw a potential 30 percent increase in international bookings resulting in a 25 percent increase in revenue on economy routes over control routes. With a better understanding of how fare conditions influence uptake, the airline is now manoeuvring further as more travel routes open.

What are your key implementations and how is it contributing in providing seamless services to your customers?

As I mentioned above, our analytics platforms enable organisations globally to make faster and better business decisions based on real-time, anonymised and aggregated transaction data and proprietary analysis by:
• Providing customers across industries and geographies with a tailored portfolio of solutions to address pain points across their businesses
• Harnessing the power of anonymized and aggregated transaction data, analytics and expertise to create global, actionable insights, enable more intelligent decisions and drive predictive capabilities
• Prioritising customer-centricity and the user experience by delivering convenience: speed, ease of use and personalisation

To cite an example; our application Intelligent Targeting helps boost efficiency, effectiveness and ease of acquisition by leveraging insights and expertise to design, execute and optimise acquisition campaigns for high-value customers and Business Locator provides the most accurate, up-to-date view of Mastercard-accepting merchants open for business on any given day.

Similarly, Acquirer Intelligence Center enables pre-defined analytics on portfolio performance across volume, fraud, and authorisation, compared to custom benchmarks for a full view of business performance and actionable insights.

In the last few months, we have been committed to leverage this expertise and help retailers, restaurants, consumer brands and many others navigate the challenges of the pandemic. Our customers need our services more than they ever needed as they need to act fast while they are taking multiple recovery actions across the globe. Since cross border travel has been hit the hardest in this pandemic, we are trying to help the customers on gradually building up domestic solutions as alternatives before the international travel across the world resumes.

For instance, we used data analytics on transactional data to help a financial institution count and counter the Covid-19 impact with a report outlining year-over-year shifts in spending broken down by day relevant subcategories was provided along with a quantitative financial assessment on clients’ businesses. This was followed by drawing up likely scenarios in terms of mitigating actions and growth opportunities.

We recently introduced a portal, www.shopopenings.com in UK to help people and businesses manage the transition from the lockdown by providing searchable information about merchant establishments that have re-opened. This is based on successful Mastercard card transactions at the relevant stores within the last 48 hours and up to the previous seven days.

Another instance, when an Asian country wanted to understand the impact of the lockdown on its economy it chose to analyse transaction data. Changing consumption patterns indicated the sectors that had been most affected.

Please share how you are setting high benchmarks in terms of data security and safety ? If you can give some examples.

We live in an increasingly interconnected world. On an average, a household has around 8 connected devices and an organisation has 100s. The attack surface that a cybercriminal can exploit grows with every device and so does the complexity of managing cyber security.

Mastercard strives to deliver best in class user experience that is safe, secure and fits well with customer’s needs. We have been leveraging latest mobile and AI tools to build an environment with highest levels of security. Our latest acquisition, RiskRecon, can monitor the cybersecurity performance of organisations using open-source intelligence by deploying passive, non-interfering techniques to discover organisation’s public systems and to analyse the cybersecurity risk posture of those systems. These scans can be completed easily without any technical help without any access to data or integration with existing systems. This helps in increasing the frequency of the scans without any disruption to business. We believe that these capabilities will help organisation identify gaps before the criminals and also address risks with the (aforementioned) expanding scope of cyber security.

Your views / any other significant factor.

We established ‘Data & Services Centre of Excellence’ in India, in 2013, to support Mastercard foreign group entities. The aforesaid support includes providing of data insights and strategic payment solutions by identifying spending trends derived from the billions of anonymous transactions processed by Mastercard every year.

Our Centre has recruited qualified analytics talent with payments industry, retail, technology and media experience to provide hands-on support to Mastercard foreign group entities on custom analytics.

The COE provides data insights by analysing spending trends derived from the 73 billion anonymous transactions processed by Mastercard and specific customer shared data elements, every year. Along with insights and analytics, this group also supports in data driven consulting.

New-age technology is witnessing a steady growth in India. Keeping that in mind, we (under the guidance and instructions of the Mastercard foreign group entities) established Artificial Intelligence Garage at the start of last year with an aim to enhance existing solutions as well as create new ones. With this, we embarked on an expansion and reskilling programme focused on AI where we are hiring experienced AI professionals as well as fresh computer science graduates and training them in this new-age technology.

The post Mastercard leveraging big data analytics for business in a post-Covid world appeared first on Artificial Intelligence.

Instead of relying on customers to protect their vulnerable smart home devices from being used in cyberattacks, Ben-Gurion University of the Negev (BGU) and National University of Singapore (NUS) researchers have developed a new method that enables telecommunications and internet service providers to monitor these devices.

According to their new study, the ability to launch massive DDoS attacks via a botnet of compromised devices is an exponentially growing risk in the Internet of Things (IoT). Such attacks, possibly emerging from IoT devices in home networks, impact the attack target, as well as the infrastructure of telcos.

“Most home users don’t have the awareness, knowledge, or means to prevent or handle ongoing attacks,” says Yair Meidan, a Ph.D. candidate at BGU. “As a result, the burden falls on the telcos to handle. Our method addresses a challenging real-world problem that has already caused challenging attacks in Germany and Singapore, and poses a risk to telco infrastructure and their customers worldwide.”

Each connected device has a unique IP address. However, home networks typically use gateway routers with NAT functionality, which replaces the local source IP address of each outbound data packet with the household router’s public IP address. Consequently, detecting connected IoT devices from outside the home network is a challenging task.

The researchers developed a method to detect connected, vulnerable IoT models before they are compromised by monitoring the data traffic from each smart home device. This enables telcos to verify whether specific IoT models, known to be vulnerable to exploitation by malware for cyberattacks are connected to the home network. It helps telcos identify potential threats to their networks and take preventive actions quickly.

By using the proposed method, a telco can detect vulnerable IoT devices connected behind a NAT, and use this information to take action. In the case of a potential DDoS attack, this method would enable the telco to take steps to spare the company and its customers harm in advance, such as offloading the large volume of traffic generated by an abundance of infected domestic IoT devices. In turn, this could prevent the combined traffic surge from hitting the telco’s infrastructure, reduce the likelihood of service disruption, and ensure continued service availability.

“Unlike some past studies that evaluated their methods using partial, questionable, or completely unlabeled datasets, or just one type of device, our data is versatile and explicitly labeled with the device model,” Meidan says. “We are sharing our experimental data with the scientific community as a novel benchmark to promote future reproducible research in this domain.” This dataset is available here.

This research is a first step toward dramatically mitigating the risk posed to telcos’ infrastructure by domestic NAT IoT devices. In the future, the researchers seek to further validate the scalability of the method, using additional IoT devices that represent an even broader range of IoT models, types and manufacturers.

“Although our method is designed to detect vulnerable IoT devices before they are exploited, we plan to evaluate the resilience of our method to adversarial attacks in future research,” Meidan says. “Similarly, a spoofing attack, in which an infected device performs many dummy requests to IP addresses and ports that are different from the default ones, could result in missed detection.”

The post New defense method enables telecoms, ISPs to protect consumer IoT devices appeared first on Artificial Intelligence.

The global Big Data Analytics (BDA) market will witness double-digit growth in the post-pandemic COVID-19 era, according to new research from Frost & Sullivan.

The analyst firm’s new report, Post-pandemic Growth Opportunity Analysis of the Big Data Analytics Market examines what global markets could look like under the scenario of the COVID-19 virus being contained by August 2020, and that global markets will be able to recover by the end of the year.

In that scenario, Frost & Sullivan forecasts the market is to expand at a compound annual growth rate (CAGR) of 28.9%, reaching $68.09 billion by 2025 from $14.85 billion in 2019. 

Under the conservative forecast scenario, the market is likely to reach $41.84 billion by 2025, at a CAGR of 18.8%. 

Depending on the development and availability of a vaccine, the conservative forecast includes a market slowdown and recovery period of 18 to 24 months.

“BDA use will continue to grow because it is currently being utilised to manage, diagnose, and develop a cure for COVID-19,” says Deviki Gupta, Information & Communication Technologies senior industry analyst at Frost & Sullivan. 

“Additionally, considering the benefits of BDA solutions in both the government and intelligence (G&I) and non-governmental organisation (NGO) sectors, there will be an increase in demand for analytics as it has promising features, such as mitigating risk in business planning, improving operations, and better serving customer needs,” he says.

“Despite the current crisis, BDA continues to be among the top three deployment priorities for enterprises, after data security and replacing legacy systems. 

“Further, major competitive factors will include a vendors ability to serve advanced use cases and its ability to play a consultative role for customers, such as helping them better understand their hardware and software needs to achieve these use cases in both BDA market segments data discovery and visualisation (DDV) and advanced analytics (AA).”

According to the report, the lengthening of sales cycles and decrease in customer spending on BDA solutions as budgets are frozen or diverted to meet urgent operational needs are likely to restrain the growth of the BDA market. 

However,  Frost & Sullivan says market participants should consider the following key growth opportunities:

• Organisations aim to reduce latency and take timely action, especially during the pandemic, will encourage vendors to focus more on intuitive searchability powered by a deeper metadata schema and machine learning. 
• Increasing BDA investment in disaster preparedness, operational management, and diagnostic use cases across the globe will create exponential growth prospects for vendors. 
• Big Data, cloud computing, and consumer demand for personalised and context-based services are driving the implementation of AI/ML applications. 
• APAC presents continued high potential for growth with the increasing use of BDA, AI, and robotics in China, Singapore, Taiwan, and Japan to contain COVID-19.

The post Double digit growth expected for big data analytics market – report appeared first on Artificial Intelligence.

The debate between cloud and edge computing strategies remains a point of contention for many controls engineers in the packaging industry. However, most agree that smart factories in an Industry 4.0 context must efficiently collect, visualize and analyze data from machines and production lines to enhance equipment performance and production processes. Using advanced analytics algorithms, companies can sift through this mass of information, or Big Data, to identify areas for improvement.

To some, edge computing devices may seem to create an unnecessary step when all data can simply be handled in the cloud. Microsoft Azure, Amazon Web Services (AWS) and other cloud platforms offer limitless space for this purpose. Moreover, MQTT encryption and data security built into the OPC UA protocol ensure that all data remain secure while in transport. When it comes to analytics and simple data management, however, edge computing presents important advantages to closely monitor equipment health and maximize uptime in production.

Because of the massive amount of data that modern machines can produce, bandwidth can severely limit cloud computing or push costs to unacceptable levels. New software solutions for PC-based controllers, such as TwinCAT Analytics from Beckhoff, allow controls engineers to leverage advanced algorithms locally in addition to data pre-processing and compression. As a result, a key advance in analytical information is the concept to process data on the edge first, which enables individual packaging machines and lines to identify inefficiencies on their own and make improvements before using the cloud for further analysis across the enterprise.

Bandwidth Burdens when Streaming Machine Data

Performing Big Data analytics in the cloud exclusively often proves expensive in terms of storage space. However, the more difficult proposition is first getting your data there. Managing bandwidth can create a serious issue for factories, since the average Ethernet connection speed across the globe is 7.2 Mbps, according to the most recent connectivity report from Akamai.

When one machine sends data to the cloud, much less multiple machines, little to no bandwidth is available for the rest of the operation. Two use cases published in a 2017 article by Kloepfer, Koch, Bartel and Friedmann illustrate this point. In the first, the structural dynamics of wind turbines using 50 sensors at a 100-hertz sampling rate required 2.8 Mbps bandwidth for standard JavaScript Object Notation (JSON) to stream all data to the cloud. The second case, condition monitoring of assets in intralogistics, used 20 sensors at a 1,000 hertz sampling rate and required 11.4 Mbps JSON. This is quite a relevant test as JSON is a common format to send data to the cloud or across the web.

Without compression or pre-processing mechanisms, an average 7.2 Mbps Internet connection can’t stream data from three or more large machines that require advanced measurement, condition monitoring and traceability of production. A factory must use a connection that is much larger than normal or multiple connections, or it can leverage advanced analytics on the edge.

Edge Devices and Advanced Algorithms

In the past, most programmable logic controllers (PLCs) were capable of handling repetitive tasks in machines, but possessed the computing prowess of a smart toaster. Today’s Industrial PCs (IPCs) feature ample storage and powerhouse processors, such as the Intel® Core™ i7 or Intel® Xeon® offerings, with four or as many as 40 cores. TwinCAT 3 automation software, for example, offers a complete IPC platform that runs alongside Windows, easily supports third-party applications and enables remote access. Most importantly, PC-based control software can provide advanced algorithms to manage data, such as pre-processing, compression, measurement and condition monitoring. This doesn’t require a separate, stand-alone software platform.

Condition monitoring performs many operations locally, such as converting raw accelerometer data into the frequency domain. This can be done on an edge device or within the actual machine controllers’ PLC program. When analyzing vibration, for example, the information is often collected as a 0-10 volt or 4-20 mA signal. This can be changed to a more usable format on the controller through a Fast Fourier Transform (FFT) algorithm. More extensive evaluations of machine vibrations are possible using DIN ISO 10816-3. To monitor bearing life and other specific components, algorithms are readily available to add to a PLC program for calculating the envelope spectrum first and then the power spectrum. Many common machine conditions and predictive maintenance algorithms can be evaluated within the machine control, or on an edge device.

To optimize a Big Data strategy from the ground level upwards, automation software should offer built-in algorithms to process both deterministic and stochastic data. If the data is deterministic, controllers using pre-processing algorithms could send certain values only upon a change, so the recipient should know the mathematic correlation and be able to reconstruct the original signal if desired. For stochastic data, the controller can send statistical information, such as the average value. Although the original signal is unknown, the recipient can still use compressed, statistical information.

It also is possible to implement algorithms on the IPC to monitor process data over a set sequence. This includes writing input data periodically, according to a configured number of learned points, to a file or to a database. After storing standard values, such as torque for a motion operation, algorithms compare cycle values against them. Ensuring the data are within a configured bandwidth creates a type of process window monitoring, which can readjust immediately since the local controller reacts in real-time.

Big Data on Both Edge and Cloud

Running advanced algorithms on a local edge device reduces cloud bandwidth requirements and offers an efficient solution for process optimization guided by Big Data. However, that does not mean a packaging plant should disconnect from the cloud. In the age of IIoT, it is essential to gather and easily access data across an operation, even if many analysis and decision-making tasks can be completed on local hardware first.

To decide what needs to be sent to the cloud and what can be processed or pre-processed locally, make sure to ask a few key questions. First, what are the goals your operation wants to achieve through data acquisition in this instance? Next, which data sets from which machines need to be analyzed in order to achieve these goals? Finally, what types of data insights does the operation need to improve efficiency and profitability?

Local monitoring with edge computing often works most efficiently to improve the operation of individual machines. However, the cloud provides the best platform to compare separate machines, production lines or manufacturing sites against each other. Implementing both allows an operation to maximize its Big Data strategies.

The post Augment Big Data Strategy with Advanced Analytics appeared first on Artificial Intelligence.

Some of the biggest challenges with data management and analytics efforts is security.

Databricks, based in San Francisco, is well aware of the data security challenge, and recently updated its Databricks’ Unified Analytics Platform with enhanced security controls to help organizations minimize their data analytics attack surface and reduce risks. Alongside the security enhancements, new administration and automation capabilities make the platform easier to deploy and use, according to the company.

Organizations are embracing cloud-based analytics for the promise of elastic scalability, supporting more end users, and improving data availability, said Mike Leone, a senior analyst at Enterprise Strategy Group. That said, greater scale, more end users and different cloud environments create myriad challenges, with security being one of them, Leone said.

“Our research shows that security is the top disadvantage or drawback to cloud-based analytics today. This is cited by 40% of organizations,” Leone said. “It’s not only smart of Databricks to focus on security, but it’s warranted.”

He added that Databricks is extending foundational security in each environment with consistency across environments and the vendor is making it easy to proactively simplify administration.As organizations turn to the cloud to enable more end users to access more data, they’re finding that security is fundamentally different across cloud providers.Mike LeoneSenior analyst, Enterprise Strategy Group

“As organizations turn to the cloud to enable more end users to access more data, they’re finding that security is fundamentally different across cloud providers,” Leone said. “That means it’s more important than ever to ensure security consistency, maintain compliance and provide transparency and control across environments.”

Additionally, Leone said that with its new update, Databricks provides intelligent automation to enable faster ramp-up times and improve productivity across the machine learning lifecycle for all involved personas, including IT, developers, data engineers and data scientists.

Gartner said in its February 2020 Magic Quadrant for Data Science and Machine Learning Platforms that Databricks Unified Analytics Platform has had a relatively low barrier to entry for users with coding backgrounds, but cautioned that “adoption is harder for business analysts and emerging citizen data scientists.”

Bringing Active Directory policies to cloud data management

Data access security is handled differently on-premises compared with how it needs to be handled at scale in the cloud, according to David Meyer, senior vice president of product management at Databricks.

Meyer said the new updates to Databricks enable organizations to more efficiently use their on-premises access control systems, like Microsoft Active Directory, with Databricks in the cloud. A member of an Active Directory group becomes a member of the same policy group with the Databricks platform. Databricks then maps the right policies into the cloud provider as a native cloud identity.

Databricks uses the open source Apache Spark project as a foundational component and provides more capabilities, said Vinay Wagh, director of product at Databricks.

“The idea is, you, as the user, get into our platform, we know who you are, what you can do and what data you’re allowed to touch,” Wagh said. “Then we combine that with our orchestration around how Spark should scale, based on the code you’ve written, and put that into a simple construct.”

Protecting personally identifiable information

Beyond just securing access to data, there is also a need for many organizations to comply with privacy and regulatory compliance policies to protect personally identifiable information (PII).

“In a lot of cases, what we see is customers ingesting terabytes and petabytes of data into the data lake,” Wagh said. “As part of that ingestion, they remove all of the PII data that they can, which is not necessary for analyzing, by either anonymizing or tokenizing data before it lands in the data lake.”

In some cases, though, there is still PII that can get into a data lake. For those cases, Databricks enables administrators to perform queries to selectively identify potential PII data records.

Improving automation and data management at scale

Another key set of enhancements in the Databricks platform update are for automation and data management.

Meyer explained that historically, each of Databricks’ customers had basically one workspace in which they put all their users. That model doesn’t really let organizations isolate different users, however, and has different settings and environments for various groups.

To that end, Databricks now enables customers to have multiple workspaces to better manage and provide capabilities to different groups within the same organization. Going a step further, Databricks now also provides automation for the configuration and management of workspaces.

Delta Lake momentum grows

Looking forward, the most active area within Databricks is with the company’s Delta Lake and data lake efforts.

Delta Lake is an open source project started by Databrick and now hosted at the Linux Foundation. The core goal of the project is to enable an open standard around data lake connectivity.

“Almost every big data platform now has a connector to Delta Lake, and just like Spark is a standard, we’re seeing Delta Lake become a standard and we’re putting a lot of energy into making that happen,” Meyer said.

Other data analytics platforms ranked similarly by Gartner include Alteryx, SAS, Tibco Software, Dataiku and IBM. Databricks’ security features appear to be a differentiator.

The post Databricks bolsters security for data analytics tool appeared first on Artificial Intelligence.

Fresh off raising an additional $7 million in funding, Concentric has launched a tool that employs deep learning algorithms to enable cybersecurity teams to identify documents and repositories where sensitive data has been stored.

Company CEO Karthik Krishnan said the Semantic Intelligence platform takes the place of relying on users to remember where documents containing personally identifiable information (PII) data might be stored, for example. The technology takes less than 30 minutes to install and about a week to learn the overall environment. Once the deep learning algorithms are trained to identify sensitive data, cybersecurity teams can focus their efforts on securing the documents and repositories where that data is stored, he said.

Krishnan said Concentric’s deep learning algorithms, also known as neural networks, have already been employed by beta customers to find millions of unprotected or inappropriately shared documents accessible by thousands of employees.

A business on average has nearly 10 million documents, he said, with 1.2 million documents deemed business-critical. Of those business-critical documents, more than 15% are at risk because of improper sharing with users and groups or inadequate/incorrect data classification. In addition to the inherent cybersecurity risks those documents represent, Krishnan noted organizations could be fined millions of dollars for breaching any number of compliance mandates.

Cybersecurity and compliance teams have wrestled with data classification issues for decades. Most processes are deeply flawed because they rely on end users to determine how sensitive a document might be. Over time, the number of documents that are misclassified even though they may include, for example, Social Security numbers starts to multiply. Rather than rely on end users, Concentric is making the case for employing artificial intelligence (AI) in the form of deep learning algorithms to determine the appropriate level of classification for any document and identify which policies should be applied, said Krishnan.

The existence of an AI tool to classify data should not absolve end users of the responsibility to classify data. However, given that people make mistakes or simply forget, an AI tool will enable cybersecurity and compliance teams to enforce policies at a more granular level without having to disrupt the business. That capability is becoming a fundamental requirement as regulations such as the General Data Protection Rule (GDPR) and California Consumer Privacy Act (CCPA) raise the stakes involving almost any type of data breach or even accidental sharing of sensitive data.

While there’s obviously a lot of hype surrounding AI these days, it’s arguably in the realm of rote tasks where algorithms can prove most effective. The more narrowly focused the task, the more accurate algorithms tend to become. Of course, the more the humans employed to train algorithms know about a process, the faster the AI system tends become effective. The challenge now is cutting through all the hype to identify practical use cases for AI. Arguably, data classification along with other data security and data management functions make an ideal area of initial focus.

The post Concentric Applies Deep Learning Algorithms to Data Security appeared first on Artificial Intelligence.

Data, data, data. As our lives get evermore entwined into the world of digital, so the importance of data skyrockets: the importance of acquiring it, storing it and – crucially – interpreting it. In fact, last year reports abound that so important is data to us now, it has become a more valuable resource than oil…

And while that may remain open to debate, it highlights the inescapable truth that data is crucial to every day life.

So, if you are thinking about a career change – or looking to start your first – what’s stopping you from considering data science?

“I didn’t decide that I wanted to be a data scientist until a few months before I started working as one,” says Thordis Thorsteins, security data scientist at cyber company Panaseer. “[I] found that skills that I’d gathered through other experiences were very applicable.”

Then there’s the money – it can be a highly lucrative field to work in. Glassdoor, for instance, lists ‘Data Scientist’ as one of its best jobs for 2020, citing the median base salary as £45,188.

Interest sufficiently piqued? Here, Thorsteins explains what data science is and how you can make an entree.

What is data science?

The core of data science is using theory from maths and power from computer science to answer questions. These questions can range from ‘what events this weekend might I be most interested in?’ to ‘which security challenge should my company first focus on?’.

It’s a collection of techniques that can be applied to any domain (that has sufficient data) to see trends and patterns that may not be obvious at all.

When coupled with domain specific knowledge it brings enormous value as the human input helps model the problem correctly, and the data science techniques give us a view on the problem we wouldn’t be able to get without it.

Why is data science an exciting career?

The possibilities are endless and the potential benefits are clear. From helping diagnose diseases correctly, to helping avoid cyber-attacks – the field is not bound to any single industry, so the variety of options is far-reaching.

Something that may not be clear is that there are different types of data scientists – for example ‘product data scientists’ and ‘research scientists’. These different roles involve various different levels of design, working with people that are, and are not, data scientists, and studying the state-of-the-art techniques.

Lastly, the field is constantly evolving, so the possibility to learn is never-ending. It’s an exciting field that people with different backgrounds can bring a lot to.

What skills or qualifications do you need?

In my opinion, the key ingredients when starting out are a problem-solving mindset and practicality. With time and experience you will pick up coding skills (mainly what open source projects to use) and learn practical things about how to work with data, but these skills are easier to pick up as you go.

Other skills that come with experience are learning to balance what can be done with what brings value, learning to ask the right questions so that a computer can answer it and preventing pitfalls like overlooking bias in data or ways to mis-use your solution.

Many people in the field come from a STEM background, but this is by no means the only way to get into data science. The online resources are plentiful and there is a big community of meet-up groups that will help anyone interested get up to speed.

Is it too late to get started at 30, 40, 50 plus – if not, how might you go about it?

It’s never too late to get started in data science. The resources available online are of high quality and very extensive which means that upskilling (regardless of age or previous experience) is a lot more manageable than it might seem.

It is worth reaching out to someone you look up to in the field for specific advice if you feel like you’ll be out of place, but I can assure you that additional experience in other areas comes in surprisingly useful.

You are likely to have many things to offer that people who started earlier in the field do not and you should embrace that.

Is it true women are under-represented in data science?

There is no reason why they should be, but this is sadly still the case. It’s worth noting that it’s not only women that are under-represented in data science, and efforts should be made to increase representation from all groups.

I think the under-representation is a bit of a chicken and egg situation. The fact that there is a lack of women in field in some cases discourages women that are interested and means that people that are in or hiring into the field don’t realise how to make the workplace a good one for people of all genders, and this is turn means that the lack of women persists.

What can be done to change this?

I think there are a variety of things. As someone involved in hiring, you can advertise open posts on sites that encourage diversity and eliminate discrimination.

You should make sure that the language in your job ads isn’t biased and that you don’t exaggerate experience and qualifications needed for a job (it’s rare to truly need 10 years of A.I. experience, a PhD and publications, and research shows that women are less likely to apply for positions they don’t think they meet all the requirements for than men are).

As someone in data science, you can offer to mentor individuals from underrepresented groups that want help getting started in the field and share achievements of, for example, women around you to help other women find role models. As a member of society, you should be mindful of your language and assumptions, and encourage individuals around you.

A good thing to do is to follow communities that encourage diversity to learn how you can help drive positive change.

What trends do you see moving into 2020?

I think the data science community is going to put more focus on data ethics in 2020. This was a hot topic in 2019, and I think we’re getting better at spotting and pointing out downfalls that make data science solutions unfair to some groups.

I believe people are going to be more mindful of this when developing solutions in 2020 because of the raised visibility of the issue. I also believe that the data science community will become better at clearly communicating necessary details about data science and how it works to data science consumers.​

For instance, while not all the technical details are relevant for the average person who wants to know the quickest way to get to work, they may benefit from knowing that this solution performs best in the morning or doesn’t take into account that you may not want to walk down all streets at night when it’s dark.

Sharing the necessary information in way that’s easy to understand is not an easy task, but I believe that we can do better and this in turn will help us make the solutions better when the users can point out issues.

The post What is data science and how much can you earn as a data scientist? appeared first on Artificial Intelligence.

As firms move towards the adoption of machine learning, Artificial Intelligence (AI) is generating substantial security risks.

One of the most significant risks associated with AI remains the ML-based models operating as “black boxes.” The deep learning models composed of artificial neural networks have complicated the process of deriving automated inferences. These complications increase the risks associated with AI models. ML-based applications may inadvertently get influenced by biases and other adverse factors while producing automated decisions. To mitigate the risks, firms are starting to demand enhanced transparency into how ML operates, focusing on the entire workflow in which models are trained, built, and deployed.

There are many frameworks for maintaining the algorithmic transparency of AI models to ensure explainability, interpretability, and accountability. Business demands flexibility, but IT needs control. This has pushed the need of firms to rely on different frameworks to secure   algorithm transparency. All these tools and techniques assist the data scientists in generating explanations to understand which data inputs drove different algorithmic inferences under various circumstances. However, sadly, these frameworks can be easily hacked, thereby reducing trust in the explanations they generate and exposing the risks they create:

Algorithmic deceptions may sneak into the public record – Dishonest parties may hack the narrative explanations generated by these algorithms to obscure or misrepresent any biases. In other words, “perturbation-based” approaches can be tricked into creating “safe” reasons for algorithmic behaviors that are definitely biased.

Technical vulnerabilities may get disclosed accidentally – Revealing information about machine learning algorithms can make them highly vulnerable to attacks. Complete transparency into how machine learning models function will expose them to attacks designed either to trick the inferences from live operational data or by injecting bogus data into their training workflows.

Intellectual property theft may be encouraged – Entire ML algorithms and training data sets can get stolen through their APIs and other features. Transparency regarding how ML models operate may enable the underlying models to be reconstructed with full reliability. Similarly, transparency will also make it possible to partially or entirely reconstruct training data sets, which is an attack known as “model inversion.”

Privacy violations may run rampant. ML transparency may make it possible for unauthorized third parties to ascertain a particular individual’s data record through a “membership inference attack,” to enable hackers to unlock considerable amounts of privacy-sensitive data.

To mitigate such technical risks of algorithmic transparency, enterprise data professionals need to adhere to the below strategies:

• Firms should have control access to model outputs and monitor to prevent data abuse.
• Add controlled amounts of “perturbations” into the data used to train transparent ML models to make it difficult for adversarial hackers to use model manipulations to gain insight into the original raw data itself.
• Insert intermediary layers between the final transparent ML models and the raw data, making it difficult for an unauthorized third party to recover the full training data from the explanations generated against final models.
• In addition to these risks of a technical nature, enterprises get exposed to more lawsuits and regulatory scrutiny.

Without sacrificing  ML transparency, firms need to have a clear objective of mitigating these broader business risks  .

Enterprises will need to monitor these explanations for irregularities continually, to derive evidence that they or the models have been hacked. This is a critical concern because trust in the AI technology will come tumbling down if the enterprises that build and train ML models can’t vouch for the transparency of the models’ official documentation.

The post Fighting the Risks Associated with Transparency of AI Models appeared first on Artificial Intelligence.

It has been reported that some people are stealing data about tigers in Indian wildlife reserves, raising concerns over data abuse.

Wildlife data tracking used to rely on field observations recorded by experts. However, the lack of standard formats makes it difficult to process such data.

Fortunately, technological advancements in tracking systems, such as the global positioning system (GPS) and other digital sensor devices, have made large, continuous and high-frequency data sets available.

These new technologies have made data collection, storage, evaluation and visualization a lot more easy. However, they have also increased the risk of data misuse.

The importance of information sharing in the push for innovations has been recognized by many people.

For example, geospatial data is essential in wildlife research, particularly in the study of animal behavior, habits and demand. Such research is used in animal conservation and prevention of disease transmission.

But there is always a need to prevent big data abuse while extracting the benefits of data mining.

While some animal monitoring centers have fully opened access to their real-time geospatial data, others only provide a small fraction of their data for fear that the data could be misused.

To solve this problem, researchers at the Massachusetts Institute of Technology have come up with Open Algorithms (OPAL), which makes a broad array of data available for inspection and analysis without violating personal data privacy.

OPAL also allows the repository owner to control raw data: only aggregate answers or “safe answers” are returned. The system can also stipulate that data must be in an encrypted state while being transmitted and during computations.

Data security is the cornerstone for innovative development in big data. I believe technology is the best solution to tackle data security issues, not more stringent regulation.

How to best utilize data resources while protecting individual privacy presents a great challenge, but it is also a great opportunity.

The post Why harnessing big data technology is a delicate balancing act appeared first on Artificial Intelligence.

IT service security has many layers. The IT security layer; firewalls, intrusion detection and access controls. The infrastructure layer; power, network, server health and cooling. And, most important, the people layer. The right people with the right processes, tools and measures to ensure everything else is in working order. Artificial intelligence (AI) will by far have the biggest impact on the tools and measures that people use by amplifying capabilities, streamlining processes and increasing efficiencies.

AI and deep learning will become a necessity in parsing and analyzing the mountain of data generated within a data center to more effectively manage service delivery while mitigating risks like outages. This stems from the recent transformation in how we deliver application workloads.

Too much data?

In the last 10 years, we’ve moved from mostly single server single applications to distributed applications that run in containers. These are now being delivered by micro-services running on-premise and in the cloud–all managed by automation tools. Infrastructure has become part of the application, while other applications have become part of the infrastructure. If you are using a platform like Amazon S3 or Google Maps as an integral component of your service delivery, then you are experiencing this transformation first-hand.

The resulting impact on data center management is significant with power and cooling becoming just a fraction of what needs regular attention. Environmental controls, physical devices, virtual machines and public clouds all need to be monitored and managed round-the-clock to achieve efficiencies in cost and performance. Understanding where and when to move specific workloads becomes paramount.

The amount of data an enterprise collects, monitors and analyzes today to ensure business continuity has exploded. Consider the data generated just from sensors, applications, access control systems, power distribution units, UPS, generators, and solar panels. Add to that external data sources like application vulnerability information, power rates and weather forecasts. Robust data center infrastructure management (DCIM) tools are needed to store all of this data, analyze it and turn it into actionable intelligence. You can try to compartmentalize some of this, but it is becoming increasingly difficult.

AI and deep learning are becoming integral in data center and critical infrastructure management. Here are some of the more notable areas:

• Situational awareness
  Active dashboards with trends, correlations analysis and recommended actions.
• Preventive maintenance
  Deep learning used to identify and correlate data that predicts a failure in power, storage or network connection. This allow operators to mobilize and pro-actively move workloads to safer zones, while maintenance is being performed.
• Root cause analysis
  Machine learning used to trace the failure of several services to a root cause. This becomes learned and used for future preventive maintenance.
• Network security and intrusion detection
  Machine learning and deep neural networks used to spot unusual patterns in application sensors, access control systems and network systems–and provide better signal-to-noise and pro-active mitigations. Learning neural networks are used to continuously improve the enterprise’s security posture and ability to manage related issues.
• Automation
  A “Narrow AI” equipped with various automated mitigation techniques and resulting actions similar to a car applying the brakes if it sees an imminent collision.

Deep neural networks and machine learning algorithms will improve over time, allowing for higher efficiency and performance to match fast growing application workloads. With all of this on the horizon, there’s little doubt that AI will have a massive impact on how enterprises manage their data center.

The post How AI can enhance data center security appeared first on Artificial Intelligence.