Researchers will now be able to uncover new connections and make new discoveries using ProQuest’s TDM Studio service, a pioneering end-to-end solution for text and data mining. TDM Studio puts the power of text and data mining directly into the researcher’s hands, from their initial idea to their final output.

With this new solution, creating a content set has been reduced to hours, rather than the months required with traditional approaches. TDM Studio gives researchers the freedom to use the content, methods, and tools they prefer – and to collaborate on projects both within and outside their university.

TDM Studio unlocks a vast collection of current and historical ProQuest content (including news, journals dissertations and theses, primary sources and more) for TDM. Researchers also have the option to incorporate content from other sources, and to utilize their preferred methods with open source programming languages such as R and Python – along with methods provided by ProQuest – for analysis and visualization.

Transport and Logistics IoT Data is Creating a New Wave of Possibilities

“The use of text and data mining in academia is enabling researchers in all disciplines to make breakthroughs that have never before been possible,” said Mindy Pozenel, Director of Product Management for ProQuest TDM Studio. “With its flexibility and ease of use, TDM Studio helps researchers bypass the cumbersome mechanics of text and data mining and get straight to the point: answering their research questions. TDM Studio has also been designed as an ideal solution for teaching and learning.”

Librarians can use TDM Studio to further leverage their existing wealth of content, creating more ways to partner with research teams and enhance teaching and learning.

Caleb Rawson, Assistant Professor of Accounting at the University of Arkansas, is using TDM Studio for a project that analyzes how public firms communicate information and how that information is reported in the media. “One of my greatest research challenges is working with a corpus of millions of news articles, organizing the relevant content and matching it to other data,” he said. “TDM Studio is helping me find patterns in data in a way that’s consistent, reliable, and in a way that makes sense.”

AI Changing the Society to Make World a Better Place

“The use of text and data mining greatly increases the number of important questions we can answer,” said John Eric Humphries, Assistant Professor of Economics at Yale University, who is using TDM Studio to analyze dissertations for a project related to human capital. “ProQuest has been an amazing partner in setting up a TDM infrastructure that is both powerful and easy to use.”

“In the medical field, we often don’t have the manpower to go through tedious manual processes to analyze data, which can be frustrating,” said Sunmoo Yoon, Associate Research Scientist at Columbia University, who is working on a project that uses Twitter to provide culturally sensitive support to dementia caregivers. “By text mining a large corpus of material, TDM Studio is helping us learn the right words and phrases to help target caregivers with self-care and self-management messages, reducing their risk of loneliness and depression.”

In addition to Yale, Columbia and the University of Arkansas, nine other institutions globally partnered with ProQuest and contributed to the development of TDM Studio. Several early-access customers are already using TDM Studio, and ProQuest is currently recruiting additional early users.

The post ProQuest’s TDM Studio™ Service Transforms Text and Data Mining with Efficiency, Flexibility and Power appeared first on Artificial Intelligence.

Last week, the Python team released NumPy version 1.17.0. This version has many new features, improvements and changes to increase the performance of NumPy.

The major highlight of this release includes a new extensible numpy.random module, new radix sort & timsort sorting methods and a NumPy pocketfft FFT implementation for accurate transforms and better handling of datasets of prime length. Overriding of numpy functions has also been made possible by default.

NumPy 1.17.0 will support Python versions 3.5 – 3.7. Python 3.8b2 will work with the new release source packages, but may not find support in future releases. Python version 2.7 has been officially dropped.

What’s new in NumPy 1.17.0?

New extensible numpy.random module with selectable random number generators

NumPy 1.17.0 has a new extensible numpy.random module. It also includes four selectable random number generators and improved seeding designed for use in parallel processes. PCG64 is the new default numpy.random module while MT19937 is retained for backwards compatibility.

Timsort and radix sort have replaced mergesort for stable sorting

Both the radix sort and timsort have been implemented and can be used instead of mergesort. The sorting kind options ‘stable’ and ‘mergesort’ have been made aliases of each other with the actual sort implementation for maintaining backward compatibility. Radix sort is used for small integer types of 16 bits or less and timsort is used for all the remaining types of bits.

empty_like and related functions now accept a shape argument

Functions like empty_like, full_like, ones_like and zeros_like will now accept a shape keyword argument, which can be used to create a new array as the prototype and overriding its shape also. These functions become extremely useful when combined with the __array_function__ protocol, as it allows the creation of new arbitrary-shape arrays from NumPy-like libraries.

User-defined LAPACK detection order

numpy.distutils now uses an environment variable, comma-separated and case insensitive detection order to determine the detection order for LAPACK libraries. This aims to help users with MKL installation to try different implementations.

.npy files support unicode field names

A new format version of .npy files has been introduced. This enables structured types with non-latin1 field names. It can be used automatically when needed.

New mode “empty” for pad

The new mode “empty” pads an array to a desired shape without initializing any new entries.

New Deprications in NumPy 1.17.0

numpy.polynomial functions warn when passed float in place of int

Previously, functions in numpy.polynomial module used to accept float values. With the latest NumPy version 1.17.0, using float values is deprecated for consistency with the rest of NumPy. In future releases, it will cause a TypeError.

Deprecate numpy.distutils.exec_command and temp_file_name

The internal use of these functions has been refactored for better alternatives such as replace exec_command with subprocess. Also, replace Popen and temp_file_name with tempfile.mkstemp.

Writeable flag of C-API wrapped arrays

When an array is created from the C-API to wrap a pointer to data, the writeable flag set during creation indicates the read-write nature of the data. In the future releases, it will not be possible to convert the writeable flag to True from python as it is considered dangerous.

Other improvements and changes

Replacement of the fftpack based fft module by the pocketfft library

pocketfft library contains additional modifications compared to fftpack which helps in improving accuracy and performance. If FFT lengths has large prime factors then pocketfftuses Bluestein’s algorithm, which maintains O(N log N) run time complexity instead of deteriorating towards O(N*N) for prime lengths.

Array comparison assertions include maximum differences

Error messages from array comparison tests such as testing.assert_allclos now include “max absolute difference” and “max relative difference” along with previous “mismatch” percentage. This makes it easier to update absolute and relative error tolerances.

median and percentile family of functions no longer warn about nan

Functions like numpy.median, numpy.percentile, and numpy.quantile are used to emit a RuntimeWarning when encountering a nan. Since these functions return the nan value, the warning is redundant and hence has been removed.

timedelta64 % 0 behavior adjusted to return NaT

The modulus operation with two np.timedelta64 operands now returns NaT in case of division by zero, rather than returning zero.

Though users are happy with NumPy 1.17.0 features, some are upset over the Python version 2.7 being officially dropped.

The post NumPy 1.17.0 is here, officially drops Python 2.7 support pushing forward Python 3 adoption appeared first on Artificial Intelligence.