Text mining and analytics: how does it work?

Expert System Team - 13 January 2017

Data is being generated in a variety of formats, and at a rate that shows no signs of slowing down. Identifying information that can support decision making among thousands of documents, web pages and social media feeds is a complex and time-consuming process. Because it’s simply too much to read quickly, this information is often ignored or only partially used to support organizational processes.

Text analytics software helps people digest text data more efficiently. Combined with text mining, together they offer the opportunity to discover interesting patterns and the ability to turn text data into actionable knowledge that can be useful for decision making.

But how do text mining and text analytics work?

They can be viewed as the two subsequent steps in a process designed to gain knowledge and support strategic decision making by using the data contained in text. The text data (keywords, concepts, verbs, nouns, adjectives, etc.) are extracted through the text mining process and are then used in the text analysis step to extract insight from the data. Here, it decides which pieces of content need to be further reviewed by people. The text analysis part works in a similar fashion. A text processing engine that has a deep and rich understanding of natural language will usually provide better, more insightful text data analysis. .

For example, let’s consider how text mining and text analytics work for product managers or marketing managers. Each of these roles may be interested in knowing what customers think about their products, and how their products compare with those of competitors. If they can use text mining techniques and software to tap into sources like social media or verbatim text from customer surveys, they can extract this information to improve future releases.

Or, let’s consider how text mining and text analytics can work for scientists. For example, scientists are interested in research topics and newdiscoveries. Because different scientist communities tend to use different terminologies, they face similar issues when it comes to distinguishing the relevant from the irrelevant. Combining the terminology in a single, unique view can speed up discovery of important information. Once again, the result of text mining can be used via text analytics to give a representation of the data to speed up the comprehension of the knowledge extracted.

Here is another simple example of how text mining and text analytics work together. The output of text mining is a list of terms with attributes like frequency or, for more advanced semantic technologies like Cogito, their relative semantic relevancy inside a text or a collection of texts. The analytic parts of the process instead consist of the representation of this list. For example, this could show us clusters of terms based on the semantic vicinity of the words extracted. With tools like our Cogito Intelligence Platform, this representation can be made more interesting using different visual structures, such as bubble graphs, polarization graphs, heat maps, etc. Anyone can use the visual representation to easily navigate to the most relevant documents or sections of the documents to extract knowledge automatically or through further manual analysis.

When people write about how text mining and text analytics work, they tend to consider them as synonyms. Instead, they are two separate pieces of the same process that is designed to extract knowledge from information. Today, most text analytics or cognitive computing software can perform both tasks with an increasing level of effectiveness, and for businesses, that’s what matters most.

Discover Expert System solutions!

Related Reading