The Micro-services architecture-based Text Analysis Engine

Main Article Content

Swaswati Dutta
Parvej Saleh
Srivathshan KS


As the hype for social media and various other web applications increases, the need for modern scalable data management has become a sheer need to cope with the huge bulks of data. In the domain of text analytics, the data collected from various social media needs to be collected, cleaned, processed and visualized for providing various insights. This paper presents an
architecture relying on the microservice approach for creating a data management backend for text analysis. The microservices-based data caching, data processing, data analysis, and data visualization methods can be applied to enhance the available data for providing efficient management services to the users. As of 2018, the micro-blogging site Twitter averaged at 321 million monthly active users[1]. Twitter provides a strong emphasis on real-time information. Information relating to geolocation and entities such as author id, author name/id, source, the reaction of people towards an event, etc can be extracted, stored and further analyzed by various Big Data Tools. This work aims at applying machine learning while adopting microservice approach for analyzing events from the Twitter platform in real-time based on keywords and geolocation, and finally, propose a user-friendly visualization based on the data. The tweets are stored and fetched using the Elasticsearch search engine. Indexing and standardizing of the Elasticsearch framework are used for large scale text mining. The results obtained from the query-based search engine are finally visualized using the powerful d3.js library.

Article Details

How to Cite
DuttaS., SalehP., & KSS. (2020). The Micro-services architecture-based Text Analysis Engine. Probyto AI Journal, 1(01). Retrieved from


[2] “Sensing Trending Topics in Twitter” Luca Maria Aiello, Georgios Petkos, Carlos Martin, David Corney,
Symeon Papadopoulos, Ryan Skraba, Ayse Göker, Ioannis Kompatsiaris, Senior Member, IEEE, and Alejandro
[3] S. Petrovic and e. al., "Streaming first story detection with application to Twitter," in Proceedings of the 11th
Annual Conference of the North American Chapter of the Association for Computational Linguistics, 2010
[4]M. Fowler and J. Lewis. Microservices a definition of this new architectural term. URL:
[5] Research on Architecting Microservices: Trends, Focus, and Potential for Industrial Adoption Paolo Di
Francesco , Patricia Lago , Ivano Malavolta Gran Sasso Science Institute, L’Aquila, Italy
Jose Ignacio Fern ´ andez-Villamor, Carlos ´ A. Iglesias, Mercedes Garijo ´ Departamento de Ingenier´ıa de
Sistemas Telematicos, Universidad Polit ´ ecnica de Madrid
[7] A Systematic Mapping Study in Microservice Architecture Nuha Alshuqayran, Nour Ali and Roger Evans
Computing Engineering and Mathematics University of Brighton Brighton, UK
[8] Text Classification made easy with Elasticsearch By Saskia Vola
[10] 2019 Indian general election , Wikipedia, the free encyclopedia.
[11] Fake News Detection on Social Media: A Data Mining Perspective Kai Shu , Amy Sliva , Suhang Wang ,
Jiliang Tang , and Huan Liu.
[12] Measuring User Credibility in Social Media, Mohammad-Ali Abbasi and Huan Liu

Most read articles by the same author(s)

1 2 > >>