Topic identification, topic modelling

IA161 Advanced NLP Course?, Course Guarantee: Aleš Horák

Prepared by: Jirka Materna

State of the Art

Topic modeling is a statistical approach for discovering abstract topics hidden in text documents. A document usually consists of multiple topics with different weights. Each topic can be described by typical words belonging to the topic. The most frequently used methods of topic modeling are Latent Semantic Analysis and Latent Dirichlet Allocation.


  1. David M. Blei, Andrew Y. Ng, and Michael I. Jordan. Latent Dirichlet Allocation. Journal of Machine Learning Research, 3:993 – 1022, 2003.
  2. Yee W. Teh, Michael I. Jordan, Matthew J. Beal, and David M. Blei. Hierarchical Dirichlet processes . Journal of the American Statistical Association, 101:1566 – 1581, 2006.
  3. S. T. Dumais, G. W. Furnas, T. K. Landauer, S. Deerwester, and R. Harshman. Using Latent Semantic Analysis to Improve Access to Textual Information. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’88, pages 281–285, New York, NY, USA, 1988. ACM. ISBN 0-201-14237-6.
  4. Castellanos, A., Juan Cigarrán, and Ana García-Serrano. "Formal concept analysis for topic detection: a clustering quality experimental analysis." Information Systems 66 (2017): 24-42.
  5. K. H. Lim, S. Karunasekera and A. Harwood, "ClusTop?: A clustering-based topic modelling algorithm for twitter using word networks," 2017 IEEE International Conference on Big Data (Big Data), Boston, MA, 2017, pp. 2009-2018, doi: 10.1109/BigData.2017.8258147.
  6. Xie, Pengtao, and Eric P. Xing. "Integrating document clustering and topic modeling." arXiv preprint arXiv:1309.6874 (2013).

Practical Session

In this session we will use Gensim to model latent topics of Wikipedia documents. We will focus on Latent Semantic Analysis and Latent Dirichlet Allocation models.

  1. Gensim is already installed on and it also offers faster model processing.
  2. Download and extract the corpus of Czech Wikipedia documents: wiki corpus.
  3. Train LSA and LDA models of the corpus for various numbers of topics using Gensim. You can use this template:
  4. For both LSA and LDA select the best model (by looking at the data or by computing perplexity of a test set for LDA).
  5. Select 5 most important topics with 10 most important words, give them a name, save it into a text file and upload it into odevzdavarna.

You can save the files in your home directory on NLP computers and they will be accessible on the server.

Last modified 3 years ago Last modified on Aug 31, 2021, 2:11:21 PM

Attachments (1)

Download all attachments as: .zip