private/NlpInPracticeCourse/TopicModelling: Topic_Modeling_with_gensim.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# Topic Modeling with `gensim`"
      ],
      "metadata": {
        "id": "A87--djCFZCr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#import logging, gensim\n",
        "import gensim\n",
        "import pprint"
      ],
      "metadata": {
        "id": "bvU4kUXlEYMN"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Get the data\n",
        "Download text from English Wikipedia. The texts are in form of Matrix Market Corpus (see https://radimrehurek.com/gensim/corpora/mmcorpus.html for more details). The TF-IDF are already calculated."
      ],
      "metadata": {
        "id": "Fnq-K0XBGTmp"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!wget https://nlp.fi.muni.cz/trac/research//chrome/site/bigdata/wiki_en.tar.bz2"
      ],
      "metadata": {
        "id": "DkzzClAhErV7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!bzip2 -d wiki_en.tar.bz2"
      ],
      "metadata": {
        "id": "jtT2eEXEExSh"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!tar -xvf wiki_en.tar"
      ],
      "metadata": {
        "id": "QmEWSOhdGCIX"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KXzXpxGf7ULr"
      },
      "outputs": [],
      "source": [
        "\n",
        "#logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s', level=logging.INFO)\n",
        "\n",
        "# load id->word mapping (the dictionary), one of the results of step 2 above\n",
        "id2word = gensim.corpora.Dictionary.load_from_text('wiki_en/wiki_en_wordids.txt.bz2')\n",
        "\n",
        "# load corpus iterator\n",
        "mm = gensim.corpora.MmCorpus('wiki_en/wiki_en_tfidf.mm')"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Corpus Parameters"
      ],
      "metadata": {
        "id": "kA-yYllwICo-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "len(mm) # num of documents"
      ],
      "metadata": {
        "id": "OEzeR4bnHAk7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "len(id2word) # vocabulary size"
      ],
      "metadata": {
        "id": "uLyV7NnuHK9P"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "id2word[1123] # example mapping between word id and words"
      ],
      "metadata": {
        "id": "U77CVjH-HF0a"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "[(i, len(mm[i])) for i in range(0,20)] # document lengths for n first documents"
      ],
      "metadata": {
        "id": "IyWB629kHaop"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "mm[19] # example document with TF-IDF scores"
      ],
      "metadata": {
        "id": "9FCrOVlzHlzd"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "mm[0][:10]"
      ],
      "metadata": {
        "id": "rQULIYLPQrMm"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "[id2word[t[0]] for t in mm[19]]"
      ],
      "metadata": {
        "id": "rZlsTBuUG3Ca"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "[id2word[t[0]] for t in mm[0][:10]]"
      ],
      "metadata": {
        "id": "rY0KiTSVQyBi"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# extract 10 LSA topics; use the default one-pass algorithm\n",
        "lsa = gensim.models.lsimodel.LsiModel(corpus=mm, id2word=id2word, num_topics=10)\n",
        "pprint.pprint(lsa.show_topics())"
      ],
      "metadata": {
        "id": "7SspSpnXG0bA"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# extract 10 LDA topics, using 1 pass and updating once every 1 chunk (5,000 documents)\n",
        "lda = gensim.models.ldamodel.LdaModel(corpus=mm, id2word=id2word, num_topics=10, update_every=1, chunksize=5000, passes=1)\n",
        "pprint.pprint(lda.show_topics())"
      ],
      "metadata": {
        "id": "H9VqaRrlJMrz"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Coherence Score\n",
        "Resources about different coherence measures:\n",
        "https://datascience.oneoffcoder.com/topic-modeling-gensim.html\n",
        "overview of coherence score implementations:\n",
        "https://github.com/dice-group/Palmetto/wiki/Coherences\n"
      ],
      "metadata": {
        "id": "AmaNGQ_hZyRm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# compute coherence for LDA model\n",
        "cm = gensim.models.coherencemodel.CoherenceModel(model=lda, corpus=mm, coherence='u_mass')\n",
        "print(cm.get_coherence())"
      ],
      "metadata": {
        "id": "XMhx6gISJZcr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "list(zip(range(0,lda.num_topics), cm.get_coherence_per_topic()))"
      ],
      "metadata": {
        "id": "NPmOkDrxmh8w"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "lda.show_topic(3)"
      ],
      "metadata": {
        "id": "o5qToERFm2Kn"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "lda.show_topic(1)"
      ],
      "metadata": {
        "id": "zVxjvDv0nMH5"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}