Changes between Version 3 and Version 4 of private/NlpInPracticeCourse/LanguageModelling

Timestamp:

Aug 24, 2015, 8:55:54 AM (8 years ago)

Author:

Vít Baisa

Comment:

--

private/NlpInPracticeCourse/LanguageModelling

@@ -5 +5 @@
 Prepared by: Vít Baisa
 
-== State of the Art =
+== State of the Art ==
 
-The goal of language model is to a) predict a following word or phrase based on a given text history and b) assign a probability (=score) to any possible input sentence. This was done mainly by n-gram models known since WWII. But recently, the buzzword deep learning penetrated also into language modelling and it turned out neural networks beat classic n-gram models.
+The goal of a language model is a) to predict a following word or phrase based on a given history and b) to assign a probability (= score) to any possible input sentence. In the past, this was achieved mainly by n-gram models known since WWII. But recently, the buzzword deep learning penetrated also into language modelling and it turned out to be substantially better than Markov's n-gram models.
 
 === References ===
-
-Approx 3 current papers (preferably from best NLP conferences/journals, eg. [[https://www.aclweb.org/anthology/|ACL Anthology]]) that will be used as a source for the one-hour lecture:
 
  1. Bengio, Yoshua, et al. "A neural probabilistic language model." The Journal of Machine Learning Research 3 (2003): 1137-1155.
@@ -21 +19 @@
 
 We will build a simple language model (skip-gram) which has very interesting properties. When trained properly, the vectors of words obey simple space arithmetics, e.g.
-vector "king" − vector "man" + vector "woman" ~= vector of "queen".
-We will train this model on a large Czech and English corpora and evaluate the result.
+vector "king" − vector "man" + vector "woman" ~= vector "queen".
+We will train this model on a large Czech and English corpora and evaluate the results.