Version 5 (modified by Zuzana Nevěřilová, 5 years ago) (diff)


Named Entity Recognition

IA161 Advanced NLP Course, Course Guarantee: Aleš Horák

Prepared by: Zuzana Nevěřilová

State of the Art

NER aims to recognize and classify names of people, locations, organizations, products, artworks, sometimes dates, money, measurements (numbers with units), law or patent numbers etc. Known issues are ambiguity of words (e.g. May can be a month, a verb, or a name), ambiguity of classes (e.g. HMS Queen Elisabeth can be a ship), and the inherent incompleteness of lists of NEs.

Named entity recognition (NER) is used mainly in information extraction (IE) but it can significantly improve other NLP tasks such as syntactic parsing.

Example from IE

In 2003, Hannibal Lecter (as portrayed by Hopkins) was chosen by the American Film Institute as the #1 movie villain.

Hannibal Lecter <-> Hopkins

Example concerning syntactic parsing

Wish You Were Here is the ninth studio album by the English progressive rock group Pink Floyd.


Wish_You_Were_Here is the ninth studio album by the English progressive rock group Pink Floyd.


  1. David Nadeau, Satoshi Sekine: A survey of named entity recognition and classification. In Satoshi Sekine and Elisabete Ranchhod (eds.) Named Entities: Recognition, classification and use. Lingvisticæ Investigationes 30:1. 2007. pp. 3–26
  2. Charles Sutton and Andrew McCallum: An Introduction to Conditional Random Fields. Foundations and Trends in Machine Learning 4 (4). 2012.

Practical Session

Czech Named Entity Recognition

In this workshop, we train a new NER application for the Czech language. We work with free resources & software tools: the Czech NE Corpus (CNEC) and the Stanford NER application.

Requirements: Java 8, python, gigabytes of memory

  1. Create <YOUR_FILE>, a text file named ia161-UCO-03.txt where UCO is your university ID.
  2. get the data: download CNEC from LINDAT/Clarin repository (
  3. open the NE hierarchy: acroread cnec1.1/doc/ne-type-hierarchy.pdf
  4. the data is organized into 3 disjoint datasets: the training data is called train, the development test data is called dtest and the final evaluation data is called etest.
  5. convert the train data to the Stanford NER format: python named_ent_train.xml > named_ent_train.tsv. Note that we removed documents that did not contain NEs. You can experiment with this option later.
  6. download the Stanford NE recognizer (and read about it)
  7. train the model using the default settings (cnec.prop), N.B. that the only recognizes PERSON, LOCATION and ORGANIZATION, you can extend the markup conversion later: java -cp stanford-ner.jar -prop cnec.prop
  8. convert the test data to the Stanford NER format: python named_ent_dtest.xml > named_ent_dtest.tsv
  9. evaluate the model on dtest:

java -cp stanford-ner.jar -loadClassifier cnec-3class-model.ser.gz -testFile named_ent_dtest.tsv. You should see results like:

CRFClassifier tagged 12120 words in 441 documents at 8145.16 words per second.
         Entity	P	R	F1	TP	FP	FN
       LOCATION	0.7962	0.7849	0.7905	332	85	91
   ORGANIZATION	0.7059	0.6019	0.6497	192	80	127
         PERSON	0.8062	0.8592	0.8319	470	113	77
         Totals	0.7814	0.7711	0.7763	994	278	295

In the output, the first column is the input tokens, the second column is the correct (gold) answers. Observe the differences.

  1. evaluate the model on dtest with only NEs that are not present in the train data: java -cp stanford-ner.jar -loadClassifier cnec-3class-model.ser.gz -testFile named_ent_dtest_unknown.tsv
  2. test on your own input: java -mx600m -cp stanford-ner.jar -loadClassifier cnec-3class-model.ser.gz -textFile sample.txt

(optional) 12. try to improve the train data suggestions: set useKnownLCWords to false, add gazetteers, remove punctuation, try to change the wordshape (something following the pattern: dan[12](bio)?(UseLC)?, jenny1(useLC)?, chris[1234](useLC)?, cluster1) or word shape features (see the documentation) (optional) 13. evaluate the model on dtest, final evaluation on etest

Attachments (1)

Download all attachments as: .zip