Changes between Version 16 and Version 17 of private/NlpInPracticeCourse/NamedEntityRecognition
- Timestamp:
- Oct 9, 2017, 11:44:01 AM (8 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
private/NlpInPracticeCourse/NamedEntityRecognition
v16 v17 56 56 1. train the model using the default settings (cnec.prop), N.B. that the `convert_cnec_stanford.py` only recognizes PERSON, LOCATION and ORGANIZATION, you can extend the markup conversion later: 57 57 {{{ 58 java -cp stanford-ner .jar edu.stanford.nlp.ie.crf.CRFClassifier \58 java -cp stanford-ner-2017-06-09/stanford-ner.jar edu.stanford.nlp.ie.crf.CRFClassifier \ 59 59 -prop cnec.prop 60 60 }}} 61 61 1. convert the test data to the Stanford NER format: 62 62 {{{ 63 python convert_cnec_stanford.py named_ent_dtest.xml \63 python convert_cnec_stanford.py cnec2.0/data/xml/named_ent_dtest.xml \ 64 64 > named_ent_dtest.tsv 65 65 }}} 66 66 1. evaluate the model on `dtest`: 67 67 {{{ 68 java -cp stanford-ner .jar edu.stanford.nlp.ie.crf.CRFClassifier \68 java -cp stanford-ner-2017-06-09/stanford-ner.jar edu.stanford.nlp.ie.crf.CRFClassifier \ 69 69 -loadClassifier cnec-3class-model.ser.gz \ 70 70 -testFile named_ent_dtest.tsv … … 83 83 10. evaluate the model on `dtest` with only NEs that are not present in the train data: 84 84 {{{ 85 java -cp stanford-ner .jar edu.stanford.nlp.ie.crf.CRFClassifier \85 java -cp stanford-ner-2017-06-09/stanford-ner.jar edu.stanford.nlp.ie.crf.CRFClassifier \ 86 86 -loadClassifier cnec-3class-model.ser.gz \ 87 87 -testFile named_ent_dtest_unknown.tsv … … 91 91 11. test on your own input: 92 92 {{{ 93 java -mx600m -cp stanford-ner .jar edu.stanford.nlp.ie.crf.CRFClassifier \93 java -mx600m -cp stanford-ner-2017-06-09/stanford-ner.jar edu.stanford.nlp.ie.crf.CRFClassifier \ 94 94 -loadClassifier cnec-3class-model.ser.gz -textFile sample.txt 95 95 }}}