Changes between Version 2 and Version 3 of en/WordLevelAnalysis

Timestamp:

Jun 5, 2014, 10:51:18 AM (10 years ago)

Author:

xkocinc

Comment:

--

en/WordLevelAnalysis

@@ -32 +32 @@
 Various types of analyses:
  * word form => lemma (many types of searching/indexation)
-   * nebral => brát/nebrat (úplatky)
-   * nejstaršího => nejstarší/starý (člověk)
-   * chladnička => chladničky (as a class)
-   * bavlna => bavlněný (word derivation)
+   * ''nebral => brát/nebrat (úplatky)''
+   * ''nejstaršího => nejstarší/starý (člověk)''
+   * ''chladnička => chladničky'' (as a class)
+   * ''bavlna => bavlněný'' (word derivation)
  * word form/lemma + gram. info. => word form
-   * e.g. salutation generation: pane Procházko
+   * e.g. salutation generation: ''pane Procházko''
  * word form/lemma => all word forms
  * word form => lemma + full/partial grammatical information
 
 The analysis is very fast - approx. 1 million word forms per second
+
+
+== Processing Unknown Words ==
+
+Some word forms in processed texts are unknown:
+ * terms ''polydaktylie'', neologisms ''klausoviny'', typos ''bizardního'', colloquial words ''plaťáky'', etc.
+
+An ending of the word form is able to determine e.g.
+ * lemma: ''klausoviny => klausovina''
+ * grammatical information: ''bizardního'' => genitive, etc.
+ * derivational relations: ''plaťáky => plaťákový''
+
+Texts from a particular domain allows grouping of unknown word forms:
+ * ''polydaktylie, polydaktiliích, polydaktylií, ... <=> polydaktylie''
+ * => extension of data or more precise “guessing”