Changes between Initial Version and Version 1 of en/AdvancedNlpCourse2020/CorpusIndexing


Ignore:
Timestamp:
Aug 31, 2021, 2:11:17 PM (5 months ago)
Author:
Ales Horak
Comment:

copied from private/AdvancedNlpCourse/CorpusIndexing

Legend:

Unmodified
Added
Removed
Modified
  • en/AdvancedNlpCourse2020/CorpusIndexing

    v1 v1  
     1= Indexing and Searching Very Large Texts =
     2
     3[[https://is.muni.cz/auth/predmet/fi/ia161|IA161]] [[en/AdvancedNlpCourse|Advanced NLP Course]], Course Guarantee: Aleš Horák
     4
     5Prepared by: Miloš Jakubíček
     6
     7== State of the Art ==
     8
     9=== References ===
     10
     11 1. RYCHLÝ, Pavel, et al. Korpusové manažery a~ jejich efektivní implementace. 2000.
     12 1. JAKUBÍCEK, Miloš; KILGARRIFF, Adam; RYCHLÝ, Pavel. Effective Corpus Virtualization. In: Challenges in the Management of Large Corpora (CMLC-2) Workshop Programme. p. 7.
     13 1. JAKUBICEK, Milos, et al. Fast Syntactic Searching in Very Large Corpora for Many Languages. In: PACLIC. 2010. p. 741-747.
     14
     15== Practical Session ==
     16
     17Compare search through (A) plain text using grep, (B) an indexed corpus using Manatee, (C) a corpus indexed in an arbitrary SQL database
     18Use vertical text for BNC available at aurora:/corpora/vert/bnc/bnc.vert.xz.
     19
     20Search for the phrase "test case", display context of 10 words before and after each occurrence of the search phrase.
     21
     22(A) plain
     23
     24Hint: use grep -C to display context
     25
     26(B)
     27
     28Corpus is already indexed on Manatee, try:
     29
     30time corpquery bnc '[word="test"] [word="case"]'
     31
     32(C)
     33
     34Use your favourite SQL database, on aurora you can use sqlite3.
     35Hint how to import vertical text:
     36
     37https://stackoverflow.com/questions/26065872/how-to-import-a-tsv-file-with-sqlite3
     38
     39For (A), (B) and (C), submit the commands you used and how long the search took to evaluate.