| | 1 | = Indexing and Searching Very Large Texts = |
| | 2 | |
| | 3 | [[https://is.muni.cz/auth/predmet/fi/ia161|IA161]] [[en/AdvancedNlpCourse|Advanced NLP Course]], Course Guarantee: Aleš Horák |
| | 4 | |
| | 5 | Prepared by: Miloš Jakubíček |
| | 6 | |
| | 7 | == State of the Art == |
| | 8 | |
| | 9 | === References === |
| | 10 | |
| | 11 | 1. RYCHLÝ, Pavel, et al. Korpusové manažery a~ jejich efektivní implementace. 2000. |
| | 12 | 1. JAKUBÍCEK, Miloš; KILGARRIFF, Adam; RYCHLÝ, Pavel. Effective Corpus Virtualization. In: Challenges in the Management of Large Corpora (CMLC-2) Workshop Programme. p. 7. |
| | 13 | 1. JAKUBICEK, Milos, et al. Fast Syntactic Searching in Very Large Corpora for Many Languages. In: PACLIC. 2010. p. 741-747. |
| | 14 | |
| | 15 | == Practical Session == |
| | 16 | |
| | 17 | Compare search through (A) plain text using grep, (B) an indexed corpus using Manatee, (C) a corpus indexed in an arbitrary SQL database |
| | 18 | Use vertical text for BNC available at aurora:/corpora/vert/bnc/bnc.vert.xz. |
| | 19 | |
| | 20 | Search for the phrase "test case", display context of 10 words before and after each occurrence of the search phrase. |
| | 21 | |
| | 22 | (A) plain |
| | 23 | |
| | 24 | Hint: use grep -C to display context |
| | 25 | |
| | 26 | (B) |
| | 27 | |
| | 28 | Corpus is already indexed on Manatee, try: |
| | 29 | |
| | 30 | time corpquery bnc '[word="test"] [word="case"]' |
| | 31 | |
| | 32 | (C) |
| | 33 | |
| | 34 | Use your favourite SQL database, on aurora you can use sqlite3. |
| | 35 | Hint how to import vertical text: |
| | 36 | |
| | 37 | https://stackoverflow.com/questions/26065872/how-to-import-a-tsv-file-with-sqlite3 |
| | 38 | |
| | 39 | For (A), (B) and (C), submit the commands you used and how long the search took to evaluate. |
