Changes between Initial Version and Version 1 of en/AdvancedNlpCourse2020/Stylometry


Ignore:
Timestamp:
Aug 31, 2021, 2:11:05 PM (10 months ago)
Author:
Ales Horak
Comment:

copied from private/AdvancedNlpCourse/Stylometry

Legend:

Unmodified
Added
Removed
Modified
  • en/AdvancedNlpCourse2020/Stylometry

    v1 v1  
     1= Stylometry =
     2
     3[[https://is.muni.cz/auth/predmet/fi/ia161|IA161]] [[en/AdvancedNlpCourse|Advanced NLP Course]], Course Guarantee: Aleš Horák
     4
     5Prepared by: Honza Rygl, Aleš Horák
     6
     7== State of the Art ==
     8
     9The analysis of author's characteristic
     10writing style and vocabulary has been used to uncover author's traits such as authorship, age, or gender
     11documents by both manual linguistic approaches and automatic algorithmic methods.
     12
     13The most common approach to stylometry problems
     14is to combine stylistic analysis with machine learning techniques:
     15 1. specific style markers are extracted,
     16 2. a classification procedure is applied to extracted markers
     17
     18
     19=== References ===
     20
     21 1. Bevendorff, Janek, et al.(2020), Overview of PAN 2020: Authorship Verification, Celebrity Profiling, Profiling Fake News Spreaders on Twitter, and Style Change Detection. International Conference of the Cross-Language Evaluation Forum for European Languages. Springer, Cham. [https://pan.webis.de/downloads/publications/papers/bevendorff_2020.pdf pdf]
     22 1. Stamatatos, E. (2009), A Survey of Modern Authorship Attribution Methods (2009), Journal of the American Society for Information Science and Technology, 60(3), 538-556. [[http://www.clips.ua.ac.be/~walter/educational/material/Stamatatos_survey2009.pdf | pdf]]
     23 2. Kestemont, M. (2014), Function Words in Authorship Attribution From Black Magic to Theory? Proceedings of the 3rd Workshop on Computational Linguistics for Literature, EACL 2014, 59–66 [[http://aclweb.org/anthology/W14-0908 | pdf]]
     24 1. Daelemans, W. (2013). Explanation in computational stylometry. In International conference on intelligent text processing and computational linguistics (pp. 451-462). Springer, Berlin, Heidelberg. [https://www.clips.uantwerpen.be/sites/default/files/daelemans2013.pdf pdf]
     25
     26== Practical Session ==
     27
     28Students will work with the ''Style & Identity Recognition'' (SIR) tool. They will test this tool on prepared data.
     29The goal will be to implement a small function to extract style markers from a text.
     30
     311. go to `asteria04.fi.muni.cz` server:
     32{{{
     33ssh asteria04.fi.muni.cz
     34}}}
     352. Download a  [[htdocs:bigdata/stylometry-assignment.zip|ZIP with python packages of the assignment]]
     36{{{
     37wget https://nlp.fi.muni.cz/trac/research/chrome/site/bigdata/stylometry-assignment.zip
     38}}}
     393. Unzip the downloaded file
     40{{{
     41unzip stylometry-assignment.zip
     42}}}
     434. Go to the unziped folder
     44{{{
     45cd sir-assignment
     46}}}
     475. Test the prepared program that analyses data from on-line dating services to distinguish gender (masculine/feminine) by text style features
     48{{{
     49./run.sh
     50}}}
     51
     52`run.sh` can have two optional parameters:
     53{{{
     54./run.sh  [number_of_testing_cycles]  [show_first_N_erroneously_predicted_documents]
     55}}}
     56The default values, i.e. running `./run.sh` without parameters, are `10` cycles and `no documents` (`./run.sh 10 0`). With longer feature testing `./run.sh 100` could provide better results (but not necessarily).
     57
     58Example with document output (second parameter `>0`):
     59{{{
     60[xrygl@asteria04:~/temp/sir-assignment]$ ./run.sh 10 1
     61pos: 5
     62expected: on
     63predicted: ona
     64text: Ahoj, (nejen) pro výlety do víru podivnězimního velkoměsta, či divočiny
     65      venkova, hledá se partnerka přiměřených rozměrů, tvarů a úrovně. Slečny
     66      veselé povahy preferovány; ona je to nejspíš nutnost :-)
     67morphology: [
     68     1. <s>           <s>
     69     2. Ahoj          N.N.I.S.1.-.-.-.-.-.A.-.-.-.-
     70     3. ,             Z.:.-.-.-.-.-.-.-.-.-.-.-.-.-
     71     4. (             Z.:.-.-.-.-.-.-.-.-.-.-.-.-.-
     72     5. nejen         T.T.-.-.-.-.-.-.-.-.-.-.-.-.-
     73     6. )             Z.:.-.-.-.-.-.-.-.-.-.-.-.-.-
     74     7. pro           R.R.-.-.4.-.-.-.-.-.-.-.-.-.-
     75     8. výlety        N.N.I.P.4.-.-.-.-.-.A.-.-.-.-
     76     9. do            R.R.-.-.2.-.-.-.-.-.-.-.-.-.-
     77    10. víru          N.N.I.S.2.-.-.-.-.-.A.-.-.-.-
     78    11. podivnězimního  A.A.N.S.2.-.-.-.-.1.A.-.-.-.-
     79    12. velkoměsta    N.N.N.S.2.-.-.-.-.-.A.-.-.-.-
     80    13. ,             Z.:.-.-.-.-.-.-.-.-.-.-.-.-.-
     81    14. či            J.^.-.-.-.-.-.-.-.-.-.-.-.-.-
     82    15. divočiny      N.N.F.S.2.-.-.-.-.-.A.-.-.-.-
     83    16. venkova       N.N.I.S.2.-.-.-.-.-.A.-.-.-.-
     84    17. ,             Z.:.-.-.-.-.-.-.-.-.-.-.-.-.-
     85    18. hledá         V.B.-.S.-.-.-.3.P.-.A.A.-.-.-
     86    19. se            P.7.-.X.4.-.-.-.-.-.-.-.-.-.-
     87    20. partnerka     N.N.F.S.1.-.-.-.-.-.A.-.-.-.-
     88    21. přiměřených   A.A.I.P.2.-.-.-.-.1.A.-.-.-.-
     89    22. rozměrů       N.N.I.P.2.-.-.-.-.-.A.-.-.-.-
     90    23. ,             Z.:.-.-.-.-.-.-.-.-.-.-.-.-.-
     91    24. tvarů         N.N.I.P.2.-.-.-.-.-.A.-.-.-.-
     92    25. a             J.^.-.-.-.-.-.-.-.-.-.-.-.-.-
     93    26. úrovně        N.N.F.S.2.-.-.-.-.-.A.-.-.-.-
     94    27. .             Z.:.-.-.-.-.-.-.-.-.-.-.-.-.-
     95    28. <s>           <s>
     96    29. Slečny        N.N.F.P.1.-.-.-.-.-.A.-.-.-.-
     97    30. veselé        A.A.N.S.1.-.-.-.-.1.A.-.-.-.-
     98    31. povahy        N.N.F.S.2.-.-.-.-.-.A.-.-.-.-
     99    32. preferovány   V.s.T.P.-.-.-.X.X.-.A.P.-.-.-
     100    33. ;             Z.:.-.-.-.-.-.-.-.-.-.-.-.-.-
     101    34. ona           P.P.F.S.1.-.-.3.-.-.-.-.-.-.-
     102    35. je            V.B.-.S.-.-.-.3.P.-.A.A.-.-.-
     103    36. to            P.D.N.S.1.-.-.-.-.-.-.-.-.-.-
     104    37. nejspíš       D.g.-.-.-.-.-.-.-.3.A.-.-.-.-
     105    38. nutnost       N.N.F.S.1.-.-.-.-.-.A.-.-.-.-
     106    39. :             Z.:.-.-.-.-.-.-.-.-.-.-.-.-.-
     107    40. -             Z.:.-.-.-.-.-.-.-.-.-.-.-.-.-
     108    41. )             Z.:.-.-.-.-.-.-.-.-.-.-.-.-.-
     109]
     110Acc: 73.8 +- 2.1% (baseline 50.0%, 10 iterations)
     111}}}
     112You may print more details in `http_server/basic_task.py` after `# print explanation(s)` comment.
     113
     114=== Task ===
     115Examine files in  `stylometry_features` folder.
     116Modify the `assignment.py` file to increase the accuracy of methods (use `vi` or `nano` command for editing).
     117You can create another classes inside, change their names and class names to improve the accuracy score. Don't forget to add new classes into the `assignments` list in this file.
     118
     119Suggestions for your inspiration include:
     120 * diacritics usage (yes/no), a regular expression will be needed
     121 * sentence endings (number of sentences, or typical endings)
     122 * repetitions of words in sentences/in the text
     123 * usage of uppercase letters
     124 * length of sentences/text
     125 * POS tags (n-grams)
     126 * word n-grams
     127 * character n-grams
     128
     129Each modification can be tested by running `./run.sh` again.
     130The first call of `run.sh` can be slower, because documents are morphologically analysed during the first run.
     131
     132Write the resulting `Acc:` line into the top comment of the `assignment.py` file. [[br]]
     133Submit your `assignment.py` file.