= A Human-Annotated Dataset of Scanned Images and OCR Texts from Medieval Documents =
This is an open dataset of scanned images and OCR texts from 19th and 20th century letterpress reprints of documents from the Hussite era. The dataset contains human annotations for layout analysis, OCR evaluation, and language identification.

You can [https://hdl.handle.net/11234/1-4615 download the dataset from 2021] and [https://nlp.fi.muni.cz/projects/ahisto/ocr-texts-supplementary.zip supplementary materials from 2022] in the LINDAT/CLARIAH-CZ repository.

== Contents ==
[https://hdl.handle.net/11234/1-4615 The dataset from 2021] is structured as follows:

 * The archive [https://lindat.mff.cuni.cz/repository/xmlui/bitstream/handle/11234/1-4615/scanned-images.zip?sequence=7&isAllowed=y scanned-images.zip] (47.13 GB) contains 51,351 high-resolution scanned images.
 * The archive [https://lindat.mff.cuni.cz/repository/xmlui/bitstream/handle/11234/1-4615/ocr-texts.zip?sequence=5&isAllowed=y ocr-texts.zip] (5.09 GB) contains 51,351 OCR texts in three formats:
   1. HOCR documents from the Tesseract 4 OCR engine.
   1. JSON documents from the [https://cloud.google.com/vision Google Vision AI] OCR engine.
   1. TXT documents that combine Tesseract and Google outputs to achieve maximum accuracy on different types of layout.
 * The archive [https://lindat.mff.cuni.cz/repository/xmlui/handle/11234/1-4615#file_file_7686 annotations-ocr.zip] (178.62 KB) contains 120 annotations for the evaluation of OCR.[[BR]]The archive is divided into two subdirectories for the evaluation of layout analysis:
   1. The subdirectory `with-columns` contains annotations for 17 multi-column pages.
   1. The subdirectory `without-columns` contains annotations for 103 single-column pages.
 * The archive [https://lindat.mff.cuni.cz/repository/xmlui/bitstream/handle/11234/1-4615/annotations-language-identification.zip?sequence=3&isAllowed=y annotations-language-identification.zip] (1.1 MB) contains 122 annotations for the evaluation of language identification.

[https://nlp.fi.muni.cz/projects/ahisto/ocr-texts-supplementary.zip The supplementary materials from 2022] are structured as follows:

 * The archive [https://nlp.fi.muni.cz/projects/ahisto/ocr-texts-supplementary.zip ocr-texts-supplementary.zip] (23.2 MB) contains 110 OCR texts for which we have both high-resolution scanned images and annotations for OCR evaluation.[[BR]]The archive is divided into a number of subdirectories with outputs of different OCR engines:
   * The subdirectory `google-vision-ai-old` contains JSON and TXT documents from the Google Vision AI OCR engine from 2020-10-02.
   * The subdirectory `google-vision-ai` contains JSON and TXT documents from the Google Vision AI OCR engine from 2022-08-11.
   * The subdirectory `pero-demo` contains PAGE and TXT documents from [https://pero-ocr.fit.vutbr.cz/ the web demo of the PERO OCR engine].
   * The subdirectory `pero-github` contains PAGE and TXT documents from [https://github.com/DCGM/pero-ocr the open-source variant of the PERO OCR engine] using [https://www.fit.vut.cz/~ihradis/pero/pero_eu_cz_print_newspapers_2020-10-09.tar.gz public pretrained models].
   * The subdirectory `tesseract` contains HOCR and TXT documents from the Tesseract 4 OCR engine.
   * The subdirectory `tesseract-and-google-vision-ai-old` contains TXT documents that combine `tesseract` and `google-vision-ai-old` documents.
   * The subdirectory `tesseract-and-google-vision-ai` contains TXT documents that combine `tesseract` and `google-vision-ai` documents.
   * The subdirectory `tesseract-and-pero-github` contains TXT documents that combine `tesseract` and `pero-github` documents.

== Citing ==
If you use our dataset in your work, please cite the following articles:

  Novotný, V., Seidlová, K., Vrabcová, T., Horák, A.: When Tesseract Brings Friends: Layout Analysis, Language Identification, and Super-Resolution in the Optical Character Recognition of Medieval Texts. In: Horák, A., Rychlý, P., Rambousek, A. (eds.) ''                   Proceedings of Recent Advances in Slavonic Natural Language Processing, RASLAN 2021''     . pp. 91–100. ISSN 2336-4289. ISBN 978-80-263-1600-8. Tribun EU (2021). Available also from WWW: https://nlp.fi.muni.cz/raslan/2021/paper10.pdf

  Novotný, V., Horák, A.: When Tesseract Meets PERO: Open-Source Optical Character Recognition of Medieval Texts. In: Horák, A., Rychlý, P., Rambousek, A. (eds.) ''     Proceedings of Recent Advances in Slavonic Natural Language Processing, RASLAN 2022''     . pp. 157–160. ISSN 2336-4289. ISBN 978-80-263-1752-4. Tribun EU (2022). Available also from WWW: https://nlp.fi.muni.cz/raslan/2022/paper12.pdf

If you use LaTeX, you can use the following BibTeX entries:

{{{
@inproceedings{novotny2021when,
  title = {When Tesseract Brings Friends: Layout Analysis, Language
           Identification, and Super-Resolution in the Optical Character
           Recognition of Medieval Texts},
  author = {Vít Novotný and Kristýna Seidlová and Tereza Vrabcová and
            Aleš Horák},
  editor = {Aleš Horák and Pavel Rychlý and Adam Rambousek},
  booktitle = {Proceedings of Recent Advances in Slavonic Natural
               Language Processing, {RASLAN} 2021},
  publisher = {Tribun {EU}},
  pages = {91-100},
  year = {2021},
  issn = {2336-4289},
  isbn = {978-80-263-1600-8},
  url = {https://nlp.fi.muni.cz/raslan/2021/paper10.pdf},
}
}}}
{{{
@inproceedings{novotny2022when,
  title = {When Tesseract Meets {PERO}: Open-Source Optical Character
           Recognition of Medieval Texts},
  author = {Vít Novotný and Aleš Horák},
  editor = {Aleš Horák and Pavel Rychlý and Adam Rambousek},
  booktitle = {Proceedings of Recent Advances in Slavonic Natural
               Language Processing, {RASLAN} 2022},
  publisher = {Tribun {EU}},
  pages = {157-160},
  year = {2022},
  issn = {2336-4289},
  isbn = {978-80-263-1752-4},
  url = {https://nlp.fi.muni.cz/raslan/2022/paper12.pdf},
}
}}}
== Acknowledgements ==
This work was funded by TAČR Éta, [https://starfos.tacr.cz/en/project/TL03000365 project number TL03000365].