Skip to content

Latest commit

 

History

History

docs

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
collection_specialties_of_corpora.md
collection_specialties_of_corpora.md
 
 
corpus_publishing_tasklist.md
corpus_publishing_tasklist.md
 
 
howto_archive_original_data.md
howto_archive_original_data.md
 
 
howto_archive_pkgs_ida.md
howto_archive_pkgs_ida.md
 
 
howto_archive_scripts.md
howto_archive_scripts.md
 
 
howto_archive_signed_agreements.md
howto_archive_signed_agreements.md
 
 
howto_cleanup_after_publishing.md
howto_cleanup_after_publishing.md
 
 
howto_convert_data_to_hrt.md
howto_convert_data_to_hrt.md
 
 
howto_data_storage.md
howto_data_storage.md
 
 
howto_download_package.md
howto_download_package.md
 
 
howto_korp_configuration.md
howto_korp_configuration.md
 
 
howto_korp_news.md
howto_korp_news.md
 
 
howto_korp_publish.md
howto_korp_publish.md
 
 
howto_korpmake.md
howto_korpmake.md
 
 
howto_landing_page.md
howto_landing_page.md
 
 
howto_license_page.md
howto_license_page.md
 
 
howto_maintain_resource_lists_database.md
howto_maintain_resource_lists_database.md
 
 
howto_maintain_resources_lists.md
howto_maintain_resources_lists.md
 
 
howto_metadata_record.md
howto_metadata_record.md
 
 
howto_nertagging.md
howto_nertagging.md
 
 
howto_notes-for-the-user.md
howto_notes-for-the-user.md
 
 
howto_parse.md
howto_parse.md
 
 
howto_portal_news.md
howto_portal_news.md
 
 
howto_pos_attributes.md
howto_pos_attributes.md
 
 
howto_request_pid.md
howto_request_pid.md
 
 
howto_resource_group_page.md
howto_resource_group_page.md
 
 
howto_tag_encoding_decoding.md
howto_tag_encoding_decoding.md
 
 
howto_tokenize.md
howto_tokenize.md
 
 
howto_unpublish_corpus.md
howto_unpublish_corpus.md
 
 
howto_validate_vrt.md
howto_validate_vrt.md
 
 
model_download-package-README.md
model_download-package-README.md
 
 
naming_inline_elements_hrt.md
naming_inline_elements_hrt.md
 
 
readme.md
readme.md
 
 
tool_publishing_tasklist.md
tool_publishing_tasklist.md
 
 

readme.md

Kielipankki-utilities/docs

This folder contains the pipeline instructions for preparing text corpora. Described are all steps needed from converting the original data to publishing the corpus in Korp or in the download service.

In addition, the file corpus_publishing_tasklist.md contains checklists for tasks in the corpus publishing pipeline that can be copied to the description of a Jira ticket for publishing (a version of) a corpus for keeping track of the progress of the publication process.

The instructions are accessible through the GitHub browser interface or in a cloned Kielipankki-utilities Git repository (e.g., on Puhti). You should update your own copy of the repository with git pull to see the latest changes. A third option would be to use the desktop client of GitHub.

The instructions are organized in several files, all stored in this subfolder docs. The order of tasks can be seen from the checklists.

The instructions are written in Markdown format. (For more information please see: https://help.github.com/en/articles/about-writing-and-formatting-on-github.) The browser interface of GitHub displays this nicely and makes it easy to read and edit the text.

Please feel free to give feedback, correct and edit the instructions where needed, and add what you think is missing. You can also add new files. You might find placeholders in the instructions for still missing information (e.g. a guideline for testing text corpora in Korp) and of course everybody is welcome to fill them.