Annotations in Pacte

N.B. This page is intended only for a small group of researchers working to improve the named entity recognition and geolocation for the upcoming Dreamscape tool.

Pacte is a platform for the collaborative annotation of document. We will use it to evaluate and improve the geolocalization of place names in a collection of TCP-EEBO texts.

Once you have a Pacte account (instructions provided separately) you can login at https://patx-pacte.crim.ca/ and change the interface language if need be (note that some of the subsequent interface may still be in French, but most of it is available in English; also, there were cases during my testing when I had to re-select English for the environment).

Important note: Pacte seems to work reliably in Chrome but there are issues in other browsers, so please be sure to use a desktop (not mobile) version of Chrome.

After successful login, you should see your dashboard, including near the bottom of the page, your assigned tasks. You can always return to the dashboard by clicking on the PACTE } login in the top left-hand corner. In the right part of the tasks grid a play button will appear when hovering over the actions column.

The next screen should show a document. By default the automatic annotations for that document are probably not loaded, so you need to click on the checkbox for "Groups and types" on the right side of the page.

That will cause the annotations to load (if not, and you're still using Chrome, please let me know).

You can click once on the hovering NELinking bubble that appears over an annotated word. This will cause the annotation information to be displayed on the right. Consider especially the Adm Geoname field, which should provide a full description of the annotated location.

With existing annotations there are three main possibilities:

1. the annotation is a location and properly geolocated
2. the annotation is a location but incorrectly geolocated
3. the annotation is not a location

If the annotation seems correct, you can just skip it and go to the next.

If the annotation is a location but incorrectly geolocated (or you have some doubt about the geolocation), then double-click on the NELinking bubble to call up a dialog box.

If you're unsure about the geolocation you can simply provide a confidence value and then click the save button. The confidence value is actually more of an accuracy value: put 0 if it's wrong (you're confident that it's wrong), 100 if it's right (you're confident that it's right), or something in between (you don't need to overthink the value, just estimate).

If you're fairly certain that the geolocation is wrong, then click the InvalidLinking checkbox.

If you think you can provide a better value, open up http://www.geonames.org in another window and put in a search term with as much information as you can (especially city, country).

Geonames should provide a list of possibilities on the next page. The best value may not be the first one. The link we want is not the label/link, it's the icon that's under the number. Select that link (right-click and copy the URL or visit and copy the URL) and then put that value in the CorrectedGeonameLink field:

Then save the annotation.

If the annotation is completely wrong (as is the case with "Bishop" in the annotations screenshot above, then be sure to click InvalidLinking and in the notes section write "not a location".

You can use the Notes section to convey any other information you think would be useful.

Don't correct the same problem multiple times if it recurs in the text, only the first time. Be aware that the same surface form (words in the text) may have different annotations and geolocations depending on other factors.

We've covered the scenarios with existing annotations, there's also the possibility that an obvious location has been missed. Important note: for now we're most interested in towns and cities, not every type of location needs to be annotated.

In this case you need to select the words, which should automatically cause a dialog box to appear. You don't need to fill in all the fields, though the following would be useful:

• YourConfidence
• CorrectedGeonameLink (if you can find a value in geonames.com)
• Notes if there's any to note

Then save the annotation and continue. If you incorrectly selected something, you can double-click on the annotation bubble and then use the delete button. Please note: only delete annotations that you've inadvertently created, we cannot easily recover a system-generated annotation if you remove it.

Again, you only need to annotate a surface form the first time, not every time.

Once you've completed a document, you can scroll to the top of the page and select "Conclude document" to keep track of which document you finished. It will switch automatically to the next document. Also the next time you login and work on your task, it will skip all the completed documents and display the next one to work on.