V IV Inf @FS-IMV #7->6
"<.>"
- "." CLB #8->8
+ "." CLB #8->8
+
-
Muitaleastte
- / \ \
+ / \ \
fal munnje \
sáhttá
/ \
doavttir boahtit.
\
- goas
-
+ goas
Our dependency structure is based upon a compromise between the Saami
grammatical tradition and the conventions used within [the visl
project](http://beta.visl.sdu.dk/visl/da/parsing/automatic/dependency.php).
-Verb tags
-=========
+## Verb tags
The Saami disambiguation file `disambiguator.cg3` adds dependency tags to each
cohort. The CG verb tags are substituted with these tags:
-- <mv> <aux>
+- <mv> <aux>
There are main verbs and auxiliary verbs.
-- @FMV @FAUX @IMV @IAUX
+- @FMV @FAUX @IMV @IAUX
In main clauses: Finite main verb and auxiliary verb, and infinite main
verb and auxiliary verb.
-- @FS-STA @FS-N<
+- @FS-STA @FS-N<
@FS is a finite verb in a subclause. The first one in a subclause which
functions as a statement, the latter one in a relative subclause.
-- @FS-IMV @FS-IAUX @FS-N<IMV @FS-N<IAUX
+- @FS-IMV @FS-IAUX @FS-N<IMV @FS-N<IAUX
These are infinite main verbs and auxiliary verbs in an ordinary
subclause and in a relative subclause.
-- <ctjHead>
+- <ctjHead>
This tag helps in coordination contexts.
-Dependency tags alphabetically
-==============================
+## Dependency tags alphabetically
Dependency tags look different from syntactic grammar tags.
-- **@>A:**
- Modifier of an adjective to the left.
- - **nu (Adv):**
- *Gulahallan Sámedikkiin dán gažaldagas šaddá *nu*
- konkrehtalažžan go vejolaš. - 'The discussion in the Saami
- Parliament about this issue gets *as* concrete as possible.'*
-- **@A<:**
- Modifier of an adjective to the right.
- - **:**
-- **@>Adv:**
- Modifier of an adverb.
- - **:**
-- **@Adv<:**
- Complement of an adverb.
- - **:**
-- **@ADVL:**
- Sentence adverbial.
- - **dál (Adv):**
- **Dál* lea Bireha vuorru. - 'It is Biret's turn *now*.'*
-- **@>ADVL:**
- Modifier of an adverbial.
- - **Man (Pron):**
- **Man* dávjá don lávet fitnat doppe? - '*How* often do you
- usually go there?'*
-- **@<ADVL:**
- adverbial to the right of the finite verb
- - **beaivvážis (N):**
- *Gávpot ii dárbbaš čuovgga *beaivvážis* ii ge mánus. - 'The city
- does not need light *from the sun* and not the from the moon
- either.*
-- **@ADVL<:**
- Complement of an adverbial.
- - **vahkus (N):**
- *Mun málestan guktii *vahkus*. - 'I make food twice a *week*.'*
-- **@ADVL>:**
- Adverbial to the left of the finite verb.
- - **lasttain (N):**
- *Ja muora *lasttain* ožžot álbmogat dearvvašvuođa. - 'And from
- the tree's *leaves*, the people get health.'*
-- **@ADVL>CS:**
- adverbial modifying a conjunction
- - **dallah (Adv):**
- **Dallah* goh Jeesuse tjaetseste tjuedtjele, dellie vuajna Elmie
- rihpesåvva jih Voejkene altasasse goh ledtie suaja. - '*Right*
- after Jesus stood up from the water, he sees that heaven opens
- and the holy spirit flies to him like a bird.'
- (*sma*)*
- - **dan dihte (Adv):**
- *Muhto go lassánedje olbmot, de bohte čáhppesbivttasolbmot fas
- dohko, gosa ledje sámit vuohččan ballán, ja dahke orohagaid jur
- dasa gos sámit ledje orrume, *dan dihte* go sii oidne, ahte das
- leai čáppa gieddi, maid ledje bohccot dutken, gožžan ja baikán —
- gos ledje sámit orron mánga olmmošbuolvva.*
-- **@ADVL<OBJ:**
- - **:**
-- **@ADVL>SUBJ:**
- - **:**
-- **@AGENS>:**
- kal
- - **atorfilittanit:**
- *Attartortumiit piginnittumut aaqqissuussineq
- namminersornerusuni atorfilittanit politikerinillu nuimasunit
- isertortumik atornerlunneqarsimammat illoqarfinni anginerni
- pingasuni attartortut nalinginnaasut pillarneqartussanngorput.*
-- **@APP-ADVL<:**
- Apposition to an adverbial to the left. If the apposition consists
- of more than one word, the head will get this tag.
- - **ovdal (Pr):**
- *Dolin, *ovdal* soađi, olbmot lávejedje vuovdit joŋaid. - 'In
- old times, *before* the war, people used to sell cowberries.'*
-- **@APP-N<:**
- Apposition to a noun to the left of it. If the apposition is more
- than one constituent, the head will get this tag.
- - **eatnigiela (N):**
- *Viimmat mun ohppen čállit sámegiela, mu *eatnigiela*. -
- 'Finally, I learned to write in Sámi, my *mother tongue*.'*
-- **@APP-Num<:**
- Apposition to a numeral to the left.
- - **suinniid (N):**
- *Juohke heasta borrá sullii 6 kilu *suinniid* beaivái. - 'Every
- horse eats approximately 6 kilograms of *grass* a day.'*
-- **@APP>Pron:**
- Apposition to a pronoun to the right. If the apposition is more than
- one constituent, the head will get this tag.
- - **Turner (N Prop):**
- *Muhto diet Will *Turner*, son nai lea fiinna olmmái. - 'But
- this Will *Turner*, he is also a nice guy.'*
-- **@APP-Pron<:**
- Apposition to a pronoun to the left. If the apposition is more than
- one constituent, the head will get this tag.
- - **olbmái (N):**
- *Dan mun muitalan dušše dutnje, mu buoremus *olbmái*. - 'This I
- tell only you, my best *friend*.'*
-- **@>CC:**
- modifier of CC
- - **sihke (CC):**
-- **@>CC:**
- modifier of CC
- - **sihke (CC):**
-- **@CL-ADVL>:**
- - **:**
-- **@CL-<ADVL:**
- - **:**
-- **@CMPND:**
- First part of a compound followed by a hyphen
- - **skaehtie-:**
- *Reerenasse galka båetije stoerredigkieboelhkesne jåerhkedh dam
- *skaehtie-* jïh åasadaltesem mij lea daelie, jïh daennie
- daltesisnie hov lea nuepie buerebe joekedimmiem darjodh.*
-- **@CNP:**
- Local conjunction or subjunction.
- - **ja (CC):**
- *Sihke Mázes *ja* Guovdageainnus leat boarrásat viššalit finadan
- doaibmaguovddážiin. - 'Both in Máze *and* Guovdageaidnu, the
- oldest people frequently got to the activitycentre.'*
- - **go (CS):**
- *Sámi geavaheaddjit hállet dávjá metaforaiguin ja sis leat ollu
- eará gulahallanvuogit *go* giella. - 'Saami users speak often in
- metaphores and the have many other ways of comunicating *than*
- by means of language.'*
-- **@COMPL-CS<:**
- Complement of subjunction.
- - **vejolaš (A):**
- *Gulahallan Sámedikkiin dán gažaldagas šaddá nu konkrehtalažžan
- go *vejolaš*. - 'The contact with the Saami Parliament about
- this issue gets as concrete as *possible*.'*
-- **@CVP:**
- Conjunction or subjunction that conjoins finite verb phrases
- - **ja (CC):**
- *Bealatjogas leat dološ rájes leamaš bálvvossajit *ja* dát
- golbma sieiddi ledje dovddus gitta olgoriikii. - 'Long since,
- there have been sacrificial sites at Bealatjohka *and* the three
- 'sieidi' (cult images) were known even abroad.*
- - **go (CS):**
- *Leago guhkes áigi dassá *go* Máreha oidnet? - 'Has it been a
- long time *since* you have seen Máret?'*
-- **@FAUX:**
- finite auxiliary
- - **ledje (V):**
- *Gávpotmuvrra vuođđogeađggit *ledje* čiŋahuvvon juohke lágán
- divrras geđggiiguin. - \`The cornerstones of the wall *were*
- decorated with every kind of expensive stones.'*
-- **@-F<ADVL:**
- Adverbial of infinite verb outside of the predicate
- - **árbbolaččain (N):**
- *Danne dárbbašit mii oažžut lobi Nils Aslak Valkeapää
- *árbbolaččain* almmuhit dán guokte lávlaga min sálbma-CD:s. -
- \`Therefore we need to get permission from Nils Aslak
- Valkeapää's *heirs* to release these two songs on our
- psalm-CD.'*
-- **@-FADVL>:**
- Adverbial of infinite verb outside the predicate
- - **várrogasat (Adv):**
- *Dihkkadeaddji rávve skohtervuddjiid *várrogasat* mátkkoštit.
- \`The roadman warns snowscooter drivers to drive *carefully*.'*
-- **@FMV:**
- finite mainverb
- - **lei (V):**
- *Gávpot lei njealječiegat, seammá guhkki go govdat. - \`The city
- *was* a square, same width as length.'*
-- **@FMVdic:**
- - **muitala (V):**
- *Ja go geassit eret dábálaš goluid, de lea buhtes sisaboahtu
- sullii 100 000 ruvnnu, *muitala* Eriksen. - \`And when we take
- away/subtract? the regular expenses, there is a remaining income
- of about 100 000 crowns, *says* Eriksen.'*
-- **@-F<OBJ:**
- Object of infinite verb outside the verbal to the right of it.
- - **govaid (N):**
- *Boađe mu lusa geahččat *govaid*! - \`Come to me and look *at
- the pictures*!'*
-- **@-FOBJ>:**
- Object of infinite verb outside the verbal to the left of it.
- - **váldovuoittuid (N):**
- *Valáštallanhálla lei njealjehas dievva olbmuiguin geat vurde
- *váldovuoittuid* fasket. - \`The gymn was to a quarter full of
- people that wait to grab *the main prizes*.'*
-- **@-F<SPRED:**
- - **duhtavaččat (A):**
- *IL Nordlysa beaivválaš jođiheaddji, Nils Peder Eriksen, lohká
- iežaset leat oalle *duhtavaččat* dán jagáš básárdoaluin.*
-- **@FM-SPRED<:**
- main clause functioning as a subject predicate to the right of
- another main clause
- - **ii (V):**
- *Ja dasa lea dát sivva: go sápmelaš boahtá moskkus gámmirii, de
- son ii *ii* ipmir ii báljo maidege, go ii biegga beasa bossut
- njuni vuostá. - \`And this is the reason: if a Saami comes ...,
- then he does *not* understand ...'*
-- **@FS-ADVL>:**
- subclause functioning as an adverbial to the finite verb of the main
- clause to the right of it.
- - **bohtet (V):**
- *Ja mo jos Muhtinlágan Stálu ustibat *bohtet* fitnat. - \`And
- what if the friends of some-kind-of troll *come* for a visit.'*
-- **@FS-<ADVL:**
- subclause functioning as an adverbial to the finite verb of the main
- clause to the left of it.
- - **galggai (V):**
- *Go *galggai* bargat rehkenastimiin sus šattai álo
- oaivvebávččas. - \`When they *should* work with arithmetics, she
- always got a headache.'*
-- **@FS-IAUX:**
- subclause infinite auxiliary
- - **sáhte:**
- *Mun in *sáhte* muitalit dán dutnje. - \`I *can*not tell you
- this.'*
-- **@FS-IMV:**
- subclause infinite mainverb
- - **ohcamin (V):**
- *Naba jos eadni lea sudno *ohcamin*, iige gávnna. - \`And if
- mother is *searching* for them, she will not find them.'*
-- **@FS-N<:**
- finite verb (either an auxiliary or main verb) of a relative
- subclause (with a noun (N) antecedent)
- - **lea (V):**
- *De son viežžá liegga liema ruittus mii *lea* oapmana alde.
- \`Then he fetched warm broth from the pot that *is* on the
- stove.'*
-- **@FS-N<IAUX:**
- infinite auxiliary of a (relative) subclause
- - **sáhttán (V):**
- *Mun oidnen nieidda gii ii *sáhttán* boahtit. - \`I saw the girl
- that *could* not come.'*
-- **@FS-N<IMV:**
- infinite mainverb of a (relative) subclause
- - **bargan (V):**
- *Mon lean okta sápmelaš, guhte lean *bargan* visot sámi bargguid
- ja mon dovddan visot sámi dili. - \`I am a Sámi, who has
- *worked* in all Saami occupations and I know all Saami
- affairs.'*
-- **@FS-OBJ:**
- finite verb of the subclause that has an object function
- - **leahkkasii (V):**
- *Arne ii fuobmán ahte uksa *leahkkasii*. - \`Arne did not notice
- that the door *opened*.'*
-- **@FS-OBJ>:**
- finite verb of a subclause that has object function used for kal
- e.g.
- - **pigilissagaa (V):**
- *Nunani issittuni sila pillugu tunngaviusumik ilisimasalernissaq
- silamut ilisimatusarfiup pigilissagaa ministerit marluullutik
- isumaqatigiipput, silallu allanngoriartornerata sunniutai
- maluginiarneqassasut.*
-- **@FS-P<:**
- finite verb of a subclause
- - **eru (V):**
- *Tað er ikki longur pláss fyri, at lutir og kenslur bara eru. -
- 'There is no longer any space for that things and feelings
- simply are.'*
- - **:**
- *(fao)*
-- **@FS-P<IMV:**
- finite verb of a subclause in fao
- - **:**
-- **@-F<OPRED:**
- - **:**
-- **@-FSUBJ>:**
- subject of a verbal infinitival object
- - **mánáid (N):**
- *Muhtinlágan Stállu cáhpá goikebierggu sudnuide ja dáhttu
- *mánáid* boradit. - 'Some-sort-of troll cuts dried meat for them
- and asks *the children* to eat.'*
-- **@FS-SUBJ:**
- finite verb head of a subordinate clause functioning as a subject
- - **boađát (V):**
- *Dehálaš lea ahte don maid *boađát*. - 'It is important that you
- *come* too.'*
-- **@FS-VFIN<:**
- - **eai (V):**
- *Idja ii leat šat, *eai* ge sii dárbbaš lámppá dahje beaivváža
- čuovgga, dasgo Hearrá Ipmil lea sin čuovga. - \`The night is not
- anymore, they do *not* need the lamp- or day- light either,
- because God the Lord is their light.'*
-- **@HAB:**
- Habitive, for a human target in illative or locative in a habitive
- construction (copula), is translated as "have". Possible verbs in a
- habitive construction are "boahtit" 'come', "leat" 'be', "goallut"
- 'pass', "heaŋgát", 'hang', "jápmit" 'die', "šaddat" 'become'.
- - **Máhtes (N):**
- **Máhtes* lea beana. - '*Máhtte* has a dog.'*
- - **sus (Pron):**
- **Sus* šattai álo oaivvebávččas go galggai bargat
- rehkenastimiin. - '*She* always got a headache when she was
- supposed to work with arithmetics.'*
-- **@HNOUN:**
- Stray noun in sentence fragments.
- - **boddu (N):**
- *Vuosttaš *boddu*. - 'First *lesson*.'*
-- **@IAUX:**
- non-finite auxiliary
- - **veaje (V):**
- *Dattetge ii *veaje* oađđit. - \`Still she did not *manage* to
- sleep.'*
-- **@ICL-ADVL:**
- infinitival clause adverbial
- - **árvvoštallat (V):**
- *Son namuha ahte sii leat gal ávžžuhan olbmuid geat dihtet ahte
- dárbbašit jođánit beassat buohccevissui, nugo ovdamearkka dihte
- áhpehis nissonolbmuid geain lahkona riegádahttináigi,
- *árvvoštallat* galget go ovdal go buohccájit juo vuolgit
- buohccevissui.*
-- **@ICL-OBJ:**
- infinitival clause object
- - **boradit (V):**
- *Muhtinlágan Stállu cáhpá goikebierggu sudnuide ja dáhttu mánáid
- *boradit*. - 'Some-sort-of troll cuts dried meat for them and
- asks the children *to eat*.'*
-- **@ICL-P<:**
- infinitival complement of a preposition
- - **skriva (V):**
- *Kenslan gav Unn íblástur til at skriva nakrar yrkingar um
- næstrakærleika.*
- - **:**
- *(fao)*
-- **@ICL-SUBJ:**
- infinitival subject
- - **sløkkja (V):**
- *Men tað er líka skjótt at sløkkja ljósið, lata eygu og oyru
- aftur og rista ábyrgdina av okkum.*
- - **:**
- *(fao)*
-- **@IM:**
- fao
- - **at (IM):**
- *Tá koyrdi Harrin Guð hann út úr aldingarðinum í Eden og setti
- hann til at dyrka ta jørð, sum hann var sjálvur tikin av.*
-- **@IMV:**
- non-finite mainverb
- - **čiŋahuvvon (V):**
- *Gávpotmuvrra vuođđogeađggit ledje *čiŋahuvvon* juohke lágán
- divrras geđggiiguin. - \`The cornerstones of the wall were
- *decorated* with every kind of expensive stones.'*
-- **@INF->N:**
- kal
- - **pillugu (V):**
- *Nunani issittuni sila pillugu tunngaviusumik ilisimasalernissaq
- silamut ilisimatusarfiup pigilissagaa ministerit marluullutik
- isumaqatigiipput, silallu allanngoriartornerata sunniutai
- maluginiarneqassasut.*
-- **@INTERJ:**
- Interjection.
- - **maid (Interj):**
- **Maid*, iigo leat boahtán? - '*What*, hasn't he/she come?'*
-- **@<IOBJ:**
- indirect object to the right of the finite verb.
- - **(N):**
-- **@IOBJ>:**
- Indirect object to the left of the finite verb.
- - **(N):**
-- **@MIK-OBJ:**
- kal
- - **illunik (N):**
- *Namminersornerusut Nuummi illunik ima amerlatigisunik
- tunisaqarsimalerput aningaasanut inatsit
- iluatsitaariniarfigisariaqalerlugu inissianik isatereriarlutik
- nutaanik sanaartortariaqaleramik atorfilittatik naammaginartunik
- inissaqartissinnaajumallugit.*
-- **@>N:**
- Prenominal modifier to the left
- - **geavatlaš (A):**
- *Ráđđehussii lea *geavatlaš* politihkka deaŧalaš. - 'For the
- government, *practical* politics is important.'*
- - **oahppo-:**
- **Oahppo-* ja dutkanministtar dat lea ráhkadan dieđáhusa alit
- sámi oahpu ja dutkama birra. - 'The secretary for *education*
- and research has given a notice about Saami higher education and
- research.'*
- - **rektor (N):**
- **Rektor* Tove Bull álgaga mielde... - 'According to *principal*
- Tove Bull ...'*
- - **Tove (N Prop):**
- *Rektor *Tove* Bull álgaga mielde... - 'According to principal
- *Tove* Bull ...'*
-- **@N<:**
- Modifier of the noun to the left.
- - **33 (Num):**
- *Mun lean ilus go beasan ovdanbuktit St.dieđ. nr. *33*. - 'I am
- happy that I get the opportunity to present the parliament
- notice number *33*.'* (In this case *33* modifies *St.dieđ.*.)
- - **vihtta (Num):**
- *Mun boađán diibmu *vihtta*. - 'I will come at *five* o'clock.'*
-- **@>Num:**
- Attributes of numeral to the right.
- - **nr (N):**
- *Mun lean ilus go beasan ovdanbuktit St.dieđ. *nr.* 33. - 'I am
- happy that I get the opportunity to present the parliament
- notice *number* 33.'*
-- **@Num<:**
- Attributes of numeral to the left.
- - **jagi (N):**
- *Son lea guoktelogi *jagi* boaris. - 'She/he is twenty *years*
- old.'*
-- **@<OBJ:**
- Direct object to the right of the finite verb.
- - **áiggi (N):**
- *Dat gáibida ollu *áiggi*. - 'That demands a lot of *time*.'*
-- **@OBJ>:**
- Direct object to the left of the finite verb.
- - **maid:**
- *Filbma lea oassi prošeavttas *maid* Sámi instituhtta lea
- ruthadan. - 'The film is a part of the project that the Saami
- institute has financed.'*
-- **@OPRED>:**
- Object predicative to the left of the finite verb.
- - **luoikkasin (N):**
- *Gaup dojii stivrrana hárjehallamiin, muhto oaččui *luoikkasin*
- eará stivrrana. - 'Gaup broke the handlebars during the
- practises, but got to *borrow* another steering.'*
-- **@<OPRED:**
- Object predicative.
- - **buriid (A):**
- *Gáhkkuid son ráhkada hui *buriid*. - 'Cakes, she/he makes
- really *good ones*.'*
- - **sámegielhállin (N):**
- *Dagat iežat *sámegielhállin*. - 'You make yourself *a Saami
- speaker*.'*
-- **@>P:**
- Complement of postposition to the left of it.
- - **oahpu (N), dutkama (N):**
- *Oahppo- ja dutkanministtar dat lea ráhkadan dieđáhusa alit sámi
- *oahpu* ja *dutkama* birra. - 'The secretary for education and
- research has given a notice about Saami higher *education* and
- *research*.'*
-- **@P<:**
- Complement of preposition to the right of it.
- - **oasálaččaid (N):**
- *Finnmárkkus ii goassige leat leamaš ságastallan gaskal muhtun
- muddui seammadássásaš *oasálaččaid*. - 'There has never been a
- discussion in Finnmark between somehow equal *parts*.'*
-- **@PCLE:**
- Particle.
- - **amma (Pcle):**
- **Amma* mii eat leat máksán? - 'We haven't paid, *have we*?'*
-- **@POSS>:**
- kal
- - **Jiisusi-Kristusip (N):**
- **Jiisusi-Kristusip*, Daavip ernerata, Aaperaap ernerata,
- eqqarliisa allassimaffiat.*
-- **@PPRED:**
- a predicative with a predicative as its head
- - **reaŋgan (N):**
- *Máhtes lea Jovnna *reaŋgan*. - 'Máhtte has Jovnna *as a
- searvant*.'*
-- **@>Pron:**
- Modifier of a pronoun to the left of it.
- - **buot (Pron):**
- *Mun, Johanas, lean dat guhte lean gullan ja oaidnán *buot*
- dán. - 'I, Johanas, am the one who has heard and seen *all* of
- it.'*
-- **@Pron<:**
- Modifier of pronoun to the right of it.
- - **ipmašiid (N):**
- *Maid *ipmašiid* doppe dagat? - 'What *the heck* are you doing
- there?'*
- - **golmmas (N):**
- *Mii *golmmas* oktan du vieljain finaimet Niillas-čeazi
- geahčen. - 'We *three* together with your brother visited uncle
- Niillas.'*
-- **@SPRED:**
- Subject predicative in elliptical sentences.
- - **nommh (N):**
- *Die maa onterligksh nommh, ih goh tuhtjh, men die ligan
- onterligksh nierretjh aaj. - ' '*
- - **:**
- *(sma)*
-- **@<SPRED:**
- Subject predicative to the right of the finite verb.
- - **galbmasat (A):**
- *Mus leat gieđat nu *galbmasat*. - 'My hands are so *cold*.'*
- - **beana (N):**
- *Mus lea *beana*. - 'I have *a dog*.'*
-- **@SPRED>:**
- Subject predicative to the right of the finite verb.
- - **vuođđun (N):**
- *Kommišuvnna evttohusaid *vuođđun* lea guohtundilalašvuođaid
- vuđolaš čielggadeapmi, man fágalávdegotti ášše-dovdit dahke.*
-- **@SPRED<OBJ:**
- - **:**
-- **@SUBJ:**
- Elliptical subject.
- - **ålma (N):**
- *Dennie synnagovgesne jis akte ålma maam doenh-aajmoe
- doerelamme. - ' '*
-- **@SUBJ>:**
- Subject to the left of the finite verb.
- - **son (Pron):**
- **Son* lea mu oabbá. - '*She* is my sister.'*
- - **luopmánat (N):**
- *Jeakkis leat *luopmánat*. - 'There are *cloudberries* in the
- swamp.'*
-- **@<SUBJ:**
- Subject to the right of the finite verb.
- - **ollusat (Pron):**
- *...ja dan vejolašvuođa orro gal *ollusat* geavahan. - '...and
- this opportunity, *many* seem to make use of.'*
-- **@SUBJ<ADVL:**
- - **:**
-- **@SUBJ\_COMP:**
- fao
- - **:**
-- **@<SUBJ\_COMP:**
- predicate of a subject/ subject complement (kal)
- - **:**
-- **@SUBJ<OBJ:**
- - **:**
-- **@tSUBJ:**
- Elliptical subject.
- - **tað (Pron):**
- **Tað* er ikki longur pláss fyri, at lutir og kenslur bara eru.*
-- **@i-ADVL>:**
- kal
- - **Babylonimut (N):**
- *Josijap Jekonja qatanngutaalu Babylonimut aallarussaanerup
- nalaani.*
-- **@i-><ADVL:**
- kal
- - **pruffiitikkut (N):**
- *Tamakku tamarmik pipput Naalakkap pruffiitikkut oqaaserisaa
- eqquuteqqullugu, oqarmat: »Takuat, niviarsiaq naartulissaaq
- ernertaassallunilu, atsissavaallu Immanuelimik« – imaappoq:
- Guuti ilagaarput.*
-- **@i->N:**
- kal
- - **naammaginartunik (N):**
- *Namminersornerusut Nuummi illunik ima amerlatigisunik
- tunisaqarsimalerput aningaasanut inatsit
- iluatsitaariniarfigisariaqalerlugu inissianik isatereriarlutik
- nutaanik sanaartortariaqaleramik atorfilittatik naammaginartunik
- inissaqartissinnaajumallugit.*
-- **@i->V:**
- kal
- - **tutinneq (N):**
- *Ernertaartinnaguli tutinneq ajorpaa.*
-- **>@V:**
- kal
- - **:**
-- **@VOC:**
- Vocative.
- - **hearrá:**
- **Hearrá*, du ráhkis ustit lea buohcci. - '*Lord*, your beloved
- friend is ill.'*
-- **<ctjHead>:**
- coordinated head, can be of different PoS' (V, A, N ...). The PoS
- taking part in coordination do not necessarily be of the same kind.
- The tag is useful if the coordinated part does not directly follow
- it's predecessor.
- - **geahččá:**
- *Dat *geahččá* nuppiid stáluide girkes čalmmiiguin ja reašká
- romet. - \`He/she *looks* at the other trolls with clear eyes
- and laughs hideously.'*
- - **stuorrát:**
- *Gieđat leat *stuorrát* dego steaikabánnot ja guolgan. - \`The
- hands are as *big* as fryingpans and covered with hair.'*
- - **soalsin:**
- *Skávžá lea buot *soalsin* ja njuoskkas. - \`The beard is all
- *covered with spit* and wet.'*
-- **<mv>:**
- main verb, especially useful in cases where the verb can be both a
- main verb and an auxiliary
- - **dárbbašit, oažžut:**
- *Danne *dárbbašit* mii *oažžut* lobi Nils Aslak Valkeapää
- árbbolaččain almmuhit dán guokte lávlaga min sálbma-CD:s. -
- \`Therefore we *need* to *get* permission from Nils Aslak
- Valkeapää's heirs to release these two songs on our psalm-CD.'*
-- **<vdic>:**
- verba dicendi, those that introduce direct speech, typically words
- of communication such as lohkat, cealkat, dadjat, oaivildit
- - **celkkii:**
- *Eŋgel *celkkii* munnje: Dát leat luohtehahtti ja duohta
- sánit. - \`The angel *told* me: These are trustworthy and true
- words.'*
-
-Coordination
-============
+- **@>A:**
+ Modifier of an adjective to the left.
+ - **nu (Adv):**
+ - Gulahallan Sámedikkiin dán gažaldagas šaddá _nu_
+ konkrehtalažžan go vejolaš. - 'The discussion in the Saami
+ Parliament about this issue gets _as_ concrete as possible.'
+- **@A<:**
+ Modifier of an adjective to the right.
+ - **:**
+- **@>Adv:**
+ Modifier of an adverb.
+ - **:**
+- **@Adv<:**
+ Complement of an adverb.
+ - **:**
+- **@ADVL:**
+ Sentence adverbial.
+ - **dál (Adv):**
+ _Dál_ lea Bireha vuorru. - 'It is Biret's turn _now_.'
+- **@>ADVL:**
+ Modifier of an adverbial.
+ - **Man (Pron):**
+ _Man_ dávjá don lávet fitnat doppe? - '_How_ often do you
+ usually go there?'
+- **@<ADVL:**
+ adverbial to the right of the finite verb
+ - **beaivvážis (N):**
+ - Gávpot ii dárbbaš čuovgga _beaivvážis_ ii ge mánus. - 'The city
+ does not need light _from the sun_ and not the from the moon
+ either.
+- **@ADVL<:**
+ Complement of an adverbial.
+ - **vahkus (N):**
+ Mun málestan guktii _vahkus_. - 'I make food twice a _week_.'
+- **@ADVL>:**
+ Adverbial to the left of the finite verb.
+ - **lasttain (N):**
+ Ja muora _lasttain_ ožžot álbmogat dearvvašvuođa. - 'And from
+ the tree's _leaves_, the people get health.'
+- **@ADVL>CS:**
+ adverbial modifying a conjunction
+ - **dallah (Adv):**
+ _Dallah_ goh Jeesuse tjaetseste tjuedtjele, dellie vuajna Elmie
+ rihpesåvva jih Voejkene altasasse goh ledtie suaja. - '_Right_
+ after Jesus stood up from the water, he sees that heaven opens
+ and the holy spirit flies to him like a bird.'
+ (_sma_)
+ - **dan dihte (Adv):**
+ Muhto go lassánedje olbmot, de bohte čáhppesbivttasolbmot fas
+ dohko, gosa ledje sámit vuohččan ballán, ja dahke orohagaid jur
+ dasa gos sámit ledje orrume, _dan dihte_ go sii oidne, ahte das
+ leai čáppa gieddi, maid ledje bohccot dutken, gožžan ja baikán —
+ gos ledje sámit orron mánga olmmošbuolvva.
+- **@ADVL<OBJ:**
+ - **:**
+- **@ADVL>SUBJ:**
+ - **:**
+- **@AGENS>:**
+ kal
+ - **atorfilittanit:**
+ _Attartortumiit piginnittumut aaqqissuussineq
+ namminersornerusuni atorfilittanit politikerinillu nuimasunit
+ isertortumik atornerlunneqarsimammat illoqarfinni anginerni
+ pingasuni attartortut nalinginnaasut pillarneqartussanngorput._
+- **@APP-ADVL<:**
+ Apposition to an adverbial to the left. If the apposition consists
+ of more than one word, the head will get this tag.
+ - **ovdal (Pr):**
+ Dolin, _ovdal_ soađi, olbmot lávejedje vuovdit joŋaid. - 'In
+ old times, _before_ the war, people used to sell cowberries.'
+- **@APP-N<:**
+ Apposition to a noun to the left of it. If the apposition is more
+ than one constituent, the head will get this tag.
+ - **eatnigiela (N):**
+ Viimmat mun ohppen čállit sámegiela, mu _eatnigiela_. -
+ 'Finally, I learned to write in Sámi, my _mother tongue_.'
+- **@APP-Num<:**
+ Apposition to a numeral to the left.
+ - **suinniid (N):**
+ Juohke heasta borrá sullii 6 kilu _suinniid_ beaivái. - 'Every
+ horse eats approximately 6 kilograms of _grass_ a day.'
+- **@APP>Pron:**
+ Apposition to a pronoun to the right. If the apposition is more than
+ one constituent, the head will get this tag.
+ - **Turner (N Prop):**
+ Muhto diet Will _Turner_, son nai lea fiinna olmmái. - 'But
+ this Will _Turner_, he is also a nice guy.'
+- **@APP-Pron<:**
+ Apposition to a pronoun to the left. If the apposition is more than
+ one constituent, the head will get this tag.
+ - **olbmái (N):**
+ Dan mun muitalan dušše dutnje, mu buoremus _olbmái_. - 'This I
+ tell only you, my best _friend_.'
+- **@>CC:**
+ modifier of CC
+ - **sihke (CC):**
+- **@>CC:**
+ modifier of CC
+ - **sihke (CC):**
+- **@CL-ADVL>:**
+ - **:**
+- **@CL-<ADVL:**
+ - **:**
+- **@CMPND:**
+ First part of a compound followed by a hyphen
+ - **skaehtie-:**
+ Reerenasse galka båetije stoerredigkieboelhkesne jåerhkedh dam
+ _skaehtie-_ jïh åasadaltesem mij lea daelie, jïh daennie
+ daltesisnie hov lea nuepie buerebe joekedimmiem darjodh.
+- **@CNP:**
+ Local conjunction or subjunction.
+ - **ja (CC):**
+ Sihke Mázes _ja_ Guovdageainnus leat boarrásat viššalit finadan
+ doaibmaguovddážiin. - 'Both in Máze _and_ Guovdageaidnu, the
+ oldest people frequently got to the activitycentre.'
+ - **go (CS):**
+ Sámi geavaheaddjit hállet dávjá metaforaiguin ja sis leat ollu
+ eará gulahallanvuogit _go_ giella. - 'Saami users speak often in
+ metaphores and the have many other ways of comunicating _than_
+ by means of language.'
+- **@COMPL-CS<:**
+ Complement of subjunction.
+ - **vejolaš (A):**
+ Gulahallan Sámedikkiin dán gažaldagas šaddá nu konkrehtalažžan
+ go _vejolaš_. - 'The contact with the Saami Parliament about
+ this issue gets as concrete as _possible_.'
+- **@CVP:**
+ Conjunction or subjunction that conjoins finite verb phrases
+ - **ja (CC):**
+ Bealatjogas leat dološ rájes leamaš bálvvossajit _ja_ dát
+ golbma sieiddi ledje dovddus gitta olgoriikii. - 'Long since,
+ there have been sacrificial sites at Bealatjohka _and_ the three
+ 'sieidi' (cult images) were known even abroad.
+ - **go (CS):**
+ Leago guhkes áigi dassá _go_ Máreha oidnet? - 'Has it been a
+ long time _since_ you have seen Máret?'
+- **@FAUX:**
+ finite auxiliary
+ - **ledje (V):**
+ Gávpotmuvrra vuođđogeađggit _ledje_ čiŋahuvvon juohke lágán
+ divrras geđggiiguin. - 'The cornerstones of the wall _were_
+ decorated with every kind of expensive stones.'
+- **@-F<ADVL:**
+ Adverbial of infinite verb outside of the predicate
+ - **árbbolaččain (N):**
+ Danne dárbbašit mii oažžut lobi Nils Aslak Valkeapää
+ _árbbolaččain_ almmuhit dán guokte lávlaga min sálbma-CD:s. -
+ 'Therefore we need to get permission from Nils Aslak
+ Valkeapää's _heirs_ to release these two songs on our
+ psalm-CD.'
+- **@-FADVL>:**
+ Adverbial of infinite verb outside the predicate
+ - **várrogasat (Adv):**
+ Dihkkadeaddji rávve skohtervuddjiid _várrogasat_ mátkkoštit.
+ 'The roadman warns snowscooter drivers to drive _carefully_.'
+- **@FMV:**
+ finite mainverb
+ - **lei (V):**
+ Gávpot lei njealječiegat, seammá guhkki go govdat. - 'The city
+ _was_ a square, same width as length.'
+- **@FMVdic:**
+ - **muitala (V):**
+ Ja go geassit eret dábálaš goluid, de lea buhtes sisaboahtu
+ sullii 100 000 ruvnnu, _muitala_ Eriksen. - 'And when we take
+ away/subtract? the regular expenses, there is a remaining income
+ of about 100 000 crowns, _says_ Eriksen.'
+- **@-F<OBJ:**
+ Object of infinite verb outside the verbal to the right of it.
+ - **govaid (N):**
+ Boađe mu lusa geahččat _govaid_! - 'Come to me and look _at
+ the pictures_!'
+- **@-FOBJ>:**
+ Object of infinite verb outside the verbal to the left of it.
+ - **váldovuoittuid (N):**
+ Valáštallanhálla lei njealjehas dievva olbmuiguin geat vurde
+ _váldovuoittuid_ fasket. - 'The gymn was to a quarter full of
+ people that wait to grab _the main prizes_.'
+- **@-F<SPRED:**
+ - **duhtavaččat (A):**
+ IL Nordlysa beaivválaš jođiheaddji, Nils Peder Eriksen, lohká
+ iežaset leat oalle _duhtavaččat_ dán jagáš básárdoaluin.
+- **@FM-SPRED<:**
+ main clause functioning as a subject predicate to the right of
+ another main clause
+ - **ii (V):**
+ Ja dasa lea dát sivva: go sápmelaš boahtá moskkus gámmirii, de
+ son ii _ii_ ipmir ii báljo maidege, go ii biegga beasa bossut
+ njuni vuostá. - 'And this is the reason: if a Saami comes ...,
+ then he does _not_ understand ...'
+- **@FS-ADVL>:**
+ subclause functioning as an adverbial to the finite verb of the main
+ clause to the right of it.
+ - **bohtet (V):**
+ Ja mo jos Muhtinlágan Stálu ustibat _bohtet_ fitnat. - 'And
+ what if the friends of some-kind-of troll _come_ for a visit.'
+- **@FS-<ADVL:**
+ subclause functioning as an adverbial to the finite verb of the main
+ clause to the left of it.
+ - **galggai (V):**
+ Go _galggai_ bargat rehkenastimiin sus šattai álo
+ oaivvebávččas. - 'When they _should_ work with arithmetics, she
+ always got a headache.'
+- **@FS-IAUX:**
+ subclause infinite auxiliary
+ - **sáhte:**
+ Mun in _sáhte_ muitalit dán dutnje. - 'I *can*not tell you
+ this.'
+- **@FS-IMV:**
+ subclause infinite mainverb
+ - **ohcamin (V):**
+ Naba jos eadni lea sudno _ohcamin_, iige gávnna. - 'And if
+ mother is _searching_ for them, she will not find them.'
+- **@FS-N<:**
+ finite verb (either an auxiliary or main verb) of a relative
+ subclause (with a noun (N) antecedent)
+ - **lea (V):**
+ De son viežžá liegga liema ruittus mii _lea_ oapmana alde.
+ 'Then he fetched warm broth from the pot that _is_ on the
+ stove.'
+- **@FS-N<IAUX:**
+ infinite auxiliary of a (relative) subclause
+ - **sáhttán (V):**
+ Mun oidnen nieidda gii ii _sáhttán_ boahtit. - 'I saw the girl
+ that _could_ not come.'
+- **@FS-N<IMV:**
+ infinite mainverb of a (relative) subclause
+ - **bargan (V):**
+ Mon lean okta sápmelaš, guhte lean _bargan_ visot sámi bargguid
+ ja mon dovddan visot sámi dili. - 'I am a Sámi, who has
+ _worked_ in all Saami occupations and I know all Saami
+ affairs.'
+- **@FS-OBJ:**
+ finite verb of the subclause that has an object function
+ - **leahkkasii (V):**
+ Arne ii fuobmán ahte uksa _leahkkasii_. - 'Arne did not notice
+ that the door _opened_.'
+- **@FS-OBJ>:**
+ finite verb of a subclause that has object function used for kal
+ e.g.
+ - **pigilissagaa (V):**
+ _Nunani issittuni sila pillugu tunngaviusumik ilisimasalernissaq
+ silamut ilisimatusarfiup pigilissagaa ministerit marluullutik
+ isumaqatigiipput, silallu allanngoriartornerata sunniutai
+ maluginiarneqassasut._
+- **@FS-P<:**
+ finite verb of a subclause
+ - **eru (V):**
+ _Tað er ikki longur pláss fyri, at lutir og kenslur bara eru. -
+ 'There is no longer any space for that things and feelings
+ simply are.'_
+ - **:**
+ _(fao)_
+- **@FS-P<IMV:**
+ finite verb of a subclause in fao
+ - **:**
+- **@-F<OPRED:**
+ - **:**
+- **@-FSUBJ>:**
+ subject of a verbal infinitival object
+ - **mánáid (N):**
+ Muhtinlágan Stállu cáhpá goikebierggu sudnuide ja dáhttu
+ _mánáid_ boradit. - 'Some-sort-of troll cuts dried meat for them
+ and asks _the children_ to eat.'
+- **@FS-SUBJ:**
+ finite verb head of a subordinate clause functioning as a subject
+ - **boađát (V):**
+ Dehálaš lea ahte don maid _boađát_. - 'It is important that you
+ _come_ too.'
+- **@FS-VFIN<:**
+ - **eai (V):**
+ Idja ii leat šat, _eai_ ge sii dárbbaš lámppá dahje beaivváža
+ čuovgga, dasgo Hearrá Ipmil lea sin čuovga. - 'The night is not
+ anymore, they do _not_ need the lamp- or day- light either,
+ because God the Lord is their light.'
+- **@HAB:**
+ Habitive, for a human target in illative or locative in a habitive
+ construction (copula), is translated as "have". Possible verbs in a
+ habitive construction are "boahtit" 'come', "leat" 'be', "goallut"
+ 'pass', "heaŋgát", 'hang', "jápmit" 'die', "šaddat" 'become'.
+ - **Máhtes (N):**
+ _Máhtes_ lea beana. - '_Máhtte_ has a dog.'
+ - **sus (Pron):**
+ _Sus_ šattai álo oaivvebávččas go galggai bargat
+ rehkenastimiin. - '_She_ always got a headache when she was
+ supposed to work with arithmetics.'
+- **@HNOUN:**
+ Stray noun in sentence fragments.
+ - **boddu (N):**
+ Vuosttaš _boddu_. - 'First _lesson_.'
+- **@IAUX:**
+ non-finite auxiliary
+ - **veaje (V):**
+ Dattetge ii _veaje_ oađđit. - 'Still she did not _manage_ to
+ sleep.'\*
+- **@ICL-ADVL:**
+ infinitival clause adverbial
+ - **árvvoštallat (V):**
+ Son namuha ahte sii leat gal ávžžuhan olbmuid geat dihtet ahte
+ dárbbašit jođánit beassat buohccevissui, nugo ovdamearkka dihte
+ áhpehis nissonolbmuid geain lahkona riegádahttináigi,
+ árvvoštallat\* galget go ovdal go buohccájit juo vuolgit
+ buohccevissui.
+- **@ICL-OBJ:**
+ infinitival clause object
+ - **boradit (V):**
+ Muhtinlágan Stállu cáhpá goikebierggu sudnuide ja dáhttu mánáid
+ _boradit_. - 'Some-sort-of troll cuts dried meat for them and
+ asks the children _to eat_.'
+- **@ICL-P<:**
+ infinitival complement of a preposition
+ - **skriva (V):**
+ _Kenslan gav Unn íblástur til at skriva nakrar yrkingar um
+ næstrakærleika._
+ - **:**
+ _(fao)_
+- **@ICL-SUBJ:**
+ infinitival subject
+ - **sløkkja (V):**
+ _Men tað er líka skjótt at sløkkja ljósið, lata eygu og oyru
+ aftur og rista ábyrgdina av okkum._
+ - **:**
+ _(fao)_
+- **@IM:**
+ fao
+ - **at (IM):**
+ _Tá koyrdi Harrin Guð hann út úr aldingarðinum í Eden og setti
+ hann til at dyrka ta jørð, sum hann var sjálvur tikin av._
+- **@IMV:**
+ non-finite mainverb
+ - **čiŋahuvvon (V):**
+ Gávpotmuvrra vuođđogeađggit ledje _čiŋahuvvon_ juohke lágán
+ divrras geđggiiguin. - 'The cornerstones of the wall were
+ _decorated_ with every kind of expensive stones.'
+- **@INF->N:**
+ kal
+ - **pillugu (V):**
+ _Nunani issittuni sila pillugu tunngaviusumik ilisimasalernissaq
+ silamut ilisimatusarfiup pigilissagaa ministerit marluullutik
+ isumaqatigiipput, silallu allanngoriartornerata sunniutai
+ maluginiarneqassasut._
+- **@INTERJ:**
+ Interjection.
+ - **maid (Interj):**
+ _Maid_, iigo leat boahtán? - '_What_, hasn't he/she come?'
+- **@<IOBJ:**
+ indirect object to the right of the finite verb.
+ - **(N):**
+- **@IOBJ>:**
+ Indirect object to the left of the finite verb.
+ - **(N):**
+- **@MIK-OBJ:**
+ kal
+ - **illunik (N):**
+ Namminersornerusut Nuummi _illunik_ ima amerlatigisunik
+ tunisaqarsimalerput aningaasanut inatsit
+ iluatsitaariniarfigisariaqalerlugu inissianik isatereriarlutik
+ nutaanik sanaartortariaqaleramik atorfilittatik naammaginartunik
+ inissaqartissinnaajumallugit.
+- **@>N:**
+ Prenominal modifier to the left
+ - **geavatlaš (A):**
+ Ráđđehussii lea _geavatlaš_ politihkka deaŧalaš. - 'For the
+ government, _practical_ politics is important.'
+ - **oahppo-:**
+ _Oahppo-_ ja dutkanministtar dat lea ráhkadan dieđáhusa alit
+ sámi oahpu ja dutkama birra. - 'The secretary for _education_
+ and research has given a notice about Saami higher education and
+ research.'
+ - **rektor (N):**
+ _Rektor_ Tove Bull álgaga mielde... - 'According to _principal_
+ Tove Bull ...'
+ - **Tove (N Prop):**
+ Rektor _Tove_ Bull álgaga mielde... - 'According to principal
+ _Tove_ Bull ...'
+- **@N<:**
+ Modifier of the noun to the left.
+ - **33 (Num):**
+ Mun lean ilus go beasan ovdanbuktit St.dieđ. nr. _33_. - 'I am
+ happy that I get the opportunity to present the parliament
+ notice number _33_.'\* (In this case _33_ modifies _St.dieđ._.)
+ - **vihtta (Num):**
+ Mun boađán diibmu _vihtta_. - 'I will come at _five_ o'clock.'
+- **@>Num:**
+ Attributes of numeral to the right.
+ - **nr (N):**
+ Mun lean ilus go beasan ovdanbuktit St.dieđ. _nr._ 33. - 'I am
+ happy that I get the opportunity to present the parliament
+ notice _number_ 33.'
+- **@Num<:**
+ Attributes of numeral to the left.
+ - **jagi (N):**
+ Son lea guoktelogi _jagi_ boaris. - 'She/he is twenty _years_
+ old.'
+- **@<OBJ:**
+ Direct object to the right of the finite verb.
+ - **áiggi (N):**
+ Dat gáibida ollu _áiggi_. - 'That demands a lot of _time_.'
+- **@OBJ>:**
+ Direct object to the left of the finite verb.
+ - **maid:**
+ Filbma lea oassi prošeavttas _maid_ Sámi instituhtta lea
+ ruthadan. - 'The film is a part of the project that the Saami
+ institute has financed.'
+- **@OPRED>:**
+ Object predicative to the left of the finite verb.
+ - **luoikkasin (N):**
+ Gaup dojii stivrrana hárjehallamiin, muhto oaččui _luoikkasin_
+ eará stivrrana. - 'Gaup broke the handlebars during the
+ practises, but got to _borrow_ another steering.'
+- **@<OPRED:**
+ Object predicative.
+ - **buriid (A):**
+ Gáhkkuid son ráhkada hui _buriid_. - 'Cakes, she/he makes
+ really _good ones_.'
+ - **sámegielhállin (N):**
+ Dagat iežat _sámegielhállin_. - 'You make yourself _a Saami
+ speaker_.'
+- **@>P:**
+ Complement of postposition to the left of it.
+ - **oahpu (N), dutkama (N):**
+ Oahppo- ja dutkanministtar dat lea ráhkadan dieđáhusa alit sámi
+ _oahpu_ ja _dutkama_ birra. - 'The secretary for education and
+ research has given a notice about Saami higher _education_ and
+ _research_.'
+- **@P<:**
+ Complement of preposition to the right of it.
+ - **oasálaččaid (N):**
+ Finnmárkkus ii goassige leat leamaš ságastallan gaskal muhtun
+ muddui seammadássásaš _oasálaččaid_. - 'There has never been a
+ discussion in Finnmark between somehow equal _parts_.'
+- **@PCLE:**
+ Particle.
+ - **amma (Pcle):**
+ _Amma_ mii eat leat máksán? - 'We haven't paid, _have we_?'
+- **@POSS>:**
+ kal
+ - **Jiisusi-Kristusip (N):**
+ _Jiisusi-Kristusip_, Daavip ernerata, Aaperaap ernerata,
+ eqqarliisa allassimaffiat.
+- **@PPRED:**
+ a predicative with a predicative as its head
+ - **reaŋgan (N):**
+ Máhtes lea Jovnna _reaŋgan_. - 'Máhtte has Jovnna _as a
+ searvant_.'
+- **@>Pron:**
+ Modifier of a pronoun to the left of it.
+ - **buot (Pron):**
+ Mun, Johanas, lean dat guhte lean gullan ja oaidnán _buot_
+ dán. - 'I, Johanas, am the one who has heard and seen _all_ of
+ it.'
+- **@Pron<:**
+ Modifier of pronoun to the right of it.
+ - **ipmašiid (N):**
+ Maid _ipmašiid_ doppe dagat? - 'What _the heck_ are you doing
+ there?'
+ - **golmmas (N):**
+ Mii _golmmas_ oktan du vieljain finaimet Niillas-čeazi
+ geahčen. - 'We _three_ together with your brother visited uncle
+ Niillas.'
+- **@SPRED:**
+ Subject predicative in elliptical sentences.
+ - **nommh (N):**
+ _Die maa onterligksh nommh, ih goh tuhtjh, men die ligan
+ onterligksh nierretjh aaj. - ' '_
+ - **:**
+ _(sma)_
+- **@<SPRED:**
+ Subject predicative to the right of the finite verb.
+ - **galbmasat (A):**
+ Mus leat gieđat nu _galbmasat_. - 'My hands are so _cold_.'
+ - **beana (N):**
+ Mus lea _beana_. - 'I have _a dog_.'
+- **@SPRED>:**
+ Subject predicative to the right of the finite verb.
+ - **vuođđun (N):**
+ Kommišuvnna evttohusaid _vuođđun_ lea guohtundilalašvuođaid
+ vuđolaš čielggadeapmi, man fágalávdegotti ášše-dovdit dahke.
+- **@SPRED<OBJ:**
+ - **:**
+- **@SUBJ:**
+ Elliptical subject.
+ - **ålma (N):**
+ _Dennie synnagovgesne jis akte ålma maam doenh-aajmoe
+ doerelamme. - ' '_
+- **@SUBJ>:**
+ Subject to the left of the finite verb.
+ - **son (Pron):**
+ _Son_ lea mu oabbá. - '_She_ is my sister.'
+ - **luopmánat (N):**
+ Jeakkis leat _luopmánat_. - 'There are _cloudberries_ in the
+ swamp.'
+- **@<SUBJ:**
+ Subject to the right of the finite verb.
+ - **ollusat (Pron):**
+ ...ja dan vejolašvuođa orro gal _ollusat_ geavahan. - '...and
+ this opportunity, _many_ seem to make use of.'
+- **@SUBJ<ADVL:**
+ - **:**
+- **@SUBJ_COMP:**
+ fao
+ - **:**
+- **@<SUBJ_COMP:**
+ predicate of a subject/ subject complement (kal)
+ - **:**
+- **@SUBJ<OBJ:**
+ - **:**
+- **@tSUBJ:**
+ Elliptical subject.
+ - **tað (Pron):**
+ _Tað_ er ikki longur pláss fyri, at lutir og kenslur bara eru.
+- **@i-ADVL>:**
+ kal
+ - **Babylonimut (N):**
+ _Josijap Jekonja qatanngutaalu Babylonimut aallarussaanerup
+ nalaani._
+- **@i-><ADVL:**
+ kal
+ - **pruffiitikkut (N):**
+ Tamakku tamarmik pipput Naalakkap _pruffiitikkut_ oqaaserisaa
+ eqquuteqqullugu, oqarmat: »Takuat, niviarsiaq naartulissaaq
+ ernertaassallunilu, atsissavaallu Immanuelimik« – imaappoq:
+ Guuti ilagaarput.
+- **@i->N:**
+ kal
+ - **naammaginartunik (N):**
+ Namminersornerusut Nuummi illunik ima amerlatigisunik
+ tunisaqarsimalerput aningaasanut inatsit
+ iluatsitaariniarfigisariaqalerlugu inissianik isatereriarlutik
+ nutaanik sanaartortariaqaleramik atorfilittatik _naammaginartunik_
+ inissaqartissinnaajumallugit.
+- **@i->V:**
+ kal
+ - **tutinneq (N):**
+ Ernertaartinnaguli _tutinneq_ ajorpaa.
+- **>@V:**
+ kal
+ - **:**
+- **@VOC:**
+ Vocative.
+ - **hearrá:**
+ _Hearrá_, du ráhkis ustit lea buohcci. - _Lord_, your beloved
+ friend is ill.'
+- **<ctjHead>:**
+ coordinated head, can be of different PoS' (V, A, N ...). The PoS
+ taking part in coordination do not necessarily be of the same kind.
+ The tag is useful if the coordinated part does not directly follow
+ it's predecessor.
+ - **geahččá:**
+ Dat _geahččá_ nuppiid stáluide girkes čalmmiiguin ja reašká
+ romet. - 'He/she _looks_ at the other trolls with clear eyes
+ and laughs hideously.'
+ - **stuorrát:**
+ Gieđat leat _stuorrát_ dego steaikabánnot ja guolgan. - 'The
+ hands are as _big_ as fryingpans and covered with hair.'
+ - **soalsin:**
+ Skávžá lea buot _soalsin_ ja njuoskkas. - 'The beard is all
+ _covered with spit_ and wet.'
+- **<mv>:**
+ main verb, especially useful in cases where the verb can be both a
+ main verb and an auxiliary
+ - **dárbbašit, oažžut:**
+ Danne _dárbbašit_ mii _oažžut_ lobi Nils Aslak Valkeapää
+ árbbolaččain almmuhit dán guokte lávlaga min sálbma-CD:s. -
+ 'Therefore we _need_ to _get_ permission from Nils Aslak
+ Valkeapää's heirs to release these two songs on our psalm-CD.'
+- **<vdic>:**
+ verba dicendi, those that introduce direct speech, typically words
+ of communication such as lohkat, cealkat, dadjat, oaivildit
+ - **celkkii:**
+ Eŋgel _celkkii_ munnje: Dát leat luohtehahtti ja duohta
+ sánit. - 'The angel _told_ me: These are trustworthy and true
+ words.'
+
+## Coordination
Here are some examples of our coordination-analysis:
"Náhkiin sii gorro roavgguid, dorkkaid ja gápmagiid."
From the skin they were sewing furs, coats and shoes.
-
+
gorro
- / | \
+ / | \
Náhkiin sii roavgguid,
/ \
doarkkaid gápmagiid.
/
- ja
-
+ ja
+
+
-
"Bárdni válddii niibbi ja čuohpai ráiggi sehkkii ja luittii mánáid olggos."
The boy took the knife and cut a hole in the bag and let the children out.
-
+
válddii_____
/ / \ \
Bárdni niibbi \ luittii
| čuohpai | \
ja / | mánáid olggos.
- / |
+ / |
ráiggi sehkkii
- |
+ |
ja
-
-
-Complex sentences
-=================
+## Complex sentences
Here are some examples:
@@ -694,7 +685,7 @@ Here are some examples:
vuoitá,__
/ | \ \
- Jus stállu de \
+ Jus stállu de \
goddá
/ \
son olbmo.
@@ -704,8 +695,8 @@ Here are some examples:
it __
/ | \
- don beasa \
- / \ \
+ don beasa \
+ / \ \
Dasto ruoktot borat
| \
jus luhtte.
@@ -724,9 +715,9 @@ Here are some examples:
| / | \
ja iđeda fas dan.
/
- nuppi
+ nuppi
- No verb in the main clause:
+ No verb in the main clause:
"Ovdal buorida Ipmil dálkkiidis go neavrres olmmoš dábiidis."
Rather does God improve the weather than a miserable person his habits.
@@ -737,16 +728,14 @@ Here are some examples:
/ | \
go neavrres dábiidis.
-Punctuation
-===========
+## Punctuation
Punctuation such as ".", "," and ";" also receive dependency tags. The
sentence "Arvigoahtá. - It starts raining" actually consists of two
elements, the finite verb and the punctuation. The full stop is also
interpreted as a dependent of the root "\#2->0".
-Arguments and adjuncts
-======================
+## Arguments and adjuncts
Subcategorized arguments such as "beatnagis - of the dog" in the
sentence "Balat go beatnagis? - Are you afraid of the dog" are
diff --git a/lang/common/docu-sme-grammartags.md b/lang/common/docu-sme-grammartags.md
index a0d31b8a..18e6f12e 100644
--- a/lang/common/docu-sme-grammartags.md
+++ b/lang/common/docu-sme-grammartags.md
@@ -1,8 +1,7 @@
On the bottom of this page is a list with all tags in alphabetical
order.
-Overview
-========
+# Overview
All the words are analysed with dictionary form + grammatical tags. Each
tag is introduced with a "+" sign. We thus have
@@ -21,8 +20,7 @@ Nouns (+N), adjectives (+A), verbs (+V), pronouns (+Pron), adverbs
(+Adv), particles (+Pcle), subjunctions (+CS), conjunctions (+CC),
postpositions (+Po), prepositions (+Pr) and interjections (+Interj).
-The nouns
-=========
+## The nouns
The string is
+N+(Subclass)+(Semclass)+Number+Case(+Possessivesuffix)(+Clitic)". The
@@ -34,7 +32,7 @@ analysis. Note that the grammatical categories in parentheses can be
omitted.
| | |
-|---------------------|----------------------------------------------------------------|
+| ------------------- | -------------------------------------------------------------- |
| Part of speech | +N |
| Subclass | +Prop, +G3, +NomAg |
| Semantic class | +Sem/Hum, +Sem/Plc, +Sem/Veh (see list) |
@@ -43,13 +41,12 @@ omitted.
| (Possessive suffix) | +PxSg1 +PxSg2 +PxSg3 +PxDu1 +PxDu2 +PxDu3 +PxPl1 +PxPl2 +PxPl3 |
| (Clitic) | +Qst +Foc/ |
-The adjectives
-==============
+## The adjectives
Used non-attributively the adjective resembles the noun:
| | |
-|----------------|------------------------------------|
+| -------------- | ---------------------------------- |
| Part of speech | +A |
| (Grade) | +Comp, +Superl |
| Number | +Sg, +Pl |
@@ -59,19 +56,18 @@ Used non-attributively the adjective resembles the noun:
Used attributively the adjective has a quite simple tag scheme:
| | |
-|----------------|----------------------|
+| -------------- | -------------------- |
| Part of speech | +A |
| Attribute | +Attr |
| (Clitic) | e.g. +Qst (see list) |
-The verbs
-=========
+## The verbs
Finite and infinite verb forms have quite distinct paradigms. Finite
first:
| | |
-|----------------|-------------------------------------------------------|
+| -------------- | ----------------------------------------------------- |
| Part of speech | +V |
| (Derivation) | +Der/PassL, +Der/PassS, +Der/h (see list) |
| Mood | +Ind, +Pot, +Cond, +Imprt |
@@ -82,7 +78,7 @@ first:
Infinite verb forms:
| | |
-|-------------------|----------------------------------------------------|
+| ----------------- | -------------------------------------------------- |
| Part of speech | +V |
| (Derivation) | +Der/PassL |
| Nominal verb form | +Inf, +Act, +Ger, +PrsPrc, +PrfPrc, +VGen, +VAbess |
@@ -91,7 +87,7 @@ Infinite verb forms:
Other derived verb forms:
| | |
-|---------------------|----------------------------------------------------------------|
+| ------------------- | -------------------------------------------------------------- |
| Part of speech | +V |
| Part of speech | +N |
| Derivation | +NomAg |
@@ -101,8 +97,7 @@ Other derived verb forms:
Here is an example: `oahppit` > `oahppi+N+NomAg+Pl+Nom`
-The pronouns
-============
+## The pronouns
The personal, demonstrative and interrogative pronouns:
@@ -114,230 +109,223 @@ underlying form: `mun+Pron+Pers+Sg1+Com`, surface form: `muinna`
The reflexive pronouns:
-baseform+Pron+pronoun\_type+Case(+possessive suffix)
+baseform+Pron+pronoun_type+Case(+possessive suffix)
Example: underlying form: `ieš+Pron+Refl+Loc+PxDu1`, surface form:
`alddáme`
-The indeclinable words
-======================
+## The indeclinable words
These have their POS tag as their only tag:
underlying form: `birra+Pr` or `birra+Po`, surface form: `birra`
-Alphabetic list over the tags
-=============================
-
-Part of speech and subclass
-===========================
-
-- **+A** adjective
- - **+Ord** ordinal
-- **+Adv** adverb
-- **+CC** conjunction
-- **+CS** subjunction
-- **+Interj** interjection
-- **+N** noun
- - **+Prop** proper noun
- - **+NomAg** agent noun
- - **+G3** geminat grade 3, e.g. s's
-- **+Num** numeral
- - **+Card** cardinal number
-- **+Pcle** particle
-- **+Po** postposition
-- **+Pr** preposition
-- **+Pron** pronoun
- - **+Dem** demonstrative pronoun
- - **+Indef** indefinite pronoun
- - **+Interr** interrogative pronoun
- - **+Pers** person pronoun
- - **+Refl** reflecsive pronoun
- - **+Recipr** reciprocal pronoun
- - **+Rel** relative pronoun
-- **+N, +Adv**for several parts of speech
- - **+ABBR** abbreviation
- - **+ACR** acronym
-
-Grammatical properties
-======================
-
-- **+Acc** Accusative
-- **+Actio** Actio form of the verb
-- **+Attr** Attributive
-- **+CLB** Clause boundary
-- **+Cmpnd** Compound (inconsistent with other notation for this tag)
-- **+Cmpnd+** Compound (left tag)(inconsistent with other notation for
- this tag)
-- **+Com** Comitative
-- **+Comp** Comperative
-- **+ConNeg** Negationform of the verb
-- **+Cond** Conditional
-- **+Du**Dual
-- **+Du1** Dual 1. person
-- **+Du2** Dual 2. person
-- **+Du3** Dual 3. person
-- **+Ess** Essive
-- **+Foc/ba** Focusclitic (-)ba
-- **+Foc/bat** Focusclitic (-)bat
-- **+Foc/be** Focusclitic (-)be
-- **+Foc/ge** Focusclitic (-)ge
-- **+Foc/gen** Focusclitic (-)gen
-- **+Foc/ges** Focusclitic (-)ges
-- **+Foc/gis** Focusclitic (-)gis
-- **+Foc/hal** Focusclitic (-)hal
-- **+Foc/han** Focusclitic (-)han
-- **+Foc/naj** Focusclitic (-)nai
-- **+Foc/naj+Qst** Focusclitic (-)naigo
-- **+Foc/son** Focusclitic (-)son
-- **+Gen** Genitive
-- **+Ger** Gerund
-- **+IV** Intrasitive verb
-- **+Ill** Illative
-- **+Imprt** Imperative
-- **+Ind** Indicative
-- **+Inf** Infinitive
-- **+LEFT** Left parenthesis
-- **+Loc** Locative
-- **+Neg** Negationverb
-- **+Nom** Nominative
-- **+PUNCT** Punctuation other than clause boundaries*"/", "(", ")",
- "+"*
-- **+Pl** Plural
-- **+Pl1** Plural 1. person
-- **+Pl2** Plural 2. person
-- **+Pl3** Plural 3. person
-- **+Cmp/PlGen** Plural genitive compound
-- **+Pot** Potential
-- **+PrfPrc** Perfect participle
-- **+Prs** Present tense
-- **+PrsPrc** Present participle
-- **+Prt** Preteritum
-- **+PxDu1** Possessivesuffix dual 1. person
-- **+PxDu2** Possessivesuffix dual 2. person
-- **+PxDu3** Possessivesuffix dual 3. person
-- **+PxPl1** Possessivesuffix plural 1. person
-- **+PxPl2** Possessivesuffix plural 2. person
-- **+PxPl3** Possessivesuffix plural 3. person
-- **+PxSg1** Possessivesuffix singular 1. person
-- **+PxSg2** Possessivesuffix singular 2. person
-- **+PxSg3** Possessivesuffix singular 3. person
-- **+Qst** Question clitic *(-)go*
-- **+Qst+Foc/son** Question clitic
-- **+RIGHT** Right parenthesis
-- **+Sg** Singular
-- **+Sg1** Singular 1. person
-- **+Sg2** Singular 2. person
-- **+Sg3** Singular 3. person
-- **+Cmp/Sg** Singular compound
-- **+Cmp/SgGen** Singular genitive compound
-- **+Cmp/SgNom** Singular nominative compound
-- **+Sup** Supinum
-- **+Superl** Superlative
-- **+TV** Transitive verb
-- **+VAbess** Verb abessive
-- **+VGen** Verb genitive
-
-Derivational suffix tags
-========================
-
-- **+Der/Dimin** Diminutive
-- **+Der/adda** suffix
-- **+Der/ahtti** suffix
-- **+Der/alla** suffix
-- **+Der/amoš** suffix
-- **+Der/asti** suffix
-- **+Der/aš** suffix
-- **+Der/d** suffix
-- **+Der/duohkai** suffix
-- **+Der/duohke** suffix
-- **+Der/NomAg** agent noun
-- **+Der/eamoš** suffix
-- **+Der/NomAct** action noun
-- **+Der/easti** suffix
-- **+Der/g** suffix
-- **+Der/geahtes** suffix
-- **+Der/goahti** suffix
-- **+Der/h** suffix
-- **+Der/halla** suffix
-- **+Der/hat** suffix
-- **+Der/heapmi** suffix
-- **+Der/hudda** suffix
-- **+Der/huhtti** suffix
-- **+Der/huvva** suffix
-- **+Der/j** suffix
-- **+Der/l** suffix
-- **+Der/las** suffix
-- **+Der/laš** suffix
-- **+Der/lágan** suffix
-- **+Der/meahttun** suffix
-- **+Der/muš** suffix
-- **+Der/PassL** passive verb, long form
-- **+Der/PassS** passive verb, short form
-- **+Der/st** suffix
-- **+Der/stuvva** suffix
-- **+Der/supmi** suffix
-- **+Der/upmi** suffix
-- **+Der/us** suffix
-- **+Der/viđi** suffix
-- **+Der/viđá** suffix
-- **+Der/vuohta** suffix
-- **+Der/vuolde** suffix
-- **+Der/vuollai** suffix
-- **+Der/vuolle** suffix
-- **+Der/š** suffix
-
-Semantic tags
-=============
+## Alphabetic list over the tags
+
+## Part of speech and subclass
+
+- **+A** adjective
+ - **+Ord** ordinal
+- **+Adv** adverb
+- **+CC** conjunction
+- **+CS** subjunction
+- **+Interj** interjection
+- **+N** noun
+ - **+Prop** proper noun
+ - **+NomAg** agent noun
+ - **+G3** geminat grade 3, e.g. s's
+- **+Num** numeral
+ - **+Card** cardinal number
+- **+Pcle** particle
+- **+Po** postposition
+- **+Pr** preposition
+- **+Pron** pronoun
+ - **+Dem** demonstrative pronoun
+ - **+Indef** indefinite pronoun
+ - **+Interr** interrogative pronoun
+ - **+Pers** person pronoun
+ - **+Refl** reflecsive pronoun
+ - **+Recipr** reciprocal pronoun
+ - **+Rel** relative pronoun
+- **+N, +Adv**for several parts of speech
+ - **+ABBR** abbreviation
+ - **+ACR** acronym
+
+## Grammatical properties
+
+- **+Acc** Accusative
+- **+Actio** Actio form of the verb
+- **+Attr** Attributive
+- **+CLB** Clause boundary
+- **+Cmpnd** Compound (inconsistent with other notation for this tag)
+- **+Cmpnd+** Compound (left tag)(inconsistent with other notation for
+ this tag)
+- **+Com** Comitative
+- **+Comp** Comperative
+- **+ConNeg** Negationform of the verb
+- **+Cond** Conditional
+- **+Du**Dual
+- **+Du1** Dual 1. person
+- **+Du2** Dual 2. person
+- **+Du3** Dual 3. person
+- **+Ess** Essive
+- **+Foc/ba** Focusclitic (-)ba
+- **+Foc/bat** Focusclitic (-)bat
+- **+Foc/be** Focusclitic (-)be
+- **+Foc/ge** Focusclitic (-)ge
+- **+Foc/gen** Focusclitic (-)gen
+- **+Foc/ges** Focusclitic (-)ges
+- **+Foc/gis** Focusclitic (-)gis
+- **+Foc/hal** Focusclitic (-)hal
+- **+Foc/han** Focusclitic (-)han
+- **+Foc/naj** Focusclitic (-)nai
+- **+Foc/naj+Qst** Focusclitic (-)naigo
+- **+Foc/son** Focusclitic (-)son
+- **+Gen** Genitive
+- **+Ger** Gerund
+- **+IV** Intrasitive verb
+- **+Ill** Illative
+- **+Imprt** Imperative
+- **+Ind** Indicative
+- **+Inf** Infinitive
+- **+LEFT** Left parenthesis
+- **+Loc** Locative
+- **+Neg** Negationverb
+- **+Nom** Nominative
+- **+PUNCT** Punctuation other than clause boundaries*"/", "(", ")",
+ "+"*
+- **+Pl** Plural
+- **+Pl1** Plural 1. person
+- **+Pl2** Plural 2. person
+- **+Pl3** Plural 3. person
+- **+Cmp/PlGen** Plural genitive compound
+- **+Pot** Potential
+- **+PrfPrc** Perfect participle
+- **+Prs** Present tense
+- **+PrsPrc** Present participle
+- **+Prt** Preteritum
+- **+PxDu1** Possessivesuffix dual 1. person
+- **+PxDu2** Possessivesuffix dual 2. person
+- **+PxDu3** Possessivesuffix dual 3. person
+- **+PxPl1** Possessivesuffix plural 1. person
+- **+PxPl2** Possessivesuffix plural 2. person
+- **+PxPl3** Possessivesuffix plural 3. person
+- **+PxSg1** Possessivesuffix singular 1. person
+- **+PxSg2** Possessivesuffix singular 2. person
+- **+PxSg3** Possessivesuffix singular 3. person
+- **+Qst** Question clitic _(-)go_
+- **+Qst+Foc/son** Question clitic
+- **+RIGHT** Right parenthesis
+- **+Sg** Singular
+- **+Sg1** Singular 1. person
+- **+Sg2** Singular 2. person
+- **+Sg3** Singular 3. person
+- **+Cmp/Sg** Singular compound
+- **+Cmp/SgGen** Singular genitive compound
+- **+Cmp/SgNom** Singular nominative compound
+- **+Sup** Supinum
+- **+Superl** Superlative
+- **+TV** Transitive verb
+- **+VAbess** Verb abessive
+- **+VGen** Verb genitive
+
+## Derivational suffix tags
+
+- **+Der/Dimin** Diminutive
+- **+Der/adda** suffix
+- **+Der/ahtti** suffix
+- **+Der/alla** suffix
+- **+Der/amoš** suffix
+- **+Der/asti** suffix
+- **+Der/aš** suffix
+- **+Der/d** suffix
+- **+Der/duohkai** suffix
+- **+Der/duohke** suffix
+- **+Der/NomAg** agent noun
+- **+Der/eamoš** suffix
+- **+Der/NomAct** action noun
+- **+Der/easti** suffix
+- **+Der/g** suffix
+- **+Der/geahtes** suffix
+- **+Der/goahti** suffix
+- **+Der/h** suffix
+- **+Der/halla** suffix
+- **+Der/hat** suffix
+- **+Der/heapmi** suffix
+- **+Der/hudda** suffix
+- **+Der/huhtti** suffix
+- **+Der/huvva** suffix
+- **+Der/j** suffix
+- **+Der/l** suffix
+- **+Der/las** suffix
+- **+Der/laš** suffix
+- **+Der/lágan** suffix
+- **+Der/meahttun** suffix
+- **+Der/muš** suffix
+- **+Der/PassL** passive verb, long form
+- **+Der/PassS** passive verb, short form
+- **+Der/st** suffix
+- **+Der/stuvva** suffix
+- **+Der/supmi** suffix
+- **+Der/upmi** suffix
+- **+Der/us** suffix
+- **+Der/viđi** suffix
+- **+Der/viđá** suffix
+- **+Der/vuohta** suffix
+- **+Der/vuolde** suffix
+- **+Der/vuollai** suffix
+- **+Der/vuolle** suffix
+- **+Der/š** suffix
+
+## Semantic tags
These are tags used for classifying names and nouns, e.g. +Prop+Sem/Fem
-- **+Sem/Ani:**
- Animal name
-- **+Sem/Fem:**
- Female name
-- **+Sem/Mal:**
- Male name
-- **+Sem/Obj:**
- Object
-- **+Sem/Org:**
- Organisation name
-- **+Sem/Plc:**
- Place
-- **+Sem/Sur:**
- Surname
-- **+Sem/WEB:**
- Web addresse
-
-A critical discussion of some particular tags
-=============================================
-
-**Determiner or Pronoun**
+- **+Sem/Ani:**
+ Animal name
+- **+Sem/Fem:**
+ Female name
+- **+Sem/Mal:**
+ Male name
+- **+Sem/Obj:**
+ Object
+- **+Sem/Org:**
+ Organisation name
+- **+Sem/Plc:**
+ Place
+- **+Sem/Sur:**
+ Surname
+- **+Sem/WEB:**
+ Web addresse
+
+## A critical discussion of some particular tags
+
+### Determiner or Pronoun
the POSs of words like buot, dat, etc. get different terms in the
literature.
Nickel:
-- indefinite pronouns ("ubestemte pronomen"): vaikko mii, mihkkege,
- buot, buohkat, eanas...
+- indefinite pronouns ("ubestemte pronomen"): vaikko mii, mihkkege,
+ buot, buohkat, eanas...
Vesa Guttorm:
-- determiner ("pronomendeterminatiiva")
+- determiner ("pronomendeterminatiiva")
Magga 1980:
-- demonstrative pronouns ("čujuheaddji pronomenat"): dat, dát, diet,
- duot, dot
-- interrogative pronouns ("gažaldatpronomenat")
-- relative pronouns ("relatiivapronomenat")
-- personal pronouns ("persovnnalaš pronomenat")
-- indefinite pronouns ("indefinihtta (mearritkeahtes) pronomenat"):
- muhtun, soames, goappašagat
-- reciprocal pronouns ("resiprohka pronomenat: goabbat buoibmame,
- guhtet guoimmiset")
+- demonstrative pronouns ("čujuheaddji pronomenat"): dat, dát, diet,
+ duot, dot
+- interrogative pronouns ("gažaldatpronomenat")
+- relative pronouns ("relatiivapronomenat")
+- personal pronouns ("persovnnalaš pronomenat")
+- indefinite pronouns ("indefinihtta (mearritkeahtes) pronomenat"):
+ muhtun, soames, goappašagat
+- reciprocal pronouns ("resiprohka pronomenat: goabbat buoibmame,
+ guhtet guoimmiset")
Outi Kilpimaa:
-- determiner ("dat-determinánttalš"): dat,...
+- determiner ("dat-determinánttalš"): dat,...
diff --git a/lang/common/docu-sme-syntaxtags.md b/lang/common/docu-sme-syntaxtags.md
index f985ff04..24bc5dd6 100644
--- a/lang/common/docu-sme-syntaxtags.md
+++ b/lang/common/docu-sme-syntaxtags.md
@@ -1,5 +1,4 @@
-Documentation of the syntactic tags
-===============
+# Documentation of the syntactic tags
See also separate pages on [compound](CompoundTags.html),
[semantic](SemanticTags.html), [morphological](MorphologicalTags.html)
@@ -8,19 +7,18 @@ and [dependency](docu-deptags.html) tags.
On the bottom of this page there is a list with all tags in alphabetical
order.
-Syntactic tags
-=========================
+## Syntactic tags
Our syntactic tags, or grammatical function tags, like @SUBJ>, @OBJ>,
etc., are based upon a balanced compromise between 3 principles:
-1. use the same tags across *giellalt* languages
-1. use the conventions from within within constraint grammar (CG), e.g. as found in [the visl project](http://visl.sdu.dk/) for interactive syntax learning
-1. take the grammatical tradition of the language in question into account
+1. use the same tags across _giellalt_ languages
+1. use the conventions from within within constraint grammar (CG), e.g. as found in [the visl project](http://visl.sdu.dk/) for interactive syntax learning
+1. take the grammatical tradition of the language in question into account
The main difference between the CG tradition (both giellalt and visl CG) and other descriptions is that CG is a linear system, where tags are given to **wordforms**, and not to **phrases**.
-Thus, in a sentence like the Saami equivalent of *Peter's dog barks*
-only the word *dog* will get the tag @SUBJ>. The word *Peter's* gets
+Thus, in a sentence like the Saami equivalent of _Peter's dog barks_
+only the word _dog_ will get the tag @SUBJ>. The word _Peter's_ gets
the tag @>N, or "modifying a noun to its right". It is then up to the
reader (or to further computer processing) to interpret the combination
of @>N and @SUBJ> as a phrase (phrase information will also be available via the [dependency tags](docu-deptags.html) when they are present).
@@ -43,45 +41,40 @@ distinguish them from morphological tags, which do not have such a
prefix. In the analysis, the syntactic tags are printed at the end of
the tag string.
-The syntactic tags for Saami
-===========================
+## The syntactic tags for Saami
-We present here the tags used for the Saami languages (the best developed languages in the *Giellalt* infrastructure), but linguists working on other languages will find the presentation useful. The rules assigning tags are found in the file `lang-xxx/src/cg3/disambiguation.cg3`, where xxx is the ISO code of your language.
+We present here the tags used for the Saami languages (the best developed languages in the _Giellalt_ infrastructure), but linguists working on other languages will find the presentation useful. The rules assigning tags are found in the file `lang-xxx/src/cg3/disambiguation.cg3`, where xxx is the ISO code of your language.
+### The verb tags
-The verb tags
--------------
-
-- @+FAUXV @+FMAINV @-FAUXV @-FMAINV
+- @+FAUXV @+FMAINV @-FAUXV @-FMAINV
These tags are self-explanatory: there are finite and infinite main and
auxiliary verbs.
-The major function tags
------------------------
+### The major function tags
-- @<SUBJ (@<SUBJ @<ext>) @SUBJ> @SUBJ @<SPRED
- @SPRED> @SPRED @<OBJ @OBJ> @OBJ @<OPRED @OPRED>
- @OPRED
+- @<SUBJ (@<SUBJ @<ext>) @SUBJ> @SUBJ @<SPRED
+ @SPRED> @SPRED @<OBJ @OBJ> @OBJ @<OPRED @OPRED>
+ @OPRED
The four main functions for subject, object and their predicatives are
obvious.
-- @-FSUBJ> @-FOBJ> @-F<OBJ
+- @-FSUBJ> @-FOBJ> @-F<OBJ
These are tags for the same functions of infinite verbs outside the
-verbal: *mu* gets @-FSUBJ> in *Diet dáhpáhuvai mu dieđikeahttá* (the
-infinite verb gets @<ADVL) and *girjji* gets @-F<OBJ in *Munnje
-lei lossat lohkat girjji.* (the infinite verb gets @<SPRED).
+verbal: _mu_ gets @-FSUBJ> in _Diet dáhpáhuvai mu dieđikeahttá_ (the
+infinite verb gets @<ADVL) and _girjji_ gets @-F<OBJ in _Munnje
+lei lossat lohkat girjji._ (the infinite verb gets @<SPRED).
-The adverbial tags
-------------------
+### The adverbial tags
-- @ADVL> @<ADVL @ADVL
-- @-FADVL
-- @PCLE @COMP-CS<
-- @P< @>P
-- @ADVL< @>ADVL
+- @ADVL> @<ADVL @ADVL
+- @-FADVL
+- @PCLE @COMP-CS<
+- @P< @>P
+- @ADVL< @>ADVL
The @ADVL> @<ADVL @ADVL tags mark adverbials (many, but not all of
the adverbials are adverbs). The two first ones indicate the direction
@@ -101,377 +94,371 @@ or is a complement of the adverbial to the left, respectively. Note that
these tags mark modifyers of adverbials rather than adverbials
themselves.
-The NP-internal modifiers
--------------------------
+### The NP-internal modifiers
The other syntactic tags for modifiers tell which word they modify, and
whether they modify to the left or to the right.
-- @>N @>A @>Num @>Pron
-- @Pron< @N< @Num<
+- @>N @>A @>Num @>Pron
+- @Pron< @N< @Num<
The morphological tag will tell what kind of part of speech the
constituent itself is.
The @Pron< tag is for eg. numerals modifying pronouns to their left,
-like in *Mii golmmas finaimet máná geahčen*.
+like in _Mii golmmas finaimet máná geahčen_.
-The @Num< tag is for complements of numerals, like *máná* in *Sudnos
-leat golbma máná*.
+The @Num< tag is for complements of numerals, like _máná_ in _Sudnos
+leat golbma máná_.
-Appositions
------------
+### Appositions
-- @APP-N< @APP-Pron< @APP-Num< @APP-ADVL<
-- @APP>Pron
+- @APP-N< @APP-Pron< @APP-Num< @APP-ADVL<
+- @APP>Pron
The apposition tag marks whether it is an apposition of a noun, a
pronoun, a numeral or an adverbial.
-The function words
-------------------
+### The function words
-- @CNP @CVP
+- @CNP @CVP
Conjunction connecting NPs and VPs.
-Sentence-external tags
-----------------------
+### Sentence-external tags
-- @HNOUN @INTERJ @VOC
+- @HNOUN @INTERJ @VOC
Stray noun in sentence fragment, interjection and vocative.
-The @X tag
-----------
+### The @X tag
-- @X
+- @X
An @X tag is assigned to mark that no tag has been assigned (because of
omissions in our rule component)
-The tags, listed alphabetically
-===============================
+## The tags, listed alphabetically
Here is a list of the tags, with a definition or description, and one or
more examples following each of them
-- **@+FAUXV:**
- Finite auxiliary verb.
- - **ferte (V):**
- *Sámi geavaheddjiid bálvalusaid vuođđun *ferte* leat
- sámegielmáhttu ja sámi kulturmáhttu. - 'Saami user services
- *need* to be based on Saami language competence and Saami
- cultural competence.'*
-- **@+FMAINV:**
- Finite main verb.
- - **Boađe (V):**
- **Boađe* boahtte vahku. - '*Come* next week.'*
-- **@-F<ADVL:**
- - **árbbolaččain (N):**
- *Danne dárbbašit mii oažžut lobi Nils Aslak Valkeapää
- *árbbolaččain* almmuhit dán guokte lávlaga min sálbma-CD:s. -
- 'Therefore we need to get permission from Nils Aslak Valkeapää's
- *heirs* to release these two songs on our psalm-CD.'*
-- **@-F<OBJ:**
- Object of infinite verb outside the verbal.
- - **govaid (N):**
- *Boađe mu lusa geahččat *govaid*! - 'Come to me and look at *the
- pictures*!'*
-- **@-F<OPRED:**
- Object predicative of infinite verb outside the verbal.
- - **xxx:**
- *xxx*
-- **@-F<SPRED:**
- Subject predicative of infinite verb outside the verbal.
- - **xxx:**
- *xxx*
-- **@>A:**
- Modifier of an adjective to the left.
- - **nu (Adv):**
- *Gulahallan Sámedikkiin dán gažaldagas šaddá *nu*
- konkrehtalažžan go vejolaš. - 'The discussion in the Saami
- Parliament about this issue gets *as* concrete as possible.'*
-- **@A<:**
- Modifier of an adjective to the right.
- - **básárdoaluin (N):**
- *IL Nordlysa beaivválaš jođiheaddji, Nils Peder Eriksen, lohká
- iežaset leat oalle duhtavaččat dán jagáš *básárdoaluin*. - 'The
- business manager of IL Nordlys, Nils Peder Eriksen, says he is
- really satisfied with this year's *bazar arrangment*.'*
-- **@ADVL:**
- Sentence adverbial, @ADVL> or @<ADVL.
-- **@>ADVL:**
- Modifier of an adverbial.
- - **Man (Adv):**
- **Man* dávjá don lávet fitnat doppe? - '*How* often do you
- usually go there?'*
-- **@<ADVL:**
- adverbial to the right of the finite verb
- - **beaivvážis (N):**
- *Gávpot ii dárbbaš čuovgga *beaivvážis* ii ge mánus. - 'The city
- does not need light *from the sun* and not the from the moon
- either.*
-- **@ADVL>:**
- Adverbial to the left of the finite verb.
- - **lasttain (N):**
- *Ja muora *lasttain* ožžot álbmogat dearvvašvuođa. - 'And from
- the tree's *leaves*, the people get health.'*
-- **@ADVL<:**
- Complement of an adverbial to the right of its head.
- - **vahkus (N):**
- *Mun málestan guktii *vahkus*. - 'I make food twice a *week*.'*
-- **@ADVL>CS:**
- adverbial modifying a conjunction
- - **dallah (Adv):**
- **Dallah* goh Jeesuse tjaetseste tjuedtjele, dellie vuajna Elmie
- rihpesåvva jih Voejkene altasasse goh ledtie suaja. - '*Right*
- after Jesus stood up from the water, he sees that heaven opens
- and the holy spirit flies to him like a bird.'*
- - **:**
- *(sma)*
-- **@APP-ADVL<:**
- Apposition to an adverbial to the left. If the apposition consists
- of more than one word, the head will get this tag.
- - **ovdal (Pr):**
- *Dolin, *ovdal* soađi, olbmot lávejedje vuovdit joŋaid. - 'In
- old times, *before* the war, people used to sell cowberries.'*
-- **@APP-N<:**
- Apposition to a noun to the left of it. If the apposition is more
- than one word, the head will get this tag.
- - **eatnigiela (N):**
- *Viimmat mun ohppen čállit sámegiela, mu *eatnigiela*. -
- 'Finally, I learned to write in Sámi, my *mother tongue*.'*
-- **@APP-Num<:**
- Apposition to a numeral to the left.
- - **suinniid (N):**
- *Juohke heasta borrá sullii 6 kilu *suinniid* beaivái. - 'Every
- horse eats approximately 6 kilograms of *grass* a day.'*
-- **@APP>Pron:**
- Apposition to a pronoun to the right. If the apposition is more than
- one constituent, the head will get this tag.
- - **Turner (N Prop):**
- *Muhto diet Will *Turner*, son nai lea fiinna olmmái. - 'But
- this Will *Turner*, he is also a nice guy.'*
-- **@APP-Pron<:**
- Apposition to a pronoun to the left. If the apposition is more than
- one constituent, the head will get this tag.
- - **olbmái (N):**
- *Dan mun muitalan dušše dutnje, mu buoremus *olbmái*. - 'This I
- tell only you, my best *friend*.'*
-- **@CMPND:**
- First part of a compound followed by a hyphen
- - **skaehtie-:**
- *Reerenasse galka båetije stoerredigkieboelhkesne jåerhkedh dam
- *skaehtie-* jïh åasadaltesem mij lea daelie, jïh daennie
- daltesisnie hov lea nuepie buerebe joekedimmiem darjodh.*
-- **@CNP:**
- Local conjunction or subjunction.
- - **ja (CC):**
- *Sihke Mázes *ja* Guovdageainnus leat boarrásat viššalit finadan
- doaibmaguovddážiin. - 'Both in Máze *and* Guovdageaidnu, the
- oldest people frequently got to the activitycentre.'*
- - **go (CS):**
- *Sámi geavaheaddjit hállet dávjá metaforaiguin ja sis leat ollu
- eará gulahallanvuogit *go* giella. - 'Saami users speak often in
- metaphores and the have many other ways of comunicating *than*
- by means of language.'*
-- **@COMP-CS<:**
- Complement of subjunction.
- - **vejolaš (A):**
- *Gulahallan Sámedikkiin dán gažaldagas šaddá nu konkrehtalažžan
- go *vejolaš*. - 'The contact with the Saami Parliament about
- this issue gets as concrete as *possible*.'*
-- **@CVP:**
- Conjunction or subjunction that conjoins finite verb phrases.
- - **ja (CC):**
- *Bealatjogas leat dološ rájes leamaš bálvvossajit *ja* dát
- golbma sieiddi ledje dovddus gitta olgoriikii. - 'Long since,
- there have been sacrificial sites at Bealatjohka *and* the three
- 'sieidi' (cult images) were known even abroad.*
- - **go (CS):**
- *Leago guhkes áigi dassá *go* Máreha oidnet? - 'Has it been a
- long time *since* you have seen Máret?'*
-- **@-FADVL>:**
- Complement of infinite verb outside the verbal.
- - **várrogasat (Adv):**
- *Dihkkadeaddji rávve skohtervuddjiid *várrogasat* mátkkoštit.
- 'The roadman warns snowscooter drivers to drive *carefully*.'*
-- **@-FAUXV:**
- Infinite auxiliary verb.
- - **sáhte (V):**
- *Eat mii *sáhte* vuolgit. - 'We *can*not leave.'*
-- **@-FMAINV:**
- Infinite main verb.
- - **geargan (V):**
- *Ja Biret-Elle lea easka skuvllas *geargan*. - 'And Biret-Elle
- has just *finished* school.'*
-- **@-FOBJ>:**
- Object of infinite verb outside the verbal.
- - **váldovuoittuid (N):**
- *Valáštallanhálla lei njealjehas dievva olbmuiguin geat vurde
- *váldovuoittuid* fasket. - 'The gymn was to a quarter full of
- people that wait to grab *the main prizes*.'*
-- **@-FSUBJ>:**
- Subject of infinite verb outside the verbal.
- - **mu (Pron):**
- *Diet dáhpáhuvai *mu* dieđikeahttá. - 'It happened without *me*
- knowing about it.'*
-- **@ADVL> <hab>:**
- Habitive to the left of the finite verb.
- - **Máhtes (N):**
- **Máhtes* lea beana. - '*Máhtte* has a dog.'*
-- **@<ADVL <hab>:**
- Habitive to the right of the finite verb.
- - **dus (Pron):**
- *Leago *dus* ruhta? - 'Do *you* have money?'*
-- **@HNOUN:**
- Stray noun in sentence fragments.
- - **boddu (N):**
- *Vuosttaš *boddu*. - 'First *lesson*.'*
-- **@INTERJ:**
- Interjection.
- - **maid (Interj):**
- **Maid*, iigo leat boahtán? - '*What*, hasn't he/she come?'*
-- **@>N:**
- Prenominal modifier to the left
- - **geavatlaš (A):**
- *Ráđđehussii lea *geavatlaš* politihkka deaŧalaš. - 'For the
- government, *practical* politics is important.'*
- - **oahppo-:**
- **Oahppo-* ja dutkanministtar dat lea ráhkadan dieđáhusa alit
- sámi oahpu ja dutkama birra. - 'The secretary for *education*
- and research has given a notice about Saami higher education and
- research.'*
- - **rektor (N):**
- **Rektor* Tove Bull álgaga mielde... - 'According to *principal*
- Tove Bull ...'*
- - **Tove (N Prop):**
- *Rektor *Tove* Bull álgaga mielde... - 'According to principal
- *Tove* Bull ...'*
-- **@N<:**
- Modifier of the noun to the left.
- - **33 (Num):**
- *Mun lean ilus go beasan ovdanbuktit St.dieđ. nr. *33*. - 'I am
- happy that I get the opportunity to present the parliament
- notice number *33*.'* (In this case *33* modifies *St.dieđ.*.)
- - **vihtta (Num):**
- *Mun boađán diibmu *vihtta*. - 'I will come at *five* o'clock.'*
-- **@>Num:**
- Attributes of numeral to the right.
- - **nr (N):**
- *Mun lean ilus go beasan ovdanbuktit St.dieđ. *nr.* 33. - 'I am
- happy that I get the opportunity to present the parliament
- notice *number* 33.'*
-- **@Num<:**
- Attributes of numeral to the left.
- - **jagi (N):**
- *Son lea guoktelogi *jagi* boaris. - 'She/he is twenty *years*
- old.'*
-- **@<OBJ:**
- Direct object to the right of the finite verb.
- - **áiggi (N):**
- *Dat gáibida ollu *áiggi*. - 'That demands a lot of *time*.'*
-- **@OBJ>:**
- Direct object to the left of the finite verb.
- - **maid (Pron):**
- *Filbma lea oassi prošeavttas *maid* Sámi instituhtta lea
- ruthadan. - 'The film is a part of the project *that* the Saami
- institute has financed.'*
-- **@OPRED>:**
- Object predicative to the left of the finite verb.
- - **luoikkasin (N):**
- *Gaup dojii stivrrana hárjehallamiin, muhto oaččui *luoikkasin*
- eará stivrrana. - 'Gaup broke the handlebars during the
- practises, but got to *borrow* another steering.'*
-- **@<OPRED:**
- Object predicative to the right of the finite verb.
- - **buriid (A):**
- *Gáhkkuid son ráhkada hui *buriid*. - 'Cakes, she/he makes
- really *good ones*.'*
- - **sámegielhállin (N):**
- *Dagat iežat *sámegielhállin*. - 'You make yourself *a Saami
- speaker*.'*
-- **@>P:**
- Complement of postposition to the left of it.
- - **oahpu (N), dutkama (N):**
- *Oahppo- ja dutkanministtar dat lea ráhkadan dieđáhusa alit sámi
- *oahpu* ja *dutkama* birra. - 'The secretary for education and
- research has given a notice about Saami higher *education* and
- *research*.'*
-- **@P<:**
- Complement of preposition to the right of it.
- - **oasálaččaid (N):**
- *Finnmárkkus ii goassige leat leamaš ságastallan gaskal muhtun
- muddui seammadássásaš *oasálaččaid*. - 'There has never been a
- discussion in Finnmark between somehow equal *parts*.'*
-- **@PCLE:**
- Particle.
- - **amma (Pcle):**
- **Amma* mii eat leat máksán? - 'We haven't paid, *have we*?'*
-- **@<PPRED:**
- a predicative with a predicative as its head
- - **reaŋgan (N):**
- *Máhtes lea Jovnna *reaŋgan*. - 'Máhtte has Jovnna *as a
- searvant*.'*
-- **@>Pron:**
- Modifier of a pronoun to the left of it.
- - **buot (Pron):**
- *Mun, Johanas, lean dat guhte lean gullan ja oaidnán *buot*
- dán. - 'I, Johanas, am the one who has heard and seen *all* of
- it.'*
-- **@Pron<:**
- Modifier of pronoun to the right of it.
- - **ipmašiid (N):**
- *Maid *ipmašiid* doppe dagat? - 'What *the heck* are you doing
- there?'*
- - **golmmas (N):**
- *Mii *golmmas* oktan du vieljain finaimet Niillas-čeazi
- geahčen. - 'We *three* together with your brother visited uncle
- Niillas.'*
-- **@SPRED:**
- Subject predicative in elliptical sentences.
- - **nommh (N):**
- *Die maa onterligksh nommh, ih goh tuhtjh, men die ligan
- onterligksh nierretjh aaj.*
- - **:**
- *(sma)*
-- **@<SPRED:**
- Subject predicative to the right of the finite verb.
- - **galbmasat (A):**
- *Mus leat gieđat nu *galbmasat*. - 'My hands are so *cold*.'*
-- **@SPRED>:**
- Subject predicative to the left of the finite verb.
- - **bargu (N):**
- *Sin *bargun* lei váldit fáŋgan Gonagasa. - 'Their *job* was to
- capture the King.'*
-- **@SUBJ:**
- Elliptical subject.
- - **ålma (N):**
- *Dennie synnagovgesne jis akte ålma maam doenh-aajmoe
- doerelamme.*
-- **@SUBJ>:**
- Subject to the left of the finite verb.
- - **son (Pron):**
- **Son* lea mu oabbá. - '*She* is my sister.'*
-- **@<SUBJ:**
- Subject to the right of the finite verb.
- - **ollusat (Pron):**
- *...ja dan vejolašvuođa orro gal *ollusat* geavahan. - '...and
- this opportunity, *many* seem to make use of.'*
-- **@<SUBJ <ext>:**
- Subject to the right of the finite verb, in a habitive or extencial
- construction.
- - **beana (N):**
- *Mus lea *beana*. - 'I have *a dog*.'*
- - **luopmánat (N):**
- *Jeakkis leat *luopmánat*. - 'There are *cloudberries* in the
- swamp.'*
-- **@VOC:**
- Vocative.
- - **hearrá:**
- **Hearrá*, du ráhkis ustit lea buohcci. - '*Lord*, your beloved
- friend is ill.'*
-- **@X:**
- A dummy tag assigned when no tag assignment rule has hit. This tag
- is useful for finding the flaws in the tag mapping section.
+- **@+FAUXV:**
+ Finite auxiliary verb.
+ - **ferte (V):**
+ Sámi geavaheddjiid bálvalusaid vuođđun _ferte_ leat
+ sámegielmáhttu ja sámi kulturmáhttu. - 'Saami user services
+ _need_ to be based on Saami language competence and Saami
+ cultural competence.'
+- **@+FMAINV:**
+ Finite main verb.
+ - **Boađe (V):**
+ _Boađe_ boahtte vahku. - '_Come_ next week.'
+- **@-F<ADVL:**
+ - **árbbolaččain (N):**
+ Danne dárbbašit mii oažžut lobi Nils Aslak Valkeapää
+ _árbbolaččain_ almmuhit dán guokte lávlaga min sálbma-CD:s. -
+ 'Therefore we need to get permission from Nils Aslak Valkeapää's
+ _heirs_ to release these two songs on our psalm-CD.'
+- **@-F<OBJ:**
+ Object of infinite verb outside the verbal.
+ - **govaid (N):**
+ Boađe mu lusa geahččat _govaid_! - 'Come to me and look at _the
+ pictures_!'
+- **@-F<OPRED:**
+ Object predicative of infinite verb outside the verbal.
+ - **xxx:**
+ _xxx_
+- **@-F<SPRED:**
+ Subject predicative of infinite verb outside the verbal.
+ - **xxx:**
+ _xxx_
+- **@>A:**
+ Modifier of an adjective to the left.
+ - **nu (Adv):**
+ Gulahallan Sámedikkiin dán gažaldagas šaddá _nu_
+ konkrehtalažžan go vejolaš. - 'The discussion in the Saami
+ Parliament about this issue gets _as_ concrete as possible.'
+- **@A<:**
+ Modifier of an adjective to the right.
+ - **básárdoaluin (N):**
+ IL Nordlysa beaivválaš jođiheaddji, Nils Peder Eriksen, lohká
+ iežaset leat oalle duhtavaččat dán jagáš _básárdoaluin_. - 'The
+ business manager of IL Nordlys, Nils Peder Eriksen, says he is
+ really satisfied with this year's _bazar arrangment_.'
+- **@ADVL:**
+ Sentence adverbial, @ADVL> or @<ADVL.
+- **@>ADVL:**
+ Modifier of an adverbial.
+ - **Man (Adv):**
+ _Man_ dávjá don lávet fitnat doppe? - '_How_ often do you
+ usually go there?'
+- **@<ADVL:**
+ adverbial to the right of the finite verb
+ - **beaivvážis (N):**
+ Gávpot ii dárbbaš čuovgga _beaivvážis_ ii ge mánus. - 'The city
+ does not need light _from the sun_ and not the from the moon
+ either.
+- **@ADVL>:**
+ Adverbial to the left of the finite verb.
+ - **lasttain (N):**
+ Ja muora _lasttain_ ožžot álbmogat dearvvašvuođa. - 'And from
+ the tree's _leaves_, the people get health.'
+- **@ADVL<:**
+ Complement of an adverbial to the right of its head.
+ - **vahkus (N):**
+ Mun málestan guktii _vahkus_. - 'I make food twice a _week_.'
+- **@ADVL>CS:**
+ adverbial modifying a conjunction
+ - **dallah (Adv):**
+ _Dallah_ goh Jeesuse tjaetseste tjuedtjele, dellie vuajna Elmie
+ rihpesåvva jih Voejkene altasasse goh ledtie suaja. - '_Right_
+ after Jesus stood up from the water, he sees that heaven opens
+ and the holy spirit flies to him like a bird.'
+ - **:**
+ _(sma)_
+- **@APP-ADVL<:**
+ Apposition to an adverbial to the left. If the apposition consists
+ of more than one word, the head will get this tag.
+ - **ovdal (Pr):**
+ Dolin, _ovdal_ soađi, olbmot lávejedje vuovdit joŋaid. - 'In
+ old times, _before_ the war, people used to sell cowberries.'
+- **@APP-N<:**
+ Apposition to a noun to the left of it. If the apposition is more
+ than one word, the head will get this tag.
+ - **eatnigiela (N):**
+ Viimmat mun ohppen čállit sámegiela, mu _eatnigiela_. -
+ 'Finally, I learned to write in Sámi, my _mother tongue_.'
+- **@APP-Num<:**
+ Apposition to a numeral to the left.
+ - **suinniid (N):**
+ Juohke heasta borrá sullii 6 kilu _suinniid_ beaivái. - 'Every
+ horse eats approximately 6 kilograms of _grass_ a day.'
+- **@APP>Pron:**
+ Apposition to a pronoun to the right. If the apposition is more than
+ one constituent, the head will get this tag.
+ - **Turner (N Prop):**
+ Muhto diet Will _Turner_, son nai lea fiinna olmmái. - 'But
+ this Will _Turner_, he is also a nice guy.'
+- **@APP-Pron<:**
+ Apposition to a pronoun to the left. If the apposition is more than
+ one constituent, the head will get this tag.
+ - **olbmái (N):**
+ Dan mun muitalan dušše dutnje, mu buoremus _olbmái_. - 'This I
+ tell only you, my best _friend_.'
+- **@CMPND:**
+ First part of a compound followed by a hyphen
+ - **skaehtie-:**
+ Reerenasse galka båetije stoerredigkieboelhkesne jåerhkedh dam
+ \*skaehtie-_ jïh åasadaltesem mij lea daelie, jïh daennie
+ daltesisnie hov lea nuepie buerebe joekedimmiem darjodh._
+- **@CNP:**
+ Local conjunction or subjunction.
+ - **ja (CC):**
+ Sihke Mázes _ja_ Guovdageainnus leat boarrásat viššalit finadan
+ doaibmaguovddážiin. - 'Both in Máze _and_ Guovdageaidnu, the
+ oldest people frequently got to the activitycentre.'
+ - **go (CS):**
+ Sámi geavaheaddjit hállet dávjá metaforaiguin ja sis leat ollu
+ eará gulahallanvuogit _go_ giella. - 'Saami users speak often in
+ metaphores and the have many other ways of comunicating _than_
+ by means of language.'
+- **@COMP-CS<:**
+ Complement of subjunction.
+ - **vejolaš (A):**
+ Gulahallan Sámedikkiin dán gažaldagas šaddá nu konkrehtalažžan
+ go _vejolaš_. - 'The contact with the Saami Parliament about
+ this issue gets as concrete as _possible_.'
+- **@CVP:**
+ Conjunction or subjunction that conjoins finite verb phrases.
+ - **ja (CC):**
+ Bealatjogas leat dološ rájes leamaš bálvvossajit _ja_ dát
+ golbma sieiddi ledje dovddus gitta olgoriikii. - 'Long since,
+ there have been sacrificial sites at Bealatjohka _and_ the three
+ 'sieidi' (cult images) were known even abroad.
+ - **go (CS):**
+ Leago guhkes áigi dassá _go_ Máreha oidnet? - 'Has it been a
+ long time _since_ you have seen Máret?'
+- **@-FADVL>:**
+ Complement of infinite verb outside the verbal.
+ - **várrogasat (Adv):**
+ Dihkkadeaddji rávve skohtervuddjiid _várrogasat_ mátkkoštit.
+ 'The roadman warns snowscooter drivers to drive _carefully_.'
+- **@-FAUXV:**
+ Infinite auxiliary verb.
+ - **sáhte (V):**
+ Eat mii _sáhte_ vuolgit. - 'We *can*not leave.'
+- **@-FMAINV:**
+ Infinite main verb.
+ - **geargan (V):**
+ Ja Biret-Elle lea easka skuvllas _geargan_. - 'And Biret-Elle
+ has just _finished_ school.'
+- **@-FOBJ>:**
+ Object of infinite verb outside the verbal.
+ - **váldovuoittuid (N):**
+ Valáštallanhálla lei njealjehas dievva olbmuiguin geat vurde
+ _váldovuoittuid_ fasket. - 'The gymn was to a quarter full of
+ people that wait to grab _the main prizes_.'
+- **@-FSUBJ>:**
+ Subject of infinite verb outside the verbal.
+ - **mu (Pron):**
+ Diet dáhpáhuvai _mu_ dieđikeahttá. - 'It happened without _me_
+ knowing about it.'
+- **@ADVL> <hab>:**
+ Habitive to the left of the finite verb.
+ - **Máhtes (N):**
+ _Máhtes_ lea beana. - '_Máhtte_ has a dog.'
+- **@<ADVL <hab>:**
+ Habitive to the right of the finite verb.
+ - **dus (Pron):**
+ Leago _dus_ ruhta? - 'Do _you_ have money?'
+- **@HNOUN:**
+ Stray noun in sentence fragments.
+ - **boddu (N):**
+ Vuosttaš _boddu_. - 'First _lesson_.'
+- **@INTERJ:**
+ Interjection.
+ - **maid (Interj):**
+ _Maid_, iigo leat boahtán? - '_What_, hasn't he/she come?'
+- **@>N:**
+ Prenominal modifier to the left
+ - **geavatlaš (A):**
+ Ráđđehussii lea _geavatlaš_ politihkka deaŧalaš. - 'For the
+ government, _practical_ politics is important.'
+ - **oahppo-:**
+ _Oahppo-_ ja dutkanministtar dat lea ráhkadan dieđáhusa alit
+ sámi oahpu ja dutkama birra. - 'The secretary for _education_
+ and research has given a notice about Saami higher education and
+ research.'
+ - **rektor (N):**
+ _Rektor_ Tove Bull álgaga mielde... - 'According to _principal_
+ Tove Bull ...'
+ - **Tove (N Prop):**
+ Rektor _Tove_ Bull álgaga mielde... - 'According to principal
+ _Tove_ Bull ...'
+- **@N<:**
+ Modifier of the noun to the left.
+ - **33 (Num):**
+ Mun lean ilus go beasan ovdanbuktit St.dieđ. nr. _33_. - 'I am
+ happy that I get the opportunity to present the parliament
+ notice number _33_.'\* (In this case _33_ modifies _St.dieđ._.)
+ - **vihtta (Num):**
+ Mun boađán diibmu _vihtta_. - 'I will come at _five_ o'clock.'
+- **@>Num:**
+ Attributes of numeral to the right.
+ - **nr (N):**
+ Mun lean ilus go beasan ovdanbuktit St.dieđ. _nr._ 33. - 'I am
+ happy that I get the opportunity to present the parliament
+ notice _number_ 33.'
+- **@Num<:**
+ Attributes of numeral to the left.
+ - **jagi (N):**
+ Son lea guoktelogi _jagi_ boaris. - 'She/he is twenty _years_
+ old.'
+- **@<OBJ:**
+ Direct object to the right of the finite verb.
+ - **áiggi (N):**
+ Dat gáibida ollu _áiggi_. - 'That demands a lot of _time_.'
+- **@OBJ>:**
+ Direct object to the left of the finite verb.
+ - **maid (Pron):**
+ Filbma lea oassi prošeavttas _maid_ Sámi instituhtta lea
+ ruthadan. - 'The film is a part of the project _that_ the Saami
+ institute has financed.'
+- **@OPRED>:**
+ Object predicative to the left of the finite verb.
+ - **luoikkasin (N):**
+ Gaup dojii stivrrana hárjehallamiin, muhto oaččui _luoikkasin_
+ eará stivrrana. - 'Gaup broke the handlebars during the
+ practises, but got to _borrow_ another steering.'
+- **@<OPRED:**
+ Object predicative to the right of the finite verb.
+ - **buriid (A):**
+ Gáhkkuid son ráhkada hui _buriid_. - 'Cakes, she/he makes
+ really _good ones_.'
+ - **sámegielhállin (N):**
+ Dagat iežat _sámegielhállin_. - 'You make yourself _a Saami
+ speaker_.'
+- **@>P:**
+ Complement of postposition to the left of it.
+ - **oahpu (N), dutkama (N):**
+ Oahppo- ja dutkanministtar dat lea ráhkadan dieđáhusa alit sámi
+ _oahpu_ ja _dutkama_ birra. - 'The secretary for education and
+ research has given a notice about Saami higher _education_ and
+ _research_.'
+- **@P<:**
+ Complement of preposition to the right of it.
+ - **oasálaččaid (N):**
+ Finnmárkkus ii goassige leat leamaš ságastallan gaskal muhtun
+ muddui seammadássásaš _oasálaččaid_. - 'There has never been a
+ discussion in Finnmark between somehow equal _parts_.'
+- **@PCLE:**
+ Particle.
+ - **amma (Pcle):**
+ _Amma_ mii eat leat máksán? - 'We haven't paid, _have we_?'
+- **@<PPRED:**
+ a predicative with a predicative as its head
+ - **reaŋgan (N):**
+ Máhtes lea Jovnna _reaŋgan_. - 'Máhtte has Jovnna _as a
+ searvant_.'
+- **@>Pron:**
+ Modifier of a pronoun to the left of it.
+ - **buot (Pron):**
+ Mun, Johanas, lean dat guhte lean gullan ja oaidnán _buot_
+ dán. - 'I, Johanas, am the one who has heard and seen _all_ of
+ it.'
+- **@Pron<:**
+ Modifier of pronoun to the right of it.
+ - **ipmašiid (N):**
+ Maid _ipmašiid_ doppe dagat? - 'What _the heck_ are you doing
+ there?'
+ - **golmmas (N):**
+ Mii _golmmas_ oktan du vieljain finaimet Niillas-čeazi
+ geahčen. - 'We _three_ together with your brother visited uncle
+ Niillas.'
+- **@SPRED:**
+ Subject predicative in elliptical sentences.
+ - **nommh (N):**
+ _Die maa onterligksh nommh, ih goh tuhtjh, men die ligan
+ onterligksh nierretjh aaj._
+ - **:**
+ _(sma)_
+- **@<SPRED:**
+ Subject predicative to the right of the finite verb.
+ - **galbmasat (A):**
+ Mus leat gieđat nu _galbmasat_. - 'My hands are so _cold_.'
+- **@SPRED>:**
+ Subject predicative to the left of the finite verb.
+ - **bargu (N):**
+ Sin _bargun_ lei váldit fáŋgan Gonagasa. - 'Their _job_ was to
+ capture the King.'
+- **@SUBJ:**
+ Elliptical subject.
+ - **ålma (N):**
+ _Dennie synnagovgesne jis akte ålma maam doenh-aajmoe
+ doerelamme._
+- **@SUBJ>:**
+ Subject to the left of the finite verb.
+ - **son (Pron):**
+ _Son_ lea mu oabbá. - '_She_ is my sister.'
+- **@<SUBJ:**
+ Subject to the right of the finite verb.
+ - **ollusat (Pron):**
+ ...ja dan vejolašvuođa orro gal _ollusat_ geavahan. - '...and
+ this opportunity, _many_ seem to make use of.'
+- **@<SUBJ <ext>:**
+ Subject to the right of the finite verb, in a habitive or extencial
+ construction.
+ - **beana (N):**
+ Mus lea _beana_. - 'I have _a dog_.'
+ - **luopmánat (N):**
+ Jeakkis leat _luopmánat_. - 'There are _cloudberries_ in the
+ swamp.'
+- **@VOC:**
+ Vocative.
+ - **hearrá:**
+ _Hearrá_, du ráhkis ustit lea buohcci. - '_Lord_, your beloved
+ friend is ill.'
+- **@X:**
+ A dummy tag assigned when no tag assignment rule has hit. This tag
+ is useful for finding the flaws in the tag mapping section.
diff --git a/lang/common/flag-diacritics.md b/lang/common/flag-diacritics.md
index c2e6031e..1f27b579 100644
--- a/lang/common/flag-diacritics.md
+++ b/lang/common/flag-diacritics.md
@@ -1,8 +1,6 @@
-Flag diacritics
-===============
+# Flag diacritics
-Introduction
-------------
+## Introduction
The use of flag diacritics is documented in chapter 8 of the Xerox book.
The present page documents the flag diacritics format, and the use of
@@ -13,29 +11,28 @@ remove illegal compounds, and in order to handle automatic downcasing of
proper names when they are converted to e.g. adjectives. See the
documentation for each language for an overview.
-Flag diacritics format
-----------------------
+## Flag diacritics format
There are four types of flag diacritics, all of them with the format
@operator.feature.value@ or @operator.feature@:
-- **U or Unification flags, @U.feature.value@:**
- U is the unification operator, and the form is accepted if, for the
- relevant feature, the two flags in the derivation string have the
- same value.
-- **P or Positive (Re)Setting, @P.feature.value@:**
- Sets or resets the feature to the given value.
-- **N or Negative (Re)Setting, @N.feature.value@:**
- Sets or resets the feature to the negation of the given value.
-- **R or Require Test, @R.feature.value@:**
- For this diacritic, a test is performed, and it succeeds iff feature
- is currently set to value, otherwise the path is blocked.
-- **D or Disallow Test, @D.feature.value@:**
- A test is performed that succeds if feature is neutral or set to a
- value that is incompatible with value.
-- **C or Clear Feature, @C.feature@:**
- For this flag, the value of feature is reset to neutral.
-- **U or Unification Test, @U.feature.value@:**
- If feature is currently neutra, this diacritic causes feature to be
- set to value. Else if feature is currently set, then the test
- succeeds iff value is compatible with the current value of feature.
+- **U or Unification flags, @U.feature.value@:**
+ U is the unification operator, and the form is accepted if, for the
+ relevant feature, the two flags in the derivation string have the
+ same value.
+- **P or Positive (Re)Setting, @P.feature.value@:**
+ Sets or resets the feature to the given value.
+- **N or Negative (Re)Setting, @N.feature.value@:**
+ Sets or resets the feature to the negation of the given value.
+- **R or Require Test, @R.feature.value@:**
+ For this diacritic, a test is performed, and it succeeds iff feature
+ is currently set to value, otherwise the path is blocked.
+- **D or Disallow Test, @D.feature.value@:**
+ A test is performed that succeds if feature is neutral or set to a
+ value that is incompatible with value.
+- **C or Clear Feature, @C.feature@:**
+ For this flag, the value of feature is reset to neutral.
+- **U or Unification Test, @U.feature.value@:**
+ If feature is currently neutra, this diacritic causes feature to be
+ set to value. Else if feature is currently set, then the test
+ succeeds iff value is compatible with the current value of feature.
diff --git a/lang/common/index.md b/lang/common/index.md
index 9ab9153f..b138764c 100644
--- a/lang/common/index.md
+++ b/lang/common/index.md
@@ -1,50 +1,45 @@
-Language models (transducers)
-=============================
+# Language models (transducers)
Working with LEXC, TWOLC and Constraint Grammar
-
-Transducers
-===========
+## Transducers
- [Transducer infrastructure](../../infra/Infrastructure.md)
- [Tutorials for lexc, twolc and constraint grammar](Tutorials.html)
-- [Test scripts and routines for use when working on the tools](developingwork.html)
-- [Handling morphological variation in lexc](Variation_in_lexc.html)
-- [Principles for common (language-independent) lexicon entries](PrinciplesForCommonTagsAndLexiconEntries.html)
+- [Test scripts and routines for use when working on the tools](developingwork.html)
+- [Handling morphological variation in lexc](Variation_in_lexc.html)
+- [Principles for common (language-independent) lexicon entries](PrinciplesForCommonTagsAndLexiconEntries.html)
+
+## Shared resources
-Shared resources
-================
Description of [how to set up](SharedResources.md) shared resources.
-Documentation of tags
-=====================
+## Documentation of tags
+
These links document the different types of tags used in the grammar models.
-- [How the different tags are interacting with the FSTs](DifferentFSTs.html)
-- [Harmonising the most frekvent derivations in Saami languages](DerivationOverview.html)
-- [Compoundtags](CompoundTags.html)
-- [Morphological tags](MorphologicalTags.html)
-- [Derivational tags](DerivationOverview.html)
-- [Syntax](docu-sme-syntaxtags.html)
-- [Dependency](docu-deptags.html)
-- [Semantic tags](SemanticTags.html)
+- [How the different tags are interacting with the FSTs](DifferentFSTs.html)
+- [Harmonising the most frekvent derivations in Saami languages](DerivationOverview.html)
+- [Compoundtags](CompoundTags.html)
+- [Morphological tags](MorphologicalTags.html)
+- [Derivational tags](DerivationOverview.html)
+- [Syntax](docu-sme-syntaxtags.html)
+- [Dependency](docu-deptags.html)
+- [Semantic tags](SemanticTags.html)
-Language-specific documentation
-===============================
+## Language-specific documentation
-- [Work on each languages is documented on their respective pages](https://giellalt.github.io/LanguageModels.html)
-- [Page for improving our linguistic analysis for the Saami languages](../smi/index.html)
+- [Work on each languages is documented on their respective pages](https://giellalt.github.io/LanguageModels.html)
+- [Page for improving our linguistic analysis for the Saami languages](../smi/index.html)
-Obsolete documentation
-======================
+## Obsolete documentation
-Here we keep some documentation that *now is obsolete*, but that we
+Here we keep some documentation that _now is obsolete_, but that we
don't want to throw away. Sometimes looking at how things were before
help us understand the present situation, or it may support our memory.
-- [The original sme flowchart over the old
- infra](../sme/docu-sme-flowchart.html)
-- [The makefile setup in our old infra](../sme/docu-sme-makefile.html)
-- [Our oldinfra system for flag
- diacritics"](../sme/docu-sme-flag-diacritics.html)
+- [The original sme flowchart over the old
+ infra](../sme/docu-sme-flowchart.html)
+- [The makefile setup in our old infra](../sme/docu-sme-makefile.html)
+- [Our oldinfra system for flag
+ diacritics"](../sme/docu-sme-flag-diacritics.html)
diff --git a/lang/common/korp-enkel.md b/lang/common/korp-enkel.md
index 546d1940..ac299afd 100644
--- a/lang/common/korp-enkel.md
+++ b/lang/common/korp-enkel.md
@@ -1,4 +1,4 @@
-# Søk med søkeboksen *Enkel* i Korp
+# Søk med søkeboksen _Enkel_ i Korp
Gå til et av Korp-grensesnitta, f.eks. [det samiske](http://gtweb.uit.no/korp/). Trykk på fliken **Enkel** rett under **KORP**-kogoen.
@@ -7,17 +7,16 @@ Gå til et av Korp-grensesnitta, f.eks. [det samiske](http://gtweb.uit.no/korp/)
Boksen har ett søkefelt. Skriv inn ordform, og trykk **Søk**. Merk nedfallsmenyen til høyre for ordet **Søk**: Det er mulik å lagre søket du har gjort, og deretter bruke det på nytt i sammenligning med andre søk.
Under søkefeltet er det 4 alternativ:
+
- i rekkefølge og også som
-- prefiks
-- suffiks
-- skiller ikke mellom store/små
+- prefiks
+- suffiks
+- skiller ikke mellom store/små
-Dette gir et enkelt regulært uttrykk, f.eks. «alle ord på *-guin*».
+Dette gir et enkelt regulært uttrykk, f.eks. «alle ord på _-guin_».
## Muligheter med enkelt søk
### Ordbild
Fjern avkrysninga på alle de fire alternativa, men kryss av for **Ordbild** til høyre i den blå linja under søkefeltet. Skriv inn f.eks. et verb i infinitiv, og søk. Søket vil ta flere minutt, men resultatet blir et oversyn over argumenta knytta til verbet (eller to: Et med argument før og et med argument etter verbet).
-
-
diff --git a/lang/common/korp-extended.md b/lang/common/korp-extended.md
index 673df721..4165a057 100644
--- a/lang/common/korp-extended.md
+++ b/lang/common/korp-extended.md
@@ -1,59 +1,63 @@
-# Search with the search box *Extended* in Korp
+# Search with the search box _Extended_ in Korp
Go to one of the Korp interfaces, e.g. [the Sami](http://gtweb.uit.no/korp/). Tap the **Extended** tab right below the **KORP** cogo.
-
-
-# The search box itself
+## The search box itself
(picture in English forthcoming)
+### Simple use of the search box
+
+The box has 9 different search modes, _word, Part-of-speech, Grammatical analysis, Baseform, Dependency relation, Domain, Translated from, Title, date, time interval_. We go through them one by one than:
-## Simple use of the search box
+#### word
-The box has 9 different search modes, *word, Part-of-speech, Grammatical analysis, Baseform, Dependency relation, Domain, Translated from, Title, date, time interval*. We go through them one by one than:
+Here you enter _word form_. Mark alternative to the right, e.g. _is, is not, ..._ The option _is not_ only makes sense with the use of multiple search boxes.
-### word
-Here you enter *word form*. Mark alternative to the right, e.g. *is, is not, ...* The option *is not* only makes sense with the use of multiple search boxes.
+#### Part of speech
-### Part of speech
Here there are predefined options, one for each Part of speech.
-### Grammatical analysis
-Here you enter the grammatical tag. The dropdown menu immediately to the right says **contains**, because the tag is only a part of the string *word form + analysis*. If you want to search for several tags, e.g. locative singular, type **Sg.Loc** in the search field.
+#### Grammatical analysis
+
+Here you enter the grammatical tag. The dropdown menu immediately to the right says **contains**, because the tag is only a part of the string _word form + analysis_. If you want to search for several tags, e.g. locative singular, type **Sg.Loc** in the search field.
+
+#### Baseform
+
+Here you search for the **lexeme**. Selecting _sátni_ here gives hit on the inflected forms _sátni, sáni, sániid, ..._
-### Baseform
-Here you search for the **lexeme**. Selecting *sátni* here gives hit on the inflected forms *sátni, sáni, sániid, ...*
+#### Dependency relation
-### Dependency relation
-Here you can search for tags for syntactic function, e.g. **deprel_←OBJ** (in u\_corp it only says **deprel_←OBJ**). The drop-down menu provides a list of available function tags. Here is an [explanation of the tags for syntactic function](https://giellalt.uit.no/lang/sme/docu-sme-syntaxtags.html).
+Here you can search for tags for syntactic function, e.g. **deprel_←OBJ** (in u*corp it only says \*\*deprel*←OBJ\*\*). The drop-down menu provides a list of available function tags. Here is an [explanation of the tags for syntactic function](https://giellalt.uit.no/lang/sme/docu-sme-syntaxtags.html).
+
+#### Domain
-### Domain
This is the set of corpus domains: **administration, bible, facts, ficti, news, ...** This does not seem to be implemented to work in search. On the other hand, it is possible to sort hits by domain during a search on **Statistics**.
-### Title
+#### Title
+
This is the **title** of the document. This doesn't seem to be implemented to work in search. On the other hand, it is possible to sort hits by domain during a search on **Statistics**.
-### Translated from
+#### Translated from
+
Metadata is very poor here, and it also does not seem to be implemented in search.
-## Combine several wrap legs in the same search box
+### Combine several wrap legs in the same search box
-It is possible to copy searches with the operators **AND** and **OR**. Press **or** at the bottom of the box to search for the union of two or more requirements (eg search for *noun or pronoun*). Press **and** to get a new part of the same search box, to search for an intersection of two requirements (eg search for a word that is *plural and object*).
+It is possible to copy searches with the operators **AND** and **OR**. Press **or** at the bottom of the box to search for the union of two or more requirements (eg search for _noun or pronoun_). Press **and** to get a new part of the same search box, to search for an intersection of two requirements (eg search for a word that is _plural and object_).
-# Combining multiple conditions in the same search box
-(The picture shows Norwegian as metalanguage)
+## Combining multiple conditions in the same search box
+
+(The picture shows Norwegian as metalanguage)
By pressing **⨁** to the right of the search box, you get another search box, so you can search for word combinations. Here it might also be a good idea to search for **Part of speech is not**.
-# Search for more words and show statistics
+## Search for more words and show statistics
-Search for two words (mark the empty box between verb ob object), and select **Statistics**. The result is a frequency-sorted statistic of *verb + object*.
+Search for two words (mark the empty box between verb ob object), and select **Statistics**. The result is a frequency-sorted statistic of _verb + object_.
-
-
diff --git a/lang/common/korp-simple.md b/lang/common/korp-simple.md
index 2f722bac..803e5a56 100644
--- a/lang/common/korp-simple.md
+++ b/lang/common/korp-simple.md
@@ -1,4 +1,4 @@
-# Search with the search box *Simple* in Korp
+# Search with the search box _Simple_ in Korp
Go to one of the corpus collections, e.g. [the Saami one](http://gtweb.uit.no/korp/). Press the tab **Simple** just underneath the **KORP** logo.
@@ -13,11 +13,10 @@ Under the search field there are 4 options:
- final part and
- case-insensitive
-This gives the possibility to use simple regular expressions, e.g. "all words in *-guin*".
+This gives the possibility to use simple regular expressions, e.g. "all words in _-guin_".
-## Possibilities with *simple search*
+## Possibilities with _simple search_
### Word image
Uncheck all four alternatives, then tick **Word picture** on the right in the blue line below the search field. Enter e.g. a verb in the infinitive, and search. The search will take several minutes, but the result will be an overview of arguments linked to the verb (or two: One with an argument before and one with an argument after the verb).
-
diff --git a/lang/common/korp-utvidet.md b/lang/common/korp-utvidet.md
index 2e98e59e..6344983c 100644
--- a/lang/common/korp-utvidet.md
+++ b/lang/common/korp-utvidet.md
@@ -1,55 +1,59 @@
-# Søk med søkeboksen *Utvidet* i Korp
+# Søk med søkeboksen _Utvidet_ i Korp
Gå til et av Korp-grensesnitta, f.eks. [det samiske](http://gtweb.uit.no/korp/). Trykk på fliken **Utvidet** rett under **KORP**-kogoen.
+## Selve søkeboksen
+
-# Selve søkeboksen
+### Enkel bruk av søkeboksen
-
+Boksen har 9 ulike sækemodi, _ord, ordklasse, grunnform, dependensrelasjon, domain, tittel, translated from, tidsintervall_. Vi går gjennom dem en etter enn:
+#### ord
-## Enkel bruk av søkeboksen
+Her skriver du inn _ordform_. Merk alternativa til høyre, f.eks. _er, er ikke, ..._ Alternativet _er ikke_ gir bare mening med bruk av flere søkebokser.
-Boksen har 9 ulike sækemodi, *ord, ordklasse, grunnform, dependensrelasjon, domain, tittel, translated from, tidsintervall*. Vi går gjennom dem en etter enn:
+#### msd (morphosyntactic description)
-### ord
-Her skriver du inn *ordform*. Merk alternativa til høyre, f.eks. *er, er ikke, ...* Alternativet *er ikke* gir bare mening med bruk av flere søkebokser.
+Her skriver du inn grammatisk tagg. Menyen til venstre står på **inneholder**, fordi taggen bare er en del av _ordform + analyse_. Viss målet t.d. er lokativ entall, skriv **Sg.Loc** i søkefeltet.
-### msd (morphosyntactic description)
-Her skriver du inn grammatisk tagg. Menyen til venstre står på **inneholder**, fordi taggen bare er en del av *ordform + analyse*. Viss målet t.d. er lokativ entall, skriv **Sg.Loc** i søkefeltet.
+#### ordklasse
-### ordklasse
Her er det ferdigdefinerte alternativ, et for hver ordklasse.
-### grunnform
-Her kan du søke på leksemet. Å velge *sátni* her gir *sátni, sáni, sániid, ...*
+#### grunnform
+
+Her kan du søke på leksemet. Å velge _sátni_ her gir _sátni, sáni, sániid, ..._
+
+#### dependensrelasjon
-### dependensrelasjon
-Her kan du søke på tagger for syntaktisk funksjon, f.eks. **deprel_←OBJ** (i u_korp står det bare **deprel_←OBJ**). Nedfallsmenyen gir ei liste over tilgjengelig funksjonstagger. Her er ei [forklaring av taggene for syntaktisk funksjon](https://giellalt.uit.no/lang/sme/docu-sme-syntaxtags.html).
+Her kan du søke på tagger for syntaktisk funksjon, f.eks. **deprel_←OBJ** (i u*korp står det bare \*\*deprel*←OBJ\*\*). Nedfallsmenyen gir ei liste over tilgjengelig funksjonstagger. Her er ei [forklaring av taggene for syntaktisk funksjon](https://giellalt.uit.no/lang/sme/docu-sme-syntaxtags.html).
+
+#### domain
-### domain
Dette er korpusdomena **administration, bible, facts, ficti, news, ...** Det ser ikke ut til at dette er implementert til å fungere i søk. Derimot er det mulig å sortere treff etter domene under søk på **Statistikk**.
-### tittel
+#### tittel
+
Dette er **tittelen** til dokumentet. Det ser ikke ut til at dette er implementert til å fungere i søk. Derimot er det mulig å sortere treff etter domene under søk på **Statistikk**.
-### translated from
+#### translated from
+
Her er metadata svært dårlig, og det ser heller ikke ut til at dette er implementert i søk.
-## Kombinere flere viklår i samme søkeboks
+### Kombinere flere viklår i samme søkeboks
-Det er mulig å kopiere søk med operatorene **OG** og **ELLER**. Trykk på **eller** nederst i boksen for å få søke etter unionen av to eller flere krav (f.eks. søk etter *substantiv eller pronomen*). Trykk på **og** for å få en ny del av samme søkeboks, for å søke etter et snitt av to krav (f.eks. søk etter et ord som er *plural og objekt*).
+Det er mulig å kopiere søk med operatorene **OG** og **ELLER**. Trykk på **eller** nederst i boksen for å få søke etter unionen av to eller flere krav (f.eks. søk etter _substantiv eller pronomen_). Trykk på **og** for å få en ny del av samme søkeboks, for å søke etter et snitt av to krav (f.eks. søk etter et ord som er _plural og objekt_).
-# Kombinere flere søkebokser
+## Kombinere flere søkebokser
-
Med å trykke på **⨁** til høyre for søkeboksen får du en søkeboks til, slik at du kan du søke på ordkombinasjoner. Her kan det også væerre en god idé å søke på **ordklasse er ikke**.
-# Søk på flere ord og vis statistikk
+## Søk på flere ord og vis statistikk
-Søk ett er to ord (merk den tomme boksen mellom verb ob objekt), og velg **Statistikk**. Resultatet blir en frekvenssortert statistikk over *verb + objekt*.
+Søk ett er to ord (merk den tomme boksen mellom verb ob objekt), og velg **Statistikk**. Resultatet blir en frekvenssortert statistikk over _verb + objekt_.
diff --git a/lang/docu-makefile.md b/lang/docu-makefile.md
index c47a8ee7..daa4194b 100644
--- a/lang/docu-makefile.md
+++ b/lang/docu-makefile.md
@@ -1,5 +1,4 @@
-The common Makefile and scripts
-===============================
+# The common Makefile and scripts
The Makefile is used to compile the xfst and aspell source files, i.e.
to make the programs. It is put to use by (being in `gt/`) writing the
@@ -11,8 +10,7 @@ project is found in Appendix C of the Beesley and Karttunen book. The
makefiles for the other languages follow the same layout, but they are
simpler.
-Makefile structure
-------------------
+## Makefile structure
The makefile contains variables defining tools and files to be used in
compiling the programs. In the beginning of makefile are commonly used
@@ -21,8 +19,7 @@ tools and files, and after those there are language specific variables.
The rest of the makefile is documented in [sme
makefile](sme/docu-sme-makefile.html).
-Common scripts
---------------
+## Common scripts
Common scripts to all languages are in `gt/common/src/`, and binaries of
these scripts are in `gt/common/bin/`.
diff --git a/lang/index.md b/lang/index.md
index 69138688..758190f0 100644
--- a/lang/index.md
+++ b/lang/index.md
@@ -3,153 +3,147 @@ here](../infra/infraremake/NewinfraCatalogues.html). Languages with no
work done so far are marked with an asterisk (\*). There are
[tutorials](common/Tutorials.html) explaining the grammar format.
-Saami languages
-===============
+# Saami languages
-- [North](sme/j-sme.html), [Lule](smj/j-smj.html),
- [South](sma/j-sma.html), [Inari](smn/j-smn.html),
- [Kildin](sjd/index.html), [Pite](sje/PiteSaamiDocumentation.html),
- [Skolt](sms/j-sms.html) // [Common for all Saami
- languages](smi/index.html)
+- [North](sme/j-sme.html), [Lule](smj/j-smj.html),
+ [South](sma/j-sma.html), [Inari](smn/j-smn.html),
+ [Kildin](sjd/index.html), [Pite](sje/PiteSaamiDocumentation.html),
+ [Skolt](sms/j-sms.html) // [Common for all Saami
+ languages](smi/index.html)
-Other Finnic languages
-======================
+## Other Finnic languages
-- [Estonian (version 1)](est/EstonianDocumentation.html), [Estonian
- (version 2)](experimentest/EstonianDocumentation.html),
- [Finnish](fin/j-fin.html), [Ingrian](izh/IngrianDocumentation.html),
- [Kven](fkv/KvenDocumentation.html),
- [Livonian](liv/LivonianDocumentation.html),
- [Meänkieli\*](fit/MeankieliDocumentation.html),
- [Olonetsian](olo/OlonetsianDocumentation.html),
- [Veps](vep/VepsDocumentation.html),
- [Võro](vro/VoroDocumentation.html),
+- [Estonian (version 1)](est/EstonianDocumentation.html), [Estonian
+ (version 2)](experimentest/EstonianDocumentation.html),
+ [Finnish](fin/j-fin.html), [Ingrian](izh/IngrianDocumentation.html),
+ [Kven](fkv/KvenDocumentation.html),
+ [Livonian](liv/LivonianDocumentation.html),
+ [Meänkieli\*](fit/MeankieliDocumentation.html),
+ [Olonetsian](olo/OlonetsianDocumentation.html),
+ [Veps](vep/VepsDocumentation.html),
+ [Võro](vro/VoroDocumentation.html),
-Other Uralic languages
-======================
+## Other Uralic languages
-- [Eastern Mari](mhr/EasternMariDocumentation.html),
- [Erzya](myv/ErzyaDocumentation.html),
- [Khanty](kca/KhantyDocumentation.html), [Komi](kom/index.html),
- [Komi Permyak](koi/KomiPermyakDocumentation.html),
- [Moksha](mdf/MokshaDocumentation.html),
- [Nganasan](nio/NganasanDocumentation.html), [Northern
- Mansi](mns/NorthernMansiDocumentation.html),
- [Selkup](sel/SelkupDocumentation.html), [Tundra
- Nenets](yrk/TundraNenetsDocumentation.html),
- [Udmurt](udm/UdmurtDocumentation.html), [Western
- Mari](mrj/WesternMariDocumentation.html)
+- [Eastern Mari](mhr/EasternMariDocumentation.html),
+ [Erzya](myv/ErzyaDocumentation.html),
+ [Khanty](kca/KhantyDocumentation.html), [Komi](kom/index.html),
+ [Komi Permyak](koi/KomiPermyakDocumentation.html),
+ [Moksha](mdf/MokshaDocumentation.html),
+ [Nganasan](nio/NganasanDocumentation.html), [Northern
+ Mansi](mns/NorthernMansiDocumentation.html),
+ [Selkup](sel/SelkupDocumentation.html), [Tundra
+ Nenets](yrk/TundraNenetsDocumentation.html),
+ [Udmurt](udm/UdmurtDocumentation.html), [Western
+ Mari](mrj/WesternMariDocumentation.html)
-American languages
-==================
+## American languages
-- [Apuriña\*](apu/ApurinaDocumentation.html), [Central Alaskan
- Yupik\*](esu/CentralAlaskanYupikDocumentation.html), [Central
- Siberian Yupik\*](ess/CentralSiberianYupikDocumentation.html),
- [Cherokee\*](chr/CherokeeDocumentation.html),
- [Dogrib\*](dgr/DogribDocumentation.html),
- [Greenlandic](kal/index.html), [Iñupiaq](ipk/index.html),
- [Kiowa\*](kio/KiowaDocumentation.html), [Northern
- Haida](hdn/NorthernHaidaDocumentation.html),
- [Ojibwa](oji/OjibwaDocumentation.html), [Ojibwe
- (Chippewa)](ciw/OjibweDocumentation.html), [Plains
- Cree](crk/PlainsCreeDocumentation.html), [Southern Puget Sound
- Salish
- (Lushootseed)](lut/SouthernPugetSoundSalishDocumentation.html),
- [Tsuut’ina (Sarcee)](srs/TsuutinaDocumentation.html), [Upper Necaxa
- Totonac](tku/UpperNecaxaTotonacDocumentation.html), [Upper
- Tanana](tau/UpperTananaDocumentation.html)
+- [Apuriña\*](apu/ApurinaDocumentation.html), [Central Alaskan
+ Yupik\*](esu/CentralAlaskanYupikDocumentation.html), [Central
+ Siberian Yupik\*](ess/CentralSiberianYupikDocumentation.html),
+ [Cherokee\*](chr/CherokeeDocumentation.html),
+ [Dogrib\*](dgr/DogribDocumentation.html),
+ [Greenlandic](kal/index.html), [Iñupiaq](ipk/index.html),
+ [Kiowa\*](kio/KiowaDocumentation.html), [Northern
+ Haida](hdn/NorthernHaidaDocumentation.html),
+ [Ojibwa](oji/OjibwaDocumentation.html), [Ojibwe
+ (Chippewa)](ciw/OjibweDocumentation.html), [Plains
+ Cree](crk/PlainsCreeDocumentation.html), [Southern Puget Sound
+ Salish
+ (Lushootseed)](lut/SouthernPugetSoundSalishDocumentation.html),
+ [Tsuut’ina (Sarcee)](srs/TsuutinaDocumentation.html), [Upper Necaxa
+ Totonac](tku/UpperNecaxaTotonacDocumentation.html), [Upper
+ Tanana](tau/UpperTananaDocumentation.html)
-Other languages
-===============
+## Other languages
-- [Bashkir](bak/BashkirDocumentation.html),
- [Buryaad](bxr/BuryadDocumentation.html),
- [Chukchi\*](ckt/ChukchiDocumentation.html),
- [Cornish](cor/CornishDocumentation.html),
- [Evenki\*](evn/EvenkiDocumentation.html), [Faroese](fao/index.html),
- [Finnish Romani\*](rmf/FinnishRomaniDocumentation.html),
- [Irish\*](gle/IrishDocumentation.html), [Kalderash
- Romani\*](rmy/KalderashRomaniDocumentation.html), [Khalkha
- Mongolian\*](khk/KhalkhaMongolianDocumentation.html),
- [Khakas\*](kjh/KhakasDocumentation.html),
- [Latvian](lav/LatvianDocumentation.html), [Norwegian
- Bokmål](nob/j-nob.html), [Romanian](ron/RomanianDocumentation.html),
- [Aromanian](rup/AromanianDocumentation.html),
- [Russian](rus/RussianDocumentation.html),
- [Somali](som/SomaliDocumentation.html),
- [Klingon\*](tlh/KlingonDocumentation.html),
- [Tuvan\*](tyv/TuvanDocumentation.html),
- [Kalmyk\*](xal/KalmykDocumentation.html), [Todo
- Oirat\*](xwo/TodoOiratDocumentation.html),
+- [Bashkir](bak/BashkirDocumentation.html),
+ [Buryaad](bxr/BuryadDocumentation.html),
+ [Chukchi\*](ckt/ChukchiDocumentation.html),
+ [Cornish](cor/CornishDocumentation.html),
+ [Evenki\*](evn/EvenkiDocumentation.html), [Faroese](fao/index.html),
+ [Finnish Romani\*](rmf/FinnishRomaniDocumentation.html),
+ [Irish\*](gle/IrishDocumentation.html), [Kalderash
+ Romani\*](rmy/KalderashRomaniDocumentation.html), [Khalkha
+ Mongolian\*](khk/KhalkhaMongolianDocumentation.html),
+ [Khakas\*](kjh/KhakasDocumentation.html),
+ [Latvian](lav/LatvianDocumentation.html), [Norwegian
+ Bokmål](nob/j-nob.html), [Romanian](ron/RomanianDocumentation.html),
+ [Aromanian](rup/AromanianDocumentation.html),
+ [Russian](rus/RussianDocumentation.html),
+ [Somali](som/SomaliDocumentation.html),
+ [Klingon\*](tlh/KlingonDocumentation.html),
+ [Tuvan\*](tyv/TuvanDocumentation.html),
+ [Kalmyk\*](xal/KalmykDocumentation.html), [Todo
+ Oirat\*](xwo/TodoOiratDocumentation.html),
-All languages listed alphabetically
-===================================
+## All languages listed alphabetically
-- [Apuriña](apu/ApurinaDocumentation.html)
-- [Aromanian](rup/AromanianDocumentation.html)
-- [Bashkir](bak/BashkirDocumentation.html)
-- [Central Alaskan Yupik\*](esu/CentralAlaskanYupikDocumentation.html)
-- [Central Siberian
- Yupik\*](ess/CentralSiberianYupikDocumentation.html)
-- [Cherokee\*](chr/CherokeeDocumentation.html)
-- [Chukchi\*](cor/ChukchiDocumentation.html)
-- [Cornish](cor/CornishDocumentation.html)
-- [Dogrib\*](dgr/DogribDocumentation.html)
-- [Eastern Mari](mhr/EasternMariDocumentation.html)
-- [Erzya Mordvin](myv/ErzyaDocumentation.html)
-- [Estonian (version 1)](est/EstonianDocumentation.html)
-- [Estonian (version 2)](experimentest/EstonianDocumentation.html)
-- [Evenki](evn/EvenkiDocumentation.html)
-- [Faroese](fao/index.html)
-- [Finnish](fin/j-fin.html)
-- [Finnish Romani\*](rmf/FinishRomaniDocumentation.html)
-- [Greenlandic](kal/index.html)
-- [Inari Saami](smn/j-smn.html)
-- [Ingrian](izh/IngrianDocumentation.html)
-- [Iñupiaq](ipk/index.html)
-- [Irish\*](gle/IrishDocumentation.html)
-- [Kalderash Romani\*](rmy/KalderashRomaniDocumentation.html)
-- [Kalmyk](xal/KalmykDocumentation.html)
-- [Khalkha Mongolian](khk/KhalkhaMongolianDocumentation.html)
-- [Khakas](kjh/KhakasDocumentation.html)
-- [Khanty](kca/KhantyDocumentation.html)
-- [Kiowa](kio/KiowaDocumentation.html)
-- [Klingon](tlh/KlingonDocumentation.html)
-- [Komi](kom/index.html)
-- [Komi Permyak](koi/KomiPermyakDocumentation.html)
-- [Kven](fkv/KvenDocumentation.html)
-- [Latvian](lav/LatvianDocumentation.html)
-- [Livonian](liv/LivonianDocumentation.html)
-- [Lule Saami](smj/j-smj.html)
-- [Moksha Mordvin](mdf/MokshaDocumentation.html)
-- [Meänkieli\*](mdf/MeankieliDocumentation.html)
-- [Nganasan](nio/NganasanDocumentation.html)
-- [North Saami](sme/j-sme.html)
-- [Northern Haida](hdn/NorthernHaidaDocumentation.html)
-- [Northern Mansi](mns/NorthernMansiDocumentation.html)
-- [Norwegian Bokmål](nob/j-nob.html)
-- [Ojibwa](oji/OjibwaDocumentation.html)
-- [Ojibwe (Chippewa)](ciw/OjibweDocumentation.html)
-- [Olonetsian](olo/OlonetsianDocumentation.html)
-- [Kildin Saami](sjd/index.html)
-- [Pite Saami](sje/PiteSaamiDocumentation.html)
-- [Plains Cree](crk/PlainsCreeDocumentation.html)
-- [Romanian](ron/RomanianDocumentation.html)
-- [Russian](rus/RussianDocumentation.html)
-- [Selkup\*](sel/SelkupDocumentation.html)
-- [South Saami](sma/j-sma.html)
-- [Southern Puget Sound Salish
- (Lushootseed)](lut/SouthernPugetSoundSalishDocumentation.html)
-- [Skolt Saami](sms/j-sms.html)
-- [Todo Oirat](xwo/TodoOiratDocumentation.html)
-- [Tsuut’ina](srs/TsuutinaDocumentation.html)
-- [Tundra Nenets](yrk/TundraNenetsDocumentation.html)
-- [Tuvan](tyv/TuvanDocumentation.html)
-- [Udmurt](udm/UdmurtDocumentation.html)
-- [Upper Necaxa Totonac](tku/UpperNecaxaTotonacDocumentation.html)
-- [Upper Tanana](tau/UpperTananaDocumentation.html)
-- [Võro](vro/VoroDocumentation.html)
-- [Western Mari](mrj/WesternMariDocumentation.html)
+- [Apuriña](apu/ApurinaDocumentation.html)
+- [Aromanian](rup/AromanianDocumentation.html)
+- [Bashkir](bak/BashkirDocumentation.html)
+- [Central Alaskan Yupik\*](esu/CentralAlaskanYupikDocumentation.html)
+- [Central Siberian
+ Yupik\*](ess/CentralSiberianYupikDocumentation.html)
+- [Cherokee\*](chr/CherokeeDocumentation.html)
+- [Chukchi\*](cor/ChukchiDocumentation.html)
+- [Cornish](cor/CornishDocumentation.html)
+- [Dogrib\*](dgr/DogribDocumentation.html)
+- [Eastern Mari](mhr/EasternMariDocumentation.html)
+- [Erzya Mordvin](myv/ErzyaDocumentation.html)
+- [Estonian (version 1)](est/EstonianDocumentation.html)
+- [Estonian (version 2)](experimentest/EstonianDocumentation.html)
+- [Evenki](evn/EvenkiDocumentation.html)
+- [Faroese](fao/index.html)
+- [Finnish](fin/j-fin.html)
+- [Finnish Romani\*](rmf/FinishRomaniDocumentation.html)
+- [Greenlandic](kal/index.html)
+- [Inari Saami](smn/j-smn.html)
+- [Ingrian](izh/IngrianDocumentation.html)
+- [Iñupiaq](ipk/index.html)
+- [Irish\*](gle/IrishDocumentation.html)
+- [Kalderash Romani\*](rmy/KalderashRomaniDocumentation.html)
+- [Kalmyk](xal/KalmykDocumentation.html)
+- [Khalkha Mongolian](khk/KhalkhaMongolianDocumentation.html)
+- [Khakas](kjh/KhakasDocumentation.html)
+- [Khanty](kca/KhantyDocumentation.html)
+- [Kiowa](kio/KiowaDocumentation.html)
+- [Klingon](tlh/KlingonDocumentation.html)
+- [Komi](kom/index.html)
+- [Komi Permyak](koi/KomiPermyakDocumentation.html)
+- [Kven](fkv/KvenDocumentation.html)
+- [Latvian](lav/LatvianDocumentation.html)
+- [Livonian](liv/LivonianDocumentation.html)
+- [Lule Saami](smj/j-smj.html)
+- [Moksha Mordvin](mdf/MokshaDocumentation.html)
+- [Meänkieli\*](mdf/MeankieliDocumentation.html)
+- [Nganasan](nio/NganasanDocumentation.html)
+- [North Saami](sme/j-sme.html)
+- [Northern Haida](hdn/NorthernHaidaDocumentation.html)
+- [Northern Mansi](mns/NorthernMansiDocumentation.html)
+- [Norwegian Bokmål](nob/j-nob.html)
+- [Ojibwa](oji/OjibwaDocumentation.html)
+- [Ojibwe (Chippewa)](ciw/OjibweDocumentation.html)
+- [Olonetsian](olo/OlonetsianDocumentation.html)
+- [Kildin Saami](sjd/index.html)
+- [Pite Saami](sje/PiteSaamiDocumentation.html)
+- [Plains Cree](crk/PlainsCreeDocumentation.html)
+- [Romanian](ron/RomanianDocumentation.html)
+- [Russian](rus/RussianDocumentation.html)
+- [Selkup\*](sel/SelkupDocumentation.html)
+- [South Saami](sma/j-sma.html)
+- [Southern Puget Sound Salish
+ (Lushootseed)](lut/SouthernPugetSoundSalishDocumentation.html)
+- [Skolt Saami](sms/j-sms.html)
+- [Todo Oirat](xwo/TodoOiratDocumentation.html)
+- [Tsuut’ina](srs/TsuutinaDocumentation.html)
+- [Tundra Nenets](yrk/TundraNenetsDocumentation.html)
+- [Tuvan](tyv/TuvanDocumentation.html)
+- [Udmurt](udm/UdmurtDocumentation.html)
+- [Upper Necaxa Totonac](tku/UpperNecaxaTotonacDocumentation.html)
+- [Upper Tanana](tau/UpperTananaDocumentation.html)
+- [Võro](vro/VoroDocumentation.html)
+- [Western Mari](mrj/WesternMariDocumentation.html)
\*) = Languages with no real work done, found in `startup-langs/`.
diff --git a/lang/parallel_names.md b/lang/parallel_names.md
index eb694345..f24265e7 100644
--- a/lang/parallel_names.md
+++ b/lang/parallel_names.md
@@ -1,44 +1,32 @@
# Names and multilinguality
-
Meeting between **Sjur, Thomas, Trond** on Nov. 14, 2006.
-
1. Fyrste problem:
-
-* All names in all languages will likely be misunderstood if the material is published in
+- All names in all languages will likely be misunderstood if the material is published in
risten.no.
-* "foreign" names can be as much noise as they are valuable, and including them must be
+- "foreign" names can be as much noise as they are valuable, and including them must be
done carefully
-
-
-
We need a more principled approach to this.
-
Background: the name lexicon is getting attention from the SD name/terminology
sections, and they would like to use our name lexicon also for public searching.
-
Observations:
+1. Multilinguality is always optional.
-1) Multilinguality is always optional.
+2. We can observe that "foreign" names in texts follows a domination pattern:
+ majority language forms can be found in minority language texts as real names
+ ("Kautokeino produkter"), whereas minority language names _almost always_
+ occur in majority language texts as citations. And citations should not be
+ considered a natural part of the text.
+3. When looking at our name classification, multilinguality varies according to:
-2) We can observe that "foreign" names in texts follows a domination pattern:
-majority language forms can be found in minority language texts as real names
-("Kautokeino produkter"), whereas minority language names *almost always*
-occur in majority language texts as citations. And citations should not be
-considered a natural part of the text.
-
-
-3) When looking at our name classification, multilinguality varies according to:
-
-
-```
+```text
Ani - weak/none? (pet, myth anim. names)
Fem - weak (informative)
Mal - weak (informative)
@@ -49,48 +37,51 @@ Sur - none
Tit - strong (titles)
```
-
-
-
-
-
Suggestion:
-
-We need to reconsider the *all names in all languages* policy. That policy is
+We need to reconsider the _all names in all languages_ policy. That policy is
valid only for `Fem, Mal,` and `Sur` (and Ani and Tit?). For
`Obj, Org, Plc` the rule should be that if they have multilingual names, each
name should only be used in it's own language. Then we need a modification
saying that majority language names can be included in minority language
lexicons **if attested** in our corpus.
-
Also, the majority language varies
according to country (obviously), which means that in a speller context, we
might consider tailoring spellers for each country, leaving out noise relating
to majority language names from another country.
+## TODO
-**TODO:**
-# finish first version of the editing (**Sjur, Tomi**)
-# add @type=secondary and @excl=speller,hyph to all names marked with !SUB (**Saara**)
-# test editing of the xml files. If ok, then: (**Sjur, Thomas, Trond**)
-# make terms-smX.xml <=== automatically from propernoun-sme-lex.xml (add nob as well)
- (den morfologiske delen skal vere intakt i t.d. propernoun-sme-morph.txt) (**Sjur**)
-# convert propernoun-($lang)-lex.txt to a derived file from common xml files
- (**Sjur, Tomi, Saara**)
-# Rens terms-sme.xml slik at alle namn har rett tagging for ulik bruk (@type=secondary)
- (**Thomas, Maaren, linguists**)
-# Slå i hop stadnamn som ikkje er i same termposten: Helsinki, Helsingfors, Helsset
- (**linguists**)
-# Gjer namnematerialet søkbart i risten.no (**Sjur**)
-# Legg til evt. manglande parallellnamn (stadnamn) (**linguists**)
-# Lag koplingar mellom Niillas og Nils (**linguists**)
+- finish first version of the editing (**Sjur, Tomi**)
+- add @type=secondary and @excl=speller,hyph to all names marked with !SUB (**Saara**)
+- test editing of the xml files. If ok, then: (**Sjur, Thomas, Trond**)
+- make terms-smX.xml <=== automatically from propernoun-sme-lex.xml (add nob as well)
-```
+(den morfologiske delen skal vere intakt i t.d. propernoun-sme-morph.txt) (**Sjur**)
+
+- convert propernoun-($lang)-lex.txt to a derived file from common xml files
+
+(**Sjur, Tomi, Saara**)
+
+- Rens terms-sme.xml slik at alle namn har rett tagging for ulik bruk (@type=secondary)
+
+(**Thomas, Maaren, linguists**)
+
+- Slå i hop stadnamn som ikkje er i same termposten: Helsinki, Helsingfors, Helsset
+
+(**linguists**)
+
+- Gjer namnematerialet søkbart i risten.no (**Sjur**)
+
+- Legg til evt. manglande parallellnamn (stadnamn) (**linguists**)
+
+- Lag koplingar mellom Niillas og Nils (**linguists**)
+
+```text
======= termcenter.xml =========
@@ -163,25 +154,25 @@ After merge:
plc
-
+
plc
-
+
mal
-
+
mal
-
+
diff --git a/lang/smi/AvgrenseAvleiing.md b/lang/smi/AvgrenseAvleiing.md
index e350dec6..4857891d 100644
--- a/lang/smi/AvgrenseAvleiing.md
+++ b/lang/smi/AvgrenseAvleiing.md
@@ -1,52 +1,43 @@
-Avgrense avleiing
-=========
+# Avgrense avleiing
Sjå [møtedokumentet](https://divvungiellatekno.github.io/giellalt.uit.no/admin/linguists/200423_AvgrenseAvleiing.html) (frå 23.4. 2020)
-
Her held vi fram frå møtet:
-
-# Konklusjon frå møtet
-* Vi skal endre +Comp og +Superl til +DerN+Der/Comp+A og +DerN+Der/Superl+A (nummeret på DerN må vi sjå på)
-** dette medfører endringer i filene for (signer når det er gjort)
-*** NDS
-*** Apertium
-*** Gramchecker?
-*** cgi paradigmegenerator
-*** andre?
-* Vi bør observere og deretter forbetre 12345-grammatikken, og evt andre begrensninger
-* måtar å avgrensa derivasjonane på:
-** Bare for normativ HFST: ved hjelp av Der12345-grammatikken
-** For all analyse/generering: ved hjelp av fortsettingsleksikoner
-** For all analyse/generering: ved hjelp av diakritiske flagg som også fungerer på desc (Px er løst slik)
-** Leksikalisere selve derivasjonen, feks. buoremusvuohta
-** Leksikalisere derivasjoner som er utgangspunkt for nye derivasjoner
-* I smj endrer vi slik at på -dibme og -ahtes ikke får Comp og Superl (ahtes skal få nummer likt eller etter Comp/Superl)
-
-
-# Steg framover
-
+## Konklusjon frå møtet
+
+- Vi skal endre +Comp og +Superl til +DerN+Der/Comp+A og +DerN+Der/Superl+A (nummeret på DerN må vi sjå på)
+ - dette medfører endringer i filene for (signer når det er gjort)
+ - NDS
+ - Apertium
+ - Gramchecker?
+ - cgi paradigmegenerator
+ - andre?
+- Vi bør observere og deretter forbetre 12345-grammatikken, og evt andre begrensninger
+- måtar å avgrensa derivasjonane på:
+ - Bare for normativ HFST: ved hjelp av Der12345-grammatikken
+ - For all analyse/generering: ved hjelp av fortsettingsleksikoner
+ - For all analyse/generering: ved hjelp av diakritiske flagg som også fungerer på desc (Px er løst slik)
+ - Leksikalisere selve derivasjonen, feks. buoremusvuohta
+ - Leksikalisere derivasjoner som er utgangspunkt for nye derivasjoner
+- I smj endrer vi slik at på -dibme og -ahtes ikke får Comp og Superl (ahtes skal få nummer likt eller etter Comp/Superl)
+
+## Steg framover
1. Legge Der/ til Comp/Superl
1. Deretter dei andre stega ovafor
+### Derivasjoner som vi bør se på
-## Derivasjoner som vi bør se på
-### Caritiv med komparativ, når brukes det?
-
-I korp finnes noen få reelle eksempler (her trenges bedre disambiguering), så vi kunne ihvertfall begrense til bare leksikaliserte -heapme:
-
-* [Korpsøk sme -heapme Comp](http://gtweb.uit.no/korp/#?stats_reduce=word&cqp=%5Bmsd%20_%3D%20%22Comp%22%20%26%20lemma%20%26%3D%20%22heapme%22%5D&search_tab=1&sort=keyword&hpp=1000&search=cqp)
-* [Korpsøk sme -heapme Superl|http://gtweb.uit.no/korp/#?stats_reduce=word&cqp=%5Bmsd%20_%3D%20%22Superl%22%20%26%20lemma%20%26%3D%20%22heapme%22%5D&search_tab=1&sort=keyword&hpp=1000&search=cqp
-* Her er det meste ikke komperlativ. Her trenges god disambiguering]
-* Eksempler på bruk:
-** Geahnohat bealli dán riiddus lea ge eahpitkeahttá palestiinnálaččat.
-** gutneheappo mielkkeheabbon návccaheappot
-
-
-## Logg over hva som blir gjort, med dato
-
+#### Caritiv med komparativ, når brukes det?
+I korp finnes noen få reelle eksempler (her trenges bedre disambiguering), så vi kunne ihvertfall begrense til bare leksikaliserte -heapme:
+- [Korpsøk sme -heapme Comp](http://gtweb.uit.no/korp/#?stats_reduce=word&cqp=%5Bmsd%20_%3D%20%22Comp%22%20%26%20lemma%20%26%3D%20%22heapme%22%5D&search_tab=1&sort=keyword&hpp=1000&search=cqp)
+- [Korpsøk sme -heapme Superl|http://gtweb.uit.no/korp/#?stats_reduce=word&cqp=%5Bmsd%20_%3D%20%22Superl%22%20%26%20lemma%20%26%3D%20%22heapme%22%5D&search_tab=1&sort=keyword&hpp=1000&search=cqp
+- Her er det meste ikke komperlativ. Her trenges god disambiguering]
+- Eksempler på bruk:
+ - Geahnohat bealli dán riiddus lea ge eahpitkeahttá palestiinnálaččat.
+ - gutneheappo mielkkeheabbon návccaheappot
+### Logg over hva som blir gjort, med dato
diff --git a/lang/smi/DisambiguerenBargovuohki.md b/lang/smi/DisambiguerenBargovuohki.md
index aade7c16..81348e65 100644
--- a/lang/smi/DisambiguerenBargovuohki.md
+++ b/lang/smi/DisambiguerenBargovuohki.md
@@ -1,22 +1,14 @@
-Disambigueren -- bargovuohki
-=======
-
+# Disambigueren -- bargovuohki
(dát lea álgu, dokumeanta ii leat válmmaš)
-
-
-
Go lea gávdnan cealkaga mas lea boasttoanalysa: Dás leat ideat das mo sáhttá bargat.
-
1. Makkár njuolggadus addá boasttoanalysa (= Rule1)? Kommentere dan gaskaboddosaččat eret.
1. Jus dalle oažžu rivttes analysa, de sáhttá buoridit Rule1.
-1. Jus ii leat vejolaš Rule1 divvut: de sáhtát álkidahttit cealkaga (= váldit eret modifikáhtora/advearbbaid jnv), ja analyseret ođđasit.
- 1. Jus dat lihkostuvvá, de sáhtát buoridit njuolggadusa mii vállje riekta (= Rule2) nu ahte dohkkeha modivikáhtoriid/advearbbaid, jus Rule2 boahtá árabut go Rule1.
- 1. Jus buorideapmi ii leat vejolaš, de sáhtát ráhkadit ođđa njuolggadusa (= Rule3) mii galgá boahtit árabut fiillas go Rule1.
- 1. Geahča CG-fiilla struktuvrra, ja geahččal bidjat ođđa njuolggadusaid eará seammásullásaš njuolggadusaid searvái.
+1. Jus ii leat vejolaš Rule1 divvut: de sáhtát álkidahttit cealkaga (= váldit eret modifikáhtora/advearbbaid jnv), ja analyseret ođđasit.
+ 1. Jus dat lihkostuvvá, de sáhtát buoridit njuolggadusa mii vállje riekta (= Rule2) nu ahte dohkkeha modivikáhtoriid/advearbbaid, jus Rule2 boahtá árabut go Rule1.
+ 1. Jus buorideapmi ii leat vejolaš, de sáhtát ráhkadit ođđa njuolggadusa (= Rule3) mii galgá boahtit árabut fiillas go Rule1.
+ 1. Geahča CG-fiilla struktuvrra, ja geahččal bidjat ođđa njuolggadusaid eará seammásullásaš njuolggadusaid searvái.
1. Teste ovdalgo šekket sisa: `sme$ sh script/testCGrules.sh`
1. Árvvoštala lasihit testkorpus.txt:i ja testkorpus.dis.corr.txt:i cealkaga mainna leat bargan.
-
-
diff --git a/lang/smi/Samansetjing.md b/lang/smi/Samansetjing.md
index c201900c..cb99c840 100644
--- a/lang/smi/Samansetjing.md
+++ b/lang/smi/Samansetjing.md
@@ -1,163 +1,143 @@
-Samansetjing
-========
-
-
+# Samansetjing
Her samlar vi analyser og konklusjonar for samansetjing for samiske språk.
-
Se [møtereferat](https://divvungiellatekno.github.io/giellalt.uit.no/admin/linguists/210302_Cmp_avledninger.html)
+## Dynamiske sammensetningstyper og tagging av dem
+### +N+Cmp/SgNom+Cmp\#
+- sme: skuvlahistorjá - skuvla+N+Cmp/SgNom+Cmp#historjá+N+Sg+Nom
+- sme: jorgalanbargu - jorgalit+V+TV+Der/NomAct+N+Cmp/SgNom+Cmp#bargu+N+Sg+Nom
-# Dynamiske sammensetningstyper og tagging av dem
-
-
-## +N+Cmp/SgNom+Cmp#
-* sme: skuvlahistorjá - skuvla+N+Cmp/SgNom+Cmp#historjá+N+Sg+Nom
-* sme: jorgalanbargu - jorgalit+V+TV+Der/NomAct+N+Cmp/SgNom+Cmp#bargu+N+Sg+Nom
-
-
-
-
-## +N+Cmp/SgNom+Cmp/Hyph+Cmp#
-* sme: čoahkkinbovdehus-áššelistá čoahkkinbovdehus+N+Cmp/SgNom+Cmp/Hyph+Cmp#áššelistu+v3+N+Sg+Nom
-
-
-## +Prop+Cmp/SgNom+Cmp-#
-* sme: Nils-Henrik - Nils+N+Prop+Sem/Mal+Cmp/SgNom+Cmp-#Henrik+N+Prop+Sem/Mal+Sg+Nom
-
-
-## +Cmp/Sh+Cmp# (short form)
-* sme: čoarbbeallađas - čoarbbealli+N+Cmp/Sh+Cmp#lađas+N+Sg+Nom
-* sme: justiskomitea - justiisa+N+Cmp/Sh+Cmp#komitea+N+Sg+Nom
-
-
-## +N+Cmp/SgGen+Cmp#
-* sme: sámegiella - sápmi+N+Cmp/SgGen+Cmp#giella+N+Sg+Nom
-
-
-## +N+Cmp/PlGen+Cmp#
-* sme: mánáidskuvla - mánná+N+Cmp/PlGen+Cmp#skuvla+N+Sg+Nom
-
-
-## +A+Cmp/SgNom+Cmp#
-* sme: buoridahkki - buorre+A+Cmp/SgGen+Cmp#dahkki+N+NomAg+Sg+Nom
+### +N+Cmp/SgNom+Cmp/Hyph+Cmp\#
+- sme: čoahkkinbovdehus-áššelistá čoahkkinbovdehus+N+Cmp/SgNom+Cmp/Hyph+Cmp#áššelistu+v3+N+Sg+Nom
+### +Prop+Cmp/SgNom+Cmp-\#
+- sme: Nils-Henrik - Nils+N+Prop+Sem/Mal+Cmp/SgNom+Cmp-#Henrik+N+Prop+Sem/Mal+Sg+Nom
-## +A+Cmp/PlGen+Cmp#
-* sme: čalmmehemiidlihttu - čalmmeheapme+A+Cmp/PlGen+Cmp#lihttu+N+Sg+Nom
+### +Cmp/Sh+Cmp# (short form)
+- sme: čoarbbeallađas - čoarbbealli+N+Cmp/Sh+Cmp#lađas+N+Sg+Nom
+- sme: justiskomitea - justiisa+N+Cmp/Sh+Cmp#komitea+N+Sg+Nom
-## +A+Cmp/Attr+Cmp#
-* sme: oktasaščoahkkin - oktasaš+A+Cmp/Attr+Cmp#čoahkkin+N+Sg+Nom
+### +N+Cmp/SgGen+Cmp\#
+- sme: sámegiella - sápmi+N+Cmp/SgGen+Cmp#giella+N+Sg+Nom
-## +A+Cmp/Attr+Cmp/Hyph+Cmp#
-* sme: oppalaš-ávkkálaš - oppalaš+A+Cmp/Attr+Cmp/Hyph+Cmp#ávki+N+Der/lasj+A+Sg+Nom
+### +N+Cmp/PlGen+Cmp\#
+- sme: mánáidskuvla - mánná+N+Cmp/PlGen+Cmp#skuvla+N+Sg+Nom
-## +Num+Cmp-#
-* sme: 1700-lohku - 1700+Num+Cmp-#lohku+N+Sg+Nom
+### +A+Cmp/SgNom+Cmp\#
+- sme: buoridahkki - buorre+A+Cmp/SgGen+Cmp#dahkki+N+NomAg+Sg+Nom
-## +Num+Cmp/SgNom+Cmp#
-* sme: logijahki - logi+Num+Cmp/SgNom+Cmp#jahki+N+Sg+Nom
+### +A+Cmp/PlGen+Cmp\#
+- sme: čalmmehemiidlihttu - čalmmeheapme+A+Cmp/PlGen+Cmp#lihttu+N+Sg+Nom
-## +Num+Cmp/SgGen+Cmp#
-* sme: guovttejahkásaš - guokte+Num+Cmp/SgGen+Cmp#jahki+N+Der/sasj+A+Sg+Nom
+### +A+Cmp/Attr+Cmp\#
+- sme: oktasaščoahkkin - oktasaš+A+Cmp/Attr+Cmp#čoahkkin+N+Sg+Nom
-## +Adv+Cmp#
-* sme: dáppeolmmoš - dáppe+Adv+Err/Orth+Cmp#olmmoš+N+Sg+Nom : Hvorfor Err/Orth?
+### +A+Cmp/Attr+Cmp/Hyph+Cmp\#
+- sme: oppalaš-ávkkálaš - oppalaš+A+Cmp/Attr+Cmp/Hyph+Cmp#ávki+N+Der/lasj+A+Sg+Nom
-## +ACR+Cmp-#
-* sme: EU-válggain - EU+N+ACR+Cmp-#válga+N+Pl+Loc
+### +Num+Cmp-\#
+- sme: 1700-lohku - 1700+Num+Cmp-#lohku+N+Sg+Nom
+### +Num+Cmp/SgNom+Cmp\#
+- sme: logijahki - logi+Num+Cmp/SgNom+Cmp#jahki+N+Sg+Nom
-## Eksempler på sammensetninger som ikke gir sammensetningsanalyse
+### +Num+Cmp/SgGen+Cmp\#
+- sme: guovttejahkásaš - guokte+Num+Cmp/SgGen+Cmp#jahki+N+Der/sasj+A+Sg+Nom
-### Forleddet er substantiv, ikke nominativ eller genitiv
-* sme: buorringeavaheapmi (buorri+N+Ess)
-* sme: fápmuibidjan (fápmu+N+Sg+Ill)
-* sme: árvvusatnin (árvu+N+Sg+Loc)
+### +Adv+Cmp\#
+- sme: dáppeolmmoš - dáppe+Adv+Err/Orth+Cmp#olmmoš+N+Sg+Nom : Hvorfor Err/Orth?
-### Forleddet er substantiv, nominativ eller genitiv, men sammensetninga følger ikke regler i lexc for stammevokalen i forledd
-* sme: jahkeduhát (jahki+N+Sg+Nom)
-* sme: lottibeassi (loddi+N+Sg+Gen)
+### +ACR+Cmp-\#
+- sme: EU-válggain - EU+N+ACR+Cmp-#válga+N+Pl+Loc
-### Forleddet er substantiv, men har en forkortet form
-* sme: oahpaheaibargu (oahpaheaddji+N+Sg+Nom)
+### Eksempler på sammensetninger som ikke gir sammensetningsanalyse
+#### Forleddet er substantiv, ikke nominativ eller genitiv
+- sme: buorringeavaheapmi (buorri+N+Ess)
+- sme: fápmuibidjan (fápmu+N+Sg+Ill)
+- sme: árvvusatnin (árvu+N+Sg+Loc)
+#### Forleddet er substantiv, nominativ eller genitiv, men sammensetninga følger ikke regler i lexc for stammevokalen i forledd
-### Forleddet er pronomen
-* sme: iešdovdu (ieš+Pron+Refl+Sg+Nom)
-* sme: iežasmáksu (ieš+Pron+Refl+Gen+PxSg3)
-* sme: buohkaidopmodat (buohkat+Pron+Indef+Pl+Gen)
-* sme: dasagulli (dat+Pron+Dem+Sg+Ill)
+- sme: jahkeduhát (jahki+N+Sg+Nom)
+- sme: lottibeassi (loddi+N+Sg+Gen)
+#### Forleddet er substantiv, men har en forkortet form
-### Forleddet er tallord
-* sme: duhátjahki (duhát+Num+Sg+Nom)
-* sme: guoktelogi (guokte+Num+Sg+Nom)
-* sme: guvttiidlohku (guokte+Num+Pl+Gen)
+- sme: oahpaheaibargu (oahpaheaddji+N+Sg+Nom)
+#### Forleddet er pronomen
-### Forleddet er verb
-* sme: buolleviidna (buollit+V+IV+PrsPrc)
-* sme: báhcánvuoigatvuohta (báhcit+V+IV+PrfPrc)
+- sme: iešdovdu (ieš+Pron+Refl+Sg+Nom)
+- sme: iežasmáksu (ieš+Pron+Refl+Gen+PxSg3)
+- sme: buohkaidopmodat (buohkat+Pron+Indef+Pl+Gen)
+- sme: dasagulli (dat+Pron+Dem+Sg+Ill)
+#### Forleddet er tallord
+- sme: duhátjahki (duhát+Num+Sg+Nom)
+- sme: guoktelogi (guokte+Num+Sg+Nom)
+- sme: guvttiidlohku (guokte+Num+Pl+Gen)
+#### Forleddet er verb
-### Forleddet er adjektiv
-* sme: arvvesdálki (arvves+A+Attr)
-* sme: bajimusčoahkkin (bajit+A+Superl+Attr)
+- sme: buolleviidna (buollit+V+IV+PrsPrc)
+- sme: báhcánvuoigatvuohta (báhcit+V+IV+PrfPrc)
+#### Forleddet er adjektiv
-### Forleddet er adverb
-* sme: aisttonmearka (aistton+Adv)
-* sme: oktováhnen (okto+Adv)
-* sme: bajásgeassin (bajás+Adv)
-* sme: bieđgguidássanguovlu (bieđgguid+Adv)
+- sme: arvvesdálki (arvves+A+Attr)
+- sme: bajimusčoahkkin (bajit+A+Superl+Attr)
+#### Forleddet er adverb
-### Forleddet er postposisjon
-* sme: maŋisboahtti (maŋis+Po)
+- sme: aisttonmearka (aistton+Adv)
+- sme: oktováhnen (okto+Adv)
+- sme: bajásgeassin (bajás+Adv)
+- sme: bieđgguidássanguovlu (bieđgguid+Adv)
+#### Forleddet er postposisjon
-### Forleddet er propernoun
-* sme: Gáivuonasuopman (Gáivuotna+N+Prop+Sem/Plc+Sg+Gen)
-* sme: anársápmelaš (Anár+N+Prop+Sem/Plc+Sg+Nom)
+- sme: maŋisboahtti (maŋis+Po)
+#### Forleddet er propernoun
-### Ikke tydelige lånord, men forleddet får ikke egen analyse
-* sme: bužosdákti (bužos - ikke eget lemma)
+- sme: Gáivuonasuopman (Gáivuotna+N+Prop+Sem/Plc+Sg+Gen)
+- sme: anársápmelaš (Anár+N+Prop+Sem/Plc+Sg+Nom)
+#### Ikke tydelige lånord, men forleddet får ikke egen analyse
-### Etterleddet får ikke egen analyse
-* sme: bággopántideapmi (pántideapmi - ikke eget lemma)
-* sme: báikegoddelaš (goddelaš - ikke eget lemma)
+- sme: bužosdákti (bužos - ikke eget lemma)
+#### Etterleddet får ikke egen analyse
+- sme: bággopántideapmi (pántideapmi - ikke eget lemma)
+- sme: báikegoddelaš (goddelaš - ikke eget lemma)
+#### Lånord, forleddet får ikke egen analyse
-### Lånord, forleddet får ikke egen analyse
-* sme: adoptiivaváhnen (adoptiiva - ikke eget lemma)
-* sme: allegrohápmi (allegro - ikke eget lemma)
+- sme: adoptiivaváhnen (adoptiiva - ikke eget lemma)
+- sme: allegrohápmi (allegro - ikke eget lemma)
+#### Etterleddet er adjektiv
-### Etterleddet er adjektiv
-* sme: čálaoahppavaš
+- sme: čálaoahppavaš
diff --git a/lang/smi/index.md b/lang/smi/index.md
index a2d8637e..f901514b 100644
--- a/lang/smi/index.md
+++ b/lang/smi/index.md
@@ -1,14 +1,13 @@
-Topics
-======
+# Topics
-- [Compounding in the Saami languages](Samansetjing.html)
-- [Harmonising derivational tags for Saami languages.
- Overview.](../common/DerivationOverview.html)
-- [Discussions on restricting derivation for Saami languages. Log for
- what is done.](AvgrenseAvleiing.html)
-- [Prinsipper for homonymi i lemma, varianter og subfomer](lemma.html)
-- [How to handle variation in LEXC: Main documentation in
- English](../common/Variation_in_lexc.html)
-- [Discussions on restricting generating of possessive suffixes, esp.
- North Saami](https://giellalt.github.io/lang-sme/PXdiscussion.html)
-- [Bruken av Use/-Spell taggen](minusspelltag.html)
+- [Compounding in the Saami languages](Samansetjing.html)
+- [Harmonising derivational tags for Saami languages.
+ Overview.](../common/DerivationOverview.html)
+- [Discussions on restricting derivation for Saami languages. Log for
+ what is done.](AvgrenseAvleiing.html)
+- [Prinsipper for homonymi i lemma, varianter og subfomer](lemma.html)
+- [How to handle variation in LEXC: Main documentation in
+ English](../common/Variation_in_lexc.html)
+- [Discussions on restricting generating of possessive suffixes, esp.
+ North Saami](https://giellalt.github.io/lang-sme/PXdiscussion.html)
+- [Bruken av Use/-Spell taggen](minusspelltag.html)
diff --git a/lang/smi/lemma.md b/lang/smi/lemma.md
index 7e01c105..4ac4184a 100644
--- a/lang/smi/lemma.md
+++ b/lang/smi/lemma.md
@@ -1,66 +1,39 @@
# Prinsipp for lemmatisering av samiske språk
-
-
-
-
-
## Lemma som ikkje skal inn i stavekontrollen - Err/Lex
+Bakgrunnen for dette er ord i leksikon som ikkje er skal inn i stavekontrollen, men som likevel skal bli generert. Døme på slike ord er på sørsamisk _cubanske, juni_, og det kan være behov for det i nordsamisk også.
-Bakgrunnen for dette er ord i leksikon som ikkje er skal inn i stavekontrollen, men som likevel skal bli generert. Døme på slike ord er på sørsamisk *cubanske, juni*, og det kan være behov for det i nordsamisk også.
-
-
-Desse blir merka med *+Err/Lex* i leksikon. Dei kjem med i genereringsfilene, men ikkje i den normative fila.
-
+Desse blir merka med _+Err/Lex_ i leksikon. Dei kjem med i genereringsfilene, men ikkje i den normative fila.
## Leksikalsk homonymi: identifisere riktig lemma
+Lemmaene er homonyme, men det er samantisk forskjell og forskjellige bøyningsparadigmer. I nordsamisk skiller vi de fleste med G3- og NomAg-tagger, fordi det er systematikk for store grupper av lemmaer.
-Lemmaene er homonyme, men det er samantisk forskjell og forskjellige bøyningsparadigmer. I nordsamisk skiller vi de fleste med G3- og NomAg-tagger, fordi det er systematikk for store grupper av lemmaer.
-
-
-| Nom | Gen | norsk | norm-fst-analyse
-| --- | --- | --- | ---
-| lohkki | lohki | lokk | lohkki+N+Sg+Nom
-| lohkki | lohkki | lesar | lohkki+N+NomAg+Sg+Nom
-| beassi | beasi | reir | beassi+N+Sg+Nom
-| beassi | beassi | never | beassi+G3+N+Sg+Nom
-
-
-
+| Nom | Gen | norsk | norm-fst-analyse |
+| ------ | ------ | ----- | --------------------- |
+| lohkki | lohki | lokk | lohkki+N+Sg+Nom |
+| lohkki | lohkki | lesar | lohkki+N+NomAg+Sg+Nom |
+| beassi | beasi | reir | beassi+N+Sg+Nom |
+| beassi | beassi | never | beassi+G3+N+Sg+Nom |
Når det er snakk om enkelttilfeller, gir vi disse arbitrære taggar `+Hom1, +Hom2, …` (nummerert oppover ad lib).
-Taggane blir lagt inn i leksikon før POS, men burde flyttast til etter POS
+Taggane blir lagt inn i leksikon før POS, men burde flyttast til etter POS
i kompileringa.
-
-* Eksempler fra sørsamisk:
-** govledh+Hom1 - kl. IV å høre
-** govledh+Hom2 - kl. V å høres
-
-
-
+- Eksempler fra sørsamisk:
+ - govledh+Hom1 - kl. IV å høre
+ 0 govledh+Hom2 - kl. V å høres
## Varianter under samme lemma: sortere bøyningsformer til riktig grunnform - v1, v2 osv
-
Ortografiske varianter av samme lemma, dvs. grunnform og ihvertfall deler av bøyingsparadigmet, bør i fst sorteres under samme lemma. Men vi legger til en tag for å kunne sortere bøyningsparadigmene til riktig grunnform.
-
Vi brukar taggane `+v1, +v2, …` (nummerert oppover ad lib) for å skilje mellom
dei ulike paradigmene.
-
-* Eksempler:
-** sihkar+v1:sihkar
-** sihkar+v2:sihkkar
-
+- Eksempler:
+ - sihkar+v1:sihkar
+ - sihkar+v2:sihkkar
Hvis grunnformen er den samme, men det er to mulige bøyningsparadigmer, bruker vi ikke denne merkinga.
-
-
-
-
-
-
diff --git a/lang/smi/minusspelltag.md b/lang/smi/minusspelltag.md
index 7c8f5d7d..30974e7f 100644
--- a/lang/smi/minusspelltag.md
+++ b/lang/smi/minusspelltag.md
@@ -1,13 +1,11 @@
# Bruken av +Use/-Spell for samiske språk
-
-
Dokumentasjon over bruken i lexc for samiske språk.
Møtereferat er [her](https://divvungiellatekno.github.io/giellalt.uit.no/admin/linguists/220324_Tagger_Adverber.html)
-```
-+Use/-Spell Orthographically correct, typically perifer words,
+```text
++Use/-Spell Orthographically correct, typically perifer words,
excluded in speller because they cause trouble for frequent words (fra sme root)
lang-sme lan000$ cut -d '!' -f1 src/fst/stems/* |grep 'Use/-Spell' |wc -l 33
@@ -21,65 +19,66 @@ lang-sma lan000$ cut -d '!' -f1 src/fst/affixes/* |grep 'Use/-Spell' |wc -l
lang-sms lan000$ cut -d '!' -f1 src/fst/stems/* |grep 'Use/-Spell' |wc -l 0
lang-sms lan000$ cut -d '!' -f1 src/fst/affixes/* |grep 'Use/-Spell' |wc -l 14
lang-smn: 0
- ```
+```
Linjene med denne taggen blir ikke med i normativ HFST. Vi diskuterte bruken.
-
-
## Bruken i sme
-### Oftest for å begrense generering, for å unngå å generere marginale former.
+### Oftest for å begrense generering, for å unngå å generere marginale former
Ek
-```
-LEXICON acrooblique
+
+```text
+LEXICON acrooblique
!------Der/ár------
+Der2+Der/ár+N+CmpN/SgN+CmpN/SgG+CmpN/PlG+Use/-Spell:»ár GAHPIRLONG ; !
husme
-ČSV:ár ČSV:ár+v1+N+Sg+Nom
-ČSV:ár ČSV+v1+N+Prop+Sem/Org+ACR+Der/ár+N+Sg+Nom
-SG:ár SG+N+Prop+Sem/Org+ACR+Der/ár+N+Sg+Nom
+ČSV:ár ČSV:ár+v1+N+Sg+Nom
+ČSV:ár ČSV+v1+N+Prop+Sem/Org+ACR+Der/ár+N+Sg+Nom
+SG:ár SG+N+Prop+Sem/Org+ACR+Der/ár+N+Sg+Nom
-husmeNorm
+husmeNorm
ČSV:ár
-ČSV:ár ČSV:ár+N+Sg+Nom
-SG:ár SG:ár+? inf
+ČSV:ár ČSV:ár+N+Sg+Nom
+SG:ár SG:ár+? inf
+
+```
+
+Jeg har sammenlikna med korpus. [i SIKOR finnes, med bøyningformer](https://gtweb.uit.no/korp/#?stats_reduce=word&cqp=%5Bword%20*%3D%20%22.*%5BA-Z%C4%8C%C5%A0%5D%7B2%7D:*%5Ba%C3%A1%5Dr.%7B0,5%7D%22%5D&prefix&isCaseInsensitive&search_tab=1&search=cqp) : AUF:ár, TIFF:ár, NSR:ár, SG:ár, ČSV:ár. Bare ČSV:ár er leksikalisert.
- ```
-Jeg har sammenlikna med korpus. [i SIKOR finnes, med bøyningformer](https://gtweb.uit.no/korp/#?stats_reduce=word&cqp=%5Bword%20*%3D%20%22.*%5BA-Z%C4%8C%C5%A0%5D%7B2%7D:*%5Ba%C3%A1%5Dr.%7B0,5%7D%22%5D&prefix&isCaseInsensitive&search_tab=1&search=cqp) : AUF:ár, TIFF:ár, NSR:ár, SG:ár, ČSV:ár. Bare ČSV:ár er leksikalisert.
#### Hva er argumentet mot å la alle akronymer også få :ár med alle kasus? Eller burde man lage egen sti for Sem/Org som er typiske for slik bruk?
+
f.eks. for alle politiske parti?
-numerals.lexc: +Use/-Spell+Use/Circ: NUM-PREFIXES ; ! for §34 etc.
+numerals.lexc: +Use/-Spell+Use/Circ: NUM-PREFIXES ; ! for §34 etc.
Av 28 stier for adjektiv + vuohta har 8 +Use/-Spell, kanskje fordi de er mindre produktive? Men -vuohta skulle kanskje ikke dekke over for skrivefeil?
Disse bør sjekkes og sammenliknes med korpus. (med første blikk ser de ut til å være veldig marginale, f.eks. med adjektiv i flertall før derivasjon med vuohta, cealkemeahttumatvuohta
-
-### Noen ganger for å unngå genererte former som er svært marginale og som kan dekke over skrivefeil i frekvente ord,
+### Noen ganger for å unngå genererte former som er svært marginale og som kan dekke over skrivefeil i frekvente ord
eks.
-LEXICON ENGEL Restricted denominals for speller -eŋgel
-eŋgelaš eŋgel+N+Der/Dimin+N+Sg+Nom som også er en Err/Orth av eŋgelas
+LEXICON ENGEL Restricted denominals for speller -eŋgel
+eŋgelaš eŋgel+N+Der/Dimin+N+Sg+Nom som også er en Err/Orth av eŋgelas
-### Full dokumentasjon for sme, med kommentarer:
+### Full dokumentasjon for sme, med kommentarer
-#### nouns-fila: for å begrense generering, unngå for mange irrelevante former:
+#### nouns-fila: for å begrense generering, unngå for mange irrelevante former
-sis+N+CmpN/SgN+Use/-Spell+Sem/Dummytag+Cmp/SgNom:sis%> Rreal ;
+sis+N+CmpN/SgN+Use/-Spell+Sem/Dummytag+Cmp/SgNom:sis%> Rreal ;
sisa+N+CmpN/SgN+Use/-Spell+Sem/Dummytag+Cmp/SgNom:sisa%> Rreal ;
(disse gir bare støy, svært få relevante ord mangler leksikalisering, jeg kommenterer stiene ut)
+#### 108 substantiver med dynamisk førsteledd fra adj+attr
-#### 108 substantiver med dynamisk førsteledd fra adj+attr:
-Disse har jeg sammenlikna med korpus. Dette er lite produktive stier. Jeg har kommentert dem ut, sjekka i korpus og DG-ordbok og lagt til lemmaer i adj-fila.
+Disse har jeg sammenlikna med korpus. Dette er lite produktive stier. Jeg har kommentert dem ut, sjekka i korpus og DG-ordbok og lagt til lemmaer i adj-fila.
Mange av disse får også analyse som N+Pl+Nom, og overgenerering skaper dermed støy i analysen, siden adjektivanalysen blir +Attr
Ved at det ikke er dynamisk analyse, kan vi fange dem opp i missinglist.
-LEXICON NAMATCont second-part compounds (fra adj+attr og fra arabics)
+LEXICON NAMATCont second-part compounds (fra adj+attr og fra arabics)
nuolus+N+Use/-Spell:nuollus AHKASAS "unravelled? A" ;
stávval+N+Use/-Spell:stávval AGAdjINFL "syllabled A" ; Ikke i bruk
náittot+N+CmpN/SgN+CmpN/PlG+Use/-Spell+Sem/Hum:náittog AGAdjINFL "-gamic A" ;
@@ -88,10 +87,11 @@ suttat+N+Use/-Spell+Sem/Plc:sutt AGAdj ;
dáfot+N+Use/-Spell:dáfog AGAdjINFL "faceted A" ;
osv.
-#### substantiver med dynamisk førsteledd fra arabics:
+#### substantiver med dynamisk førsteledd fra arabics
+
Disse har jeg sammenlikna med korpus. Jeg har lagt til noen som manglet. Jeg forstår ikke at disse skulle lage problemer, så jeg har fjerna Use/-Spell for dem som er produktive
-LEXICON SASCont FROM NUMERALS, gives -kilosaš etc.
+LEXICON SASCont FROM NUMERALS, gives -kilosaš etc.
buddi+N+Use/-Spell:buddás DER-SAS ;
báiki+N+Use/-Spell+Sem/Ani_Hum:báikás DER-SAS ;
dássi+N+Use/-Spell:dássás DER-SAS ;
@@ -105,27 +105,29 @@ vahkku+N+Use/-Spell+Sem/Ani_Hum:vahkkos DER-SAS ; !50-vahkkosaš
čiehka+N+Der2+Der/has+N+Use/-Spell:čiegahass JOHTOLAT ;
giella+N+Der2+Der/lasj+A+Use/-Spell:gielal AHKASAS ; !2-gielalaš
-
#### hit går f.eks. fra NAMATCont ahki+N+Sem/Ani_Hum:ag DER-AGAdj ;
+
Her fjerner jeg Use/-Spell, den hindrer fornuftige dynamiske ord. Begrensninga bør skje tidligere i stien.
LEXICON DER-AGAdj
- +Der2+Der/t+A+CmpN/SgN+CmpN/PlG+Use/-Spell: AGAdj ; (2-agat)
++Der2+Der/t+A+CmpN/SgN+CmpN/PlG+Use/-Spell: AGAdj ; (2-agat)
#### hit går f.eks. fra NAMATCont lahttu+N+Sem/Hum:laht DER-OGAdj "membered A" ;
+
Her fjerner jeg Use/-Spell, den hindrer fornuftige dynamiske ord. Begrensninga bør skje tidligere i stien.
LEXICON DER-OGAdj
- +Der2+Der/t+A+CmpN/SgN+CmpN/PlG+Use/-Spell:og AGAdjINFL ; (2-lahtot)
++Der2+Der/t+A+CmpN/SgN+CmpN/PlG+Use/-Spell:og AGAdjINFL ; (2-lahtot)
+
+#### hit går f.eks. fra NAMATCont málli+N+Sem/Ani_Hum:máll DER-EGAdj "modelled A" ;
-#### hit går f.eks. fra NAMATCont málli+N+Sem/Ani_Hum:máll DER-EGAdj "modelled A" ;
Her fjerner jeg Use/-Spell, den hindrer fornuftige dynamiske ord. Begrensninga bør skje tidligere i stien.
LEXICON DER-EGAdj
- +Der2+Der/t+A+CmpN/SgN+CmpN/PlG+Use/-Spell:eg AGAdjINFL ; (2-mállet)
++Der2+Der/t+A+CmpN/SgN+CmpN/PlG+Use/-Spell:eg AGAdjINFL ; (2-mállet)
+#### Hvorfor disse?
-#### Hvorfor disse?
dávvirvuorkásuorgi+N+Use/-Spell+Sem/Plc-abstr:dávvir#vuorká#suorºgi GOAHTI-I ;
gákcilotlohku+v1+N+Use/-Spell+Sem/Dummytag:gákci#lot#lohºku LOTLOHKU ;
gákcilotlohku+v2+N+Use/-Spell+Sem/Dummytag:gákci#loh9#lohºku LOTLOHKU ;
@@ -141,62 +143,61 @@ sábbát+v1+N+CmpN/SgN+CmpN/SgG+CmpN/PlG+Sem/Time:sabºbah GAHPIRLONG ; ! NT
sme-acronyms.lexc:iežaskap+Use/-Spell+Sem/Dummytag:iežaskap9 UNIT ; (forkortelse for iežaskapitála, ikke i bruk i SIKOR)
-
#### i AFFIX-FILER:
-LEXICON ENGEL Restricted denominals for speller -eŋgel
+LEXICON ENGEL Restricted denominals for speller -eŋgel
+
+LEXICON BUORRE For this adj only
++Use/-Spell: VUOHTA ; ! ... to A > N -vuohta derivation
++Use/-Spell:»X7 NAMAT ; ! comp-only adj. Here since buorre has no Attr, not compound.
++Use/-Spell:»X7# NAMATLAGANLAGASCont ;
-LEXICON BUORRE For this adj only
-+Use/-Spell: VUOHTA ; ! ... to A > N -vuohta derivation
-+Use/-Spell:»X7 NAMAT ; ! comp-only adj. Here since buorre has no Attr, not compound.
-+Use/-Spell:»X7# NAMATLAGANLAGASCont ;
+LEXICON RIEKTA Bisyll adj w/o obl sg forms, WeG Attr
++Use/-Spell:a VUOHTA ; ! ... to A > N -vuohta derivation
-LEXICON RIEKTA Bisyll adj w/o obl sg forms, WeG Attr
-+Use/-Spell:a VUOHTA ; ! ... to A > N -vuohta derivation
+LEXICON MEAHTTUS meahttun-adj. with comp. and superl. forms -seabbo, -seamos etc.
++Pl+Nom+Use/-Spell:m%>at VUOHTA ;
-LEXICON MEAHTTUS meahttun-adj. with comp. and superl. forms -seabbo, -seamos etc.
- +Pl+Nom+Use/-Spell:m%>at VUOHTA ;
-
-LEXICON BEAKKAN Trisyll. Non-gradating C-Adj. without Separate Attr.
- +Pl+Nom+Use/-Spell:%>at VUOHTA
-
-LEXICON BEAKKAN_NOCOMP
- +Pl+Nom+Use/-Spell:%>at VUOHTA ;
+LEXICON BEAKKAN Trisyll. Non-gradating C-Adj. without Separate Attr.
++Pl+Nom+Use/-Spell:%>at VUOHTA
-LEXICON GEARDAN Trisyll. Non-gradating C-Adj. without Separate Attr.
+LEXICON BEAKKAN_NOCOMP
+Pl+Nom+Use/-Spell:%>at VUOHTA ;
-LEXICON VUDDJII_DECLINED misses most cases
-+Use/-Spell: VUOHTA ; ! VUOHTA, without j
+LEXICON GEARDAN Trisyll. Non-gradating C-Adj. without Separate Attr.
++Pl+Nom+Use/-Spell:%>at VUOHTA ;
+
+LEXICON VUDDJII_DECLINED misses most cases
++Use/-Spell: VUOHTA ; ! VUOHTA, without j
-LEXICON JIEDNAI_DECLINED misses most cases
+LEXICON JIEDNAI_DECLINED misses most cases
+Pl+Acc+Use/-Spell:%>Y5jd K ; !riiduid, ruvsuid
-LEXICON LAS from verbs: čirrolas, bealkálas etc
+LEXICON LAS from verbs: čirrolas, bealkálas etc
+Use/-Spell: VUOHTA ;
-LEXICON DenominalAdjsV1 caritives and their derivatives (huvva, huhtti), from bisyll nouns
- +Der1+Der2+Der/laakan+A+Use/-Spell:» LAGAN ; ! ! biilalágan, noaidelágán noaiddilágán beatnagalágán beanalágán all these goes Nielsen: beatnatlágán, beatnatlágáš, beanalágáš, giđalágáš, áhččelágáš,
- +Der1+Der2+Der/laagasj+A+Use/-Spell:» LAGAS ; ! ! etc.
-LEXICON DenominalAdjsV1Long caritives and their derivatives (huvva, huhtti), from bisyll nouns without vowel shortening
- +Der1+Der2+Der/laakan+A+Use/-Spell:» LAGAN ; ! ! biilalágan, noaidelágán noaiddilágán beatnagalágán beanalágán all these goes Nielsen: beatnatlágán, beatnatlágáš, beanalágáš, giđalágáš, áhččelágáš,
- +Der1+Der2+Der/laagasj+A+Use/-Spell:» LAGAS ; ! ! etc.
-
-LEXICON DenominalAdjsV1Short caritives and their derivatives (huvva, huhtti), from bisyll nouns with vowel shortening
- +Der1+Der2+Der/laakan+A+Use/-Spell:» LAGAN ; ! ! biilalágan, noaidelágán noaiddilágán beatnagalágán beanalágán all these goes Nielsen: beatnatlágán, beatnatlágáš, beanalágáš, giđalágáš, áhččelágáš,
- +Der1+Der2+Der/laagasj+A+Use/-Spell:» LAGAS ; ! ! etc.
+LEXICON DenominalAdjsV1 caritives and their derivatives (huvva, huhtti), from bisyll nouns
++Der1+Der2+Der/laakan+A+Use/-Spell:» LAGAN ; ! ! biilalágan, noaidelágán noaiddilágán beatnagalágán beanalágán all these goes Nielsen: beatnatlágán, beatnatlágáš, beanalágáš, giđalágáš, áhččelágáš,
++Der1+Der2+Der/laagasj+A+Use/-Spell:» LAGAS ; ! ! etc.
+LEXICON DenominalAdjsV1Long caritives and their derivatives (huvva, huhtti), from bisyll nouns without vowel shortening
++Der1+Der2+Der/laakan+A+Use/-Spell:» LAGAN ; ! ! biilalágan, noaidelágán noaiddilágán beatnagalágán beanalágán all these goes Nielsen: beatnatlágán, beatnatlágáš, beanalágáš, giđalágáš, áhččelágáš,
++Der1+Der2+Der/laagasj+A+Use/-Spell:» LAGAS ; ! ! etc.
+LEXICON DenominalAdjsV1Short caritives and their derivatives (huvva, huhtti), from bisyll nouns with vowel shortening
++Der1+Der2+Der/laakan+A+Use/-Spell:» LAGAN ; ! ! biilalágan, noaidelágán noaiddilágán beatnagalágán beanalágán all these goes Nielsen: beatnatlágán, beatnatlágáš, beanalágáš, giđalágáš, áhččelágáš,
++Der1+Der2+Der/laagasj+A+Use/-Spell:» LAGAS ; ! ! etc.
-LEXICON DenominalAdjsV2_lasj from bisyllables, muoralaš, gieđalaš etc
- +Sg+Nom+PxDu2+Use/-Spell:»X6lažža%>X2t RPXADD_FLAG ; ! ! tentative.
- LEXICON RPXADD_FLAG
+LEXICON DenominalAdjsV2_lasj from bisyllables, muoralaš, gieđalaš etc
++Sg+Nom+PxDu2+Use/-Spell:»X6lažža%>X2t RPXADD_FLAG ; ! ! tentative.
+LEXICON RPXADD_FLAG
@R.Px.add@ K ;
-LEXICON acrooblique
+LEXICON acrooblique
!------Der/ár------
- +Der2+Der/ár+N+CmpN/SgN+CmpN/SgG+CmpN/PlG+Use/-Spell:»ár GAHPIRLONG ; !
++Der2+Der/ár+N+CmpN/SgN+CmpN/SgG+CmpN/PlG+Use/-Spell:»ár GAHPIRLONG ; !
#### Propernouns
+
Alle leksikon har denne: (men fra RProp kreves hyphen, hvis ikke Err/Orth, så Use/-Spell her er unødvendig, jeg kommenterer den ut)
+Cmp/SgNom+Use/-Spell:%> RProp ;
@@ -204,4 +205,3 @@ LEXICON SULLOT-plc
+N+Prop+Sem/Plc+Sg+Gen+Use/-Spell:%>Y5 VUONAT ;
LEXICON ADJAGAT-plc ! Place names
+N+Prop+Sem/Plc+Sg+Nom+Use/-Spell:X4 VUONAT ;
-
diff --git a/ling/CorpusConvertingManipulation.md b/ling/CorpusConvertingManipulation.md
index b9993396..272806b8 100644
--- a/ling/CorpusConvertingManipulation.md
+++ b/ling/CorpusConvertingManipulation.md
@@ -1,18 +1,9 @@
# How to get a better converting
-
Converting of
-* [doc-documents](DocConvertingManipulation.html)
-* [html-documents](HtmlConvertingManipulation.html)
-* [pdf-documents](PdfConvertingManipulation.html)
-* [rtf-documents](RtfConvertingManipulation.html)
-* [txt-documents](TxtConvertingManipulation.html)
-
-
-
-
-
-
-
-
+- [doc-documents](DocConvertingManipulation.html)
+- [html-documents](HtmlConvertingManipulation.html)
+- [pdf-documents](PdfConvertingManipulation.html)
+- [rtf-documents](RtfConvertingManipulation.html)
+- [txt-documents](TxtConvertingManipulation.html)
diff --git a/ling/CorpusTools.md b/ling/CorpusTools.md
index 6d090bf2..2bfb2244 100644
--- a/ling/CorpusTools.md
+++ b/ling/CorpusTools.md
@@ -1,110 +1,94 @@
-Corpus Tools
-============
-
+# Corpus Tools
Corpus Tools contains tools to manipulate a corpus in different ways.
These scripts will be installed
+## Howto install and update the tools
-# Howto install and update the tools
-
-
-## First time install
-* [Install requirements](#Requirements).
-* [Install CorpusTools](#To-own-home-directory-(recommended))
-
+### First time install
-## Update
-* [Howto update CorpusTools](#To-own-home-directory-(recommended))
+- [Install requirements](#Requirements).
+- [Install CorpusTools](<#To-own-home-directory-(recommended)>)
+### Update
+- [Howto update CorpusTools](<#To-own-home-directory-(recommended)>)
+## Use the content of the corpus
-# Use the content of the corpus
-* [convert2xml: Convert original files to giellatekno xml](CorpusTools.html#convert2xml)
-* [ccat: Print the contents of a converted corpus file as plain text](CorpusTools.html#ccat)
-* [analyse_corpus: Do syntactic analysis of converted files](CorpusTools.html#analyse_corpus)
-* [parallelize: Sentence align file pairs](CorpusTools.html#parallelize)
-* [reparallelize: Reconvert and realign a given .tmx.html file](CorpusTools.html#reparallelize)
-* [tmx2html: Convert tmx files to html files](CorpusTools.html#tmx2html)
+- [convert2xml: Convert original files to giellatekno xml](CorpusTools.html#convert2xml)
+- [ccat: Print the contents of a converted corpus file as plain text](CorpusTools.html#ccat)
+- [analyse_corpus: Do syntactic analysis of converted files](CorpusTools.html#analyse_corpus)
+- [parallelize: Sentence align file pairs](CorpusTools.html#parallelize)
+- [reparallelize: Reconvert and realign a given .tmx.html file](CorpusTools.html#reparallelize)
+- [tmx2html: Convert tmx files to html files](CorpusTools.html#tmx2html)
+## Add files to the corpus
-# Add files to the corpus
-* [add_files_to_corpus: Add file(s) to a corpus directory](CorpusTools.html#add_files_to_corpus)
-* [saami_crawler: Crawl saami sites, add files to corpus](CorpusTools.html#saami_crawler)
+- [add_files_to_corpus: Add file(s) to a corpus directory](CorpusTools.html#add_files_to_corpus)
+- [saami_crawler: Crawl saami sites, add files to corpus](CorpusTools.html#saami_crawler)
+## Manage the corpus repositories
-# Manage the corpus repositories
-* [move_corpus_file: Move or rename a file inside the corpus](CorpusTools.html#move_corpus_file)
-* [remove_corpus_file: Remove a file from the corpus](CorpusTools.html#remove_corpus_file)
-* [normalise_corpus_names: Program to normalise file names](CorpusTools.html#normalise_corpus_names)
-* [paracheck: Check if the parallel files found in the metadata files exist](CorpusTools.html#paracheck)
-* [duperemover: Remove duplicate files from the given directory](CorpusTools.html#duperemover)
-* [dupefinder: Find files with more than 90% similarity in the given directory](CorpusTools.html#dupefinder)
-* [clean_prestable:Remove files in prestable that have no original files](CorpusTools.html#clean_prestable)
-* [pick_parallel_docs: Pick out parallel files from converted to prestable/converted](CorpusTools.html#pick_parallel_docs)
-* [update_metadata: Update metadata files in given directories](CorpusTools.html#update_metadata)
+- [move_corpus_file: Move or rename a file inside the corpus](CorpusTools.html#move_corpus_file)
+- [remove_corpus_file: Remove a file from the corpus](CorpusTools.html#remove_corpus_file)
+- [normalise_corpus_names: Program to normalise file names](CorpusTools.html#normalise_corpus_names)
+- [paracheck: Check if the parallel files found in the metadata files exist](CorpusTools.html#paracheck)
+- [duperemover: Remove duplicate files from the given directory](CorpusTools.html#duperemover)
+- [dupefinder: Find files with more than 90% similarity in the given directory](CorpusTools.html#dupefinder)
+- [clean_prestable:Remove files in prestable that have no original files](CorpusTools.html#clean_prestable)
+- [pick_parallel_docs: Pick out parallel files from converted to prestable/converted](CorpusTools.html#pick_parallel_docs)
+- [update_metadata: Update metadata files in given directories](CorpusTools.html#update_metadata)
+## Miscellaneous
-# Miscellaneous
-* [pytextcat: textcat implemented in Python](CorpusTools.html#pytextcat)
-* [generate_anchor_list: Generate paired anchor list for languages lang1 and lang2](CorpusTools.html#generate_anchor_list)
-* [html_cleaner: Program to print out a nicely indented html document](CorpusTools.html#html_cleaner)
-* [epubchooser: Program to set metadata of an epub file](CorpusTools.html#epubchooser)
-* [make_training_corpus: Program to make training corpus from giella xml analysed files](CorpusTools.html#make_training_corpus)
+- [pytextcat: textcat implemented in Python](CorpusTools.html#pytextcat)
+- [generate_anchor_list: Generate paired anchor list for languages lang1 and lang2](CorpusTools.html#generate_anchor_list)
+- [html_cleaner: Program to print out a nicely indented html document](CorpusTools.html#html_cleaner)
+- [epubchooser: Program to set metadata of an epub file](CorpusTools.html#epubchooser)
+- [make_training_corpus: Program to make training corpus from giella xml analysed files](CorpusTools.html#make_training_corpus)
+## Requirements
-# Requirements
-
-
-* python3
-* pip for python3
-* pysvn (only needed for add_files_to_corpus)
-* wvHtml (only needed for convert2xml)
-* pdftohtml (only needed for convert2xml)
-* latex2html (only needed for convert2xml)
-* Java (only needed for parallelize)
-* Perl (only needed for parallelize)
-
+- python3
+- pip for python3
+- pysvn (only needed for add_files_to_corpus)
+- wvHtml (only needed for convert2xml)
+- pdftohtml (only needed for convert2xml)
+- latex2html (only needed for convert2xml)
+- Java (only needed for parallelize)
+- Perl (only needed for parallelize)
On Mac, do:
-
```
sudo port install py-pysvn py-pip wv latex2html poppler
```
-
On Debian/Ubuntu, do:
-
```
sudo apt-get install python3-svn python3-pip wv latex2html poppler-utils
```
-
On Arch Linux, do:
-
```
sudo pacman -S python3-pip wv
yaourt -S python3-pysvn
```
-
You also need to have the $GLANGS variable set to where you checked
-out *https://github.com/giellalt/CorpusTools/* (see the *Getting Started* documentation).
-
-
-## Custom version of pdftohtml (poppler)
+out *https://github.com/giellalt/CorpusTools/* (see the _Getting Started_ documentation).
+### Custom version of pdftohtml (poppler)
The standard version of pdftohtml sometimes produces invalid xml-documents.
A version that fixes this bug is found at https://github.com/albbas/poppler
and the poppler developers have been notified about the bug.
-
To install it do the following
+
```
git clone https://github.com/albbas/poppler
cd poppler
@@ -114,120 +98,96 @@ make
sudo make install
```
+## Installation
-# Installation
-
-
-## To own home directory (recommended)
-
+### To own home directory (recommended)
Install the tools for the current user by writing
-
```
cd $GTLANGS/CorpusTools
python3 setup.py install --user --install-scripts=$HOME/bin --record installed_files.txt
```
+### System wide (recommended for servers only)
-## System wide (recommended for servers only)
Install the tools for all users on a machine by
-
```
cd $GTLANGS/CorpusTools
sudo python3 setup.py install --install-scripts=/usr/local/bin --record installed_files.txt
```
+## Uninstalling
+### Remove from own home directory
-
-# Uninstalling
-
-
-## Remove from own home directory
```
cd $GTLANGS/CorpusTools
cat installed_files.txt | xargs rm -rf
```
-
-## System wide
+### System wide
```
cd $GTLANGS/CorpusTools
cat installed_files.txt | xargs sudo rm -rf
```
-
-# ccat
-
+## ccat
Convert corpus format xml to clean text.
-
ccat has three usage modes, print to stdout the content of:
-* converted files (produced by [convert2xml](CorpusTools.html#convert2xml))
-* converted files containing errormarkup (produced by [convert2xml](CorpusTools.html#convert2xml))
-* analysed files (produced by [analyse_corpus](CorpusTools.html#analyse_corpus))
+- converted files (produced by [convert2xml](CorpusTools.html#convert2xml))
+- converted files containing errormarkup (produced by [convert2xml](CorpusTools.html#convert2xml))
+- analysed files (produced by [analyse_corpus](CorpusTools.html#analyse_corpus))
-## Printing content of converted files to stdout
-
+### Printing content of converted files to stdout
To print out all sme content of all the converted files found in
$GTFREE/converted/sme/admin and its subdirectories, issue the command:
-
```
ccat -a -l sme $GTFREE/converted/sme/admin
```
-
It is also possible to print a file at a time:
-
```
ccat -a -l sme $GTFREE/converted/sme/admin/sd/other_files/vl_05_1.doc.xml
```
-
To print out the content of e.g. all converted pdf files found in a directory
and its subdirectories, issue this command:
+
```
find converted/sme/science/ -name "*.pdf.xml" | xargs ccat -a -l sme
```
-
-## Printing content of analysed files to stdout
-
+### Printing content of analysed files to stdout
The analysed files produced by
[analyse_corpus](CorpusTools.html#analyse_corpus) contain among other one
dependency element and one disambiguation element, that contain the
dependency and disambiguation analysis of the original files content.
-
```
ccat -dis sda/sda_2006_1_aikio1.pdf.xml
```
-
Prints the content of the disambiguation element.
-
```
ccat -dep sda/sda_2006_1_aikio1.pdf.xml
```
-
Prints the content of the dependency element.
-
The usage pattern for printing these elements is otherwise the same as
printing the content of converted files.
-
Printing dependency elements
```
@@ -236,7 +196,6 @@ ccat -dep $GTFREE/analysed/sme/admin/sd/other_files/vl_05_1.doc.xml
find analysed/sme/science/ -name "*.pdf.xml" | xargs ccat -dep
```
-
Printing disambiguation elements
```
@@ -245,16 +204,12 @@ ccat -dis $GTFREE/analysed/sme/admin/sd/other_files/vl_05_1.doc.xml
find analysed/sme/science/ -name "*.pdf.xml" | xargs ccat -dis
```
-
-## Printing errormarkup content
-
+### Printing errormarkup content
This usage mode is used in the speller tests. Examples of this usage pattern
is found in the make files in $GTBIG/prooftools.
-
-## The complete help text from the program:
-
+### The complete help text from the program:
```
usage: ccat [-h] [--version] [-l LANG] [-T] [-L] [-t] [-a] [-c] [-C] [-ort]
@@ -310,91 +265,66 @@ optional arguments:
Replace hyph tags with the given argument
```
-
**Source code**
-* [ccat.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/ccat.py)
-
-
-# convert2xml
+- [ccat.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/ccat.py)
+## convert2xml
Convert original files in a corpus to giellatekno/divvun xml format.
-
-
-
convert2xml depends on these external programs:
-
-* pdftotext
-* wvHtml
-
+- pdftotext
+- wvHtml
as well as various files from the Divvun/Giellatekno SVN, at least the
following files/directories need to exist under $GTHOME:
-
-* gt/dtd
-
+- gt/dtd
**Usage**
-
Convert all files in the directory $GTFREE/orig/sme and its
subdirectories.
-
```
convert2xml $GTFREE/orig/sme
```
-
The converted files are placed in $GTFREE/converted/sme
with the same directory structure as that in $GTFREE/orig/sme.
-
Convert only one file:
-
```
convert2xml $GTFREE/orig/sme/admin/sd/file1.html
```
-
The converted file is found in $GTFREE/orig/sme/admin/sd/file1.htm.xml
-
Convert all sme files in directories ending with corpus
-
```
convert2xml *corpus/orig/sme
```
-
If convert2xml is not able to convert a file these kinds of message will appear:
-
```
/home/boerre/Dokumenter/corpus/freecorpus/orig/eng/admin/depts/regjeringen.no/calendar-for-the-ministry-of-children-an.html_id=308
```
-
A log file will be found in
-
```
/home/boerre/Dokumenter/corpus/freecorpus/orig/eng/admin/depts/regjeringen.no/calendar-for-the-ministry-of-children-an.html_id=308.log
```
-
explaining what went wrong.
-
The complete help text from the program:
-
```
usage: convert2xml [-h] [--version] [--serial] [--lazy-conversion]
[--write-intermediate] [--goldstandard]
@@ -422,79 +352,62 @@ optional arguments:
--goldstandard Convert goldstandard and .correct files
```
-
**Source code**
-* [converter.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/converter.py)
-* [ccat.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/ccat.py)
-* [corpuspath.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/corpuspath.py)
-* [decode.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/decode.py)
-* [errormarkup.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/errormarkup.py)
-* [text_cat.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/text_cat.py)
-* [xslsetter.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/xslsetter.py)
-* [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
-
-
-# analyse_corpus
+- [converter.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/converter.py)
+- [ccat.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/ccat.py)
+- [corpuspath.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/corpuspath.py)
+- [decode.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/decode.py)
+- [errormarkup.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/errormarkup.py)
+- [text_cat.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/text_cat.py)
+- [xslsetter.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/xslsetter.py)
+- [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
+## analyse_corpus
Analyse converted corpus files.
-
analyse_corpus depends on these external programs:
-
-* preprocess (found in the Divvun/Giellatekno svn)
-* lookup2cg (found in the Divvun/Giellatekno svn)
-* lookup (from xfst)
-* vislcg3
-* hfst
-
+- preprocess (found in the Divvun/Giellatekno svn)
+- lookup2cg (found in the Divvun/Giellatekno svn)
+- lookup (from xfst)
+- vislcg3
+- hfst
**Usage**
-
To be able to use this program you must either use the
[nightly giella packages](https://giellalt.uit.no/infra/compiling_HFST3.html#The+simple+installation+%28you+download+ready-made+programs%29)
or build the needed resources for the supported
languages (exchange "sma" with "sme, smj" ad lib):
-
`cd $GTHOME/langs/sma`
-
Configure the language, use at least these to options `--prefix=$HOME/.local --with-hfst --enable-tokenisers`
-
```
-./configure --prefix=$HOME/.local --with-hfst --enable-tokenisers # add your own flags to taste
+./configure --prefix=$HOME/.local --with-hfst --enable-tokenisers ## add your own flags to taste
make install
```
-
Then you must convert the corpus files as explained in the [convert2xml](CorpusTools.html#convert2xml) section.
-
When this is done you can analyse all files in the directory $GTFREE/converted/sme (and sma, smj) and its subdirectories by issuing this command:
-
```
analyse_corpus -k hfst sme$GTFREE/converted/sme
```
-
-The analysed file will be found in
+The analysed file will be found in
`$GTFREE/analysed/sme`
-
To analyse only one file, issue this command:
-
```
analyse_corpus -k hfst --serial sme $GTFREE/converted/sme/file.html.xml
```
-
The complete help text from the program:
```
@@ -522,21 +435,17 @@ optional arguments:
default).
```
-
**Source code**
-* [analyser.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/analyser.py)
-* [ccat.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/ccat.py)
-* [parallelize.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/parallelize.py)
-* [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
-
-
-# add_files_to_corpus
+- [analyser.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/analyser.py)
+- [ccat.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/ccat.py)
+- [parallelize.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/parallelize.py)
+- [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
+## add_files_to_corpus
The complete help text from the program is as follows:
-
```
usage: add_files_to_corpus [-h] [-v] [-p PARALLEL_FILE] [-l LANG]
[-d DIRECTORY]
@@ -571,77 +480,54 @@ no_parallel:
The directory where the origs should be placed
```
-
Examples:
Download and add parallel files from the net to the corpus:
-
cd $GTFREE
-
**Adding the first file**
-
The command
-```add_files_to_corpus -d orig/sme/admin/sd/other_files http://www.samediggi.no/content/download/5407/50892/version/2/file/Sametingets+%C3%A5rsmelding+2013+-+nordsamisk.pdf```
-
+`add_files_to_corpus -d orig/sme/admin/sd/other_files http://www.samediggi.no/content/download/5407/50892/version/2/file/Sametingets+%C3%A5rsmelding+2013+-+nordsamisk.pdf`
Gives the message:
-```Added orig/sme/admin/sd/other_files/sametingets_ay-rsmelding_2013_-_nordsamisk.pdf```
-
+`Added orig/sme/admin/sd/other_files/sametingets_ay-rsmelding_2013_-_nordsamisk.pdf`
**Adding the parallel file**
-
-```add_files_to_corpus -p orig/sme/admin/sd/other_files/sametingets_ay-rsmelding_2013_-_nordsamisk.pdf -l nob http://www.samediggi.no/content/download/5406/50888/version/2/file/Sametingets+%C3%A5rsmelding+2013+-+norsk.pdf```
-
+`add_files_to_corpus -p orig/sme/admin/sd/other_files/sametingets_ay-rsmelding_2013_-_nordsamisk.pdf -l nob http://www.samediggi.no/content/download/5406/50888/version/2/file/Sametingets+%C3%A5rsmelding+2013+-+norsk.pdf`
Gives the message:
-```Added orig/nob/admin/sd/other_files/sametingets_ay-rsmelding_2013_-_norsk.pdf```
-
+`Added orig/nob/admin/sd/other_files/sametingets_ay-rsmelding_2013_-_norsk.pdf`
After this is done, you will have to commit the files to
the working copy, like this:
-
```
svn ci orig
```
-
**Source code**
-* [adder.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/adder.py)
-* [namechanger.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/namechanger.py)
-* [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
-* [xslsetter.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/xslsetter.py)
-
-
-
-
-
-
-
-
-# parallelize
+- [adder.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/adder.py)
+- [namechanger.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/namechanger.py)
+- [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
+- [xslsetter.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/xslsetter.py)
+## parallelize
Parallelize parallel corpus files, write the results to
.tmx and .txm.html files.
-
NB! When debugging alignment, use [reparallelize](CorpusTools.html#reparallelize), it
reconverts all files and realigns the file anew.
-
parallelize depends on various files from the Divvun/Giellatekno SVN,
at least the following directories need to exist in $GTHOME:
-
-* langs (specifically, the abbr.txt files)
-* gt/common
-* gt/script
-
+- langs (specifically, the abbr.txt files)
+- gt/common
+- gt/script
It also requires Java if you wish to use the default (included)
alignment program TCA2. For convenience, a pre-compiled version of
@@ -649,13 +535,11 @@ TCA2's alignment.jar-file is included in SVN and installed by
CorpusTools, but if you have ant installed, you can recompile it by
simply typing "ant" in corpustools/tca2.
-
Alternatively, you can align with Hunalign, if you have that installed
(or don't have Java). Hunalign is faster, and the quality is less
dependent on predefined dictionaries (though it can use those as
well). Neither system gives perfect alignments.
-
By default, it uses the $GTHOME/gt/common/src/anchor.txt file as an
anchor dictionary for alignment. If your language pair is not in this
dictionary, you can provide your own with the --dict argument. If you
@@ -663,13 +547,10 @@ do not have a dictionary, you can use "--dict=<(echo)" to provide an
"empty" dictionary – in this case, you should also use
"--aligner=hunalign".
-
**Compile dependencies**
-
XXX is one of the languages in $GTHOME/langs.
-
```
cd $GTHOME/langs/XXX
./configure --prefix="$HOME"/.local \
@@ -682,12 +563,10 @@ XXX is one of the languages in $GTHOME/langs.
make install
```
-
To prepare for parallelising e.g. nob and sme files, do the following:
-
```
-for LANG in sme nob # Replace sme and nob by languages for your own needs
+for LANG in sme nob ## Replace sme and nob by languages for your own needs
do
cd $GTHOME/langs/$LANG
./configure --prefix="$HOME"/.local \
@@ -701,8 +580,8 @@ do
done
```
-
The complete help text from the program is as follows:
+
```
usage: parallelize [-h] [--version] [-s] [-f] [-q] [-a {hunalign,tca2}]
[-d DICT] -l2 LANG2
@@ -739,39 +618,33 @@ optional arguments:
parallelised with
```
-
-
-
You run the program on the files created by convert2xml by running a command with the following syntax:
+
```
parallelize -l2 TARGET_LANGUAGE PATH/TO/THE/CONVERTED/SOURCE_LANGUAGE/FILE.xml
```
-
for instance, with nob as SOURCE_LANGUAGE and sma as TARGET_LANGUAGE
+
```
parallelize -l2 sma converted/nob/admin/ntfk/tsaekeme.html.xml
```
-
This will create a file named
tmx/nob2sma/admin/ntfk/tsaekeme.html.tmx
-
If you want to parallelize all your sma files with nob in one go, you
can do e.g.
+
```
convert2xml orig/{sma,nob}
parallelize -l2 sma converted/nob
```
-
The files will end up in corresponding directories under
tmx/nob2sma.
-
-* ****CAVEAT 1****: *If you get a message such as*
-
+- \***\*CAVEAT 1\*\***: _If you get a message such as_
```
parallelize -l2 sma converted/sma/admin/ntfk/tsaekeme.html.xml
@@ -779,29 +652,24 @@ Error reading file '/Users/xxx/freecorpus/converted/sma/admin/ntfk/.xml':
failed to load external entity "/Users/xxx/freecorpus/converted/sma/admin/ntfk/.xml"
```
-
then you gave nob als l1 but the path to a sma-file as argument.
+- \***\*CAVEAT 2\*\***: _If you get a similar error message as_
-* ****CAVEAT 2****: *If you get a similar error message as*
```
parallelize -l2 sma converted/nob/admin/ntfk/rup_2013_trykt_versjon.pdf.xml
ERROR: /Users/xxx/gtsvn/langs/nob/tools/preprocess/tokeniser-gramcheck-gt-desc.pmhfst does not exist
```
-
you have to recompile the language tool of the respective language (in the example above it is nob)
with a different configuration, as in the following example with nob as language to recompile, have
a look at the info above on how to [compile dependencies](CorpusTools.html#Compile+dependencies)
-
After that you can go back to the directory where you are working with the parallelizing files and
try to parallelize the files anew. You might recompile the language tools for ALL the languages
you are working with.
-
-* ****CAVEAT 3****: *If you get a message like*
-
+- \***\*CAVEAT 3\*\***: _If you get a message like_
```
Exception in thread "main" java.lang.UnsupportedClassVersionError: aksis/alignment/Alignment : Unsupported major.minor version 51.0
@@ -819,52 +687,37 @@ Exception in thread "main" java.lang.UnsupportedClassVersionError: aksis/alignme
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
```
-
then you need to recompile the Java parts and reinstall CorpusTools.
Make sure you have Apache ant installed, then do:
-
```
cd $GTHOME/tools/CorpusTools/corpustools/tca2
ant
```
-
Then follow the instructions on [how to install CorpusTools ](CorpusTools.html#Installation)
-
**Source code**
-* [parallelize.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/parallelize.py)
-* [generate_anchor_list.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/generate_anchor_list.py)
-* [typosfile.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/typosfile.py)
-* [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
-
-
-
-
-
-
-# saami_crawler
+- [parallelize.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/parallelize.py)
+- [generate_anchor_list.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/generate_anchor_list.py)
+- [typosfile.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/typosfile.py)
+- [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
+## saami_crawler
Add files to freecorpus from a given site.
-
Only able to crawl www.samediggi.fi now, will collect html files only for now.
-
Run it like this:
-
```
saami_crawler www.samediggi.fi
```
-
The complete help text from the program is as follows:
-
```
usage: saami_crawler [-h] [-v] sites [sites ...]
@@ -881,38 +734,29 @@ optional arguments:
-v, --version show program's version number and exit
```
-
**Source code**
-* [saami_crawler.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/saami_crawler.py)
-* [namechanger.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/namechanger.py)
-* [text_cat.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/text_cat.py)
-* [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
-* [xslsetter.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/xslsetter.py)
-
-
-
-
-# pytextcat
+- [saami_crawler.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/saami_crawler.py)
+- [namechanger.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/namechanger.py)
+- [text_cat.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/text_cat.py)
+- [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
+- [xslsetter.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/xslsetter.py)
+## pytextcat
Pytextcat is an implementation of the "N-Gram-Based Text Categorization" algorithm.
-
Original article:
-
-Cavnar, W. B. and J. M. Trenkle,
-*N-Gram-Based Text Categorization*
+Cavnar, W. B. and J. M. Trenkle,
+_N-Gram-Based Text Categorization_
In Proceedings of Third Annual Symposium on
Document Analysis and Information Retrieval, Las Vegas, NV, UNLV
Publications/Reprographics, pp. 161-175, 11-13 April 1994.
-
Original Perl implementation and article available from
[http://odur.let.rug.nl/~vannoord/TextCat/]
-
```
usage: pytextcat [-h] [--version] [-V] {proc,complm,compwm,compdir} ...
@@ -935,22 +779,11 @@ optional arguments:
-V, --verbose Print some info to stderr
```
-
**Source code**
-* [text_cat.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/text_cat.py)
-
-
-
-
-
-
-
-
-
-
-# generate_anchor_list
+- [text_cat.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/text_cat.py)
+## generate_anchor_list
```
usage: generate_anchor_list.py [-h] [-v] [--lang1 LANG1] [--lang2 LANG2]
@@ -975,19 +808,15 @@ optional arguments:
--outdir OUTDIR The output directory
```
-
**Source code**
-* [generate_anchor_list.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/generate_anchor_list.py)
-* [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
-
-
-# normalise_corpus_names
+- [generate_anchor_list.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/generate_anchor_list.py)
+- [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
+## normalise_corpus_names
Normalise the filenames of the files found in the given directories.
-
```
usage: normalise_corpus_names [-h] [--version] target_dirs [target_dirs ...]
@@ -1006,15 +835,12 @@ optional arguments:
--version show program's version number and exit
```
-
**Source code**
-* [normalise_filenames](https://github.com/giellalt/CorpusTools/blob/main/corpustools/normalise_filenames.py)
-* [namechanger.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/namechanger.py)
-
-
-# move_corpus_file
+- [normalise_filenames](https://github.com/giellalt/CorpusTools/blob/main/corpustools/normalise_filenames.py)
+- [namechanger.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/namechanger.py)
+## move_corpus_file
```
usage: move_corpus_file [-h] [-v] oldpath newpath
@@ -1034,15 +860,12 @@ optional arguments:
-v, --version show program's version number and exit
```
-
**Source code**
-* [move_files.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/move_files.py)
-* [namechanger.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/namechanger.py)
-
-
-# paracheck
+- [move_files.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/move_files.py)
+- [namechanger.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/namechanger.py)
+## paracheck
```
usage: paracheck [-h] [-v] orig_dir
@@ -1061,17 +884,14 @@ optional arguments:
-v, --version show program's version number and exit
```
-
**Source code**
-* [check_para_consistency.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/check_para_consistency.py)
-* [namechanger.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/namechanger.py)
-* [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
-* [xslsetter.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/xslsetter.py)
-
-
-# html_cleaner
+- [check_para_consistency.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/check_para_consistency.py)
+- [namechanger.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/namechanger.py)
+- [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
+- [xslsetter.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/xslsetter.py)
+## html_cleaner
```
usage: html_cleaner [-h] [-v] inhtml outhtml
@@ -1092,16 +912,13 @@ optional arguments:
-v, --version show program's version number and exit
```
-
**Source code**
-* [html_cleaner.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/html_cleaner.py)
-* [converter.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/converter.py)
-* [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
-
-
-# duperemover
+- [html_cleaner.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/html_cleaner.py)
+- [converter.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/converter.py)
+- [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
+## duperemover
```
usage: duperemover [-h] [-v] dir
@@ -1119,19 +936,14 @@ optional arguments:
-v, --version show program's version number and exit
```
-
-
-
**Source code**
-* [dupe_finder.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/dupe_finder.py)
-* [ccat.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/ccat.py)
-* [move_files.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/move_files.py)
-* [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
-
-
-# dupefinder
+- [dupe_finder.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/dupe_finder.py)
+- [ccat.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/ccat.py)
+- [move_files.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/move_files.py)
+- [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
+## dupefinder
```
usage: dupefinder [-h] [-v] dir
@@ -1149,17 +961,14 @@ optional arguments:
-v, --version show program's version number and exit
```
-
**Source code**
-* [dupe_finder.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/dupe_finder.py)
-* [ccat.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/ccat.py)
-* [move_files.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/move_files.py)
-* [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
-
-
-# move_corpus_file
+- [dupe_finder.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/dupe_finder.py)
+- [ccat.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/ccat.py)
+- [move_files.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/move_files.py)
+- [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
+## move_corpus_file
```
usage: move_corpus_file [-h] [-v] oldpath newpath
@@ -1179,15 +988,12 @@ optional arguments:
-v, --version show program's version number and exit
```
-
**Source code**
-* [move_files.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/move_files.py)
-* [namechanger.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/namechanger.py)
-
-
-# remove_corpus_file
+- [move_files.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/move_files.py)
+- [namechanger.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/namechanger.py)
+## remove_corpus_file
```
usage: remove_corpus_file [-h] [-v] oldpath
@@ -1205,15 +1011,12 @@ optional arguments:
-v, --version show program's version number and exit
```
-
**Source code**
-* [move_files.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/move_files.py)
-* [namechanger.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/namechanger.py)
-
-
-# pick_parallel_docs
+- [move_files.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/move_files.py)
+- [namechanger.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/namechanger.py)
+## pick_parallel_docs
```
usage: pick_parallel_docs [-h] [-v] -p PARALLEL_LANGUAGE --minratio MINRATIO
@@ -1238,16 +1041,13 @@ optional arguments:
--maxratio MAXRATIO The maximum ratio
```
-
**Source code**
-* [pick_parallel_docs.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/pick_parallel_docs.py)
-* [parallelize.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/parallelize.py)
-* [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
-
-
-# clean_prestable
+- [pick_parallel_docs.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/pick_parallel_docs.py)
+- [parallelize.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/parallelize.py)
+- [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
+## clean_prestable
```
usage: clean_prestable [-h] [--version] corpusdirs [corpusdirs ...]
@@ -1265,17 +1065,14 @@ optional arguments:
--version show program's version number and exit
```
-
**Source code**
-* [clean_prestable.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/clean_prestable.py)
-* [corpuspath.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/corpuspath.py)
-* [versioncontrol.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/versioncontrol.py)
-* [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
-
-
-# reparallelize
+- [clean_prestable.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/clean_prestable.py)
+- [corpuspath.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/corpuspath.py)
+- [versioncontrol.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/versioncontrol.py)
+- [util.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/util.py)
+## reparallelize
```
usage: reparallelize [-h] [--version] [--files] [--convert] tmxhtml
@@ -1300,18 +1097,15 @@ optional arguments:
converted files.
```
-
**Source code**
-* [realign.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/realign.py)
-* [corpuspath.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/corpuspath.py)
-* [converter.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/converter.py)
-* [parallelize.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/parallelize.py)
-* [tmx.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/tmx.py)
-
-
-# tmx2html
+- [realign.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/realign.py)
+- [corpuspath.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/corpuspath.py)
+- [converter.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/converter.py)
+- [parallelize.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/parallelize.py)
+- [tmx.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/tmx.py)
+## tmx2html
```
usage: tmx2html [-h] [--version] sources [sources ...]
@@ -1329,13 +1123,11 @@ optional arguments:
--version show program's version number and exit
```
-
**Source code**
-* [tmx.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/tmx.py)
+- [tmx.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/tmx.py)
-# update_metadata
-
+## update_metadata
```
usage: update_metadata [-h] [--version] directories [directories ...]
@@ -1356,15 +1148,12 @@ optional arguments:
--version show program's version number and exit
```
-
**Source code**
-* [update_metadata.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/update_metadata.py)
-* [xslsetter.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/xslsetter.py)
-
-
-# epubchooser
+- [update_metadata.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/update_metadata.py)
+- [xslsetter.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/xslsetter.py)
+## epubchooser
```
usage: epubchooser [-h] [--version] epubfile
@@ -1382,16 +1171,13 @@ optional arguments:
--version show program's version number and exit
```
-
**Source code**
-* [epubchooser.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/epubchooser.py)
-* [epubconverter.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/epubconverter.py)
-* [xslsetter.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/xslsetter.py)
-
-
-# make_training_corpus
+- [epubchooser.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/epubchooser.py)
+- [epubconverter.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/epubconverter.py)
+- [xslsetter.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/xslsetter.py)
+## make_training_corpus
```
Make training corpus from analysed giella xml files. Sentences with words
@@ -1407,13 +1193,6 @@ optional arguments:
--version show program's version number and exit
```
-
**Source code**
-* [trainingcorpusmaker.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/trainingcorpusmaker.py)
-
-
-
-
-
-
+- [trainingcorpusmaker.py](https://github.com/giellalt/CorpusTools/blob/main/corpustools/trainingcorpusmaker.py)
diff --git a/ling/DerivationalTagSystem.md b/ling/DerivationalTagSystem.md
index 7a1057c2..bf2d6077 100644
--- a/ling/DerivationalTagSystem.md
+++ b/ling/DerivationalTagSystem.md
@@ -1,36 +1,25 @@
+# Documenting the derivational tags & system
-Documenting the derivational tags & system
-============
-
-
-# The most common verb nominalisations
-
+## The most common verb nominalisations
+Der/NomAct - Nomen Actionis, Actionnoun, Handlingsnomen
-+Der/NomAg - Nomen Agentis, Agentnoun, Handlernomen
-
++Der/NomAg - Nomen Agentis, Agentnoun, Handlernomen
vs
-
+Actio - this is a verbal tag, ie when the word form is part of the verbal construction.
-
-# Position tags
-
+## Position tags
Tallene er bare navn på plassering av derivasjonene. De beskriver ikke posisjonen men rekkefølgen, i normativ forstand:
-
-* +Der1
-* +Der2
-* +Der3
-* +Der4
-
+- +Der1
+- +Der2
+- +Der3
+- +Der4
Hver enkelt derivasjonstag har en posisjonstag foran seg i LexC-koden:
-
```
LEXICON BÅETEDH
...
@@ -44,23 +33,14 @@ LEXICON BÅETEDH
+Der1+Der/ldahke:%»%^1UMLaldahk LDAHKE ;
+Der1+Der/ldh+N+SgNomCmp:%»%^1UMLeldh R ;
+Der1+Der/ldh+N+Sg+Nom:%»%^1UMLeldh FINAL1 ;
- +Der1+Der/ldh+N+Sg+Nom+Err/Sub:%»%^1UMLeld FINAL1 ;
+ +Der1+Der/ldh+N+Sg+Nom+Err/Sub:%»%^1UMLeld FINAL1 ;
```
-
-LexC-koden tillater en ikkje helt ufri kombinasjon av ulike derivasjoner, som vil gi kombinasjonar som er ugrammatiske. Posisjonsklassifiseringa er eit forsøk på å avgrensa overgenereringa, ved å seia at det er ugrammatisk å bruka posisjon 1-derivasjonar *etter* posisjon 2-derivasjonar.
-
+LexC-koden tillater en ikkje helt ufri kombinasjon av ulike derivasjoner, som vil gi kombinasjonar som er ugrammatiske. Posisjonsklassifiseringa er eit forsøk på å avgrensa overgenereringa, ved å seia at det er ugrammatisk å bruka posisjon 1-derivasjonar _etter_ posisjon 2-derivasjonar.
Oppsummert: definere de grammatiske kombinasjonene for å unngå ugrammatiske kombinasjoner! - Men, hvor produktiv skal et avledningsmønster være, for at denne skal defineres som en grammatisk avlednngskombinasjon?
-
bassat
basádallat
basáduvvat
basádaláduvvat
-
-
-
-
-
-
diff --git a/ling/DocConvertingManipulation.md b/ling/DocConvertingManipulation.md
index b844101c..731d9a76 100644
--- a/ling/DocConvertingManipulation.md
+++ b/ling/DocConvertingManipulation.md
@@ -1,23 +1,9 @@
-Converting .doc files
-=======
-
-# Skip pages
-
-
-
-
-# Skip part of pages
-
-
-
-
-
-
-# Skip lines
-
-
-# Skip words
+# Converting .doc files
+## Skip pages
+## Skip part of pages
+## Skip lines
+## Skip words
diff --git a/ling/HtmlConvertingManipulation.md b/ling/HtmlConvertingManipulation.md
index 6bd18789..c6d8804f 100644
--- a/ling/HtmlConvertingManipulation.md
+++ b/ling/HtmlConvertingManipulation.md
@@ -1,26 +1,20 @@
-Converting html pages
-==========
+# Converting html pages
## Skip pages
-...
-
+...
## Skip part of pages or lines
-
Select contains comma separated xpath path pairs.
A path pair is separated by a semicolon.
-
In this:
-``` ```
-
-
-
+``
Each path should start with .//body \\
Examples of valid pairs (fra og med - til):
+
```
.//body/div[1]/h2[1];.//body/div[3]/div[1]/h3[1]
.//body/div[5];.//body/div[8]/div[3]/h1[1], .//body/div[11]/div[2];.//body/div[11]/div[5]
@@ -28,10 +22,8 @@ Examples of valid pairs (fra og med - til):
```
-
### Comments:
-
```
.//body er xslt-introen som er måten å gje
@@ -44,12 +36,8 @@ vi vil slette området frå første p under første div under body til andre p
.//body/div[1]/p[1];.//body/div[1]/p[2]
```
-
-
-
## Skip words in
-
Change or remove problematic characters from the text.
Specify the elements to match (here all p's within
//body, that do contain text, but do NOT contain em and
@@ -61,7 +49,6 @@ of elements - then only one of them will apply. Also try
to restrict the template to nodes that do not contain
other markup, as such markup otherwise will be removed.
-
```
@@ -90,18 +77,16 @@ other markup, as such markup otherwise will be removed.
```
-
### Skip words in e.g. span
-
```
- <=
+ <=
- <=
+ <=
diff --git a/ling/ImprovementNotes2016.md b/ling/ImprovementNotes2016.md
index 5a84181d..31929a02 100644
--- a/ling/ImprovementNotes2016.md
+++ b/ling/ImprovementNotes2016.md
@@ -1,5 +1,4 @@
-Improvement notes 2016
-========
+# Improvement notes 2016
This is a document without any real content. It is supposed to be filled as we
move forward.
diff --git a/ling/LanguageIndependentTagsInTheGiellaInfra.md b/ling/LanguageIndependentTagsInTheGiellaInfra.md
index 4872b938..117e06d4 100644
--- a/ling/LanguageIndependentTagsInTheGiellaInfra.md
+++ b/ling/LanguageIndependentTagsInTheGiellaInfra.md
@@ -1,145 +1,120 @@
-Language Independent Tags In The Giella Infra
-============
+# Language Independent Tags In The Giella Infra
There are a number of classes of tags where the classes are language
independent, but the actual tags are language specific. Some examples of
such classes of tags are:
-
-* **Error tags**: tags describing parts of the language outside the established norm
-* **Dialect tags**: tags describing variation (in the written language) based on
- dialect
-* **Derivation tags**: tags describing derivational morphology
-
+- **Error tags**: tags describing parts of the language outside the established norm
+- **Dialect tags**: tags describing variation (in the written language) based on
+ dialect
+- **Derivation tags**: tags describing derivational morphology
All such classes of tags are described below. New classes will probably be added
in the future, but we'll try to keep the document updated. See also the
[documentation for each language](/lang/index.html).
-
Each class is recognised by having a **tag prefix**, a short string starting
with "**+**" (for suffix tags; prefix tags for prefixing languages end with
**+** as their last character) and ending with "**/**". Examples of such tag
prefixes are: `+Err/`, `+Dial/` etc.
-
It is assumed — and required — that all tags described here (and all other tags,
for that matter) are declared as multichar symbols in the `root.lexc` file of
each language.
-
-# Error tags
-
+## Error tags
The error tag class is defined as follows:
+- **Tag prefix**: `+Err/`
+- **Definition**: tags describing parts of the language outside the established norm
+- **FST implication**: all strings containing one or more such tags are removed from
+ all normative transducres
-* **Tag prefix**: `+Err/`
-* **Definition**: tags describing parts of the language outside the established norm
-* **FST implication**: all strings containing one or more such tags are removed from
- all normative transducres
-
-
-# Dialect tags
-
+## Dialect tags
The dialect tag class is defined as follows:
-
-* **Tag prefix**: `+Dial/`
-* **Definition**: tags describing (written) variation based on dialect
-* **FST implication**: when the `DIALECTS` variable is set in `configure.ac`, one
- filter for each dialect defined there is built automatically. Each
- filter will remove all strings tagged with a dialect different from
- the one specific to the filter. Untagged strings will be left as is.
- The dialect tags are presently only made use of in Oahpa generators.
-
+- **Tag prefix**: `+Dial/`
+- **Definition**: tags describing (written) variation based on dialect
+- **FST implication**: when the `DIALECTS` variable is set in `configure.ac`, one
+ filter for each dialect defined there is built automatically. Each
+ filter will remove all strings tagged with a dialect different from
+ the one specific to the filter. Untagged strings will be left as is.
+ The dialect tags are presently only made use of in Oahpa generators.
Other notes:
-* The first character after the **/** *must* be one of `+` or `–`,
+
+- The first character after the **/** _must_ be one of `+` or `–`,
denoting either inclusion (the entry/form is valid for the specified dialect)
or exclusion (the entry/form is NOT valid for the specified dialect - but for
all others)
-* The string following **/** and **+/–** *must* be one of the strings
+- The string following **/** and **+/–** _must_ be one of the strings
specified in `configure.ac` for the variable `DIALECTS`.
-
-# Area tags
-
+## Area tags
The area/country tag class is defined as follows:
-
-* **Tag prefix**: `+Area/`
-* **Definition**: tags describing (written) variation based on country or another
- geographical unit, as per the
- [ISO 3166](https://en.wikipedia.org/wiki/ISO_3166) standard.
-* **FST implication**: not yet actively used, but will be used to build proofing
- tools and possibly other normative fst's where strings for other
- areas than the specified one will be removed. This will e.g. make
- the Lule Sámi speller for Sweden a better tool, as all strings
- with Norwegian *æ* (except for in names) will be removed: smaller
- and faster, and with less irrelevant suggestions.
-
+- **Tag prefix**: `+Area/`
+- **Definition**: tags describing (written) variation based on country or another
+ geographical unit, as per the
+ [ISO 3166](https://en.wikipedia.org/wiki/ISO_3166) standard.
+- **FST implication**: not yet actively used, but will be used to build proofing
+ tools and possibly other normative fst's where strings for other
+ areas than the specified one will be removed. This will e.g. make
+ the Lule Sámi speller for Sweden a better tool, as all strings
+ with Norwegian _æ_ (except for in names) will be removed: smaller
+ and faster, and with less irrelevant suggestions.
Other notes:
-* The tag prefix must be followed by an
- [ISO 3166](https://en.wikipedia.org/wiki/ISO_3166) string.
-
-# Semantic tags
+- The tag prefix must be followed by an
+ [ISO 3166](https://en.wikipedia.org/wiki/ISO_3166) string.
+## Semantic tags
The semantic tag class is defined as follows:
-
-* **Tag prefix**: `+Sem/`
-* **Definition**: tags describing semantic properties of the lexeme
-* **FST implication**: all semantic tags are automatically identified, and a couple
- of filters for manipulating them are built and applied, see further
- notes below.
-
+- **Tag prefix**: `+Sem/`
+- **Definition**: tags describing semantic properties of the lexeme
+- **FST implication**: all semantic tags are automatically identified, and a couple
+ of filters for manipulating them are built and applied, see further
+ notes below.
Other notes:
-* **the raw fst:** the semantic tags are moved relative to the POS tag, to
- ensure consistent tag ordering
-* **all fst's except disambiguators and grammar checkers:**
- the semantic tags are removed.
-* **disambiguators and grammar checkers:** the tags are kept (i.e. they are
- untouched)
+- **the raw fst:** the semantic tags are moved relative to the POS tag, to
+ ensure consistent tag ordering
+- **all fst's except disambiguators and grammar checkers:**
+ the semantic tags are removed.
+- **disambiguators and grammar checkers:** the tags are kept (i.e. they are
+ untouched)
-# Derivation tags
-
+## Derivation tags
The derivation tag class is defined as follows:
+- **Tag prefix**: `+Der/`
+- **Definition**: tags describing derivational morphology
+- **FST implication**: there is no language-independent processing of these tags ATM
-* **Tag prefix**: `+Der/`
-* **Definition**: tags describing derivational morphology
-* **FST implication**: there is no language-independent processing of these tags ATM
-
-
-# Originating language tags
-
+## Originating language tags
The originating language tag class is defined as follows:
-
-* **Tag prefix**: `+OLang/`
-* **Definition**: tags describing originating language for loan words in cases where
- such information is required to get proper pronunciation in speech
- synthesis
-* **FST implication**: there is no language-independent processing of these tags ATM,
- and they are removed from all fst's; for North Sámi there is some
- language-specific processing to split the lexical fst into separate
- fst's for each defined `+OLang/` language, after which it is
- possible to apply OLang-specific phonetic rules
-
+- **Tag prefix**: `+OLang/`
+- **Definition**: tags describing originating language for loan words in cases where
+ such information is required to get proper pronunciation in speech
+ synthesis
+- **FST implication**: there is no language-independent processing of these tags ATM,
+ and they are removed from all fst's; for North Sámi there is some
+ language-specific processing to split the lexical fst into separate
+ fst's for each defined `+OLang/` language, after which it is
+ possible to apply OLang-specific phonetic rules
Other notes:
-
So far the only speech synthesis system we have built is for North Sámi. It was
furthermore built without using our text processing technology, and the features
being made possible with these tags (ie pronouncing «u» as /ʉː/ instead
diff --git a/ling/LexCIntro.md b/ling/LexCIntro.md
index cb9418fe..bc290598 100644
--- a/ling/LexCIntro.md
+++ b/ling/LexCIntro.md
@@ -5,7 +5,7 @@
1. LexC-formalismen - del to: fleirteiknssymbolar (Multichar_Symbols)
1. del tre: fortsetjingsleksikon, start og slutt
-# stutt om fst-ar - kva er dei, korleis funkar dei?
+## stutt om fst-ar - kva er dei, korleis funkar dei?
- fst = Finite state transducer
- to nivå: ordform + analyse
@@ -18,7 +18,7 @@ g å e t i e +N +Pl +Gen
g ö ö t i - - - -
```
-# LexC-formalismen - del ein: leksikonstruktur
+## LexC-formalismen - del ein: leksikonstruktur
```
lemma+Tag:stamme fortsetjingsleksikon "infostreng" ;
@@ -26,11 +26,11 @@ g ö ö t i - - - -
Dvs lemma + analyse på venstre side av kolon, (abstrakt) ordform på høgre side.
-# LexC-formalismen - del to: fleirteiknssymbolar (Multichar_Symbols)
+## LexC-formalismen - del to: fleirteiknssymbolar (Multichar_Symbols)
Kva med taggane? Alle taggar må definerast som eit fleirteiknssymbol.
-# del tre: fortsetjingsleksikon, start og slutt
+## del tre: fortsetjingsleksikon, start og slutt
- start: `LEXICON Root` - **MÅ** stå fyrst
- slutt: `#` - alle stiar **MÅ** enda opp ved `#`
diff --git a/ling/LinguisticAnalysis.md b/ling/LinguisticAnalysis.md
index 2981db87..a779edf7 100644
--- a/ling/LinguisticAnalysis.md
+++ b/ling/LinguisticAnalysis.md
@@ -1,18 +1,15 @@
-Linguistic analysis with GiellaLT models
-========================================
+# Linguistic analysis with GiellaLT models
Instead of compiling the grammatical tools yourself (as described elsewhere on these pages), you may also **download ready-compiled analysers for text analysis**. This page explains how. If you have compiled the tools on your machine **already**, we recommend [this page](../tools/docu-sme-manual.md) instead. If not, read on.
+## 1. Download the programs
-# 1. Download the programs
-
-## 1.1. Download the required *support programs*
-
-
-These commands will download the compilers *hfst* and *vislcg3*. They require a unix system. For use on Windows, see below.
+### 1.1. Download the required _support programs_
+These commands will download the compilers _hfst_ and _vislcg3_. They require a unix system. For use on Windows, see below.
**Download on Mac:**
+
```
curl http://apertium.projectjj.com/osx/install-nightly.sh > install-nightly.sh
@@ -21,7 +18,6 @@ chmod a+x install-nightly.sh
sudo ./install-nightly.sh
```
-
**Download on Linux ubuntu:**
```
@@ -38,35 +34,32 @@ curl https://apertium.projectjj.com/rpm/install-nightly.sh |sudo bash
sudo apt-get -f install apertium-all-devel
```
-
-## 1.2. Download the *analyser and disambiguator for your language:*
-
+### 1.2. Download the _analyser and disambiguator for your language:_
You will need both morphology and syntax. We use North Sámi (ISO code: **sme**) as an example:
+**Morphological analyser:**
-**Morphological analyser:**
```
curl https://gtsvn.uit.no/biggies/trunk/bin/sme/tokeniser-disamb-gt-desc.pmhfst > sme.pmhfst
```
+**Syntactic disambiguator:**
-**Syntactic disambiguator:**
```
curl https://gtsvn.uit.no/biggies/trunk/bin/sme/disambiguator.cg3 > sme.cg3
```
**NOTE!** For North Sámi (but not for the other languages) you also should run this command:
+
```
curl https://gtsvn.uit.no/biggies/trunk/bin/sme/semsets.cg3 > semsets.cg3
```
-The file *semset.cg3* should be in the same catalogue as the file *sme.cg3*.
-
+The file _semset.cg3_ should be in the same catalogue as the file _sme.cg3_.
Replace the language code **sme** with the language you want (note! the language code is mentioned **twice** in the commands above, replace both!):
-
- **fao**: Faroese
- **fin**: Finnish
- **smn**: Inari Saami
@@ -77,40 +70,35 @@ Replace the language code **sme** with the language you want (note! the language
- **rus**: Russian (Note! For Russian only morphology is available)
- **sma**: South Saami
-
More languages may be added upon request, from [this list](https://giellalt.github.io/LanguageModels.html). Feel free to contact us if your language is missing.
+## 2. Use the programs
-
-# 2. Use the programs
-
-## 2.1. Automatic grammatical analysis
+### 2.1. Automatic grammatical analysis
**Summary:** When you have downloaded the files (cf. the **Download...** links above), you will be able to run the following command in a terminal window (again with **sme** as an example):
-
```
-cat yourtextfile.txt | hfst-tokenise -cg sme.pmhfst | vislcg3 -g sme.cg3
+cat yourtextfile.txt | hfst-tokenise -cg sme.pmhfst | vislcg3 -g sme.cg3
```
+The textfile is sent through a two-step analysis: First through the morphological analyser `sme.pmhfst`,
+by using the support program `hfst-tokenise`. The flag `-cg` ensures morphological analysis in the required format.
+Thereafter the output is disambiguated with the disambiguator sme.cg3, by using the support program `vislcg3`.
+The flag `-g` identifies the file `sme.cg3` as the grammar file. In order to see more options, you may write
+`hfst-tokenise -h` and `vislcg3 -h`.
-The textfile is sent through a two-step analysis: First through the morphological analyser ``sme.pmhfst``,
-by using the support program ``hfst-tokenise``. The flag ``-cg`` ensures morphological analysis in the required format.
-Thereafter the output is disambiguated with the disambiguator sme.cg3, by using the support program ``vislcg3``.
-The flag ``-g`` identifies the file ``sme.cg3`` as the grammar file. In order to see more options, you may write
-``hfst-tokenise -h`` and ``vislcg3 -h``.
-
-You may also conduct automatic dictionary lookup, see below.
+You may also conduct automatic dictionary lookup, see below.
+## 3. Download other programs
-# 3. Download other programs
+### 3.1. Dictionaries
-## 3.1. Dictionaries
-You may also use the *Neahttadigisánit* dictionaries on the command line. **Warning!!** The program to be downloaded here gives translation equivalent only, not explanations or example sentences. For dictionary lookup the online dictionaries are thus far better, the programs presented here are good for automatic lookup.
+You may also use the _Neahttadigisánit_ dictionaries on the command line. **Warning!!** The program to be downloaded here gives translation equivalent only, not explanations or example sentences. For dictionary lookup the online dictionaries are thus far better, the programs presented here are good for automatic lookup.
-### 3.1.1. Fetching the dictionaries
+#### 3.1.1. Fetching the dictionaries
-The dictionaries are found in the catalogue of **the first language**, the language to translate **from**. Each dictionary has the file name *Lang1Lang2-all.hfst*.
+The dictionaries are found in the catalogue of **the first language**, the language to translate **from**. Each dictionary has the file name _Lang1Lang2-all.hfst_.
Here are two command examples for fetching the dictionaries.
@@ -124,36 +112,36 @@ curl https://gtsvn.uit.no/biggies/trunk/bin/fin/finsme-all.hfst > finsme.hfst
...
```
-For other dictionaries, replace *sme/smenob-all.hfst* above with *smn/smnfin-all.hfst*, *fin/finsmn-all.hfst*, *sma/smanob-all.hfst*, *nob/nobsma-all.hfst*, and correspondingly for *sme/smenob.hfst* etc.
+For other dictionaries, replace _sme/smenob-all.hfst_ above with _smn/smnfin-all.hfst_, _fin/finsmn-all.hfst_, _sma/smanob-all.hfst_, _nob/nobsma-all.hfst_, and correspondingly for _sme/smenob.hfst_ etc.
-### 3.1.2. Using the dictionaries
+#### 3.1.2. Using the dictionaries
The dictionaries may be used in two ways:
-- send a list of baseforms through it: ``cat smn-words.txt | hfst-lookup smnfin-all.hfst``
-- use the dictionary interactively: ``hfst-lookup smnfin-all.hfst``and thereafter write Inari Saami words and press ENTER. Leave the program with ``ctrl C``.
+- send a list of baseforms through it: `cat smn-words.txt | hfst-lookup smnfin-all.hfst`
+- use the dictionary interactively: `hfst-lookup smnfin-all.hfst`and thereafter write Inari Saami words and press ENTER. Leave the program with `ctrl C`.
-## 3.2. Word analysers
+### 3.2. Word analysers
```
curl https://gtsvn.uit.no/biggies/trunk/bin/smn/smn.hfstol > smn.hfstol
```
+
Use the word analysers in two ways:
a, send lists with one word per line through them: `cat wordlist | hfst-lookup smn.hfstol`
b. use the analyser interactively (put it on stand-by) with ` hfst-lookup smn.hfstol` and feed it with one word at a time (press ENTER). Leave the program with `ctrl C`.
+### 3.3. Spellers
-## 3.3. Spellers
-
-**Note** The spellers will need the *hfst-ospell* program (**TODO**: Document how to get hfst-ospell from nightly).
+**Note** The spellers will need the _hfst-ospell_ program (**TODO**: Document how to get hfst-ospell from nightly).
```
curl https://gtsvn.uit.no/biggies/trunk/bin/smn/smn.zhfst > smn.zhfst
```
-Thereafter use them as follows (presuming you have the *hfst-ospell* program:
+Thereafter use them as follows (presuming you have the _hfst-ospell_ program:
```
hfst-ospell -S -n 5 smn.zhfst
@@ -161,11 +149,8 @@ hfst-ospell -S -n 5 smn.zhfst
The flag `-S` means "present a correction suggestion", and the flag `-n 5` specifles the number of suggestions (here: 5).
+## 4. Running the analysers on Windows:
-# 4. Running the analysers on Windows:
All the above works on Linux and Mac. In order to make it work on Windows, do the following:
[Install a Linux shell](https://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/). It is not too complicated, but requires admin rights on your machine. Thereafter, execute the commands for Linux ubuntu above.
-
-
-
diff --git a/ling/Ordbild.md b/ling/Ordbild.md
index 7e41a4f0..196ae558 100644
--- a/ling/Ordbild.md
+++ b/ling/Ordbild.md
@@ -1,49 +1,31 @@
-Ordbild
-=====
+# Ordbild
The goal of this project is to make a corpus-driven valency dictionary,
along the lines of Korp's Ordbild, Gramtrans' DeepDict and Kilgarriff's
Sketch Engine. We use the Korp interface.
It is intended for researchers, students and translators.
+## Documents
+- [Plan for innholdet for samisk Ordbild](ordbild/OversiktOverOrdbild.html)
+## Meeting memos
+- [DeepDict-diskusjon 3.11.2011](/ped/nudoc/meetings/111103.html)
+## Links, (as reference)
-
-
-# Documents
-
-
-* [Plan for innholdet for samisk Ordbild](ordbild/OversiktOverOrdbild.html)
-
-
-# Meeting memos
-
-
-* [DeepDict-diskusjon 3.11.2011](/ped/nudoc/meetings/111103.html)
-
-
-
-
-# Links, (as reference)
-
-
-* [GramTrans DeepDict](http://gramtrans.com/deepdict/), in cooperation with
-[GramTrans](http://gramtrans.com).
-* [Kommentarer (til Tino) til en tidligere versjon av prosjektet](GammalKravspesifikasjon.html)
-* 6 eksempel frå den eksperimentelle Saami DeepDict testversjonen som vi sette opp tidlegare
-** V:
-*** [leat](http://gramtrans.com/deepdict/lookup.php?word=leat&class=V&lang=smi)
-*** [mannat](http://gramtrans.com/deepdict/lookup.php?word=mannat&class=V&lang=smi)
-*** [boahtit](http://gramtrans.com/deepdict/lookup.php?word=boahtit&class=V&lang=smi)
-** N:
-*** [boazu](http://gramtrans.com/deepdict/lookup.php?word=boazu&class=N&lang=smi)
-*** [eallu](http://gramtrans.com/deepdict/lookup.php?word=eallu&class=N&lang=smi)
-** ADJ:
-*** [olu](http://gramtrans.com/deepdict/lookup.php?word=olu&class=ADJ&lang=smi)
-*** [stuoris](http://gramtrans.com/deepdict/lookup.php?word=stuoris&class=ADJ&lang=smi)
-** ADV:
-*** [de](http://gramtrans.com/deepdict/lookup.php?word=de&class=ADV&lang=smi)
-*** [nu](http://gramtrans.com/deepdict/lookup.php?word=nu&class=ADV&lang=smi)
+- [GramTrans DeepDict](http://gramtrans.com/deepdict/), in cooperation with
+ [GramTrans](http://gramtrans.com).
+- [Kommentarer (til Tino) til en tidligere versjon av prosjektet](GammalKravspesifikasjon.html)
+- 6 eksempel frå den eksperimentelle Saami DeepDict testversjonen som vi sette opp tidlegare
+ ** V: \*** [leat](http://gramtrans.com/deepdict/lookup.php?word=leat&class=V&lang=smi)
+ **_ [mannat](http://gramtrans.com/deepdict/lookup.php?word=mannat&class=V&lang=smi)
+ _** [boahtit](http://gramtrans.com/deepdict/lookup.php?word=boahtit&class=V&lang=smi)
+ ** N: \*** [boazu](http://gramtrans.com/deepdict/lookup.php?word=boazu&class=N&lang=smi)
+ **\* [eallu](http://gramtrans.com/deepdict/lookup.php?word=eallu&class=N&lang=smi)
+ ** ADJ:
+ **_ [olu](http://gramtrans.com/deepdict/lookup.php?word=olu&class=ADJ&lang=smi)
+ _** [stuoris](http://gramtrans.com/deepdict/lookup.php?word=stuoris&class=ADJ&lang=smi)
+ ** ADV: \*** [de](http://gramtrans.com/deepdict/lookup.php?word=de&class=ADV&lang=smi)
+ \*\*\* [nu](http://gramtrans.com/deepdict/lookup.php?word=nu&class=ADV&lang=smi)
diff --git a/ling/ParallelCorpusCheckFix.md b/ling/ParallelCorpusCheckFix.md
index 9bd1ae96..8b7d66ba 100644
--- a/ling/ParallelCorpusCheckFix.md
+++ b/ling/ParallelCorpusCheckFix.md
@@ -1,23 +1,19 @@
-Check and fix parallel corpus
-==================
+# Check and fix parallel corpus
Do this if you find files in freecorpus that aren't parallel anyway (or how to
improve freecorpus in sixteen simple steps)
-
To find files with wrong sentence alignment:
-
1. Open a terminal. Run tmx2html.sh. This converts all .tmx files to html files.
-This makes it easier to read the parallelized files.
-
+ This makes it easier to read the parallelized files.
2. Go to:
-$GTFREE/prestable/tmx/nob2sme
-
+ $GTFREE/prestable/tmx/nob2sme
3. If you want to check in which files the word is, grep word in sentence that
-has wrong alignment:
+ has wrong alignment:
+
```
nob2sme $ grep -rl bransjep . | grep -v '.svn' | less
./admin/depts/regjeringen.no/aktuelt.html_id=166.tmx
@@ -26,22 +22,17 @@ nob2sme $ grep -rl bransjep . | grep -v '.svn' | less
./admin/depts/regjeringen.no/pressemeldinger.html_id=184.tmx
```
-
Check all the listed files
-
4. Check if correctly aligned:
-nob2sme $ open ./admin/depts/regjeringen.no/aktuelt.html_id=166.tmx.html
-Then you will see the sentence alignment and can check if it is correctly
-aligned. Use cmd+f bransjep to search for words, etc.
-
-
-
+ nob2sme $ open ./admin/depts/regjeringen.no/aktuelt.html_id=166.tmx.html
+ Then you will see the sentence alignment and can check if it is correctly
+ aligned. Use cmd+f bransjep to search for words, etc.
NOW YOU CAN START TO FIX THE FREECORPUS
-
1. Find all the files with the same id number in orig:
+
```
~ $ cd freecorpus
$ find orig -name '*id=210*' | grep -v ".svn"
@@ -55,31 +46,26 @@ orig/sme/admin/depts/regjeringen.no/bargu-ja-algu.html_id=210
orig/sme/admin/depts/regjeringen.no/bargu-ja-algu.html_id=210.xsl
```
-
2. Check if nob and sme are parallel files:
-$ see orig/nob/admin/depts/regjeringen.no/arbeid_og_velferd.html_id=210
-orig/sme/admin/depts/regjeringen.no/bargu-ja-algu.html_id=210
-
+ $ see orig/nob/admin/depts/regjeringen.no/arbeid_og_velferd.html_id=210
+ orig/sme/admin/depts/regjeringen.no/bargu-ja-algu.html_id=210
In SubEthaEdit press cmd+r to open the files in a webbrowser.
-
3. If the files are not parallel files, change the sme xsl-file and delete
-information about parallel files (line 83-96). Check also line 60-79
-(multilanguages):
-$ see orig/sme/admin/depts/regjeringen.no/bargu-ja-algu.html_id=210.xsl
-
+ information about parallel files (line 83-96). Check also line 60-79
+ (multilanguages):
+ $ see orig/sme/admin/depts/regjeringen.no/bargu-ja-algu.html_id=210.xsl
4. Convert the file xml to check if there are any errors in xsl-file:
-$ convert2xml orig/sme/admin/depts/regjeringen.no/bargu-ja-algu.html_id=210
-
+ $ convert2xml orig/sme/admin/depts/regjeringen.no/bargu-ja-algu.html_id=210
5. If converted succesfully, check in xsl-file:
-$ svn ci -m "This file doesn't have a parallel nob version"
-orig/sme/admin/depts/regjeringen.no/bargu-ja-algu.html_id=210.xsl
-
+ $ svn ci -m "This file doesn't have a parallel nob version"
+ orig/sme/admin/depts/regjeringen.no/bargu-ja-algu.html_id=210.xsl
6. Find the rest of the files with the same id number:
+
```
$ find orig -name '*id=210*' | grep -v ".svn"
orig/eng/admin/depts/regjeringen.no/engelsk-tema.html_id=210
@@ -90,41 +76,33 @@ orig/nob/admin/depts/regjeringen.no/arbeid_og_velferd.html_id=210
orig/nob/admin/depts/regjeringen.no/arbeid_og_velferd.html_id=210.xsl
```
-
7. svn rm the other languages except sme (6 files; 2 eng, 2 nno, 2 nob):
-$ svn rm orig/eng/admin/depts/regjeringen.no/engelsk-tema.html_id=210
-./eng/admin/depts/regjeringen.no/engelsk-tema.html_id=210.xsl
-./nno/admin/depts/regjeringen.no/arbeid-og-velferd-nynorsk.html_id=210
-./nno/admin/depts/regjeringen.no/arbeid-og-velferd-nynorsk.html_id=210.xsl
-./nob/admin/depts/regjeringen.no/arbeid_og_velferd.html_id=210
-./nob/admin/depts/regjeringen.no/arbeid_og_velferd.html_id=210.xsl
-
+ $ svn rm orig/eng/admin/depts/regjeringen.no/engelsk-tema.html_id=210
+ ./eng/admin/depts/regjeringen.no/engelsk-tema.html_id=210.xsl
+ ./nno/admin/depts/regjeringen.no/arbeid-og-velferd-nynorsk.html_id=210
+ ./nno/admin/depts/regjeringen.no/arbeid-og-velferd-nynorsk.html_id=210.xsl
+ ./nob/admin/depts/regjeringen.no/arbeid_og_velferd.html_id=210
+ ./nob/admin/depts/regjeringen.no/arbeid_og_velferd.html_id=210.xsl
8. check in the changes:
-svn ci -m "deleted files with no parallel Saami translation"
-
+ svn ci -m "deleted files with no parallel Saami translation"
+9. Find the rest of the files with same id number in prestable and delete
+ them:
-
-10. Find the rest of the files with same id number in prestable and delete
-them:
```
freecorpus $ find prestable -name '*id=210*' | grep -v '.svn'
```
-
11. Delete the files (5 files: 3 in converted, 1 in tmx and 1 in toktmx):
-freecorpus $ svn rm
-prestable/converted/nob/admin/depts/regjeringen.no/arbeid_og_velferd.html_id=210.xml
-prestable/converted/sme/admin/depts/regjeringen.no/bargu-ja-
-algu.html_id=210.typos
-prestable/converted/sme/admin/depts/regjeringen.no/bargu-ja-
-algu.html_id=210.xml
-prestable/tmx/nob2sme/admin/depts/regjeringen.no/arbeid_og_velferd.html_id=210.tmx
-prestable/toktmx/nob2sme/admin/depts/regjeringen.no/arbeid_og_velferd.html_id=210.toktmx
-
+ freecorpus $ svn rm
+ prestable/converted/nob/admin/depts/regjeringen.no/arbeid_og_velferd.html_id=210.xml
+ prestable/converted/sme/admin/depts/regjeringen.no/bargu-ja-
+ algu.html_id=210.typos
+ prestable/converted/sme/admin/depts/regjeringen.no/bargu-ja-
+ algu.html_id=210.xml
+ prestable/tmx/nob2sme/admin/depts/regjeringen.no/arbeid_og_velferd.html_id=210.tmx
+ prestable/toktmx/nob2sme/admin/depts/regjeringen.no/arbeid_og_velferd.html_id=210.toktmx
12. Check in the changes:
-svn ci -m "deleting useless files, they weren't parallel anyway"
-
-
+ svn ci -m "deleting useless files, they weren't parallel anyway"
diff --git a/ling/ParallelCorpusConversion.md b/ling/ParallelCorpusConversion.md
index 1ea2303d..b994fb1d 100644
--- a/ling/ParallelCorpusConversion.md
+++ b/ling/ParallelCorpusConversion.md
@@ -1,17 +1,15 @@
-Parallel corpus conversion
-=============
+# Parallel corpus conversion
This is an example on how to:
-- fetch parallel documents
-- add metadata that make the parallel documents refer to each other
-- add the parallel documents and their metadata to the corpus
- repository
-- convert them to giellateknos xml format
-- move the converted documents to prestable/converted
+- fetch parallel documents
+- add metadata that make the parallel documents refer to each other
+- add the parallel documents and their metadata to the corpus
+ repository
+- convert them to giellateknos xml format
+- move the converted documents to prestable/converted
-Fetch parallel documents
-========================
+## Fetch parallel documents
1. Open Safari, go to this address:
@@ -43,7 +41,7 @@ Fetch parallel documents
4. Open a new Terminal window and go to:
- freecorpus/orig/sme/admin/sd/other\_files/
+ freecorpus/orig/sme/admin/sd/other_files/
Fetch the saami document with this command: wget (cmd+v, to paste
the link we just copied)
@@ -63,7 +61,7 @@ Fetch parallel documents
7. Go the already opened Terminal. Press cmd+t to open a new tab and go
to:
- freecorpus/orig/nob/admin/sd/other\_files/
+ freecorpus/orig/nob/admin/sd/other_files/
Fetch the norwegian document using this command: wget (press cmd+v,
then paste the link using cmd+c)
@@ -76,8 +74,7 @@ Fetch parallel documents
(Now you have the saami pages in the left tab of Terminal and the
norwegian pages in the right tab)
-Add metadata
-============
+## Add metadata
1. Open the saami xsl file:
@@ -95,28 +92,21 @@ Add metadata
This entry has to be entered in the saami xsl file (don't fill in
"translated from" in the norwegian xsl file):
-
-
-
-->NB!! This is the full link, note that you have to replace some
characters in the link. (Paste the link into a clean SubEthaEdit
document, use the search and replace function and replace & with
(&). Don't include the paranthesis). Copy the link and paste it
into the Terminal. IMPORTANT: replace & with &
-
-
-
- --->NB! Only use translated\_from if it is a translated document!
-
+ --->NB! Only use translated_from if it is a translated document!
@@ -126,12 +116,9 @@ Add metadata
-
-
Save!
-Add the parallel documents and their metadata to the corpus repository
-======================================================================
+## Add the parallel documents and their metadata to the corpus repository
1. Rerun convert2xml in the SME Terminál window (note that you can
press up in the Terminal untill the right command appears.):
@@ -171,8 +158,7 @@ Add the parallel documents and their metadata to the corpus repository
svn ci -m "your svn message"
-Move the converted documents to prestable/converted
-===================================================
+## Move the converted documents to prestable/converted
1. In the SME Terminal, write:
@@ -191,14 +177,14 @@ Move the converted documents to prestable/converted
4. Write svn stat and the result is:
- ? prestable/converted/nob/admin/sd/other\_files/sp2012-2.pdf.xml ?
- prestable/converted/sme/admin/sd/other\_files/dc2012-2.pdf.xml
+ ? prestable/converted/nob/admin/sd/other_files/sp2012-2.pdf.xml ?
+ prestable/converted/sme/admin/sd/other_files/dc2012-2.pdf.xml
5. Write:
svn add (copy and paste both paths, remember to add a space between
- them) prestable/converted/nob/admin/sd/other\_files/sp2012-2.pdf.xml
- prestable/converted/sme/admin/sd/other\_files/dc2012-2.pdf.xml
+ them) prestable/converted/nob/admin/sd/other_files/sp2012-2.pdf.xml
+ prestable/converted/sme/admin/sd/other_files/dc2012-2.pdf.xml
6. Write:
diff --git a/ling/PdfConvertingManipulation.md b/ling/PdfConvertingManipulation.md
index 40847848..c2fb8d4f 100644
--- a/ling/PdfConvertingManipulation.md
+++ b/ling/PdfConvertingManipulation.md
@@ -1,93 +1,67 @@
-Converting PDF files
-=========
-# PDF
+# Converting PDF files
+## PDF
This is by far the most problematic format to convert to xml, often needing extensive manipulation of the variables in the metadata documents to get the wanted output in the converted documents.
-
Portable Document Format (PDF) is a digital document format developed by Adobe Systems and was introduced in in 1993. Each PDF file encapsulates a complete description of a fixed-layout flat document, including the text, fonts, graphics, and other information needed to display it.
-
A loose definition of the format could be "digital paper".
-
Extracting text from a pdf document can be approximated to that of extracting text using OCR: to retain the "story" of the document, we often need to skip pages, headers, footers, page numbers, foot notes, etc.
-
-# Converted document contains less (or no) text compared to the original document
-
+## Converted document contains less (or no) text compared to the original document
Decrease margins to 0, then compare document to the converted output.
-
Then adjust variables to taste.
-
-# Extracting individual articles from a document
-
+## Extracting individual articles from a document
Some documents contain many articles written by different authors. To correctly attribute the authors their text, we need to extract their article from the document.
-
-First download the document into a corpus, preferrably using __add_files_to_corpus__. Remove the metadata document of the downloaded document, we will not need it.
-
+First download the document into a corpus, preferrably using **add_files_to_corpus**. Remove the metadata document of the downloaded document, we will not need it.
Make a soft link to the document, e.g.
-
```
ln -s original.pdf original-author1-author2.pdf
ln -s original.pdf original-author3-author4.pdf
```
-
Run convert2xml on both the soft-linked documents to make basic metadata files belonging to these soft linked files.
-
```
convert2xml original-author1-author2.pdf
convert2xml original-author3-author4.pdf
```
+Then use **skip_pages** in the files `original-author1-author2.pdf.xsl` and `original-author3-author4.pdf.xsl` so that only the wanted pages are left in the converted documents.
-Then use __skip_pages__ in the files `original-author1-author2.pdf.xsl` and `original-author3-author4.pdf.xsl` so that only the wanted pages are left in the converted documents.
-
-
-# Order in the converted document is not retained
-
+## Order in the converted document is not retained
Run the command:
+
```
pdftohtml -hidden -enc UTF-8 -stdout -nodrm -i -xml documentname.pdf | less
```
-
to see if order of the text is contained. This is the command that is used by the pdf converter to do the first conversion from pdf to xml. It produces a xml format specific to the [poppler](https://poppler.freedesktop.org/) tools, which pdftohtml is a part of.
-
If the order of the text from the above content is different from the content of the converted document, then there is a bug in the pdf converter. File a bug on bugzilla. Use the **product**
"Corpus", **component**
"xml conversion".
+## Most of the text lines in the pdf documents are interpreted as paragraphs
-# Most of the text lines in the pdf documents are interpreted as paragraphs
-
-
-Have a look at the documentation on linespacing below.
-
-
+Have a look at the documentation on linespacing below.
+## Variables specific to pdf documents
-# Variables specific to pdf documents
-
-
-# Skipping pages
-
+## Skipping pages
Typical uses are to skip front page, pages containing tables of content, indexes, etc. In short, removing pages not relevant for the "story" of the document.
-
```
@@ -100,15 +74,10 @@ Examples:
1, 6-10, 15, 20, 25-30
```
-
-
-
-# Margins
-
+## Margins
This option is used to remove text outside a given rectangle. Typical uses are to remove page numbers, page headers, page footers, foot notes at the bottom of the page, info boxes on the left or right of the "real" document.
-
```
@@ -153,13 +122,10 @@ all=9, 8=12
1;3;8=20, 4;5;7=10
```
-
-# Removing content from a page
-
+## Removing content from a page
Typical uses of this is to remove info boxes inside the margins of a page.
-
```
@@ -172,16 +138,12 @@ as *_margin above. For a given page, all four margins
must be defined.
```
-
-# Line spacing
-
+## Line spacing
The pdf converter in CorpusTools uses guesswork to glue text lines into paragraphs. Usually documents have a line spacing of 1.5 and less. This means that from the bottom of a line to the bottom of the next line there is maximally 1.5 times larger than the font size.
-
Some documents, typically student texts, have a larger linespacing. When using the default linespacing, lines in the documents are interpreted as paragraphs leading to output like this:
-
```
This sentence
is divided
@@ -192,10 +154,8 @@ Some documents, typically student texts, have a larger linespacing. When using t
paragraph.
```
-
Increasing the value for this variable improves this situation.
-
```
@@ -227,7 +187,3 @@ odd=5, even=8, 8=15, 11=3
all=9, 8=12
1;3;8=20, 4;5;7=10
```
-
-
-
-
diff --git a/ling/RtfConvertingManipulation.md b/ling/RtfConvertingManipulation.md
index adeace23..2b6c83a5 100644
--- a/ling/RtfConvertingManipulation.md
+++ b/ling/RtfConvertingManipulation.md
@@ -1,28 +1,7 @@
-
-
-## Skip pages
-
-
-
-
-
-
-
+# Skip pages
## Skip part of pages
-
-
-
-
-
-
-
## Skip lines
-
-
-
## Skip words
-
-
diff --git a/ling/SaamiTextOnline.md b/ling/SaamiTextOnline.md
index 9e692b82..16922be1 100644
--- a/ling/SaamiTextOnline.md
+++ b/ling/SaamiTextOnline.md
@@ -1,68 +1,58 @@
-Online-teavsttat sámegillii
-=========
-
+# Online-teavsttat sámegillii
Mii háliidit nu olu teakstamateriála go vejolaš.
Prinsihppa lea, ahte mii čohkket html-siidduid
automahtalaččat, muhto pdf-siiddut manuálalaččat.
-
Bija ođđa siidduid freecorpus/urls.yaml:i.
-
-# Automáhtalaš viežžan
-* samediggi.fi
-* samediggi.no
-* yle.fi/uutiset/osasto/sapmi/
-* nrk.no/sapmi
-* avvir.no
-* samas.no
-* beaivvas.no
-* giella.org
-* paliskunnat.fi
-* regjeringen.no
-* arkivverket.no/om-oss/sami-arkiiva
-* samisklegeforening.no/
-* finnmarksarkivene.no
-
-
-
-
-# Neahttásiiddut main leat pdf
-
-
-* [Dutkanráđđi](https://www.forskningsradet.no/prognett-samisk/Sentrale_dokumenter/1229378700479)
-* [FeFo](http://www.fefo.no/sa/sider/start.aspx)
-* [Gielddat](https://nn.wikipedia.org/wiki/Samiske_kommunar) (fuom! oassi dain mis lea juo)
-* Munin uit.no
-** [PhD|https://munin.uit.no/handle/10037/281/browse?type=title], [master](https://munin.uit.no/handle/10037/159/browse?type=title) oza "ahte" site:munin.uit.no/handle
-* NDLA (Nasjonal digital læringsarena
-** [NDLA lullisámegiella vuosttaš giellan](https://ndla.no/nn/node/162357?fag=126960)
-
-
-## Eará gáldut
-
-
-* Leanat
-* Stáhtahálddahus Norggas, Ruoŧas, Suomas
-** Direktoráhtat
-** Suopmi: Metsähallitus
-* Sámedikkit
-** Preassadieđahusat
-* Sámi politihkalaš bellodagat
-* Bloggat
-* Girkkut
-
-
-## Reetta epoasttas
-
+## Automáhtalaš viežžan
+
+- samediggi.fi
+- samediggi.no
+- yle.fi/uutiset/osasto/sapmi/
+- nrk.no/sapmi
+- avvir.no
+- samas.no
+- beaivvas.no
+- giella.org
+- paliskunnat.fi
+- regjeringen.no
+- arkivverket.no/om-oss/sami-arkiiva
+- samisklegeforening.no/
+- finnmarksarkivene.no
+
+## Neahttásiiddut main leat pdf
+
+- [Dutkanráđđi](https://www.forskningsradet.no/prognett-samisk/Sentrale_dokumenter/1229378700479)
+- [FeFo](http://www.fefo.no/sa/sider/start.aspx)
+- [Gielddat](https://nn.wikipedia.org/wiki/Samiske_kommunar) (fuom! oassi dain mis lea juo)
+- Munin uit.no
+ \*\* [PhD|https://munin.uit.no/handle/10037/281/browse?type=title], [master](https://munin.uit.no/handle/10037/159/browse?type=title) oza "ahte" site:munin.uit.no/handle
+- NDLA (Nasjonal digital læringsarena
+ \*\* [NDLA lullisámegiella vuosttaš giellan](https://ndla.no/nn/node/162357?fag=126960)
+
+### Eará gáldut
+
+- Leanat
+- Stáhtahálddahus Norggas, Ruoŧas, Suomas
+ ** Direktoráhtat
+ ** Suopmi: Metsähallitus
+- Sámedikkit
+ \*\* Preassadieđahusat
+- Sámi politihkalaš bellodagat
+- Bloggat
+- Girkkut
+
+### Reetta epoasttas
Her er ei liste av nettesider som jeg fant, men ikke rukket å hente nye filer fra. Risten kan kanskje gjøre det når hun begynner på jobb igjen?
-* giella.org: lulesamisk og sørsamisk
-* paliskunnat.fi
-* saminuorra.org
-* saamivillage.fi
-* rdm.no
-* kuati.fi?
-* dutkansearvi.fi?
-* Báikkálaš sámi searvvit?
+
+- giella.org: lulesamisk og sørsamisk
+- paliskunnat.fi
+- saminuorra.org
+- saamivillage.fi
+- rdm.no
+- kuati.fi?
+- dutkansearvi.fi?
+- Báikkálaš sámi searvvit?
diff --git a/ling/TxtConvertingManipulation.md b/ling/TxtConvertingManipulation.md
index ed8376bb..f2eb4bd0 100644
--- a/ling/TxtConvertingManipulation.md
+++ b/ling/TxtConvertingManipulation.md
@@ -1,15 +1,9 @@
-Converting .txt files
-=============
+# Converting .txt files
-# txt
+## txt
+### Skip lines
-## Skip lines
-
-
-
-## Skip words
-
-
+### Skip words
diff --git a/ling/UnicodeNormalisation.md b/ling/UnicodeNormalisation.md
index 2a235ab8..574bac01 100644
--- a/ling/UnicodeNormalisation.md
+++ b/ling/UnicodeNormalisation.md
@@ -1,44 +1,29 @@
-
-
-(*or: how to fix decomposed Sami letters*)
-
+(_or: how to fix decomposed Sami letters_)
In Unicode, many glyphs (letter symbols) may either be represented
by one character, or by a sequence of many. The letter á may thus be
either one character á or two characters a and combining ´ . Normalisation
forms are used to standardise the representation.
-
1. NFKD = Normalization Form Compatibility Decomposition
1. NFKC = Normalization Form Compatibility Composition
-
-The first, NFKD, **decomposes** the characters (á as two characters),
+The first, NFKD, **decomposes** the characters (á as two characters),
whereas the second, NFKC, **composes it** (á as one character).
-
Our North Sami analysers use the **composed** representation.
-
If you get text with decomposed letters (**UnicodeChecker** will tell you that č is two characters), you must **compose** them with the following command
-
```
cat infile.txt \
| uconv -f utf8 -t utf8 -x Any-NFKC > outfile.txt
```
-
See also `man uconv`
-
The uconv program should be installed on your machine as part of
the ICU installation.
-
-
-
-* [Unicode on normalization](http://unicode.org/reports/tr15/)
-* [Exmple script where the command is used](https://github.com/redpony/cdec/blob/master/corpus/utf8-normalize.sh)
-
-
+- [Unicode on normalization](http://unicode.org/reports/tr15/)
+- [Exmple script where the command is used](https://github.com/redpony/cdec/blob/master/corpus/utf8-normalize.sh)
diff --git a/ling/WikipediaAsCorpus.md b/ling/WikipediaAsCorpus.md
index 508bcb95..9e727f8e 100644
--- a/ling/WikipediaAsCorpus.md
+++ b/ling/WikipediaAsCorpus.md
@@ -1,38 +1,33 @@
-
# Wikipedia as a Corpus
-
This page explains how to fetch whole Wikipedia editions as raw text
-
-# Do the following:
-
-
-
-
-1. Find the language code for the language you want: It is the two-letter ISO code (**se**, etc.). If the language has no two-letter code, use the 3-letter code.
-1. Go to the download page. The URL is [http://dumps.wikimedia.org/sewiki/](http://dumps.wikimedia.org/sewiki/) will give you North Sámi, exchange the **se** in *sewiki* with the language code you want.
-1. In the list that follows, choose the last one **before** *latest/*. The
- latest one is the same as the one with the last dates (it is just a stable url), the download headers are more nicely formatted in the last dated link.
-1. Download the .bz2 file found under the header
- **Articles, templates, image descriptions, and primary meta-pages.**
- This will give you the articles. \\
- If you want revision history (e.g. for spellchecker testing), you need
- *All pages with complete edit history* (this use is not documented).
+## Do the following
+
+1. Find the language code for the language you want: It is the two-letter ISO code (**se**, etc.). If the language has no two-letter code, use the 3-letter code.
+1. Go to the download page. The URL is [http://dumps.wikimedia.org/sewiki/](http://dumps.wikimedia.org/sewiki/) will give you North Sámi, exchange the **se** in _sewiki_ with the language code you want.
+1. In the list that follows, choose the last one **before** _latest/_. The
+ latest one is the same as the one with the last dates (it is just a stable url), the download headers are more nicely formatted in the last dated link.
+1. Download the .bz2 file found under the header
+ **Articles, templates, image descriptions, and primary meta-pages.**
+ This will give you the articles. \\
+ If you want revision history (e.g. for spellchecker testing), you need
+ _All pages with complete edit history_ (this use is not documented).
1. When downloaded, open the .bz2 file. (On Mac and Linux, just doubleclick on the file.)
-You now want to convert the xml files to text. Use e.g. the script [https://pypi.org/project/wikiextractor/](WikiExtractor.py). If you have downloaded the svn giellalt file tree from Tromsø, you already have this script, in ``$GTHOME/gt/script/corpus/``. If not, look at the documentation on the script's homepage. The script has a --help option explaining
- usage. Let us say you call the folder for output `outf`.
+You now want to convert the xml files to text. Use e.g. the script [https://pypi.org/project/wikiextractor/](WikiExtractor.py). If you have downloaded the svn giellalt file tree from Tromsø, you already have this script, in `$GTHOME/gt/script/corpus/`. If not, look at the documentation on the script's homepage. The script has a --help option explaining
+usage. Let us say you call the folder for output `outf`.
+
1. The output is xml. If you want clean text, you may strip the tags.
Here are two ways of stripping xml tags. First, just with sed:
- ```
- cat outf/* | sed 's/<[^>]*>//g;' | ...
- ```
+```sh
+ cat outf/* | sed 's/<[^>]*>//g;' | ...
+```
For Tromsø users we have made a script to somewhat refine this command, also that in $GTHOME/gt/script/corpus/. It is called `rydd_i_wikipedia.sh`
- ```
- cat outf/* | sh $GTHOME/gt/script/corpus/rydd_i_wikipedia.sh | ...
- ```
+```sh
+ cat outf/* | sh $GTHOME/gt/script/corpus/rydd_i_wikipedia.sh | ...
+```
diff --git a/ling/bokhylla/BrukJupyter.md b/ling/bokhylla/BrukJupyter.md
index 280d6906..766bb94a 100644
--- a/ling/bokhylla/BrukJupyter.md
+++ b/ling/bokhylla/BrukJupyter.md
@@ -1,25 +1,24 @@
-
Dokumentasjon for bruk av Bokhylla som korpus, med hjelp av Jupyter.
-* [Nasjonalbiblioteket sin dokumentasjon](https://nbviewer.jupyter.org/github/DH-LAB-NB/DHLAB/blob/master/DHLAB_ved_Nasjonalbiblioteket.ipynb)
-* [Installering av Jupyter](https://realpython.com/jupyter-notebook-introduction/)
+- [Nasjonalbiblioteket sin dokumentasjon](https://nbviewer.jupyter.org/github/DH-LAB-NB/DHLAB/blob/master/DHLAB_ved_Nasjonalbiblioteket.ipynb)
+- [Installering av Jupyter](https://realpython.com/jupyter-notebook-introduction/)
# Installering
Dette er våre notatar, sjå også Jupyter sine eigne notatar (over).
-* Det du trenger for å kjøre jupyter:
-**python3, som du kanskje har frå før (viss ikkje installer den, t.d. frå Anaconda:
-** [Anaconda](https://www.datacamp.com/community/tutorials/installing-anaconda-mac-os-x) (en pythondistribusjon)
-** Deretter jupyter, installert med desse to kommandoane: \\
-python3 -m pip setuptools \\
-python3 -m pip install jupyter
-* Start deretter jupyter, i terminalen: \\
-jupyter notebook
+- Det du trenger for å kjøre jupyter:
+ **python3, som du kanskje har frå før (viss ikkje installer den, t.d. frå Anaconda:
+ ** [Anaconda](https://www.datacamp.com/community/tutorials/installing-anaconda-mac-os-x) (en pythondistribusjon)
+ \*\* Deretter jupyter, installert med desse to kommandoane: \\
+ python3 -m pip setuptools \\
+ python3 -m pip install jupyter
+- Start deretter jupyter, i terminalen: \\
+ jupyter notebook
Merk at det øverst til høgre i sida du får i nettlesaren skal stå **Python3**.
-# Lars Johnsen sitt seminar
+## Lars Johnsen sitt seminar
[https://github.com/Yoonsen/Seminar_NB_september_2019]
@@ -53,7 +52,7 @@ Rediger denne fila (t.d. ved å fjerne irrelevante bøker), eller berre spar hen
Viss neste sesjon startar på nytt, må vi importere bokhylla på nytt (sjå ovafor).
-Deretter last korus inn att frå bokhylla med hjelp av (den redigerte) xls-fila:
+Deretter last korus inn att frå bokhylla med hjelp av (den redigerte) xls-fila:
```python
nb.restore_metadata_from_excel("korpusFriis.xls")
@@ -85,11 +84,11 @@ Translation
Reference
Topic(s)
-----
+---
Og i eksemplet med sami_count (telling for hele korpuset) må den gjøres om til dataramme først, i og med det er et Counter-object (en dict med ekstra funksjonalitet - fin å telle tekst med generelt):
-sami_count = nb.frame(sami_count) # nb.frame() er en wrapper for pandas-funksjonalitet med litt ekstra...
+sami_count = nb.frame(sami_count) ## nb.frame() er en wrapper for pandas-funksjonalitet med litt ekstra...
så kan det summeres:
@@ -163,9 +162,9 @@ Nye kommandoer og hjelpekommandoer kan du lage som du vil egentlig. Mesteparten
```python
import dhlab.module_update as mu
-mu.update('nbtext') # laster ned nbtext.py
-mu.update('nbtokenizer') # tokenisator for norsk
-mu.update('token_map') # for å gjøre navnebehandling
+mu.update('nbtext') ## laster ned nbtext.py
+mu.update('nbtokenizer') ## tokenisator for norsk
+mu.update('token_map') ## for å gjøre navnebehandling
```
Håper det her hjelper.
diff --git a/ling/cgii-writing.md b/ling/cgii-writing.md
index d7cf5299..3976bab3 100644
--- a/ling/cgii-writing.md
+++ b/ling/cgii-writing.md
@@ -1,8 +1,6 @@
-How to write disambiguation files
-=================================
+# How to write disambiguation files
-Constraint grammar
-------------------
+## Constraint grammar
The main introduction to CG-2 is Tapanainen 1996. Karlsson & al 1992
gives a good introduction to CG-1, and also the most thorough
@@ -11,18 +9,17 @@ presentation of the philosophy behind the constraint grammar framework.
The projects uses the CG-2 formalism, and this formalism is presentation
below. The concrete implementation is vislcg.
-The structure of the disambiguation file
-----------------------------------------
+## The structure of the disambiguation file
The disambiguation file has the suffix .rle, in our case it is called
sme-dis.rle, smj-dis.rle, etc. The file consists of the following
sections (an additional section CORRECTIONS may also be used, it then
follows the CONSTRAINTS sections):
-- DELIMITERS
-- SETS
-- MAPPINGS
-- CONSTRAINTS (there are several CONSTRAINT sections
+- DELIMITERS
+- SETS
+- MAPPINGS
+- CONSTRAINTS (there are several CONSTRAINT sections
### The delimiters
@@ -114,7 +111,6 @@ the rule to work since Acc is not a member of the set PRE-NP-HEAD.
The constraints of the North Saami file are documented
[here](/lang/sme/docu-sme-dis.html).
-The format of the constraint rules
-----------------------------------
+## The format of the constraint rules
tbw.
diff --git a/ling/common-regex.md b/ling/common-regex.md
index 7488bef2..25beabd3 100644
--- a/ling/common-regex.md
+++ b/ling/common-regex.md
@@ -1,47 +1,34 @@
# Utility regexes found in gt/common/src
-
The following is the beginning of documentation of the different utility regexes found in `$GTHOME/gt/common/src/`. To be extended as more is found.
-
-# tag-pos.fst
-
+## tag-pos.fst
In order to make pos.fst we need a binary tag-pos.fst
This goal depends on tag-pos.regex. The way it
is done is that all tags except the POS one are deleted.
+## tag-not-save.fst
-# tag-not-save.fst
We want to delete the +TV +IV tags for the generator (and other
tags later on. For that we need our tag-deleter.
-
-# hyphen-convert.fst
-
+## hyphen-convert.fst
No documentation yet
-
-# hyphen-remove.fst
-
+## hyphen-remove.fst
No documentation yet
-
-# tag-inclusion-filter.fst
-
+## tag-inclusion-filter.fst
No documentation yet
-
-# tag-no.fst
-
+## tag-no.fst
No documentation yet
-
-# webadr.fst
-
+## webadr.fst
This goal is to make a regex for filenames, urls and mail addresses
diff --git a/ling/common.md b/ling/common.md
index 376c56dc..d72658f9 100644
--- a/ling/common.md
+++ b/ling/common.md
@@ -1,43 +1,39 @@
-Tutorials
-=========
+# Tutorials
-- [Tutorials for working with Lexc/Twolc, CG, and UNIX
- commands](../lang/common/Tutorials.html)
-- [How to use the morphological parsers](/tools/docu-sme-manual.html)
-- [A flowchart of the analysis pipeline](global-flowchart.html)
+- [Tutorials for working with Lexc/Twolc, CG, and UNIX
+ commands](../lang/common/Tutorials.html)
+- [How to use the morphological parsers](/tools/docu-sme-manual.html)
+- [A flowchart of the analysis pipeline](global-flowchart.html)
-Linguistic issues
-=================
+## Linguistic issues
-- Preprocessing of text
- - [How to split text into tokens](preprocessor-usage.html)
- - [Documentation of the preprocessor files](preprocessor.html)
-- Morphological tagging of text, with LEXC and TWOLC
- - [Principles for common (language-independent) lexicon
- entries](../lang/common/PrinciplesForCommonTagsAndLexiconEntries.html)
- - [Handling variation in
- LEXC](../lang/common/Variation_in_lexc.html)
-- Documentation of tags
- - [Compoundtags](../lang/common/CompoundTags.html)
- - [Morphological tags](../lang/common/MorphologicalTags.html)
- - [Derivational tags in Sámi](../lang/common/DerivationOverview.html)
- - [How the different tags are interacting with the
- FSTs](../lang/common/DifferentFSTs.html)
- - [Syntax tags](../lang/common/docu-sme-syntaxtags.html)
- - [Dependency tags](../lang/common/docu-deptags.html)
- - [Semantic tags](../lang/common/SemanticTags.html)
-- Disambiguation of morphological analysis
- - [Morphological disambiguation](docu-disambiguation.html)
+- Preprocessing of text
+ - [How to split text into tokens](preprocessor-usage.html)
+ - [Documentation of the preprocessor files](preprocessor.html)
+- Morphological tagging of text, with LEXC and TWOLC
+ - [Principles for common (language-independent) lexicon
+ entries](../lang/common/PrinciplesForCommonTagsAndLexiconEntries.html)
+ - [Handling variation in
+ LEXC](../lang/common/Variation_in_lexc.html)
+- Documentation of tags
+ - [Compoundtags](../lang/common/CompoundTags.html)
+ - [Morphological tags](../lang/common/MorphologicalTags.html)
+ - [Derivational tags in Sámi](../lang/common/DerivationOverview.html)
+ - [How the different tags are interacting with the
+ FSTs](../lang/common/DifferentFSTs.html)
+ - [Syntax tags](../lang/common/docu-sme-syntaxtags.html)
+ - [Dependency tags](../lang/common/docu-deptags.html)
+ - [Semantic tags](../lang/common/SemanticTags.html)
+- Disambiguation of morphological analysis
+ - [Morphological disambiguation](docu-disambiguation.html)
-Testing
-=======
+## Testing
-- [LEXC/TWOLC work – Testscripts](../lang/common/developingwork.html)
-- Check analysis regressions, TO BE WRITTEN
-- [Testing the disambiguation](docu-distesting.html)
+- [LEXC/TWOLC work – Testscripts](../lang/common/developingwork.html)
+- Check analysis regressions, TO BE WRITTEN
+- [Testing the disambiguation](docu-distesting.html)
-Outdated documentation
-======================
+## Outdated documentation
[Analyzed corpus,](corpus_analyze.html) [Correct
corpus,](correct-dir.html) [Corpus plan,](corpus_plan.html) [The tools
diff --git a/ling/corpus_analyze.md b/ling/corpus_analyze.md
index 398d4586..2ad13684 100644
--- a/ling/corpus_analyze.md
+++ b/ling/corpus_analyze.md
@@ -1,8 +1,6 @@
-Corpus analysis
-==============
-
-# Overview and introduction
+# Corpus analysis
+## Overview and introduction
One of the goals of the giellatekno-project is to provide easy access to
the text materials for non-commercial purposes such as research. The
@@ -11,15 +9,14 @@ with which a user can fetch different types of data from the Sámi
corpora. The raw corpus material is collected in co-operation with the
owners of the documents. The documents are preprocessed so that the
texts can be used in research. The process of text collection is
-described in documents [corpus\_conversion.html](corpus_conversion.html)
-and [corpus\_conversion\_tech.html.](corpus_conversion_tech.html) This
+described in documents [corpus_conversion.html](corpus_conversion.html)
+and [corpus_conversion_tech.html.](corpus_conversion_tech.html) This
documents describes the process where the document is transferred to the
graphical corpus interface. The graphical corpus interface is developed
and maintaned by[Textlaboratoriet](http://www.hf.uio.no/tekstlab/)in the
university of Oslo.
-# How to parallelize documents
-
+## How to parallelize documents
Files that are ready to be parellellised exist in
`$GTFREE/prestable/converted`. The steps to parallelize between sme and
@@ -33,40 +30,38 @@ nob are:
5. `make`
2. Make an anchor file using the command (note that different text
domains may have different additional anchor files):
- - `generate-anchor-list.pl --lang1=sme --lang2=nob --outdir=$GTFREE $GTHOME/gt/common/src/anchor.txt $GTHOME/gt/common/src/anchor_admin.txt `
+ - `generate-anchor-list.pl --lang1=sme --lang2=nob --outdir=$GTFREE $GTHOME/gt/common/src/anchor.txt $GTHOME/gt/common/src/anchor_admin.txt `
3. Compile TCA2, the sentence aligner, using these commands:
- - `cd $GTHOME/tools/alignment-tools/tca2`
- - `ant`
+ - `cd $GTHOME/tools/alignment-tools/tca2`
+ - `ant`
The files may be parallellised in commandline mode.
1. Parallelize the files in `$GTFREE/prestable/converted/sme` and
`$GTFREE/prestable/converted/nob` using this command:
- - `` for file in `find $GTFREE/prestable/converted/sme -name \*.xml | grep -v .svn`; do corpus-parallel.pl --lang1=sme --lang2=nob $file ; done ``
+ - `` for file in `find $GTFREE/prestable/converted/sme -name \*.xml | grep -v .svn`; do corpus-parallel.pl --lang1=sme --lang2=nob $file ; done ``
2. The output is found in
`$GTFREE/prestable/tmx/smenob/.tmx`
The files may also be parallellised in graphical mode.
1. Issue the command
- - ` java -jar $GTHOME/tools/alignment-tools/tca2/dist/lib/alignment.jar `
+ - ` java -jar $GTHOME/tools/alignment-tools/tca2/dist/lib/alignment.jar `
2. To load files when starting tca2 in gui mode, issue this command:
- - ` java -jar $GTHOME/tools/alignment-tools/tca2/dist/lib/alignment.jar -anchor= -in1= -in2= `
+ - ` java -jar $GTHOME/tools/alignment-tools/tca2/dist/lib/alignment.jar -anchor= -in1= -in2= `
To parallelize the other way, exchange the values for lang1 and lang2 in
step 2 and 4, and change the find command in step 4 to
`find $GTFREE/prestable/converted/nob`
-# Analyzing the corpus text.
-
-
-## The files and formats
+## Analyzing the corpus text.
+### The files and formats
The project-internal corpus format contains the basic elements, such as
paragraphs, lists and tables that can be extracted from the original
document format. The xml-format of the Saami corpus resources is
-documented in [corpus\_conversion.html](corpus_conversion.html)
+documented in [corpus_conversion.html](corpus_conversion.html)
The original name of the document is preserved in the process with the
suffix indicating the document type, e.g. `file.doc.` When the text is
@@ -81,8 +76,7 @@ those files are indicated with suffix `.sent.xml`, e.g.
The xml-files reside in either `$GTFREE/converted` or
`$GTBOUND/converted`
-## XML-format of the analyzed corpus.
-
+### XML-format of the analyzed corpus.
The XML format of the analyzed text is basically the following:
@@ -91,7 +85,7 @@ The XML format of the analyzed text is basically the following:
-
+
@@ -99,8 +93,7 @@ The XML format of the analyzed text is basically the following:
See the description of the dtd.
-## The tool to analyze corpus files: `corpus-analyze.pl`
-
+### The tool to analyze corpus files: `corpus-analyze.pl`
Usage:
@@ -116,7 +109,7 @@ exclulded as well.
The following options regulate the exclusion and inclusion of elements:
--tables Take also -elements, which are excluded by default.
- -T
+ -T
--lists Take also -elements, which are excluded by default.
-L
--all Take all elements.
@@ -148,7 +141,7 @@ use the following command:
The files in the corpus hierarchy do not contain sentence elements
(<s>). Sentence elements are the basic units of analysis and have
-to be added with --add\_sentences or -s. If this option is not given,
+to be added with --add_sentences or -s. If this option is not given,
the <s> tags are assumed to be already added. The
<s>-elements may be added without analysis, with command:
@@ -157,8 +150,7 @@ the <s> tags are assumed to be already added. The
At the same time when <s>-tags are added, the sentences are
numbered and given id's. These id's are used in alignment process.
-# Parallel files
-
+## Parallel files
Each xml-document in the corpus database contains field for parallel
documents, which exist in the corpus hierarchy. For example, the header
@@ -174,12 +166,11 @@ parallel file, which is assumed to be found in the same subdirectory as
the file `file.doc.xml`. The information of the parallel files can be
updated to the xml-document by editing the file-specific xsl-file, see
the document
-[corpus\_conversion.html](corpus_conversion.html#Parallel+texts) for
+[corpus_conversion.html](corpus_conversion.html#Parallel+texts) for
instructions. These fields conduct the search and processing of the
parallel texts.
-## Preprocessing for alignment
-
+### Preprocessing for alignment
Parallel files are processed by the script `corpus-parallel.pl`. Usage:
@@ -215,8 +206,7 @@ detection have to be changed in that script. The resulting files are
stored in `$GTFREE/tmp`, the resulting file names are:
`file.doc.sent.xml, file_n.doc.sent.xml.`
-## Aligning documents
-
+### Aligning documents
The documents are aligned using [TRANSLATION CORPUS ALIGNER (TCA)
2](http://gandalf.aksis.uib.no/tca2/)by Knut Hofland and Øystein Reigem,
@@ -227,42 +217,40 @@ s-elements:`file.doc.sent.xml` and `file_n.doc.sent.xml`.
The program outputs three files:
-- one file that contains the pairings of the sentence-id's. The file
- name is generally `file.doc.sent.xml_file_n.doc.sent.xml.xml` which
- indicates the order of the paired id lists.
-- a file pair where each line corresponds to the parellellised content
- in the other file. These files are named `file.doc.sent_new.txt` and
- `file_n.doc.sent_new.txt`
-
-# Morphological information
+- one file that contains the pairings of the sentence-id's. The file
+ name is generally `file.doc.sent.xml_file_n.doc.sent.xml.xml` which
+ indicates the order of the paired id lists.
+- a file pair where each line corresponds to the parellellised content
+ in the other file. These files are named `file.doc.sent_new.txt` and
+ `file_n.doc.sent_new.txt`
+## Morphological information
The file korpustags.txt contains the list of tags and their internal
distrbution. The list below list is not up-to-date, please see the file
korpustags.txt in cvs.
-## Corpus tags
-
+### Corpus tags
There are the following tag categories:
-- Part of speech: N A V Adv Pron CS CC Adp Po Pr Interj Pcle Num
-- Number or Person/Number: Sg Pl Sg1 Sg2 Sg3 Du1 Du2 Du3 Pl1 Pl2 Pl3
- ConNeg
-- Case: Ess Nom Gen Acc Ill Loc Com
-- Possessive suffix: PxSg1 PxSg2 PxSg3 PxDu1 PxDu2 PxDu3 PxPl1 PxPl2
- PxPl3
-- Clitic: Qst Foc
-- Grade: Comp Superl
-- Attribute: Attr
-- Derivation (We must consider whether to use these..): Pass h upmi
- ...
-- Polarity: Neg
-- Mood: Ind Pot Cond Imprt ImprtII
-- Tense: Prs Prt
-- Nominal verb form: Inf Act PrsPrc PrfPrc VGen VAbess Ger ConNeg
- ConNegII
-- Pronoun type: Pers Dem Interr Refl Recipr Rel Indef
-- Other: PUNCT CLB ABBR ACR
-- Punctuation type: LEFT RIGHT
-- Num type: Ord Card
+- Part of speech: N A V Adv Pron CS CC Adp Po Pr Interj Pcle Num
+- Number or Person/Number: Sg Pl Sg1 Sg2 Sg3 Du1 Du2 Du3 Pl1 Pl2 Pl3
+ ConNeg
+- Case: Ess Nom Gen Acc Ill Loc Com
+- Possessive suffix: PxSg1 PxSg2 PxSg3 PxDu1 PxDu2 PxDu3 PxPl1 PxPl2
+ PxPl3
+- Clitic: Qst Foc
+- Grade: Comp Superl
+- Attribute: Attr
+- Derivation (We must consider whether to use these..): Pass h upmi
+ ...
+- Polarity: Neg
+- Mood: Ind Pot Cond Imprt ImprtII
+- Tense: Prs Prt
+- Nominal verb form: Inf Act PrsPrc PrfPrc VGen VAbess Ger ConNeg
+ ConNegII
+- Pronoun type: Pers Dem Interr Refl Recipr Rel Indef
+- Other: PUNCT CLB ABBR ACR
+- Punctuation type: LEFT RIGHT
+- Num type: Ord Card
diff --git a/ling/corpus_bibles.md b/ling/corpus_bibles.md
index 0c357976..f43d3503 100644
--- a/ling/corpus_bibles.md
+++ b/ling/corpus_bibles.md
@@ -1,8 +1,6 @@
-The Bible as a corpus
-============
-
-# Overview and introduction
+# The Bible as a corpus
+## Overview and introduction
The bibles are an important source of text in a language. It is even
more imprtant source of parallel text, since the bible translations
@@ -12,15 +10,14 @@ Lule Sámi, Bokmål, Nynorsk, Swedish and Finnish. Since the bibles have a
specific format which requires separate processing than the other files,
there is a documentation included.
-# Bible as a text source
-
+## Bible as a text source
In general..
The bible files are part of the general conversion process, which is
described in the
-documents[corpus\_conversion.html](corpus_conversion.html) and
-[corpus\_conversion\_tech.html.](corpus_conversion_tech.html) This
+documents[corpus_conversion.html](corpus_conversion.html) and
+[corpus_conversion_tech.html.](corpus_conversion_tech.html) This
conversion process is aimed to produce texts that are ready for
linguistic analysis and research. The texts contain some structure, such
as the sections and paragraphs. Similarly, titles, lists and tables are
@@ -38,8 +35,7 @@ the original structure of the text. The bible texts can be better
compared, the text can be searched using verse and chapter number and
the structural information can be used in text alignment.
-## XML-format of the bible
-
+### XML-format of the bible
The XML format of the analyzed text is basically the following:
@@ -48,7 +44,7 @@ The XML format of the analyzed text is basically the following:
I opphavet skapte Gud himmelen og jorda.
- Jorda var aud og tom, og mørker låg over havdjupet.
+ Jorda var aud og tom, og mørker låg over havdjupet.
Men Guds Ande sveiv over vat net.
...
@@ -64,8 +60,7 @@ and taken out from the paragraph. Some versions may not have subtitles,
so the section layer may be missing. The paragraph-marks are not added
when they are not explicitely marked in the document.
-## The available files
-
+### The available files
The xml-formatted bible text is stored at
`corp/orig/lang/nt `and `ot `-directories. The files are
@@ -75,26 +70,25 @@ Most of the bible texts are under standard license conditions, which are
explained in... At the moment, the corpus hierarchy contains the
following bible texts:
-- **sme and smj:**
- New Testament
+- **sme and smj:**
+ New Testament
-- **nno and nob :**
- Genesisi and Psalms
+- **nno and nob :**
+ Genesisi and Psalms
-- **swe:**
- New Testament, Genesis and Psalms
+- **swe:**
+ New Testament, Genesis and Psalms
-- **fin:**
- New and Old Testaments, 1938 edition
-
-## The tool for converting bibles to text: `bible2xml.pl`
+- **fin:**
+ New and Old Testaments, 1938 edition
+### The tool for converting bibles to text: `bible2xml.pl`
Usage:
@@ -106,6 +100,8 @@ form longer paragraphs. A sentence many times spreads over more than one
verse, so when the verses are combined, there sentece structure is
preserved.
-## Technical details of cleaning the bible texts.
-----------------------------------------------
+### Technical details of cleaning the bible texts.
+
+---
+
(...)
diff --git a/ling/corpus_conversion.md b/ling/corpus_conversion.md
index aa3ac9cc..cd6d4ab5 100644
--- a/ling/corpus_conversion.md
+++ b/ling/corpus_conversion.md
@@ -1,7 +1,6 @@
-Corpus conversion
-============
+# Corpus conversion
-# Introduction
+## Introduction
This file provides an overview of the [corpus conversion
process](CorpusTools.html#convert2xml). Basically, when a new document
@@ -17,10 +16,9 @@ author, is stored in an xsl-file and appended to the xml-file during the
conversion. The metainformation is used elsewhere, in different corpus
applications.
-# Working with the corpus
+## Working with the corpus
-
-- [Adding content to the
- corpus](corpus_repositories.html#Adding+content)
-- [Example session on how to add parallel files to the
- corpus](ParallelCorpusConversion.html)
+- [Adding content to the
+ corpus](corpus_repositories.html#Adding+content)
+- [Example session on how to add parallel files to the
+ corpus](ParallelCorpusConversion.html)
diff --git a/ling/corpus_conversion_tech.md b/ling/corpus_conversion_tech.md
index 2eb24d49..9055fe1a 100644
--- a/ling/corpus_conversion_tech.md
+++ b/ling/corpus_conversion_tech.md
@@ -1,12 +1,11 @@
-Corpus conversion process
-==================
+# Corpus conversion process
[This text in Spanish \~ Este texto en español (translation: Maria
Ramos)](http://www.webhostinghub.com/support/es/misc/documentacion-tecnica)
This document describes the more technical side of the corpus
infrastructure. The document
-[corpus\_conversion.html](corpus_conversion.html) contains the user
+[corpus_conversion.html](corpus_conversion.html) contains the user
documentation as well as the description of the basic infrastructure.
The usage of the script convert2xml.pl is documented there, this
document describes the contents of the script.
@@ -27,36 +26,31 @@ type and the tools used. The conversion includes hyphen-detection and
language recognition as well as decoding the wrongly utf-8 -encoded
characters.
-Word documents
---------------
+### Word documents
Microsoft Word documents are converted with the program `antiword` to
produce a docbook xml, and piped to the xslt program xsltproc, that
converts to our XML-format. We have an xsl-document docbook2corpus.xsl,
that is used in converting the document.
-RTF documents
--------------
+### RTF documents
RTF documents are converted to html using
[unrtf](http://www.gnu.org/software/unrtf/), then to our xml format
using
[xhtml2xml.xsl](https://gtsvn.uit.no/langtech/trunk/gt/script/corpus/xhtml2corpus.xsl).
-Ávvir xml documents
--------------------
+### Ávvir xml documents
These documents are converted to our xml format using
[avvir2corpus.xsl](https://gtsvn.uit.no/langtech/trunk/gt/script/corpus/avvir2corpus.xsl)
-SVG documents
--------------
+### SVG documents
These documents are converted to our xml format using
[svg2corpus.xsl](https://gtsvn.uit.no/langtech/trunk/gt/script/corpus/svg2corpus.xsl)
-Plain text document
--------------------
+### Plain text document
Plain text documents that are stored to the database should have the
extension `.txt`. The encoding of a text document is solved using the
@@ -65,24 +59,21 @@ like in some newspaper texts, they are used in creating the document
structure. Otherwise, the empty lines mark paragraph breaks and short
lines beginning with numbers are treated as titles.
-Adobe document
---------------
+### Adobe document
Pdf documents are converted to plain text using
[pdftotext](http://poppler.freedesktop.org/), and then converted to our
xml format using
-[langTools::Corpus::txt\_clean](https://gtsvn.uit.no/langtech/trunk/gt/script/langTools/Corpus.pm)
+[langTools::Corpus::txt_clean](https://gtsvn.uit.no/langtech/trunk/gt/script/langTools/Corpus.pm)
-Web-document
-------------
+### Web-document
Web documents are first cleaned using the program [HTML
Tidy](http://tidy.sourceforge.net/) , using several command line
options.. The output is converted to xml using the xsl-file
`xhtml2xml.xsl`.
-Paratext document
------------------
+### Paratext document
Paratext is a file format for publishing and interchanging basic
Scripture texts in multiple languages. It is intended to be used for all
@@ -93,8 +84,7 @@ are converted using a Perl script paratext2xml.pl which forms basic
xml-structure which roughly corresponds our corpus.dtd. The files that
are added to the corpus repository should have the suffix .ptx.
-Hyphenation: add-hyph-tags.pl
-=============================
+## Hyphenation: add-hyph-tags.pl
Usage: add-hyph-tags.pl [OPTIONS] FILES
Tag the hyphenation marks.
@@ -131,8 +121,7 @@ Some text extraction tools such as antiword may create this kind of
structures. The script also reformats the text by removing white space,
moving <p>-tags and changing the place of the line break.
-Language recognition
-====================
+## Language recognition
The newly created xml-document is parsed and the language of each
paragraph is recognized using the tool `pytextcat`. The tool is
@@ -151,8 +140,7 @@ monolingual, or multilingual containg text fractions of some of the
abovementioned languages. You should set these variables in the
file-specific xsl-file.
-XSLT-scripts
-============
+## XSLT-scripts
The structural information, such as titles and paragraphs, that is
contained in MS Word of pdf document is preserved in the xml-document.
@@ -189,81 +177,79 @@ when the result of the conversion is not satisfactory. The document
could be removed from the database as well, but e.g. some newspaper
documents are considered to be part of the distribution.
-Useful scripts for corpus maintainer
-====================================
+## Useful scripts for corpus maintainer
There are several small scripts for corpus database maintenance and
cleaning. They reside in `gt/script `-catalog. The most important ones
are listed here:
-- **corpus\_summarize.sh:**
- The script is used for generating summarizing information of the
- corpus files and storing them to cvs, under the catalog
- `gt/doc/lang/corp`. The script calls the Perl-script
- ` corpus_summary.pl`, which generates the summaries.
- The file `corpus-content.xml` contains a list of all the files in
- the corpus database and some relevant information like the license
- and size. The files `corpus-susummary.xml` and
- `corpus-summary-YYYY-MM-DD.xml` contain the total count of the
- documents as well as the number of the documents in different
- subdirectories. The xml-files are further transformed to forrest
- documentation.
-- **corpus\_zip.sh:**
- The script copies the free-catalog from the corpus database and
- creates zipped tar-archives of it. An archive is created of both the
- xml files and the files that contain only the extracted text in the
- selected language. The archives are copied to the download area.
-- **corpus\_chmod.sh:**
- The script is used for correcting the group ids and permissions of
- the corpus files in different catalogs. Running the script requires
- sudo rights.
-- **change\_xsl\_generic.py:**
- for i in \`find . -name \\\*.xsl\`
- do
- change\_xsl\_generic.py translator\_fn Pirjo translator\_ln
- Paavaniemi
- translator\_gender fem translator\_nat FI $i
- done
-- **gt2ga.sh:**
- Copies the selected files to G5 for analysis, compiles the latest
- versions of the tools and runs the analysis for the whole corpus.
- The analyzed files are stored under ga/lang/genre, the whole genre
- in one file. The script is currently used only in G5.
-- **corpus\_fix\_meta.sh:**
- Some of the corpus directories contain several files with similar
- metainformation, e.g. all the newspaper texts may have the same
- collection information, the name of the newspaper. This script is
- planned for updating several file-specific xsl-files at the same
- time. The directory where the files are searched is given in
- variable inside the script. The script uses xsl-script
- `change_xsl.xsl `for the transformation. The xsl-script should be
- modified for the different uses and the variable containing the path
- to the script changed. The version control of the xsl-files is
- handled automatically, although sometimes stealing a lock is
- necessary and requires some typing.
-- **corpus\_rename.sh:**
- The names of the corpus files have to face some basic requirements:
- They should be in utf-8 encoding, in NFC, not contain any special
- characters that would harm the shell ("´? etc.), have spaces
- replaced by underscores or removed and have the correct extensions
- (.txt, .html, .doc, .pdf or .ptx). This script searches whole
- directories for filenames and changed them to confront these
- requirements. If the file already has some extension, it is not
- changed.
-- **corpus\_bad\_encoding.sh:**
- Some of the character set conversions do not succeed well enough for
- the purposes of linguistic analysis. The files may contain different
- encodings in different parts of the file, single unrecognized
- characters or the character set is not recognized at all. Those
- files can be located using this script.
-- **Makefile:**
- There is a Makefile in /usr/local/share/corp for converting files
- that are not up-to-date concerning the original or file-specific
- xsl-file. Usage: `make LANGUAGE=sme GENRE=facta` or
- `make path/to/file.xml`.
-
-corpus.dtd
-==========
+- **corpus_summarize.sh:**
+ The script is used for generating summarizing information of the
+ corpus files and storing them to cvs, under the catalog
+ `gt/doc/lang/corp`. The script calls the Perl-script
+ ` corpus_summary.pl`, which generates the summaries.
+ The file `corpus-content.xml` contains a list of all the files in
+ the corpus database and some relevant information like the license
+ and size. The files `corpus-susummary.xml` and
+ `corpus-summary-YYYY-MM-DD.xml` contain the total count of the
+ documents as well as the number of the documents in different
+ subdirectories. The xml-files are further transformed to forrest
+ documentation.
+- **corpus_zip.sh:**
+ The script copies the free-catalog from the corpus database and
+ creates zipped tar-archives of it. An archive is created of both the
+ xml files and the files that contain only the extracted text in the
+ selected language. The archives are copied to the download area.
+- **corpus_chmod.sh:**
+ The script is used for correcting the group ids and permissions of
+ the corpus files in different catalogs. Running the script requires
+ sudo rights.
+- **change_xsl_generic.py:**
+ for i in \`find . -name \\\*.xsl\`
+ do
+ change_xsl_generic.py translator_fn Pirjo translator_ln
+ Paavaniemi
+ translator_gender fem translator_nat FI $i
+ done
+- **gt2ga.sh:**
+ Copies the selected files to G5 for analysis, compiles the latest
+ versions of the tools and runs the analysis for the whole corpus.
+ The analyzed files are stored under ga/lang/genre, the whole genre
+ in one file. The script is currently used only in G5.
+- **corpus_fix_meta.sh:**
+ Some of the corpus directories contain several files with similar
+ metainformation, e.g. all the newspaper texts may have the same
+ collection information, the name of the newspaper. This script is
+ planned for updating several file-specific xsl-files at the same
+ time. The directory where the files are searched is given in
+ variable inside the script. The script uses xsl-script
+ `change_xsl.xsl `for the transformation. The xsl-script should be
+ modified for the different uses and the variable containing the path
+ to the script changed. The version control of the xsl-files is
+ handled automatically, although sometimes stealing a lock is
+ necessary and requires some typing.
+- **corpus_rename.sh:**
+ The names of the corpus files have to face some basic requirements:
+ They should be in utf-8 encoding, in NFC, not contain any special
+ characters that would harm the shell ("´? etc.), have spaces
+ replaced by underscores or removed and have the correct extensions
+ (.txt, .html, .doc, .pdf or .ptx). This script searches whole
+ directories for filenames and changed them to confront these
+ requirements. If the file already has some extension, it is not
+ changed.
+- **corpus_bad_encoding.sh:**
+ Some of the character set conversions do not succeed well enough for
+ the purposes of linguistic analysis. The files may contain different
+ encodings in different parts of the file, single unrecognized
+ characters or the character set is not recognized at all. Those
+ files can be located using this script.
+- **Makefile:**
+ There is a Makefile in /usr/local/share/corp for converting files
+ that are not up-to-date concerning the original or file-specific
+ xsl-file. Usage: `make LANGUAGE=sme GENRE=facta` or
+ `make path/to/file.xml`.
+
+## corpus.dtd
corpus.dtd contains the document type definition for the xml-structure.
It is stored in The fields are
@@ -271,99 +257,95 @@ briefly described in the following:
document (header,body)
-header
-------
+### header
The document is divided into to elements: header that contains the
metainformation and body for the document content. The header contains
the following fields:
-- **`title`:**
- The title of the document
-- **`genre`:**
- The document genre can be one of the following: admin, bible, facta,
- ficti or news.
-- **`author+`:**
- The author of the document is either person or "unknown".
- - **`person`:**
- Person consist of following attributes: firstname, lastname,
- born, sex and nationality.
-- **`translator*`:**
- If the document is translated, the translator is either "unknown" or
- person.
-- **`translated from*`:**
-- **`year?`:**
- Publishing year, or if the document is unpublished, the year the
- document was written.
-- **`place?`:**
- Publishing place, or if the document is unpublished, the country or
- place where the document was written.
-- **`publChannel`:**
- Either publication or unpublished
- - **`publication`:**
- Consists of elements publisher, ISBN and ISSN.
-- **`collection`:**
- The name of the journal, book or article collection.
-- **`wordcount`:**
-- **`availability`:**
- Either free or license
- - **`license`:**
- License type is "standard" or "other". The type "standard"
- indicates a license agreed with the basic contract
-- **`submitter?`:**
- The person or instance who stored the document to the database.
- Contains elements for name and email address.
-- **`multilingual?`:**
- If the document is multilingual, this element describes the
- languages that occur in the document.
- - **`language+`:**
- Language is given as an xml:lang attribute.
-- **`origFileName?`:**
- The original name of the file. The filename may change during the
- conversion to xml.
-- **`metada`:**
- Either "complete" or "uncomplete".
-- **`version`:**
- Contains version information of different conversion tools.
-
-body
-----
+- **`title`:**
+ The title of the document
+- **`genre`:**
+ The document genre can be one of the following: admin, bible, facta,
+ ficti or news.
+- **`author+`:**
+ The author of the document is either person or "unknown".
+ - **`person`:**
+ Person consist of following attributes: firstname, lastname,
+ born, sex and nationality.
+- **`translator*`:**
+ If the document is translated, the translator is either "unknown" or
+ person.
+- **`translated from*`:**
+- **`year?`:**
+ Publishing year, or if the document is unpublished, the year the
+ document was written.
+- **`place?`:**
+ Publishing place, or if the document is unpublished, the country or
+ place where the document was written.
+- **`publChannel`:**
+ Either publication or unpublished
+ - **`publication`:**
+ Consists of elements publisher, ISBN and ISSN.
+- **`collection`:**
+ The name of the journal, book or article collection.
+- **`wordcount`:**
+- **`availability`:**
+ Either free or license
+ - **`license`:**
+ License type is "standard" or "other". The type "standard"
+ indicates a license agreed with the basic contract
+- **`submitter?`:**
+ The person or instance who stored the document to the database.
+ Contains elements for name and email address.
+- **`multilingual?`:**
+ If the document is multilingual, this element describes the
+ languages that occur in the document.
+ - **`language+`:**
+ Language is given as an xml:lang attribute.
+- **`origFileName?`:**
+ The original name of the file. The filename may change during the
+ conversion to xml.
+- **`metada`:**
+ Either "complete" or "uncomplete".
+- **`version`:**
+ Contains version information of different conversion tools.
+
+### body
The document body contains sections and text-entities (`%text.ent`):
-- **`%text.ent`:**
- Basic text-entites are list, table, p and pre.
-- **`section `:**
- Contains sections and text-entities.
-- **`list`:**
- Consists of paragraphs `p` with type "listitem".
-- **`table`:**
- Consist of rows:
- - **row:**
- Contains paragraphs `p` with type "tablecell".
-- **`p`:**
- The paragraph type is given as an attribute: title, listitem, text
- or tablecell. Paragraph contains text and elements em, hyph and
- error:
- - **`em`:**
- Emphasis type: bold, italic, underline or delimited.
- - **`hyph`:**
- Empty hyphenation tag
- - **`error`:**
- Error marking in the text, has the correct replacement as an
- attribute.
-- **`pre`:**
- Programlisting or other unformatted text.
-
-Character decoding: samiChar::Decode.pm
-=======================================
+- **`%text.ent`:**
+ Basic text-entites are list, table, p and pre.
+- **`section `:**
+ Contains sections and text-entities.
+- **`list`:**
+ Consists of paragraphs `p` with type "listitem".
+- **`table`:**
+ Consist of rows:
+ - **row:**
+ Contains paragraphs `p` with type "tablecell".
+- **`p`:**
+ The paragraph type is given as an attribute: title, listitem, text
+ or tablecell. Paragraph contains text and elements em, hyph and
+ error:
+ - **`em`:**
+ Emphasis type: bold, italic, underline or delimited.
+ - **`hyph`:**
+ Empty hyphenation tag
+ - **`error`:**
+ Error marking in the text, has the correct replacement as an
+ attribute.
+- **`pre`:**
+ Programlisting or other unformatted text.
+
+## Character decoding: samiChar::Decode.pm
Some of the sámi characters are wrongly utf-8-encoded by the conversion
tools, like pdftotext. There is a Perl module samiChar::Decode.pm for
decoding the sámi characters.
-SYNOPSIS
---------
+### SYNOPSIS
use samiChar::Decode;
@@ -378,8 +360,7 @@ SYNOPSIS
$encoding = &guess_text_encoding($file, $outfile, $lang);
&decode_text_file($file, $encoding, $outfile);
-DESCRIPTION
------------
+### DESCRIPTION
samiChar::Decode.pm decodes characters to utf-8 byte-wise, using code
tables. It is planned for decoding the Saami characters in a situation,
@@ -397,7 +378,7 @@ file which is not utf-8 encoded but in it's original encoding. This is
the case with many text files that are not converted by any tool. Thes
functions are implemented using the iconv conversion tool.
-### Code tables
+#### Code tables
Code tables are text files with the following format: Three
space-separated columns:
@@ -409,7 +390,7 @@ space-separated columns:
Most of the code tables are available at the Unicode Consortium:
-. Some of the code tables like samimac\_roman and levi\_winsam are
+. Some of the code tables like samimac_roman and levi_winsam are
composed from two code tables, the one that is used as input encoding
and another that is used as the file was converted to utf-8.
@@ -419,7 +400,7 @@ and another that is used as the file was converted to utf-8.
levi.txt and samimac.txt are available under Trond’s home page at:
[smi-kodetabell.html](https://web.archive.org/web/20120308152538/http://www.hum.uit.no/a/trond/smi-kodetabell.html).
The codetables are composed using the function
-"combine\_two\_codings($coding1, $coding2, $outfile)" which is available
+"combine_two_codings($coding1, $coding2, $outfile)" which is available
in this package.
These encodings are available:
@@ -433,11 +414,11 @@ These encodings are available:
8859-4 => 8859-4.txt
winsam => winsam.txt
-### Guessing the input encoding
+#### Guessing the input encoding
The original input encoding is guessed by examining the text and
searching the most common characters. The unicode characters in hex are
-listed in hash %Char\_Tables for North Saami for example. The
+listed in hash %Char_Tables for North Saami for example. The
uncommented characters are the ones that take part into guessing the
encoding.
@@ -450,14 +431,12 @@ If there is no certain amount of characters found, the test returns -1,
which means that the characters should be already correctly utf-8
encoded. Or, the encoding was not found from the code tables.
-INSTALL
--------
+### INSTALL
To your own computer: copy the directory `gt/script/samiChar` to
`/System/Library/Perl/5.8.6/`. The module is now installed to victorio.
-Web upload interface
-====================
+## Web upload interface
The main page of the web upload interface is
@@ -480,7 +459,6 @@ is an error during the upload.
The file names are changed to secure ones and orig-hierarchy is checked
for a file with the same content.
-Corpus DTD
-==========
+## Corpus DTD
[Corpus DTD](corpus_dtd.html)
diff --git a/ling/corpus_conversionerrors_may11.md b/ling/corpus_conversionerrors_may11.md
index 12299a3a..9704112b 100644
--- a/ling/corpus_conversionerrors_may11.md
+++ b/ling/corpus_conversionerrors_may11.md
@@ -1,32 +1,28 @@
-Conversion errors
-=========
+# Conversion errors
**Note: The document is from 2011**
-# Errors addressed so far (May 2011):
+## Errors addressed so far (May 2011):
-
-* dårlege originalfiler - gjev ugyldig xml
-** desse blir fanga opp i dag
-* kodefeil - desse gjev gyldig xml, men meiningslause bokstavar
-** utf-som-macroman
-** utf-som-latin1
-** utf-som-html-hex
-** utf-som-html-entitet
-* skannefeil/ocr-feil - desse gjev meiningsfulle bokstavar, men meiningslaus tekst
-** đ-som ó, osv.
-* bad sentence-delimitation: one real sentence is one fragment in one language,
+- dårlege originalfiler - gjev ugyldig xml
+ \*\* desse blir fanga opp i dag
+- kodefeil - desse gjev gyldig xml, men meiningslause bokstavar
+ ** utf-som-macroman
+ ** utf-som-latin1
+ ** utf-som-html-hex
+ ** utf-som-html-entitet
+- skannefeil/ocr-feil - desse gjev meiningsfulle bokstavar, men meiningslaus tekst
+ \*\* đ-som ó, osv.
+- bad sentence-delimitation: one real sentence is one fragment in one language,
3 fragments in the other -> alignment goes bunk
-* files `freecorpus/converted/sme/admin/others/`
-** `STM200420050011000SE_PDFS.pdf.xml`
+- files `freecorpus/converted/sme/admin/others/`
+ ** `STM200420050011000SE_PDFS.pdf.xml`
`STM200420050044000SE_PDFS.pdf.xml`
have encoding errors that đ is represented as and the document is full of
's; thus these files should be deleted
-** file `OTP200620070025000SE_PDFS.pdf.xml`
+ ** file `OTP200620070025000SE_PDFS.pdf.xml`
has paragraphs with content '--------' so it should be deleted.
-** file `STM200320040010000SE_PDFA.pdf.xml`
+ ** file `STM200320040010000SE_PDFA.pdf.xml`
has so many errors, it should be rescanned
-** `uito-ohpenplana.txt.xml`
+ ** `uito-ohpenplana.txt.xml`
the original file is corrupted
-
-
diff --git a/ling/corpus_dtd.md b/ling/corpus_dtd.md
index 38150729..6937fc7c 100644
--- a/ling/corpus_dtd.md
+++ b/ling/corpus_dtd.md
@@ -1,19 +1,16 @@
-Corpus DTD
-===========
-
-# Introduction
+# Corpus DTD
+## Introduction
Draft \[X\] Proposal \[ \] Approved \[ \]
| Version | Date | Explanation | Author |
-|---------|-----------|---------------------------------------|--------|
+| ------- | --------- | ------------------------------------- | ------ |
| 0.1 | 14.3.2005 | Initial transformation from newsgroup | Tomi |
Version history
-# Content structure
-
+## Content structure
document (header, body)
@@ -32,7 +29,7 @@ Version history
place
publDesc (publisher, idno)
idno
-
+
body (text)
@@ -43,6 +40,6 @@ Version history
table (row+)
row (cell+)
-# Formal TEI conformant DTD
+## Formal TEI conformant DTD
To be added
diff --git a/ling/corpus_improve_alignment.md b/ling/corpus_improve_alignment.md
index 9c3f662b..03c87e9b 100644
--- a/ling/corpus_improve_alignment.md
+++ b/ling/corpus_improve_alignment.md
@@ -1,11 +1,9 @@
-Improve corpus alignment
-===============
+# Improve corpus alignment
Example: Sentences as paragraphs:
-
| | |
-|-----------------------------------------------------------|--------------------------------------------------------------|
+| --------------------------------------------------------- | ------------------------------------------------------------ |
| Hallitus esitteli maaliskuussa 2009 uuden | Ráđđehus ovdanbuvttii njukčamánus 2009 ođđa |
| vähemmistöpoliittisen strategian esityksessään | unnitlogopolitihkalaš strategiija proposišuvnnas Från |
| Tunnustamisesta omaan valtaan – Hallituksen kansallisia | strategiija nášuvnnalaš unnitloguid váras) (prop. |
@@ -44,8 +42,7 @@ To fix this, one has to:
The above steps improves the situation somewhat, but the sentence
alignment has not become perfect.
-General approach to improving sentence alignment
-================================================
+## General approach to improving sentence alignment
To improve the sentence alignment, one must improve the quality of the
converted xml file. The general steps are:
@@ -63,8 +60,7 @@ converted xml file. The general steps are:
6. Open <path-to-tmx-html-file> in a web browser. If alignment is
good, commit modified files. If not, repeat the steps above.
-Improve sentence alignment of pdf files
----------------------------------------
+### Improve sentence alignment of pdf files
When the sentence alignment between two pdf files is bad, it is possible
to improve it a lot by editing the [metadata](corpus_xsl_file.html) file
@@ -73,18 +69,18 @@ and setting various variables that are specific to pdf files.
The following variables in the [metadata](corpus_xsl_file.html) file
affect the content of the converted file:
-- **skip\_pages:**
- skip specific pages that are not wanted in the converted file
-- **right\_margin, left\_margin, top\_margin, bottom\_margin:**
- remove text above, below, to the right and left of the main content.
- This is typically removes page numbers, foot notes, reoccuring page
- headers and footers, etc.
-- **inner\_\*\_margin:**
- remove a rectangle shaped part of the page
-- **linespacing:**
- as the above example shows, if line spacing in a document is larger
- than the default 1.5, text lines will be converted to paragraphs. To
- improve this, set a larger value for this.
+- **skip_pages:**
+ skip specific pages that are not wanted in the converted file
+- **right_margin, left_margin, top_margin, bottom_margin:**
+ remove text above, below, to the right and left of the main content.
+ This is typically removes page numbers, foot notes, reoccuring page
+ headers and footers, etc.
+- **inner\_\*\_margin:**
+ remove a rectangle shaped part of the page
+- **linespacing:**
+ as the above example shows, if line spacing in a document is larger
+ than the default 1.5, text lines will be converted to paragraphs. To
+ improve this, set a larger value for this.
A more thorough documentation of these variables are found in [the
metadata template
diff --git a/ling/corpus_intro.md b/ling/corpus_intro.md
index 6987c0b4..56d6aa9e 100644
--- a/ling/corpus_intro.md
+++ b/ling/corpus_intro.md
@@ -1,19 +1,17 @@
-Introduction to working on corpora at Giellalt
-==============
+# Introduction to working on corpora at Giellalt
This document will teach the user how to convert files in the corpus
repositories to xml, and how to extract text from these documents.
-Converting the corpus to xml
-============================
+## Converting the corpus to xml
To be able to convert files in our repository, you have to [check out
our tools and do a basic setup](/infra/anonymous-svn.html#Preparation).
The corpus is stored using
-[git](https://en.wikipedia.org/wiki/Git).
+[git](https://en.wikipedia.org/wiki/Git).
-For each language, the corpus files are divided in a *free* and a *restricted* part. Each corpus has two directories, one for the original files (*docx, pdf, html, ...) and one for the converted text files (xml). For a language with ISO code xxx, the four repositories are:
+For each language, the corpus files are divided in a _free_ and a _restricted_ part. Each corpus has two directories, one for the original files (\*docx, pdf, html, ...) and one for the converted text files (xml). For a language with ISO code xxx, the four repositories are:
```
corpus-xxx
@@ -23,41 +21,34 @@ corpus-xxx-orig-x-private
```
Access to the `private` repositories are restricted for copyright reasons, and are given only to project employees working on the corpora. The open ones are available for all, at [giithub.com/giellalt/](https://github.com/giellalt/) (search for `corpus-xxx`, xxx being the ISO code of your language) in the search field).
-
-
-
-Extract text from the corpus
-============================
+## Extract text from the corpus
The converted xml files are found in the `corpus-xxx/converted/` catalogue. To get
all North Saami text, issue the command `ccat -a -r -l sme corpus-sme/converted`.
The options available for `ccat` are listed with the
command `ccat -h`.
-If you do not have `ccat` installed (part of our *corpustools*), you may use `cat` and get xml files.
+If you do not have `ccat` installed (part of our _corpustools_), you may use `cat` and get xml files.
-Divvun/Giellatekno corpus documentation
-=======================================
+## Divvun/Giellatekno corpus documentation
(this documentation may be obsolete)
-- The tagged corpus files
- - [How to create an analyzed corpus](corpus_analyze.html)
- - [A plan for building our external corpus](corpus_plan.html)
- - [The <Correct!> corpora](correct-dir.html)
-- The parallel corpus files
- - [ParallelCorpusConversion](ParallelCorpusConversion.html)
+- The tagged corpus files
+ - [How to create an analyzed corpus](corpus_analyze.html)
+ - [A plan for building our external corpus](corpus_plan.html)
+ - [The <Correct!> corpora](correct-dir.html)
+- The parallel corpus files
+ - [ParallelCorpusConversion](ParallelCorpusConversion.html)
-Other corpora
-=============
+## Other corpora
-- [How to use Wikipedia as a corpus](WikipediaAsCorpus.html)
-- [How to build corpora from Wikipedia
- editions](wikipedia_as_corpus.html)
+- [How to use Wikipedia as a corpus](WikipediaAsCorpus.html)
+- [How to build corpora from Wikipedia
+ editions](wikipedia_as_corpus.html)
-Old links (kept for now)
-========================
+## Old links (kept for now)
[DTD -discussion](corpus_dtd.html)
diff --git a/ling/corpus_maintenance.md b/ling/corpus_maintenance.md
index 4d1dff2a..96a6ec50 100644
--- a/ling/corpus_maintenance.md
+++ b/ling/corpus_maintenance.md
@@ -1,73 +1,60 @@
# Corpus maintenance
-
This document keeps track of measures to improve the corpus collection and conversion process.
Note also the [sentence alignment page](/tools/tca2.html), which looks
into that specific sub-part of the corpus maintenance.
+### Corpus improvement work
+#### Mappestruktur osv
+- news: muligheter for å flytte fra bound til free?
+- science: vi har filer sme/science både i free og i bound, uten noen klar deling
+- sma: eget valg for klassiske tekster
-## Corpus improvement work
-
-
-### Mappestruktur osv
-* news: muligheter for å flytte fra bound til free?
-* science: vi har filer sme/science både i free og i bound, uten noen klar deling
-* sma: eget valg for klassiske tekster
-
-
-
-
-# Tasks
-
-
-## Where do we find texts
-* [A sketch list of where to find Saami text online](SaamiTextOnline.html)
-
-
-## Parallel texts
-* [Suggestions for detecting (flaws in) parallel texts](corpus_parallel_maintenance.html)
-* [How to manipulate the conversion of different file formats, in xsl, to get the correct language](CorpusConvertingManipulation.html)
-
+## Tasks
+### Where do we find texts
+- [A sketch list of where to find Saami text online](SaamiTextOnline.html)
+### Parallel texts
+- [Suggestions for detecting (flaws in) parallel texts](corpus_parallel_maintenance.html)
+- [How to manipulate the conversion of different file formats, in xsl, to get the correct language](CorpusConvertingManipulation.html)
-# Meetings in the corpus improvement project
+## Meetings in the corpus improvement project
-
-* 2019:
-[ 7.6.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2019-06-07.html)
-* 2017:
-[ 3.3.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2017-03-03.html) //
+- 2019:
+ [ 7.6.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2019-06-07.html)
+- 2017:
+ [ 3.3.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2017-03-03.html) //
[ 25.4.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2017-04-25.html) //
- [ 6.9.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2017-09-06.html) //
+ [ 6.9.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2017-09-06.html) //
[ 5.10.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2017-10-05.html)
-* 2016:
-[ 26.10.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2016-10-26.html)//
+- 2016:
+ [ 26.10.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2016-10-26.html)//
[ 02.11.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2016-11-02.html)//
[ 16.11.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2016-11-16.html)//
[ 25.11.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2016-11-25.html)
-* 2014:
-[ 12.3.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2014-03-12.html)
-* 2012:
-[ 12.1.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2012-01-12.html) //
+- 2014:
+ [ 12.3.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2014-03-12.html)
+- 2012:
+ [ 12.1.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2012-01-12.html) //
[ 19.1.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2012-01-19.html) //
[ 25.1.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2012-01-25.html) //
- [ 1.2.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2012-02-01.html) //
- [ 7.2.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2012-02-07.html) //
+ [ 1.2.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2012-02-01.html) //
+ [ 7.2.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2012-02-07.html) //
[ 13.2.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2012-02-13.html) //
[ 17.2.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2012-02-17.html) //
[ 29.2.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2012-02-29.html) //
[ 12.3.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2012-03-12.html) //
[ 22.3.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2012-03-22.html) //
[ 31.8.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2012-08-31.html)
-* 2011:
-[ 7.4.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2011-04-07.html) //
+- 2011:
+ [ 7.4.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2011-04-07.html) //
[ 11.4.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2011-04-11.html) //
- [ 3.5.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2011-05-03.html) //
+ [ 3.5.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2011-05-03.html) //
[ 27.6.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2011-06-27.html) //
[ 12.9.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2011-09-12.html) //
[ 21.9.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2011-09-21.html) //
@@ -80,17 +67,7 @@ into that specific sub-part of the corpus maintenance.
[14.12.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2011-12-14.html) //
[20.12.](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2011-12-20.html)
+## OCR and conversion errors leftover from spring 2011
-
-
-
-
-
-
-# OCR and conversion errors leftover from spring 2011
-
-
-* [OCR error overview, May 2011](corpus_ocr_may11.html) (still open issues here)
-* [Conversion errors](corpus_conversionerrors_may11.html) (open issues here?)
-
-
+- [OCR error overview, May 2011](corpus_ocr_may11.html) (still open issues here)
+- [Conversion errors](corpus_conversionerrors_may11.html) (open issues here?)
diff --git a/ling/corpus_norwegianwhitepapers.md b/ling/corpus_norwegianwhitepapers.md
index 9f5a4a27..d4c5f551 100644
--- a/ling/corpus_norwegianwhitepapers.md
+++ b/ling/corpus_norwegianwhitepapers.md
@@ -1,35 +1,31 @@
-Norwegian Whitepapers
-==============
+# Norwegian Whitepapers
The files listed below are part of the [repository of freely available
sami texts](https://gtsvn.uit.no/freecorpus). They are found in
$GTFREE/\[nno\|nob\|sme\]/admin/(regjeringen.no)
-Stortingsmeldinger
-==================
+## Stortingsmeldinger
-PDF's
------
+### PDF's
-| sme | nob | nno |
-|--------------------------------|-------------------------------|-------------------------------|
-| STM200220030010000SE\_PDFS.pdf | x | STM200220030010000DDDPDFS.pdf |
-| STM200320040010000SE\_PDFA.pdf | STM200320040010000DDDPDFS.pdf | x |
-| STM200420050011000SE\_PDFS.pdf | STM200420050011000DDDPDFS.pdf | x |
-| STM200420050044000SE\_PDFS.pdf | STM200420050044000DDDPDFS.pdf | x |
-| STM200620070007000SE\_PDFS.pdf | STM200620070007000DDDPDFS.pdf | x |
-| STM200720080028000SE\_PDFS.pdf | STM200720080028000DDDPDFS.pdf | x |
-| STM200820090043000SE\_PDFS.pdf | STM200820090043000DDDPDFS.pdf | x |
-| STM200220030010000SE\_PDFS.pdf | STM200920100022000DDDPDFS.pdf | x |
-| STM\_TS007SA.pdf | STM\_TS007.pdf | x |
+| sme | nob | nno |
+| ----------------------------- | ----------------------------- | ----------------------------- |
+| STM200220030010000SE_PDFS.pdf | x | STM200220030010000DDDPDFS.pdf |
+| STM200320040010000SE_PDFA.pdf | STM200320040010000DDDPDFS.pdf | x |
+| STM200420050011000SE_PDFS.pdf | STM200420050011000DDDPDFS.pdf | x |
+| STM200420050044000SE_PDFS.pdf | STM200420050044000DDDPDFS.pdf | x |
+| STM200620070007000SE_PDFS.pdf | STM200620070007000DDDPDFS.pdf | x |
+| STM200720080028000SE_PDFS.pdf | STM200720080028000DDDPDFS.pdf | x |
+| STM200820090043000SE_PDFS.pdf | STM200820090043000DDDPDFS.pdf | x |
+| STM200220030010000SE_PDFS.pdf | STM200920100022000DDDPDFS.pdf | x |
+| STM_TS007SA.pdf | STM_TS007.pdf | x |
-HTML's
-------
+### HTML's
-### Stortingsmelding 55 2000-2001
+#### Stortingsmelding 55 2000-2001
| sme | nob |
-|-------------------|-------------------|
+| ----------------- | ----------------- |
| 1.html?id=458398 | 1.html?id=195309 |
| 2.html?id=458416 | 2.html?id=195310 |
| 3.html?id=458425 | 3.html?id=325829 |
@@ -51,67 +47,59 @@ HTML's
| 19.html?id=458644 | 19.html?id=326047 |
| 20.html?id=458645 | 20.html?id=326048 |
-NOU
-===
-
-| sme | nob |
-|----------------------------------|-------------------------------|
-| NAC\_1993\_34.pdf | x |
-| NAC\_1994\_21.pdf | NOU199419940021000DDDPDFA.pdf |
-| NAC\_2001\_35.pdf | NOU200120010035000DDDPDFA.pdf |
-| NOU2007\_13SamiskKOrtversjon.pdf | NOU2007\_13Kortversjon.pdf |
-
-Odelstingsproposisjoner
-=======================
-
-PDF's
------
-
-| sme | nob |
-|--------------------------------|-------------------------------|
-| otp200420050021000se\_pdts.pdf | OTP200420050021000DDDPDFS.pdf |
-| otp200420050038000se\_pdts.pdf | OTP200420050038000DDDPDFS.pdf |
-| OTP200620070025000SE\_PDFS.pdf | OTP200620070025000DDDPDFS.pdf |
-
-HTML's
-======
-
-Od.prp. nr. 114
----------------
-
-| sme | nno |
-|-----------------------------------------|------------------------------------------------------|
-| od-prp-nr-114\_2001-2002.html?id=170392 | otprp-nr-114-2001-2002-.html?id=170374 |
-| 1.html?id=170393 | 1.html?id=170375 |
-| 2.html?id=170397 | 2.html?id=170379 |
-| 3.html?id=170404 | 3.html?id=170386 |
-| 4.html?id=170408 | 4.html?id=170390, 5.html?id=313778, 6.html?id=313779 |
-
-Od.prp. nr. 34 2002-2003
-------------------------
-
-| sme | nno |
-|----------------------------------------|---------------------------------------|
-| od-prp-nr-34\_2002-2003.html?id=172555 | otprp-nr-34-2002-2003-.html?id=172546 |
-| 1.html?id=172556 | 1.html?id=172547 |
-| 2.html?id=172557 | 2.html?id=172548 |
-| 3.html?id=172562 | 3.html?id=172553 |
-
-Od.prp. nr. 26 2003-2004
-------------------------
-
-| sme | nob |
-|----------------------------------------|---------------------------------------|
-| od-prp-nr-26\_2003-2004.html?id=176794 | otprp-nr-26-2003-2004-.html?id=392931 |
-| 1.html?id=176795 | 1.html?id=392932 |
-| 2.html?id=176796 | 2.html?id=392933 |
-| 3.html?id=176801 | 3.html?id=392938 |
-
-Od.prp. nr. 43 2007-2008
-------------------------
+## NOU
+
+| sme | nob |
+| ------------------------------- | ----------------------------- |
+| NAC_1993_34.pdf | x |
+| NAC_1994_21.pdf | NOU199419940021000DDDPDFA.pdf |
+| NAC_2001_35.pdf | NOU200120010035000DDDPDFA.pdf |
+| NOU2007_13SamiskKOrtversjon.pdf | NOU2007_13Kortversjon.pdf |
+
+## Odelstingsproposisjoner
+
+### PDF's
+
+| sme | nob |
+| ----------------------------- | ----------------------------- |
+| otp200420050021000se_pdts.pdf | OTP200420050021000DDDPDFS.pdf |
+| otp200420050038000se_pdts.pdf | OTP200420050038000DDDPDFS.pdf |
+| OTP200620070025000SE_PDFS.pdf | OTP200620070025000DDDPDFS.pdf |
+
+## HTML's
+
+### Od.prp. nr. 114
+
+| sme | nno |
+| -------------------------------------- | ---------------------------------------------------- |
+| od-prp-nr-114_2001-2002.html?id=170392 | otprp-nr-114-2001-2002-.html?id=170374 |
+| 1.html?id=170393 | 1.html?id=170375 |
+| 2.html?id=170397 | 2.html?id=170379 |
+| 3.html?id=170404 | 3.html?id=170386 |
+| 4.html?id=170408 | 4.html?id=170390, 5.html?id=313778, 6.html?id=313779 |
+
+### Od.prp. nr. 34 2002-2003
+
+| sme | nno |
+| ------------------------------------- | ------------------------------------- |
+| od-prp-nr-34_2002-2003.html?id=172555 | otprp-nr-34-2002-2003-.html?id=172546 |
+| 1.html?id=172556 | 1.html?id=172547 |
+| 2.html?id=172557 | 2.html?id=172548 |
+| 3.html?id=172562 | 3.html?id=172553 |
+
+### Od.prp. nr. 26 2003-2004
+
+| sme | nob |
+| ------------------------------------- | ------------------------------------- |
+| od-prp-nr-26_2003-2004.html?id=176794 | otprp-nr-26-2003-2004-.html?id=392931 |
+| 1.html?id=176795 | 1.html?id=392932 |
+| 2.html?id=176796 | 2.html?id=392933 |
+| 3.html?id=176801 | 3.html?id=392938 |
+
+### Od.prp. nr. 43 2007-2008
| sme | nob |
-|---------------------------------------|---------------------------------------|
+| ------------------------------------- | ------------------------------------- |
| odprp-nr-43-2007-2008-.html?id=504689 | otprp-nr-43-2007-2008-.html?id=504667 |
| 1.html?id=504690 | 1.html?id=504668 |
| 2.html?id=504691 | 2.html?id=504669 |
diff --git a/ling/corpus_ocr_may11.md b/ling/corpus_ocr_may11.md
index 8170f5f6..62a8a5fe 100644
--- a/ling/corpus_ocr_may11.md
+++ b/ling/corpus_ocr_may11.md
@@ -1,121 +1,91 @@
-OCR reading
-===========
-
+# OCR reading
## Algorithm for dealing with OCR errors
-
### Finding these errors
-
-* Problem: There are document-specific conversion errors that result
+- Problem: There are document-specific conversion errors that result
in letters, not in garbage, errors that can be found only by linguistic
means.
-* Solution: identify the problematic files via error detection with fst
-
+- Solution: identify the problematic files via error detection with fst
**TODO**:
-
Tne Error Detection Algorithm runs as follows:
-
For each file:
+
1. Analyse the main language text morphologically
1. Count the missing ones
1. Register the missing/total ratio, and pick the worst files
Look at the worst files, and figure out how to mend them, or move them,
- e.g. to an OCR gold standard
+e.g. to an OCR gold standard
-
-
-
### Results of finding errors for North Sami
-
-Here is a list of errors per file in each folder in the admin directory. For each file we list *error/total ratio - total number of words - words not recognized - filename*, and we sort the file list according to error/total ratio:
-
-* [admin/depts/others](corpus_errors_admin_depts_others.txt)
-* [admin/guovda](corpus_errors_admin_guovda.txt)
-* [admin/others](corpus_errors_admin_others.txt)
-* [admin/regjering](corpus_errors_admin_regjering.txt)
-* [admin/sd/others](corpus_errors_admin_sd_others.txt)
-* [admin/sd/samediggi](corpus_errors_admin_sd_samediggi.txt)
-* [sma corpus errors](corpus_errors_sma.txt)
-* [sma corpus error analysis](corpus_errors_sma_analysis.txt)
-* [sme corpus errors analysis](corpus_errors_analysis.txt)
-* [sme corpus errors admin/](corpus_errors_sme_admin.txt)
-* [sme corpus errors analysis](corpus_errors_sme_analysis.txt)
-* [sme corpus errors guovda/](corpus_errors_sme_guovda.txt)
-* [sme corpus errors regjering/](corpus_errors_sme_regjering.txt)
-* [smj corpus errors](corpus_errors_smj.txt)
-* [smj corpus error analysis](corpus_errors_smj_analysis.txt)
-
-
-
-
-
+Here is a list of errors per file in each folder in the admin directory. For each file we list _error/total ratio - total number of words - words not recognized - filename_, and we sort the file list according to error/total ratio:
+
+- [admin/depts/others](corpus_errors_admin_depts_others.txt)
+- [admin/guovda](corpus_errors_admin_guovda.txt)
+- [admin/others](corpus_errors_admin_others.txt)
+- [admin/regjering](corpus_errors_admin_regjering.txt)
+- [admin/sd/others](corpus_errors_admin_sd_others.txt)
+- [admin/sd/samediggi](corpus_errors_admin_sd_samediggi.txt)
+- [sma corpus errors](corpus_errors_sma.txt)
+- [sma corpus error analysis](corpus_errors_sma_analysis.txt)
+- [sme corpus errors analysis](corpus_errors_analysis.txt)
+- [sme corpus errors admin/](corpus_errors_sme_admin.txt)
+- [sme corpus errors analysis](corpus_errors_sme_analysis.txt)
+- [sme corpus errors guovda/](corpus_errors_sme_guovda.txt)
+- [sme corpus errors regjering/](corpus_errors_sme_regjering.txt)
+- [smj corpus errors](corpus_errors_smj.txt)
+- [smj corpus error analysis](corpus_errors_smj_analysis.txt)
A list of error analyses can be found from [corpus error analysis](corpus_errors_analysis.txt).
-
Error typology (summarising the corpus error analysis):
-
-* Conversion errors
- * ==> Improve conversion
-* Typing errors
- * ==> Add to typos.txt, evt. move to typos gold corpus
-* Linguistic spelling errors
- * ==> Add to typos.txt, evt. move to typos gold corpus
-* Scanning errors
- * ==> Analyse the scanning errors and add search-replace to xsl file
-* Language recognition errors
- * ==> Check whether the xsl file lists the relevant languages
- * ==> Improve language rec module
-* Numbers not recognised
- * ==> Improve fst
-* Unknown words (bad fst)
- * ==> Improve fst
-* Corrupted original
- * ==> Consider removing it
-
+- Conversion errors
+ - ==> Improve conversion
+- Typing errors
+ - ==> Add to typos.txt, evt. move to typos gold corpus
+- Linguistic spelling errors
+ - ==> Add to typos.txt, evt. move to typos gold corpus
+- Scanning errors
+ - ==> Analyse the scanning errors and add search-replace to xsl file
+- Language recognition errors
+ - ==> Check whether the xsl file lists the relevant languages
+ - ==> Improve language rec module
+- Numbers not recognised
+ - ==> Improve fst
+- Unknown words (bad fst)
+ - ==> Improve fst
+- Corrupted original
+ - ==> Consider removing it
**TODO:**
-* Improve conversion according to error type, as sketched above
-
-
-
+- Improve conversion according to error type, as sketched above
### Results of finding errors for South Sami
-
-* [April 20 Analysis of sma corpus|corpus_errors_sma.txt], and [breakdown of the error types](corpus_errors_sma_analysis.txt).
-
-
-
+- [April 20 Analysis of sma corpus|corpus_errors_sma.txt], and [breakdown of the error types](corpus_errors_sma_analysis.txt).
**TODO:**
-* Sma improvement of the test results above
+- Sma improvement of the test results above
-
### Finding catalogue errors:
-
List all files in langX-catalogue with more non-langX content than
langX-content.
+**TODO:**
-**TODO:**
-
-* Still not done.
-
+- Still not done.
### Correcting OCR errors
-
-Develop algorithms for automatic correction of OCR errors. This
+Develop algorithms for automatic correction of OCR errors. This
work must be done separately for each language.
diff --git a/ling/corpus_parallel_maintenance.md b/ling/corpus_parallel_maintenance.md
index a7c42c1b..83e65d9f 100644
--- a/ling/corpus_parallel_maintenance.md
+++ b/ling/corpus_parallel_maintenance.md
@@ -1,12 +1,11 @@
-Maintenance of parallel corpus
-==============
+# Maintenance of parallel corpus
+
Many documents are parallel with the parallel content in the same file.
Other documents are simply placed in the wrong catalogues. Algorithm
to fix this:
-
1. For each file, count the number of words
1. For each file, count the number of words marked with the language of
- the catalogue
+ the catalogue
1. Estimate the ratio
1. Pick the files with a bad ratio, and investigate them. Split and reallocate.
diff --git a/ling/corpus_plan.md b/ling/corpus_plan.md
index 9bea65b6..0f24e46c 100644
--- a/ling/corpus_plan.md
+++ b/ling/corpus_plan.md
@@ -1,22 +1,19 @@
-Corpus plan
-===========
-
-# Introduction
+# Corpus plan
+## Introduction
This document describes corpus work until september 2004, as found in
the catalogue gt/cwb/. In 2005 we have made a separate corpus. This
should be documented as well.
-# Implementation plan for corpus interfaces
-
+## Implementation plan for corpus interfaces
The document is partly based on the discussion with the people in
Textlaboratoriet ([the memo](../admin/memos/oslo-2004-5.html)), and
discussions in [Saletek seminar, July
2004](http://www.ling.helsinki.fi/uhlcs/saletek/jokkmokk.shtml).
-## Overview and introduction
+### Overview and introduction
The main goal of the corpus subproject is to build an extensive and
versatile source of text materials for Saami languages. The main goals
@@ -40,7 +37,7 @@ and the query system.
All the material concerning the corpus project is currently stored in
the directory `gt/cwb/` under cvs.
-## The corpus database
+### The corpus database
The morphologically analyzed texts are stored in XML-format, which forms
the corpus database. The XML-format is used as the base format for
@@ -103,7 +100,7 @@ In this part, we rely to the help of
[Textlaboratoriet](http://www.hf.uio.no/tekstlab/) in the University of
Oslo.
-### Cleaning up the original text
+#### Cleaning up the original text
The cleaning of the corpus documents involves removing all the
formatting which is not relevant for the corpus database or
@@ -127,7 +124,7 @@ The information of wvWare can be found from packages' man page:
$ man wvWare
-### XML-format of the corpus files
+#### XML-format of the corpus files
The XML format of the analyzed text is basically the following:
@@ -140,24 +137,24 @@ The XML format of the analyzed text is basically the following:
Optionally, one can
-- enclose several texts in a corpus-tag
+- enclose several texts in a corpus-tag
+
+- use a paragraph-tag enclosing several sentence-s
-- use a paragraph-tag enclosing several sentence-s
+- use more attibutes:
-- use more attibutes:
- - on all elements: type and language
- - on tokens: morphology, syntax etc.
+ - on all elements: type and language
+ - on tokens: morphology, syntax etc.
-- have several readings for each token:
+- have several readings for each token:
-
-
-
-
-
+
+
+
+
There is a first version of the dtd corpus.dtd for the format. In
-addition, there is a file sme\_tagset.ent which contains the names of
+addition, there is a file sme_tagset.ent which contains the names of
the tag classes. This is supposed to make the dtd more flexible, since
the tag classes may change among languages.
@@ -171,7 +168,7 @@ be a good idea to install a separate xml-processor as well. Apache's
[Xerces](http://xml.apache.org/) seems to be a good and widely used tool
for xml parsing and generation.
-### Meta information
+#### Meta information
The structural information is encoded in XML-format, following for
example the CES standard. Then there would be three different categories
@@ -195,7 +192,7 @@ header information may also be consisted as a string in one attribute.
The exact format of the corpus header files is not yet planned.
-### Transforming the corpus files to IMS-format
+#### Transforming the corpus files to IMS-format
The "Corpus administrator's Manual" describes in detail, how the text
corpus is transformed to the internal representation used by the IMS
@@ -206,11 +203,11 @@ IMS have to be developed.
There will be conversion scripts from XML-format to TEI and IMS corpus
workbench, provided by Textlaboratory.
-### The web-interface
+#### The web-interface
The web-interface will be provided by Textlaboratory.
-## Files and directories
+### Files and directories
The corpus files themselves will be placed to `/usr/local/share/corp/`
for now. The subdirectory `doc` contains the original texts in their
@@ -235,7 +232,7 @@ stored to repository may grow. The size of ims-format corpus can be some
10-50 times bigger than the original raw text, depending on the amount
of tags (the number is just a hasty estimate).
-### The CWB installation and directories
+#### The CWB installation and directories
The version of the software is 3.0 and the installed archive name was
`cwb-2.2.b72-i386-linux.tar.gz`. The up-to-date information was
@@ -253,40 +250,40 @@ in one place, perhaps in the same place as the corporal, in directory
`/corpora/registry`. The environment variable `CORPUS_REGISTRY` has to
be set.
-## Information in the corpus
+### Information in the corpus
-### Morphological information in the corpus
+#### Morphological information in the corpus
The corpus contains tokens (words) and other positional attributes such
as part-of-speech tags. The tags are arranged one in each column. The
columns are separated by tab.
-#### Corpus tags
+##### Corpus tags
There are the following tag categories:
-- Part of speech: N A V Adv Pron CS CC Adp Po Pr Interj Pcle Num
-- Number or Person/Number: Sg Pl Sg1 Sg2 Sg3 Du1 Du2 Du3 Pl1 Pl2 Pl3
- ConNeg
-- Case: Ess Nom Gen Acc Ill Loc Com
-- Possessive suffix: PxSg1 PxSg2 PxSg3 PxDu1 PxDu2 PxDu3 PxPl1 PxPl2
- PxPl3
-- Clitic: Qst Foc
-- Grade: Comp Superl
-- Attribute: Attr
-- Derivation (We must consider whether to use these..): Pass h upmi
- ...
-- Polarity: Neg
-- Mood: Ind Pot Cond Imprt ImprtII
-- Tense: Prs Prt
-- Nominal verb form: Inf Act PrsPrc PrfPrc VGen VAbess Ger ConNeg
- ConNegII
-- Pronoun type: Pers Dem Interr Refl Recipr Rel Indef
-- Other: PUNCT CLB ABBR ACR
-- Punctuation type: LEFT RIGHT
-- Num type: Ord Card
-
-### Structural information
+- Part of speech: N A V Adv Pron CS CC Adp Po Pr Interj Pcle Num
+- Number or Person/Number: Sg Pl Sg1 Sg2 Sg3 Du1 Du2 Du3 Pl1 Pl2 Pl3
+ ConNeg
+- Case: Ess Nom Gen Acc Ill Loc Com
+- Possessive suffix: PxSg1 PxSg2 PxSg3 PxDu1 PxDu2 PxDu3 PxPl1 PxPl2
+ PxPl3
+- Clitic: Qst Foc
+- Grade: Comp Superl
+- Attribute: Attr
+- Derivation (We must consider whether to use these..): Pass h upmi
+ ...
+- Polarity: Neg
+- Mood: Ind Pot Cond Imprt ImprtII
+- Tense: Prs Prt
+- Nominal verb form: Inf Act PrsPrc PrfPrc VGen VAbess Ger ConNeg
+ ConNegII
+- Pronoun type: Pers Dem Interr Refl Recipr Rel Indef
+- Other: PUNCT CLB ABBR ACR
+- Punctuation type: LEFT RIGHT
+- Num type: Ord Card
+
+#### Structural information
It is possible to mark for example the beginning and end of a sentence
to the corpus file by using SGML-like markers. Whether we should use
@@ -297,19 +294,19 @@ either parsing or corpus processing.
Large units of discourse information are:
-- headings
-- lists
-- chapters
-- paragraphs?
-- footnotes etc.
+- headings
+- lists
+- chapters
+- paragraphs?
+- footnotes etc.
Smaller units:
-- sentences
-- abbreviations
-- dates
-- quotations
-- names
+- sentences
+- abbreviations
+- dates
+- quotations
+- names
We have to find out what kind of information it is possible to extract
from diffeent types of documents, and how much of the structural
@@ -324,7 +321,7 @@ bulletpoint lists is clearly a valuable resource, if we can write rules
that rely on such information (demand finite verbs form sentences, not
from titles, parenthesis fragments or bulletpoint items).
-## IMS Corpus Workbench demo
+### IMS Corpus Workbench demo
IMS Corpus Workbench is now installed to victorio and can be tested with
two demo corpuses. There is English demo corpus which consists of
@@ -352,7 +349,7 @@ corpus, type
To make a query, follow the instructions in the CQP Tutorial (path:
`/usr/local/cwb/doc/CQP-Tutorial.2up.pdf`).
-### Saami demo corpus
+#### Saami demo corpus
There is one short Saami demo corpus with limited tags, stme1029. To
make queries to it, type
@@ -374,9 +371,9 @@ directory `/usr/local/cwb/demo/stme1029` for example. The coversion from
CG2 output to word-list format is done automatically by using script
`convert2cwb`.
-## Some not so relevant documentation
+### Some not so relevant documentation
-### XCES
+#### XCES
XCES is an XML-version of CES. It allows the usage of XML-tools to
corpus files. The tag names are the same as in CES. In practise, a
@@ -390,7 +387,7 @@ problem som way, so I will trust that it is possible for us too. Next I
will go through INL's header specifications and see how they should be
modified for the Sami corpora.
-#### Tag specifications
+##### Tag specifications
INL have different kind of headers for different type of texts. We have
to see if that is necessary. We have different types of corpora, books,
@@ -410,51 +407,51 @@ bokerSA.txt **cesDoc** starts the document. It contains document id
I describe only the content elements.
-- **h.title** contains the name of the document.
-- **respStmt** is the information about the person or institute of the
- intellectual content of the text.
-- **wordCount** is the count of words in the text,
-- **byteCount** the number of bytes (the text together with its
- markup).
-- **extNote** is for additional information
-- **distributor** gives the name of institution who distributes the
- text or corpus.
-- **pubAddress** postal address of the distributor
-- **eAddress** the distributor's electronic address
-- **availability** the restrictions to the use or distribution, e.g.
- copyright
-- **pubDate** the publication date
-- If the text is published not as an independent publication, the
- following information is inside **analytic**-tag. Otherwise the tag
- **monogr** is used.
- - **h.title** full title
- - **h.author** name of the author
- - **edition** edition
-- **publisher**
-- **pubDate**
-- **pubPlace**
-- **idno** ISBN or equivalent
-- **biblScope** The scope of the bibliographic reference, e.g. page
- numbers
-- **biblNote** note
+- **h.title** contains the name of the document.
+- **respStmt** is the information about the person or institute of the
+ intellectual content of the text.
+- **wordCount** is the count of words in the text,
+- **byteCount** the number of bytes (the text together with its
+ markup).
+- **extNote** is for additional information
+- **distributor** gives the name of institution who distributes the
+ text or corpus.
+- **pubAddress** postal address of the distributor
+- **eAddress** the distributor's electronic address
+- **availability** the restrictions to the use or distribution, e.g.
+ copyright
+- **pubDate** the publication date
+- If the text is published not as an independent publication, the
+ following information is inside **analytic**-tag. Otherwise the tag
+ **monogr** is used.
+ - **h.title** full title
+ - **h.author** name of the author
+ - **edition** edition
+- **publisher**
+- **pubDate**
+- **pubPlace**
+- **idno** ISBN or equivalent
+- **biblScope** The scope of the bibliographic reference, e.g. page
+ numbers
+- **biblNote** note
The encoding description **encodingDesc** describes the relation between
the text and its original source.
-- **taxonomy** The classification taxonomy
-- **category** The category in the taxonomy
+- **taxonomy** The classification taxonomy
+- **category** The category in the taxonomy
The Profile description **profileDesc**, contains the language etc.
-- **language** The id of the language. If more than one language is
- specified, it is possible to use < foreign xml:lang=nb>
- bokm� word </foreign> tags in the text.
-- **writingSystem** specifies the writing system, there can be more
- than one specified as well.
-- **textClass** text classification scheme and keywords.
-- **catRef** defines the classification taxonomy with reference to
- **taxonomy** and the category in the taxonomy, with reference to
- **category**.
-- **translation** information about possible translations of the text.
-- **trans.loc** location of the translations.
-- **translator** name of the translator
+- **language** The id of the language. If more than one language is
+ specified, it is possible to use < foreign xml:lang=nb>
+ bokm� word </foreign> tags in the text.
+- **writingSystem** specifies the writing system, there can be more
+ than one specified as well.
+- **textClass** text classification scheme and keywords.
+- **catRef** defines the classification taxonomy with reference to
+ **taxonomy** and the category in the taxonomy, with reference to
+ **category**.
+- **translation** information about possible translations of the text.
+- **trans.loc** location of the translations.
+- **translator** name of the translator
diff --git a/ling/corpus_repositories.md b/ling/corpus_repositories.md
index 20ba4acf..9e84a9b3 100644
--- a/ling/corpus_repositories.md
+++ b/ling/corpus_repositories.md
@@ -1,23 +1,20 @@
-Corpus repositories
-===================
+# Corpus repositories
The corpus is divided in a free part, where texts that we can
redistribute are gathered (essentially, texts in the public domain), and
a bound part, where we gather texts that we can't redistribute.
-For each language, there are two corpora: One with open content and one with closed (closed due to copyright reasons). Both corpora are in two, directories, one for original files (*.docx, .pdf, .html, ...*) and one for converted text files. Search for the corpora under **Repositories** in [github.com/giellalt/](https://github.com/giellalt/). In the repository search field, write
+For each language, there are two corpora: One with open content and one with closed (closed due to copyright reasons). Both corpora are in two, directories, one for original files (_.docx, .pdf, .html, ..._) and one for converted text files. Search for the corpora under **Repositories** in [github.com/giellalt/](https://github.com/giellalt/). In the repository search field, write
```
corpus-xxx
```
-where **xxx** is the ISO code of your language
+where **xxx** is the ISO code of your language
+## The directory structure
-# The directory structure
-
-
-Inside each of the corpus directories (both *corpus-xxx* and *corpus-xxx-orig*) the structure outlined below is
+Inside each of the corpus directories (both _corpus-xxx_ and _corpus-xxx-orig_) the structure outlined below is
used.
admin/
@@ -39,30 +36,26 @@ used.
YLE
other
science/
-
-
These 7 overarching genres should be understood as follows:
-- **admin/ :**
- Texts from administrative bodies
-- **bible/ :**
- The bible and other religious texts.
-- **facta/ :**
- Factual texts, e.g. teaching books, descriptive texts. Sakprosa.
-- **ficti/ :**
- Prose, lyrics, plays and other fiction, political text.
- Skjønnlitteratur.
-- **laws/ :**
- Laws and similar juridical texts.
-- **news/ :**
- Newspaper text
-- **science/:**
- Scientific articles
-
-
-
-## Goldstandard corpus files
+- **admin/ :**
+ Texts from administrative bodies
+- **bible/ :**
+ The bible and other religious texts.
+- **facta/ :**
+ Factual texts, e.g. teaching books, descriptive texts. Sakprosa.
+- **ficti/ :**
+ Prose, lyrics, plays and other fiction, political text.
+ Skjønnlitteratur.
+- **laws/ :**
+ Laws and similar juridical texts.
+- **news/ :**
+ Newspaper text
+- **science/:**
+ Scientific articles
+
+### Goldstandard corpus files
Some parts of our corpus are used as test data for different purposes,
presently mainly for proofing tools. These files have additional markup
@@ -70,7 +63,7 @@ to add info about linguistic errors of different types and their
corrections. These files are located within a directory named
goldstandard, which has the same internal structure as shown above:
- orig/ # same orig/ as above
+ orig/ ## same orig/ as above
goldstandard/
orig/
sma/
@@ -85,9 +78,8 @@ before you add it to the repository, and make sure it is added in only
one location. A goldstandard document can always be demoted to a regular
corpus document, but the other way around is not possible.
-# Adding content
+## Adding content
To add all files found in a directory to a working copy of a corpus, you
can use the
-[add\_files\_to\_corpus](CorpusTools.html#add_files_to_corpus) program.
-
+[add_files_to_corpus](CorpusTools.html#add_files_to_corpus) program.
diff --git a/ling/corpus_xsl_file.md b/ling/corpus_xsl_file.md
index ff9a518e..4993686b 100644
--- a/ling/corpus_xsl_file.md
+++ b/ling/corpus_xsl_file.md
@@ -1,22 +1,21 @@
-XSL-files
-=========
+# XSL-files
The purpose of the xsl files is:
-- Add metadata about the original file. Examples of this is info about
- the author, publishing year, which language it has been translated
- from, who added the file to the repository, pointers to parallel
- files, etc.
-- Add info about which parts of a document to exclude or include. This
- especially handy for pdf documents, which is a fickly file format to
- convert to xml. There is also support to do the same for clean text,
- html and epub files.
-- Add processing instructions, such as the languages to which the text
- is categorized or some information of the text layout.
+- Add metadata about the original file. Examples of this is info about
+ the author, publishing year, which language it has been translated
+ from, who added the file to the repository, pointers to parallel
+ files, etc.
+- Add info about which parts of a document to exclude or include. This
+ especially handy for pdf documents, which is a fickly file format to
+ convert to xml. There is also support to do the same for clean text,
+ html and epub files.
+- Add processing instructions, such as the languages to which the text
+ is categorized or some information of the text layout.
The xsl-file is created when a file is added to a [corpus
repository](corpus_repositories.html) using
-[add\_files\_to\_corpus](/ling/CorpusTools.html#add_files_to_corpus).
+[add_files_to_corpus](/ling/CorpusTools.html#add_files_to_corpus).
The xsl-file is stored to the same directory as the original document,
with the same name as the original document, and suffix `.xsl`.
@@ -29,7 +28,7 @@ The variables are defined by expressions as the following:
-
+
The information is updated by adding the new value to the
"select"-field, inside single quotes. Most of the fields, such as the
diff --git a/ling/corpusindex.md b/ling/corpusindex.md
index 7cc25e54..32226cac 100644
--- a/ling/corpusindex.md
+++ b/ling/corpusindex.md
@@ -1,52 +1,49 @@
-The corpus page
-===============
+# The corpus page
-This page contains documentation on *corpus work* and on work on the graphic interfact *Korp*. At the end of the page there is reference to meeting memos.
+This page contains documentation on _corpus work_ and on work on the graphic interfact _Korp_. At the end of the page there is reference to meeting memos.
-# Corpus work
+## Corpus work
-## Overview and important links
+### Overview and important links
-- [Introduction to working on corpora at Giellalt](corpus_intro.html)
-- [The corpus tools](https://giellalt.github.io/CorpusTools/) (tools to collect corpora and metadata)
-- [Corpus repositories](corpus_repositories.html) (where the corpus files are stored)
-- [Metadata files](corpus_xsl_file.html) (their structure)
+- [Introduction to working on corpora at Giellalt](corpus_intro.html)
+- [The corpus tools](https://giellalt.github.io/CorpusTools/) (tools to collect corpora and metadata)
+- [Corpus repositories](corpus_repositories.html) (where the corpus files are stored)
+- [Metadata files](corpus_xsl_file.html) (their structure)
-
-## Corpus collection and maintenance
+### Corpus collection and maintenance
-- [Korpussamlerens 1-2-3 (juridisk intro for kontakt med
- tekstprodusenter)](../infra/corpus_collectors_howto.html)
-- [Corpus collector's manual (how to work)](corpus_conversion.html)
-- [Corpus maintenance](corpus_maintenance.html)
-- [Corpus conversion (technical)](corpus_conversion_tech.html)
-- [Language recognition](langrec.html)
-- [Unicode Normalisation (how to fix decomposed Sami
- letters)](UnicodeNormalisation.html)
-- [Historical corpora](hist/index.html)
-- [OCR](../tools/ocr.html)
-- [Wikipedia as a Corpus](WikipediaAsCorpus.html)
+- [Korpussamlerens 1-2-3 (juridisk intro for kontakt med
+ tekstprodusenter)](../infra/corpus_collectors_howto.html)
+- [Corpus collector's manual (how to work)](corpus_conversion.html)
+- [Corpus maintenance](corpus_maintenance.html)
+- [Corpus conversion (technical)](corpus_conversion_tech.html)
+- [Language recognition](langrec.html)
+- [Unicode Normalisation (how to fix decomposed Sami
+ letters)](UnicodeNormalisation.html)
+- [Historical corpora](hist/index.html)
+- [OCR](../tools/ocr.html)
+- [Wikipedia as a Corpus](WikipediaAsCorpus.html)
-## Sentence alignment
-
-- [Overview](../tools/tca2.html)
-- [Workflow](../tools/Bargovuohki.html)
-- [Improving PDF-files](../ling/corpus_improve_alignment.html)
-- [TCA2 parameters](../tools/TCA2_parameters.html)
-- [Graph. interface](../ling/corpus_analyze.html)
-- [Alternatives](../tools/other_aligners.html)
+### Sentence alignment
+- [Overview](../tools/tca2.html)
+- [Workflow](../tools/Bargovuohki.html)
+- [Improving PDF-files](../ling/corpus_improve_alignment.html)
+- [TCA2 parameters](../tools/TCA2_parameters.html)
+- [Graph. interface](../ling/corpus_analyze.html)
+- [Alternatives](../tools/other_aligners.html)
-# The graphical interface *Korp*
+## The graphical interface _Korp_
- [Technical documentation for Korp](../infra/korp/index.html)
- [For linguists, learners and lexicographers: how to **use** the Korp interface](../lang/common/Korp_usage.html)
+## Spoken corpora
-# Spoken corpora
- [Overview](spoken/SpokenCorpora.html)
- [LIA](spoken/LIA.html)
- [ELAN](freiburg/freiburg.html)
@@ -60,8 +57,7 @@ This page contains documentation on *corpus work* and on work on the graphic int
- [Toolbox](freiburg/Toolbox.html)
- [Transcription](freiburg/Transcription.html)
-# Meetings on corpus issues
-
+## Meetings on corpus issues
2017: [06-21](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2017-06-21.html) //
[07-04](https://divvungiellatekno.github.io/giellalt.uit.no/admin/corpus/Meeting_2017-07-04.html)
diff --git a/ling/correct-dir.md b/ling/correct-dir.md
index 943c583c..8d782f32 100644
--- a/ling/correct-dir.md
+++ b/ling/correct-dir.md
@@ -1,5 +1,4 @@
-The <Correct!> corpora
-============================
+# The <Correct!> corpora
The catalogue gt/sme/corp/ now contains a separate directory correct/.
diff --git a/ling/docu-case-allcaps.md b/ling/docu-case-allcaps.md
index 4e09873b..39343019 100644
--- a/ling/docu-case-allcaps.md
+++ b/ling/docu-case-allcaps.md
@@ -1,5 +1,4 @@
-Capitalisation
-==============
+# Capitalisation
Here, we document two processes, one accepting initial capitalisation
("Dat" as well as "dat"), and the other one full capitalisation ("DAT"
@@ -8,8 +7,7 @@ but not "DAt")
1. [Initial capitalization](#ini)
2. [Capitalization of whole words](#all)
-Initial capitalization
-----------------------
+## Initial capitalization
This is what the book says:
@@ -24,8 +22,7 @@ caseconv.fst using xfst. As a result all initial caps are downcased, but
upon generation all words are given an alternative reading with an
initial capital letter. This is not what we want.
-Capitalization of whole words
------------------------------
+## Capitalization of whole words
The key file is `allcaps.regex`. It is modelled after the book, and
works in the following way:
@@ -35,12 +32,12 @@ allacaps is defined as the set of relations 'a (->) A' etc. for all
small/capital pairs that occur in the context '.\#. upper\* \_ upper\*
.\#.', i.e. between strings of upper case letters only.
-The resulting binary files *allcaps.fst* is compiled by the Makefile. In
+The resulting binary files _allcaps.fst_ is compiled by the Makefile. In
principle, the parser sme.fst could have been composed with allcaps.fst
into a single transducer (sme.fst .o. allcaps.fst), but this is not
done, since the resulting transducer would have been very large indeed
(cf. discussion on this issue in the book). Instead, the issue is
-handled in a *lookup script file*. Ath present, this file looks as
+handled in a _lookup script file_. Ath present, this file looks as
follows (cf. the discussion on lookup script files in the book):
Er dette rett?
@@ -73,7 +70,7 @@ see
It may not solve your problem - but please read it first: towards the
end, there is reference to environment variables like
-setenv LOOKUP\_SCRIPT\_BASE ...
+setenv LOOKUP_SCRIPT_BASE ...
If it is not of enough then the interface should be improved. While it
is not a complicated matter, we are short of able people now so you may
diff --git a/ling/docu-disambiguation.md b/ling/docu-disambiguation.md
index 836f5c24..9d813775 100644
--- a/ling/docu-disambiguation.md
+++ b/ling/docu-disambiguation.md
@@ -1,5 +1,4 @@
-Our disambiguator: vislcg3
-==========================
+# Our disambiguator: vislcg3
The project uses Eckhard Bick and Tino Dideriksens **vislcg3**, a
compiler for constraint grammar, a formalism originally developed by
@@ -15,8 +14,7 @@ individual language](../lang/index.html). For each language, the **What
is This** link will indicate how much work has been done on the language
in question.
-The three generations of constraint grammar parsers
----------------------------------------------------
+## The three generations of constraint grammar parsers
1. **The original CG formalism** was developed by Fred Karlsson, at
[Dept. of Linguistics in Helsinki](http://www.ling.helsinki.fi), it
@@ -31,22 +29,20 @@ The three generations of constraint grammar parsers
3. **vislcg3**, the third generation of CG compilers, is being
developed at .
-Links to old documentation
---------------------------
-
-- [Differences between vislcg and CG-2](/tools/vislcg.html)
-
-A Constraint Grammar bibliography
----------------------------------
-
-- [English Wikipedia on constraint
- grammar](https://en.wikipedia.org/wiki/Constraint_grammar)
-- [Didriksen, Tino 2007: Constraint Grammar Manual. 3rd version of the
- CG formalism variant.](%20http://beta.visl.sdu.dk/cg3/single/)
-- [Visl Constraint Grammar
- page](https://visl.sdu.dk/constraint_grammar.html)
-- Karlsson, Fred; Atro Voutilainen, Juha Heikkilä and Arto Anttila
- 1995: Constraint Grammar. A Language-Independent System for Parsing
- Unrestricted Text. Mouton the Gruyter.
-- Tapanainen, Pasi 1996: The Constraint Grammar Parser CG-2.
- University of Helsinki Publications No. 27.
+## Links to old documentation
+
+- [Differences between vislcg and CG-2](/tools/vislcg.html)
+
+## A Constraint Grammar bibliography
+
+- [English Wikipedia on constraint
+ grammar](https://en.wikipedia.org/wiki/Constraint_grammar)
+- [Didriksen, Tino 2007: Constraint Grammar Manual. 3rd version of the
+ CG formalism variant.](%20http://beta.visl.sdu.dk/cg3/single/)
+- [Visl Constraint Grammar
+ page](https://visl.sdu.dk/constraint_grammar.html)
+- Karlsson, Fred; Atro Voutilainen, Juha Heikkilä and Arto Anttila
+ 1995: Constraint Grammar. A Language-Independent System for Parsing
+ Unrestricted Text. Mouton the Gruyter.
+- Tapanainen, Pasi 1996: The Constraint Grammar Parser CG-2.
+ University of Helsinki Publications No. 27.
diff --git a/ling/docu-distesting.md b/ling/docu-distesting.md
index 82779f65..7c6a038a 100644
--- a/ling/docu-distesting.md
+++ b/ling/docu-distesting.md
@@ -1,17 +1,17 @@
Tokens = Number of tokens in the text
Alternative 1: Process with hfst:
- cat file.txt|hfst-tokenise tools/tokenisers/tokeniser-disamb-gt-desc.pmhfst
+ cat file.txt|hfst-tokenise tools/tokenisers/tokeniser-disamb-gt-desc.pmhfst
Alternative 2: Process with perl:
cat file.txt | preprocess --abbr=bin/abbr.txt | wc -l)
(or: "ccat -l sme" instead of "cat", if input is corpus files in our xml format)
-
+
Parses = Number of parses given (number of the following command, minus the
number of tokens)
Alternative 1: process with hfst
- cat file.txt|hfst-tokenise -g tools/tokenisers/tokeniser-disamb-gt-desc.pmhfst
+ cat file.txt|hfst-tokenise -g tools/tokenisers/tokeniser-disamb-gt-desc.pmhfst
Alternative 2: process with perl:
cat file.txt | preprocess --abbr=bin/abbr.txt | lookup -flags mbTT
diff --git a/ling/docu-foreign.md b/ling/docu-foreign.md
index fa5b8800..eca63738 100644
--- a/ling/docu-foreign.md
+++ b/ling/docu-foreign.md
@@ -1,5 +1,4 @@
-The list of foreign words
-=========================
+# The list of foreign words
Incoming text contains many foreign words. Used in isolation, as
spontanous loans, they should be delt with by a POS guesser. Text chunks
@@ -17,7 +16,7 @@ stoplist of foreign words. The list was made in the following way:
3. In addition, a file `gt/script/new-foreign.txt` was added to the
cvs, containing Non-Saami words from our corpus files.
4. Each of these files were turned into fst files. Then the union of
- the two files was made into one binary file, *foreign.fst*
+ the two files was made into one binary file, _foreign.fst_
The compilation is included in the Makefile. The source files are in the
`gt/script` catalogue, whereas the binary files are in the `gt/sme/bin`
@@ -36,6 +35,6 @@ of the stop list, are included.
The list of foreign words was cut in two because compilation time for
the whole list is very long. The intention with the split is that
-*old-foreign.txt* should be left alone. All additional words should be
+_old-foreign.txt_ should be left alone. All additional words should be
added to the shorter `new-foreign.txt` file. If this file becomes too
long, it may be transferred over to old-foreign.txt.
diff --git a/ling/docu-morph-manual.md b/ling/docu-morph-manual.md
index cbeed813..beacc226 100644
--- a/ling/docu-morph-manual.md
+++ b/ling/docu-morph-manual.md
@@ -1,35 +1,31 @@
-Setting up the environment
-==========================
+# Setting up the environment
See documentation elsewhere.
-Analysing and generating words
-------------------------------
-
+## Analysing and generating words
### Analysing one word at a time:
Write
- `hfst-lookup src/analyser-gt-desc.hfstol`
+`hfst-lookup src/analyser-gt-desc.hfstol`
then write the words that shall be analysed, one word at a time, followed by RETURN. To leave lookup mode, press `ctrl C`.
If you compiled with **xfst**, write
- `lookup src/analyser-gt-desc.xfst`
-
+`lookup src/analyser-gt-desc.xfst`
### Generating words
Write
- `hfst-lookup src/generator-gt-desc.hfstol`
+`hfst-lookup src/generator-gt-desc.hfstol`
Then write words in their dictionary forms, followed by
- grammatical information. The format is given in the table in the
- file [The grammatical
- tags](/lang/sme/docu-sme-grammartags.html)
+grammatical information. The format is given in the table in the
+file [The grammatical
+tags](/lang/sme/docu-sme-grammartags.html)
Again, to leave lookup mode, press "ctrl C".
@@ -47,36 +43,34 @@ stand in the sme/ (sma/ etc.) catalogue).
echo "sentence here" \| preprocess --abbr=bin/abbr.txt \| lookup -flags
mbTT bin/sme.fst
-Analysing files:
-----------------
+## Analysing files:
For each of the languages, stand in the **lang-XXX** directory (XXX being your language code) and write the following line:
cat filename | hfst-tokenise -cg tools/tokenisers/tokeniser-disamb-gt-desc.pmhfst | less
-
Instead of just showing the result on the screen as running text (as
above), much can be done to manipulate it. Here are some examples, all
the textstrings should replace the word "less" in the command above.
-To get a frequency list of the words that the parser does not recognize:
+To get a frequency list of the words that the parser does not recognize:
`grep '\\?' | sort | uniq -c | sort -nr | less RETURN`
To get all plural nouns and save them
- to the file "plnouns"
+to the file "plnouns"
`grep '+N+Pl' > plnouns`
To get a frequency list of the lexemes that the parser
- recognizes, note that this requires that the flag TT is turned off,
- i.e. not mentioned:
-
+recognizes, note that this requires that the flag TT is turned off,
+i.e. not mentioned:
+
`grep -v '\\?' \| cut -f2 \| sort \| uniq -c \| sort -nr \| less RETURN`
-
+
To get a frequency list of the word forms that the parser does not recognize
`grep '\\+\\?' \| sort \| uniq -c \| sort -nr \| less RETURN`
To analyse more files at the same time, write their names one after
-another after the "cat" command, e.g.
+another after the "cat" command, e.g.
`cat file1 file3 file3 \|
preprocess ...`
diff --git a/ling/docu-regexps.md b/ling/docu-regexps.md
index d90d32c2..c036a94b 100644
--- a/ling/docu-regexps.md
+++ b/ling/docu-regexps.md
@@ -1,5 +1,4 @@
-Regular expressions
-===================
+# Regular expressions
To be written...
diff --git a/ling/docu-testing.md b/ling/docu-testing.md
index ebc7828d..91d3348d 100644
--- a/ling/docu-testing.md
+++ b/ling/docu-testing.md
@@ -1,8 +1,8 @@
-Testing tools for the Sámi language technology project
-======================================================
+# Testing tools for the Sámi language technology project
1. Purpose
-----------
+
+---
To test the quality of morphological analysis and generation, that is,
whether they produce what we want or not. In effect, this will also test
@@ -10,7 +10,8 @@ the correctness of the two-level rules, although one should perform more
testing than is the case at present.
2. Intro
---------
+
+---
All the tools used are language independent, and follow a pretty simple
procedure. The needed infrastructure consist of a testing directory, and
@@ -28,7 +29,8 @@ on the command line. This will run tests for all test cases in the
testing directory.
3. Easy setup and first test
-----------------------------
+
+---
**Assumption:** Somebody else has [created the needed
infrastructure](#FullSetup) (test directory, Makefile configuration,
@@ -60,8 +62,9 @@ Do the following:
each POS, one for analysis, and one for generation). See
[below](#TestReports) for more on how to interpret the test reports.
-4. Example
-----------
+5. Example
+
+---
Below is an example from South Sami, illustrating the structure of both
a word file (the actual test case) and a code file. The example is given
@@ -128,7 +131,8 @@ All these files, both the code files and the test cases, should be added
to CVS for version control.
5. More on running tests
-------------------------
+
+---
There are two types of tests available, one where the test result is
checked against a predefined correct result, and one where the output is
@@ -149,7 +153,7 @@ the premade paradigm, and any differences are highlighted in the report
(again, see [further down](#TestReports) for details).
| | |
-|------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| ---------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Command** | **Explanation** |
| `make` or `make all` | **Default test:** Run all the tests described in the rest of this table. Presently, this is the same as typing `make all-n` (see below) (this is true for North and South Sami), as there are still no test cases for verbs or adjectives. |
| `make all-n` `make all-v` `make all-a` | To run all the noun tests (or verb tests, or adjective tests). The same as typing `make n-a.summary n-g.summary` etc. |
@@ -173,9 +177,9 @@ the testing tools will generate the paradigm for the word. But it will
not generate a test report for you, and it is up to you to tell whether
the generated paradigm makes sense or not. Hopefully it does, if not,
there are one or more errors that need to be corrected somewhere. And
-that is *your* job;)
+that is _your_ job;)
-In the directory ` testing/ `, write the command
+In the directory `testing/`, write the command
make clean
@@ -183,7 +187,7 @@ Thereafter, write
make n-paradigm WORD=johka
-if you want to generate the paradigm for the noun *johka*. Similarly, if
+if you want to generate the paradigm for the noun _johka_. Similarly, if
you want to generate paradigms for verbs or adjectives, the commands
are:
@@ -201,7 +205,8 @@ paradigm is also saved in a file named '`WORD.paradigm`', where `WORD`
is the word you generated the paradigm for.
6. How to read the test reports
--------------------------------
+
+---
There are two set of test report files: Either a test report for each
test case, or a summary of all test reports for a given POS and test
@@ -209,7 +214,7 @@ category (analysis or generation). Their file names and corresponding
content is summarised in the table below:
| | |
-|--------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| ------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `*.areports` or `*.greports` | There is one such file pair for each test case. That is, if you only want to look at the test result for one specific case, have a look at these files (`*.areports` for word form analysis tests, `*.greports` for word form generation tests). The part of the file name represented by the star (\*), is similar to the test case before the ending `.txt`. |
| `n-a.summary` or `n-g.summary`, similar for verbs (`v-`) and adjectives (`a-`) | The summary reports are two files for each POS, containing **all** `*.areports` or `*.greports` for that POS. That is, if you want to look at all test results for noun word form analysis tests, look at `n-a.summary`. And similar for the other POSes. |
@@ -259,19 +264,19 @@ gives the expected output (taken from the `*.facit` file), and the
The difference indicators in the middle column should be read as
follows:
-- **<:**
- This line is only found in the test output, not in the facit file,
- there is likely more forms than expected in the output
-- **>:**
- This line is only found in the facit file, not in the test output.
- It is likely that there are some missing forms in the output.
-- **\|:**
- This line was found in both the facit file and the test output, but
- with differences between the two. It is likely that there are errors
- in the codes or in the baseform of the output.
-- **(nothing):**
- This line was found in both the facit and the test output, and they
- are identical. This line is correct.
+- **<:**
+ This line is only found in the test output, not in the facit file,
+ there is likely more forms than expected in the output
+- **>:**
+ This line is only found in the facit file, not in the test output.
+ It is likely that there are some missing forms in the output.
+- **\|:**
+ This line was found in both the facit file and the test output, but
+ with differences between the two. It is likely that there are errors
+ in the codes or in the baseform of the output.
+- **(nothing):**
+ This line was found in both the facit and the test output, and they
+ are identical. This line is correct.
In the example above, there is one instance of '\|', on the line
`aaltoe+N+Kom | aaltoe+N+Pl+Kom`. As you can see, the `+Pl` tag is
@@ -370,7 +375,8 @@ thus be able to improve the linguistic tools in a systematic and
organised way.
7. Full setup
--------------
+
+---
Follow the procedure below for the first-time setup (this "full setup"
has been conducted for all three languages, in the cvs, and as a new
@@ -398,8 +404,9 @@ user you can just go to the testing directory):
7. Continue as in ["Easy setup"](#EasySetup).
-8. How to add new test cases
-----------------------------
+8. How to add new test cases
+
+---
To create new test cases, simply make a new file, and type in all
inflections of the word you want to test, as [illustrated
@@ -407,7 +414,8 @@ above](#Example). Be consistent when naming the test case files, though,
it makes it easier in the longer run (see next for one idea).
9. File names for test cases
-----------------------------
+
+---
The test case files, one for each test, should be named with certain
restrictions. They need to end in `.txt`, they should start with one of
@@ -430,7 +438,8 @@ test cases themselves. For other Saami languages there can (and probably
should) be different naming schemes, but this is one idea.
10. How to clean up the test directory
---------------------------------------
+
+---
The testing procedure creates a lot of files, and even though many of
them are deleted upon completion of the test run (only the reports are
@@ -442,7 +451,8 @@ source files (the ones from which everything else is made), type:
It is a good habit to allways clean before running a test.
11. How to test irregular words
--------------------------------
+
+---
Routines for irregular words must be made, and then documented
@@ -455,29 +465,30 @@ the picture is more complex. We need testing routines that are flexible
enough to test the closed parts of speech.
12. Technical details
----------------------
+
+---
The intermediate files created are the following:
-- `*.testbase` - morphological codes and the test case merged into one
- file, from which both the actual tests and the expected results (the
- facit) are made.
+- `*.testbase` - morphological codes and the test case merged into one
+ file, from which both the actual tests and the expected results (the
+ facit) are made.
-- `*.atest` - test file for word form analysis
+- `*.atest` - test file for word form analysis
-- `*.gtest` - test file for word form generation
+- `*.gtest` - test file for word form generation
-- `*.afacit` - facit file for analysis test
+- `*.afacit` - facit file for analysis test
-- `*.gfacit` - facit file for generation test
+- `*.gfacit` - facit file for generation test
-- `*.aresult` - result of analysis test
+- `*.aresult` - result of analysis test
-- `*.gresult` - result of generation test
+- `*.gresult` - result of generation test
-- `*-test-script` - a script made on the fly for running the actual
- tests in xfst (one of the Xerox tools). Placed in /tmp/, and deleted
- immediately after the test run.
+- `*-test-script` - a script made on the fly for running the actual
+ tests in xfst (one of the Xerox tools). Placed in /tmp/, and deleted
+ immediately after the test run.
All the above files are deleted when the testing is complete.
diff --git a/ling/file_alignment_plan.md b/ling/file_alignment_plan.md
index 5b9343f1..3a933858 100644
--- a/ling/file_alignment_plan.md
+++ b/ling/file_alignment_plan.md
@@ -1,13 +1,10 @@
-
-File alignment plan
-=====
+# File alignment plan
#Intro
-
Plan for aligner and analysis work
- Goal: Send analysed texts to Oslo on order
+ Goal: Send analysed texts to Oslo on order
@@ -32,7 +29,7 @@ File alignment plan
To be sent:
file.nob.xml.sent (analysed in oslo, for the time being, at least)
file.sme.xml.sent.analyzed
- file.sme.xml_file.nob.xml.xml
+ file.sme.xml_file.nob.xml.xml
cor = result of the alignment
@@ -45,7 +42,7 @@ File alignment plan
Example:
dc_1_01.doc.xml_sp_1_01.doc.xml.xml
dc_1_01.doc.xml_cor.sent
- dc_1_01.doc.xml_new.txt
+ dc_1_01.doc.xml_new.txt
parallel/
namm-namn/
@@ -53,9 +50,9 @@ File alignment plan
namm.sent
namm.sent.analyzed
namm.xml_namn.xml.xml
- ...
-
+ ...
+
1. Trond: Find texts, mark the xsl files (cf. dc_ files for model), tell Saara
2. Saara: make .sent and .sent.analyzed
- Børre: make namm_namn.xml
+ Børre: make namm_namn.xml
3. Trond: Send files to Lars
diff --git a/ling/freiburg/ELAN.md b/ling/freiburg/ELAN.md
index c9523927..a8dbbb82 100644
--- a/ling/freiburg/ELAN.md
+++ b/ling/freiburg/ELAN.md
@@ -1,66 +1,48 @@
# ELAN
-This page documents conventions, standards and relevant workflows used for ELAN annotations created by the [Freiburg-Tromsø Speech Corpora](freiburg.html).
-
-## Introduction
+This page documents conventions, standards and relevant workflows used for ELAN annotations created by the [Freiburg-Tromsø Speech Corpora](freiburg.html).
+### Introduction
ELAN is a GUI tool for the creation of annotations on video and audio resources. It is used by many documentary linguists and several language documentation projects in [DOBES|http://dobes.mpi.nl/], [HRELP](http://www.hrelp.org/) and other similar programs.
-
The program allows for complex corpus searches using RegEx, multi-tier and multi-corpus (i.e. across several ELAN-files) as well as visualization of search results (concordance, frequency, etc.). For ELAN-files stored at [The Language Archive (TLA)|TLA.html], these features work also with the online tool [Trova](http://tla.mpi.nl/tools/tla-tools/trova/).
-
We use ELAN for annotating our video and audio ressources stored at TLA as well as for annoting and presenting our purely written text corpora (without links to multimedia). Here are the [ELAN Documentation Pages at TLA](http://tla.mpi.nl/tools/tla-tools/elan).
-
-
-
-## ELAN-xml
-
+### ELAN-xml
The name extension for ELAN files is **.eaf**. These are basically XML files (and can be opened as such), but they can also be read by the program ELAN for beeing presented and further edited in a GUI.
+## Workflow
+### Current praxis
+- Audio/Video raw data is stored at [TLA](TLA.html)
+- Orthographic transcription is done in ELAN
+- Translation into English and the main lingua franca (Swedish for Pite Saami, Russian for Komi, etc.) is done in ELAN
+- Occasionally, more (deeper) annotations are created manually, e.g. pos-tagging
-# Workflow
-
-
-## Current praxis
-* Audio/Video raw data is stored at [TLA](TLA.html)
-* Orthographic transcription is done in ELAN
-* Translation into English and the main lingua franca (Swedish for Pite Saami, Russian for Komi, etc.) is done in ELAN
-* Occasionally, more (deeper) annotations are created manually, e.g. pos-tagging
-
-
-## Planned extension
-* Exporting transcribed data from ELAN into the [FST](FST.html)-analyzer
-* Morphological analyzis
-* Back-importing lemmatized/pos-tagged/glossed wordforms into ELAN
+### Planned extension
+- Exporting transcribed data from ELAN into the [FST](FST.html)-analyzer
+- Morphological analyzis
+- Back-importing lemmatized/pos-tagged/glossed wordforms into ELAN
There is a script for this , at the [langdoc/elan-fst](https://github.com/langdoc/elan-fst) page at github, maintained by Niko Partanen, Joshua Wilbur and Mihael Rießler. The pipeline has been used for Komi (the Freiburg project), Pite Saami (Joshua Wilbur) and North Saami (in Oulu).
+### Planned external project (Zhivotova)
-## Planned external project (Zhivotova)
-* Morphological analyzis using [Toolbox](Toolbox.html)
-* Back-importing pos-tagged/glossed wordforms into ELAN
-
-
-
+- Morphological analyzis using [Toolbox](Toolbox.html)
+- Back-importing pos-tagged/glossed wordforms into ELAN
-# Annotation Conventions
+## Annotation Conventions
+### ELAN
-## ELAN
*Documentation page for the [ELAN tier structures used by our projects](ELANtiers.html) and links to ELAN tier template files (XML file in ELAN's .etf-format)
*Documentation page for [Transcription conventions](Transcription.html) applied by our projects
+## Related tools
-
-
-# Related tools
-*[WebLicht](http://de.clarin.eu/de/sprachressourcen/weblicht), a web-based tool to semi-automatically annotate texts for linguistics and humanities research. Interaction with WebLicht from ELAN is still only under development
-
-
+\*[WebLicht](http://de.clarin.eu/de/sprachressourcen/weblicht), a web-based tool to semi-automatically annotate texts for linguistics and humanities research. Interaction with WebLicht from ELAN is still only under development
diff --git a/ling/freiburg/ELANtiers.md b/ling/freiburg/ELANtiers.md
index d36d5a01..0fde2474 100644
--- a/ling/freiburg/ELANtiers.md
+++ b/ling/freiburg/ELANtiers.md
@@ -1,88 +1,55 @@
This page documents the ELAN tier structures used by our projects.
-
-
-
# Intro
-
The following presents an inventory of both the linguistic and the tier types used by our projects.
-*Note that adherance to these structures is necessary to use the ELAN-FST script which automatically adds annotations on word, lemma, morphological categories and part of speech.
-
-
-
-
-# ELAN Linguistic Types
-
-
-| Name | Stereotype | Controlled Vocabulary | Purpose
-| --- | --- | --- | ---
-| refT | - | - | used for ref-tiers; no stereotype because it ref-tiers are independent, root nodes, and time-alignable
-| orthT | symbolic association | - | used for orth-tiers, exact time-aligned copy of the superordinate ref-tier
-| wordT | symbolic subdivision | - | used for word-tiers, overall time-aligned copy of the orth-tier (and thus the ref-tier), but able to be divided into multiple equal parts
-| lemmaT | symbolic subdivision | - | used for lemma-tiers, overall equally-spaced, time-aligned copy of the word-tier, but able to be divided into multiple equal parts
-| morphT | symbolic subdivision | - | used for morph-tiers, overall equally-spaced, time-aligned copy of the word-tier, but able to be divided into multiple equal parts
-| posT | symbolic subdivision | pos | used for pos-tiers, overall equally-spaced, time-aligned copy of the word-tier, but able to be divided into multiple equal parts
-| ftT | symbolic association | - | used for free translation tiers, overall time-aligned copy of the orth-tier (and thus the ref-tier)
-| noteT | symbolic association | - | used for tiers adding notes to a given parent tiers, overall time-aligned copy of the parent-tier
-| langT | symbolic subdivision | languages | used for lang-tier to indicate language(s) being used in the utterance
-
-
-
+\*Note that adherance to these structures is necessary to use the ELAN-FST script which automatically adds annotations on word, lemma, morphological categories and part of speech.
-# ELAN Tiers and Tier Hierarchy
+## ELAN Linguistic Types
+| Name | Stereotype | Controlled Vocabulary | Purpose |
+| ------ | -------------------- | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
+| refT | - | - | used for ref-tiers; no stereotype because it ref-tiers are independent, root nodes, and time-alignable |
+| orthT | symbolic association | - | used for orth-tiers, exact time-aligned copy of the superordinate ref-tier |
+| wordT | symbolic subdivision | - | used for word-tiers, overall time-aligned copy of the orth-tier (and thus the ref-tier), but able to be divided into multiple equal parts |
+| lemmaT | symbolic subdivision | - | used for lemma-tiers, overall equally-spaced, time-aligned copy of the word-tier, but able to be divided into multiple equal parts |
+| morphT | symbolic subdivision | - | used for morph-tiers, overall equally-spaced, time-aligned copy of the word-tier, but able to be divided into multiple equal parts |
+| posT | symbolic subdivision | pos | used for pos-tiers, overall equally-spaced, time-aligned copy of the word-tier, but able to be divided into multiple equal parts |
+| ftT | symbolic association | - | used for free translation tiers, overall time-aligned copy of the orth-tier (and thus the ref-tier) |
+| noteT | symbolic association | - | used for tiers adding notes to a given parent tiers, overall time-aligned copy of the parent-tier |
+| langT | symbolic subdivision | languages | used for lang-tier to indicate language(s) being used in the utterance |
-## Required for each speaker:
-| Level | Name | Parent Tier | Linguistic Type | Language | Purpose
-| --- | --- | --- | --- | --- | ---
-| 0 | ref | - | refT | - (numbered) | root node, time-aligned annotation units are set here, each annotation is provided with a unique number here
-| -1 | orth | ref | orthT | vernacular | an orthographic transcription is provided here; this provides the input for the FST engine
-| -2 | word | orth | wordT | vernacular | tokenized version of the orth-tier; automatically created by ELAN-FST-script
-| -3 | lemma | word | lemmaT | vernacular | lemma (or lemmata in case of ambiguities) for word form listed in parent tier; automatically created by ELAN-FST-script
-| -3 | morph | word | morphT | English (linguistics) | morphological category (or categories in case of ambiguities) for word form listed in parent tier; automatically created by ELAN-FST-script
-| -3 | pos | word | posT | English (linguistics) | part of speech (or parts of speech in case of ambiguities) for word form listed in parent tier; adheres to 'pos'-list of controlled vocabulary; automatically created by ELAN-FST-script
+## ELAN Tiers and Tier Hierarchy
+### Required for each speaker:
-### Optional for each speaker:
-| Level | Name | Parent Tier | Linguistic Type | Language | Purpose
-| --- | --- | --- | --- | --- | ---
-| -2 | ft-XYZ | orth | ftT | a relevant lingua franca | provides a free translation of the annotated text; XYZ is replaced with a language code (e.g. eng, rus, etc.); can occur multiple times for multiple lingua francas
-| -2 | lang | orth | langT | English | indicates the language being used in an annotated utterance or part of an annotated utterance; the language name is in English; adheres to 'languages'-list of controlled vocabulary
+| Level | Name | Parent Tier | Linguistic Type | Language | Purpose |
+| ----- | ----- | ----------- | --------------- | --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| 0 | ref | - | refT | - (numbered) | root node, time-aligned annotation units are set here, each annotation is provided with a unique number here |
+| -1 | orth | ref | orthT | vernacular | an orthographic transcription is provided here; this provides the input for the FST engine |
+| -2 | word | orth | wordT | vernacular | tokenized version of the orth-tier; automatically created by ELAN-FST-script |
+| -3 | lemma | word | lemmaT | vernacular | lemma (or lemmata in case of ambiguities) for word form listed in parent tier; automatically created by ELAN-FST-script |
+| -3 | morph | word | morphT | English (linguistics) | morphological category (or categories in case of ambiguities) for word form listed in parent tier; automatically created by ELAN-FST-script |
+| -3 | pos | word | posT | English (linguistics) | part of speech (or parts of speech in case of ambiguities) for word form listed in parent tier; adheres to 'pos'-list of controlled vocabulary; automatically created by ELAN-FST-script |
+#### Optional for each speaker:
+| Level | Name | Parent Tier | Linguistic Type | Language | Purpose |
+| ----- | ------ | ----------- | --------------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| -2 | ft-XYZ | orth | ftT | a relevant lingua franca | provides a free translation of the annotated text; XYZ is replaced with a language code (e.g. eng, rus, etc.); can occur multiple times for multiple lingua francas |
+| -2 | lang | orth | langT | English | indicates the language being used in an annotated utterance or part of an annotated utterance; the language name is in English; adheres to 'languages'-list of controlled vocabulary |
+### Optional for any tier or as a root node with its own time-alignment:
-## Optional for any tier or as a root node with its own time-alignment:
-| Level | Name | Parent Tier | Linguistic Type | Language | Purpose
-| --- | --- | --- | --- | --- | ---
-| *** | note-XYZ| XYZ | noteT | anything | provide unstructured text-based notes for any given parent tier XYZ
-*all tiers for a given speaker are named using the tier name plus the @ symbol plus an short form referring to the relevant speaker, such as ref@JKW, lemma@JKW
+| Level | Name | Parent Tier | Linguistic Type | Language | Purpose |
+| ------ | -------- | ----------- | --------------- | -------- | ------------------------------------------------------------------- |
+| \*\*\* | note-XYZ | XYZ | noteT | anything | provide unstructured text-based notes for any given parent tier XYZ |
+\*all tiers for a given speaker are named using the tier name plus the @ symbol plus an short form referring to the relevant speaker, such as ref@JKW, lemma@JKW
-
-
-
-
-
-
-
-
-# ELAN Tier Template Files for Download
-
+## ELAN Tier Template Files for Download
Template files (in ELAN .etf format):
-
-* For spoken corpus data
- - [Master template](ELAN/ELAN_spoken_template.etf) (including all possible annotation tiers and linguistic types)
- - [PSDP template](ELAN/ELAN_spoken_template_PSDP.etf) (including only annotation tiers and linguistic types relevant for PSDP)
- - KSDP template (including only annotation tiers and linguistic types relevant for KSDP
- - IKDP template (including only annotation tiers and linguistic types relevant for IKDP
-*For written corpus data
- - Master template (including all possible annotation tiers and linguistic types)
- - PSDP template (including only annotation tiers and linguistic types relevant for PSDP)
- - KSDP template (including only annotation tiers and linguistic types relevant for KSDP
- - IKDP template (including only annotation tiers and linguistic types relevant for IKDP
-
-
+- For spoken corpus data - [Master template](ELAN/ELAN_spoken_template.etf) (including all possible annotation tiers and linguistic types) - [PSDP template](ELAN/ELAN_spoken_template_PSDP.etf) (including only annotation tiers and linguistic types relevant for PSDP) - KSDP template (including only annotation tiers and linguistic types relevant for KSDP - IKDP template (including only annotation tiers and linguistic types relevant for IKDP
+ \*For written corpus data - Master template (including all possible annotation tiers and linguistic types) - PSDP template (including only annotation tiers and linguistic types relevant for PSDP) - KSDP template (including only annotation tiers and linguistic types relevant for KSDP - IKDP template (including only annotation tiers and linguistic types relevant for IKDP
diff --git a/ling/freiburg/FST.md b/ling/freiburg/FST.md
index 743594c4..fb3f11ec 100644
--- a/ling/freiburg/FST.md
+++ b/ling/freiburg/FST.md
@@ -1,26 +1,14 @@
This page documents conventions, standards and relevant workflows used for the annotation of the Freiburg-Tromsø Corpora with the help of FST technology.
-
-
-
# Intro
-
FST is…
-
-
-
-# Workflows
-
+## Workflows
ELAN-->FST-->ELAN
-
-
-
-# Scripts
-
+## Scripts
*sending ELAN-data to the analyzer
*sending analyzed data back into ELAN
diff --git a/ling/freiburg/GRAID.md b/ling/freiburg/GRAID.md
index 026d4c40..1ee09d4d 100644
--- a/ling/freiburg/GRAID.md
+++ b/ling/freiburg/GRAID.md
@@ -1,29 +1,15 @@
This page documents conventions, standards and relevant workflows used for the annotation of the [Freiburg-Tromsø Corpora](freiburg.html) according to the GRAID conventions used for the investigation of "A Corpus-based microtypology of word order in varieties of Saami and Kurdish (WOSK)".
-
-
-
# Intro
+GRAID (Grammatical Relations and Animacy in Discourse) is a set of annotation conventions to be used for the cross-linguistic investigation of grammatical relations. GRAID was developed by Geoffrey Haig (Uni Bamberg) and Stefan Schnell (La Trobe Uni), see the [GRAID website](http://bamling-research.de/content/graid_main.html).
-GRAID (Grammatical Relations and Animacy in Discourse) is a set of annotation conventions to be used for the cross-linguistic investigation of grammatical relations. GRAID was developed by Geoffrey Haig (Uni Bamberg) and Stefan Schnell (La Trobe Uni), see the [GRAID website](http://bamling-research.de/content/graid_main.html).
-
-
-
-
-# Questions
-
+## Questions
GRAID annotations are written manually in a special annotation tier, e.g. in [ELAN](ELAN.html).
-*Can we produce GRAID annotations (semi-) automatically?
-
-
-
-
-# Tasks
+\*Can we produce GRAID annotations (semi-) automatically?
+## Tasks
In GRAID syntactic class (parts-of-speech) and syntactic function (direct object, predicate, etc.) are merged into one annotation tier. This seems less usefull. We should rather use separate tiers. If done manually we should also consider using closed vocabularies for inserting the annotation values. But at least the tags for syntactic classes could in principle be provided by using [FST](FST.html).
-*Create a better data structure for our GRAID-relevant ELAN-annotations!
-
-
+\*Create a better data structure for our GRAID-relevant ELAN-annotations!
diff --git a/ling/freiburg/GTcorpus.md b/ling/freiburg/GTcorpus.md
index 25ddbcd3..4cd12d59 100644
--- a/ling/freiburg/GTcorpus.md
+++ b/ling/freiburg/GTcorpus.md
@@ -1,30 +1,15 @@
This page documents conventions, standards and relevant workflows used for storing the Freiburg-Tromsø Corpora at the Giellatekno corpus repositories.
-
-
-
# Intro
-
The [Freiburg-based documentation projects|freiburg.html] archive their video and audio ressources along with catalogue and content metadata at [TLA|TLA.html]. The Giellatekno corpus repositories host copies of our [ELAN annotation files](ELAN.html).
-
[Documentation of the Giellatekno corpus repositories](../ling/corpus_repositories.html)
-
-
-
-# Workflows
-
+## Workflows
Linking corpus data to multimedia stored at TLA
-
-
-
-# Access
-
+## Access
The Giellatekno corpus repositories are divided in a free part (for texts in the public domain that we can redistribute freely) and a bound part (for texts that can be shared only upon agreement).
-
-
diff --git a/ling/freiburg/Metadata.md b/ling/freiburg/Metadata.md
index cac1201f..4f3b1a02 100644
--- a/ling/freiburg/Metadata.md
+++ b/ling/freiburg/Metadata.md
@@ -2,31 +2,20 @@
This page documents metadata categories and subcategories as well as labels we use for these metadata in the [Freiburg-Tromsø Speech Corpora](freiburg.html).
-
Project-internally we collect different kinds of metadata. Not all of them can be made public due to ethical and legal reasons. Here we document only metadata categories relevant for the corpora published through Korp. Main metadata categories describe:
-
*Actors (e.g. a recorded speaker, author, translator or annotator)
*Sessions (e.g. an annotated recording or an annotated written text)
-*Texts (e.g. modality or genre)
-
+\*Texts (e.g. modality or genre)
All publicely available metadata is stored in files separated from the [ELAN|ELAN.html] annotations in IMDI format on the Session node in the [TLA](TLA.html). A script (which does not yet exist) converts IMDI into a structure useful to be read into the Korp interface.
-
-
-
-# Actors
-
+## Actors
*Speakers (e.g. informants/consultants recorded and transcribed or authors/translators of written text included in the corpora)
*Annotators (e.g. PIs or assistants transcribing, translating or otherwise annotating recordings or written text included in the corpora)
-
-
-
-# Sessions
-
+## Sessions
*Actors
*Date
@@ -34,49 +23,35 @@ All publicely available metadata is stored in files separated from the [ELAN|ELA
*Media
*Place
*Project
-*Languages
-
-
-
+\*Languages
-# Texts
+## Texts
+### Actors
-## Actors
-
-
-## Date
+### Date
+### Language(s)
-## Language(s)
+### Modality
-
-## Modality
As a label for this category we use _Modality_ and mean here the way by which signs are transmitted by a sender. This catory has two values:
-
*oral (e.g. speech which we have recorded on audio or audio+video and transcribed or speech which is transcribed, but where there is no audio available because it is lost or the speech was transcribed without being recorded)
-*written (e.g. handwritten or printed texts, texts published online)
-
+*written (e.g. handwritten or printed texts, texts published online)
Another potential values (not relevant for our projects) are:
-
*gestured
*signed
-
Note that the kind of perception by a receiver is not relevant for our metadata categories (a written text can be received oraly if we use text-to-speech, etc.) Neither does _Modality_ in our sense refer to the actual medium (paper, video, etc.)
-
-## Language
-
+### Language
The-letter code in accordance with ISO 639-3
-
-## Genre
-
+### Genre
*poetry
*fiction
@@ -88,36 +63,23 @@ The-letter code in accordance with ISO 639-3
*idiom
*narrative
*teaching
-*story
-
-
-## Register
+\*story
+### Register
*formal
*informal
-*neutral
-
+\*neutral
-## Medium
-
-
-
-
-
-
-# Other conventions
+### Medium
+## Other conventions
Note that also file names used by us inlcude some metadata already. For instance:
*sms19610000lagercrantz318
*sjd20150609aaa-sport
where the first three letters **sms** or **sjd** - in accordance with ISO 639-3 - always mark the language (or main language) of a given session, the following eight digits **19610000** or **20150609** always mark the date of a given session in the format YYYYMMDD. If the exact date is unknown or cannot be specified (e.g. in a book publication were only the year is given) we use the digit 0.
-
-
-
-# See also
-
+## See also
XXX - ???
diff --git a/ling/freiburg/TLA.md b/ling/freiburg/TLA.md
index 92b8bb60..a608d604 100644
--- a/ling/freiburg/TLA.md
+++ b/ling/freiburg/TLA.md
@@ -2,55 +2,34 @@
This page documents conventions, standards and relevant workflows used for archiving the Freiburg-Tromsø Corpora at TLA.
-
-
-
# Intro
-
The Language Archive (TLA) is a unit of the Max Planck Institute for Psycholinguistics in Nijmegen/The Netherlands working, among other things, with archiving resources on endangered languages and cultures.
-
The [Freiburg-based documentation projects](freiburg.html) archive their video and audio ressources along with metadata at TLA.
-
[TLA Website](http://tla.mpi.nl/).
+## Workflow for Archiving at TLA
-
-
-# Workflow for Archiving at TLA
-
-
-
-
-# Data Standards for Archiving at TLA
-
+## Data Standards for Archiving at TLA
- Raw data
- Catalogue metadata
- Content metadata
-
-
-
-# Access to Data Archived at TLA
-
+## Access to Data Archived at TLA
Access to catalogue metadata at TLA is always free, although personal information on actors might be anonymized.
-
Raw data and content metadata (e.g. annotations) belongs to one of the following four types:
*Open Resources
-**Access is free (marked green)
+\*\*Access is free (marked green)
*Restricted Open Resources
**Access is free after registration (marked yellow)
-*Protected Open Resources
+\*Protected Open Resources
**Access can be requested (marked orange)
-*(Closed Resources)
-**No access (marked red)
-
+\*(Closed Resources)
+\*\*No access (marked red)
Only the first three access levels are relevant to the Freiburg-Tromsø Corpora.
-
-
diff --git a/ling/freiburg/Toolbox.md b/ling/freiburg/Toolbox.md
index 56540ced..7d23cdf4 100644
--- a/ling/freiburg/Toolbox.md
+++ b/ling/freiburg/Toolbox.md
@@ -2,22 +2,13 @@
This page documents conventions, standards and relevant workflows used for the annotation of the [Freiburg-Tromsø Speech Corpora](freiburg.html) with the help of the software Toolbox.
-
Whereas we focus on working with the [FST analyzer](FST.html), parts of our [ELAN corpora](ELAN.html) are being analyzed by external collaboration projects using Toolbox.
-
-
-
-# Intro
-
+## Intro
Toolbox (The Field Linguist’s Toolbox) is a data management and analysis tool used by many field linguists. Among other things, it can parse and interlinearize text. Here are the [Toolbox Documentation Pages](http://www-01.sil.org/computing/catalog/show_software.asp?id=79).
-
-
-
-# Workflows
-
+## Workflows
ELAN-->Toolbox-->ELAN
@@ -25,23 +16,16 @@ For a way to do this, see:
- [Simon Devylder & Lucie Laporte-Devylder: ELAN -> Toolbox -> ELAN WORKFLOW](METHOD01_ELAN_Toolbox_workflow_Interactive_Speech_Glossing.pdf)
-
-
-
-# Scripts
-
+## Scripts
- sending ELAN-data to Toolbox
- sending analyzed Toolbox-data back to ELAN
-
-
-
-# Annotation conventions and tag sets
+## Annotation conventions and tag sets
…
+## See also
-# See also
- ELAN [Transcription conventions](Transcription.html)
- Giellatekno's [tag sets for Kildin Sámi](/lang/sjd/docu-grammartags.eng.html)
diff --git a/ling/freiburg/freiburg.md b/ling/freiburg/freiburg.md
index 80e2d4f3..03aff821 100644
--- a/ling/freiburg/freiburg.md
+++ b/ling/freiburg/freiburg.md
@@ -2,70 +2,51 @@
Technical documentation for the Freiburg-Helsinki-Tromsø Speech Corpora for Sámi, Komi and other languages.
-
This document is meant for project collaborators. The documentation will now and then also be relevant for anyone wanting to use our corpora or the annotation tools created by us.
-
-
-# Intro
-
+## Intro
The Freiburg-Helsinki-Tromsø Corpora are built in collaboration between the [Freiburg Research Group for Sámi Studies](http://www.skandinavistik.uni-freiburg.de/institut/forschung/saami), [Kielipankki](https://kitwiki.csc.fi/twiki/bin/view/FinCLARIN/KielipankkiKoti) in Helsinki, [Giellatekno](http://giellatekno.uit.no) in Tromsø, and members of the respective speech communities.
-
Our approach is to combine knowledge and tools from the applied fields of Language Documentation (often focussing on ARCHIVING, rather than on annotation) and Language Technology (often focussing on WRITTEN, rather than on spoken language) with the main goal to annotating systematically and making available the largest possible variety of language samples for further corpus-based applied and theoretical research.
+## Projects
+### Language documentation and description
-
-# Projects
-
-
-## Language documentation and description
-
-
-### Pite Sámi Documentation Project (PSDP)
-
+#### Pite Sámi Documentation Project (PSDP)
Leader
- Joshua Wilbur
-
Other (current and former) main collaborators
- Miriam Hecker (MA student Freiburg)
-
-
-
-### Kola Sámi Documentation Project (KSDP)
-
+#### Kola Sámi Documentation Project (KSDP)
Leader
-- Michael Rießler
+- Michael Rießler
Other (current and former) main collaborators
+
- Maryna Litvak (student assistant Freiburg)
- Julia Reitze (MA student Freiburg)
- Evgeniya Zhivotova (PhD student Leipzig)
-
-
-
-### Izhva Komi Documentation Project (IKDP)
-
+#### Izhva Komi Documentation Project (IKDP)
[FU-Lab Project Wiki (in Russian/Komi)](http://wiki.komikyv.ru)
-
Leaders (of corpus work)
+
- Michael Rießler
- Niko Partanen (PhD student Freiburg)
-
Other (current and former) main collaborators
+
- Rogier Blokland (project leader Uppsala)
- Andrej Chemyshev (FU-Lab Syktyvkar)
- Vasili Chuprov (student assistant Syktyvkar)
@@ -73,77 +54,55 @@ Other (current and former) main collaborators
- Alexandra Kellner (student assistant Helsinki/Freiburg)
- Ënyë Lav (FU-Lab Syktyvkar)
+### Theoretical linguistics
-
-
-## Theoretical linguistics
-
-
-### A Corpus-based microtypology of word order in varieties of Sámi and Kurdish (WOSK)
-
+#### A Corpus-based microtypology of word order in varieties of Sámi and Kurdish (WOSK)
Leaders (of corpus work)
- Michael Rießler
- Hanna Thiele (PhD student)
-
-
-
-# Documentation Pages
-
+## Documentation Pages
Common documentation for our spoken data archive
- [The Language Archive (TLA)](TLA.html)
-
Common documentation for our working repository for corpus data
- [Giellatekno Corpora](GTcorpus.html)
- [ELAN Annotations](ELAN.html)
-
Common documentation for data annotation tools
- [FST Morphology Parsing](FST.html)
- [Toolbox Morphology Parsing](Toolbox.html)
-
Common documentation for metadata conventions
- [Metadata](Metadata.html)
-
Description of [GRAID Annotation Conventions](GRAID.html) (used in WOSK)
+## Meetings
+### General
-
-# Meetings
-
-
-## General
[21.05.2014](meetings/2014-05-21.html) [07.05.2014](meetings/2014-05-07.html)
+### IKDP
-## IKDP
[11.04.2014](meetings/2014-04-11.html) [18.03.2014](meetings/2014-03-18.html) [13.01.2014](meetings/2014-01-13.html)
+### WOSK
-## WOSK
[14.05.2014](meetings/2014-05-14.html) [30.04.2014](meetings/2014-04-30.html) [08.04.2014](meetings/2014-04-08.html)
-
-
-
-# Documentation of Related Projects in Collaboration with Giellatekno
-
+## Documentation of Related Projects in Collaboration with Giellatekno
- Kildin Sámi Lexicography
- Komi Lexicography
- Oahpa!-nuõrti
- Pite Sámi Lexicography
- [Skolt Sámi Lexicography](../dicts/SkoltSaami2X.html)
-
-
diff --git a/ling/freiburg/transcription.md b/ling/freiburg/transcription.md
index 86e39679..441f62c5 100644
--- a/ling/freiburg/transcription.md
+++ b/ling/freiburg/transcription.md
@@ -2,49 +2,29 @@
This page documents conventions for transcriptions and other notations applied in the [ELAN](ELAN.html) annotations created by the [Freiburg-Tromsø Speech Corpora](freiburg.html).
-
Since the audio/video remains linked to our onnotations in ELAN, we do not need very complicated transcription conventions, like marking overlapping speech, and other nuances (hesitations, different lenghts for pauses, etc.) Instead, we aim for a simple orthographic transcription and need only very few special tags in the transcription tier.
-
-
-
-# Tier "ref" (reference)
-
+## Tier "ref" (reference)
The annotations in this tier provide unique references to single linguistic units (like sentences, utterances, etc.) in the ELAN-file. The ELAN program has a function for automatic labelling und numbering of annotation units.
-
A typical label looks like:
-*sms1961lagercrantz318.003
+\*sms1961lagercrantz318.003
where **sms1961lagercrantz318** is the file name (including a hint to the language [[sms], the data of origin of the text [[1961] and additional textual explanation [[lagercrantz]. **003** marks the third sentence in the text.
-
-
-
-# Tier "orth" (orthography)
-
+## Tier "orth" (orthography)
This tier is a child of **ref**. It includes either an orthographic transcription (of spoken texts), an orthographic transliteration (of texts written in other than the standard orthography) or the original orthography (of texts written in the standard orthography).
-
-
-
-## Capitalization and punctuation
-
+### Capitalization and punctuation
In the case of transcribed spoken texts, is up to the taste of the annotator whether or not the transcription has to comply with the rules for standard orthography.
-
In the case of written texts, the original orthography, incl. capitalization and punctuation is preserved as in the original.
+If punctuation marks are used always write a whitespace before them (in order to tokenize them)! Note the (syntactic) difference between PUNCT vs. CLB in the punctuation marks!
-If punctuation marks are used always write a whitespace before them (in order to tokenize them)! Note the (syntactic) difference between PUNCT vs. CLB in the punctuation marks!
-
-
-
-
-## Other conventions
-
+### Other conventions
Non-comprehensible speech
@@ -53,41 +33,29 @@ Non-comprehensible speech
- Part(…) = unclear ending
- Part(is)anen = I am hearing "is", but I am not quite certain
-
False start and correction or incomplete articulated word
- hyphen following right after the word, like in Partisa- Partanen
-
Clear Pause
- … (after whitespace)
-
Some authers have symbols or additional explanationsin their transcriptions, e.g. Lagercrantz: (Ill.) (1234) etc. These notations needs to be kept, but marked as belonging to non-speech. Leave these signs in the text, but put them in parenthesis.
-
-
-
-# Tiers "ft" (free translation)
-
+## Tiers "ft" (free translation)
Depending on the source language, we translate into different languages, normally English and then relevant lingua franca (e.g. Russian for Kildin Sámi texts, Swedish for Pite Sámi texts, etc.). The target languages are specified by the three-letter-codes used also by Giellatekno elsewhere (e.g. deu = German, fin = Finnish).
-
Non-spoken parts in the translations are written in square brackets
-* and [[he] sailed away
-
-
-
-# ELAN tag sets
+- and [[he] sailed away
+## ELAN tag sets
Tag sets are called "controlled vocabularies" in ELAN. Here are the tag sets currently used by our projects (copied from the ELAN-xml).
-
-## Language codes
+### Language codes
- deu
- nob
@@ -99,10 +67,9 @@ Tag sets are called "controlled vocabularies" in ELAN. Here are the tag sets cur
- sje
- smj
- sme
-*kpv
-
+ \*kpv
-## PoS
+### PoS
- (??)
- (NO)
@@ -125,8 +92,6 @@ Tag sets are called "controlled vocabularies" in ELAN. Here are the tag sets cur
- Prop
- V
+## See also
-
-
-# See also
-*Giellatekno's [tag sets for Kildin Sámi](/lang/sjd/docu-grammartags.eng.html)
+\*Giellatekno's [tag sets for Kildin Sámi](/lang/sjd/docu-grammartags.eng.html)
diff --git a/ling/global-flowchart.md b/ling/global-flowchart.md
index 94dd2e9f..e17ecd0f 100644
--- a/ling/global-flowchart.md
+++ b/ling/global-flowchart.md
@@ -1,12 +1,11 @@
-Global flowchart
-================
+# Global flowchart
The flowchart shows parsing done with two different analysers: Method 1,
for hfst analysers where pre- and postprocessing is integrated in the
morphological analysis, and method 2, for xfst analysers, using perl for
pre- and postprocessing of the text.
-# Method 1 (hfst)
+## Method 1 (hfst)
Action taken.. ..by means of the commands:
************** ****************************
@@ -14,28 +13,28 @@ pre- and postprocessing of the text.
| take incoming text | cat filename.txt |
+------------------------------+
\/
- +------------------------------+
+ +------------------------------+
| divide it into sentences and |
| words and give each token |
| all possiblemorphological | hfst-tokenise -g tools/tokenisers/tokeniser-disamb-gt-desc.pmhfst |
- | analyses |
- +------------------------------+
+ | analyses |
+ +------------------------------+
\/
+------------------------------+
- | disambiguating the m-analysis|
+ | disambiguating the m-analysis|
| (= picking only the relevant | vislcg3 -g src/syntax/disambiguation.cg3 |
- | morphological analyse) |
+ | morphological analyse) |
+------------------------------+
\/
+------------------------------+
- | adding syntactic functions | vislcg3 -g src/syntax/functions.cg3 |
+ | adding syntactic functions | vislcg3 -g src/syntax/functions.cg3 |
+------------------------------+
\/
+------------------------------+
- | adding depenency relations | vislcg3 -g src/syntax/dependency.cg3
+ | adding depenency relations | vislcg3 -g src/syntax/dependency.cg3
+------------------------------+
-# Method 2 (xfst)
+## Method 2 (xfst)
Method 2 differs only in that the morphological analysis is divided in 3
separate components
@@ -48,25 +47,24 @@ separate components
\/
+--------------------------+
| preprocessing it: |
- | moving one word per line,| preprocess --abbr=bin/abbr.txt | # method 1 |
- | finding sentence bound. |
+ | moving one word per line,| preprocess --abbr=bin/abbr.txt | ## method 1 |
+ | finding sentence bound. |
+--------------------------+
\/
|-----------------------------+
| morphological analysis: |
| give each word all possible | lookup -flags mbTT -utf8 src/analyser-gt-desc.xfst |
| analyses |
- |-----------------------------+
+ |-----------------------------+
\/
|-----------------------------+
| processing the output into |
- | a format that fits the dis- | lookup2cg
- | ambiguator, w/a perlscript |
+ | a format that fits the dis- | lookup2cg
+ | ambiguator, w/a perlscript |
|-----------------------------+
\/
...
( then continue with disambiguation as shown above )
-
The commands assume you stand in the directory of the language you work
with. Method 2 may also be used with hfst (hfst-lookup), but method 1
diff --git a/ling/hist/index.md b/ling/hist/index.md
index 65feb105..ff97ad48 100644
--- a/ling/hist/index.md
+++ b/ling/hist/index.md
@@ -1,7 +1,6 @@
-Historical corpora
-==================
+# Historical corpora
In this context, by **historical corpora** we mean corpora written in earlier orthographies.
- [Overview over orthographies](orthography.html)
-- [Overview over text](texts.html)
\ No newline at end of file
+- [Overview over text](texts.html)
diff --git a/ling/hist/orthography.md b/ling/hist/orthography.md
index 4f8d9658..074693c4 100644
--- a/ling/hist/orthography.md
+++ b/ling/hist/orthography.md
@@ -1,8 +1,6 @@
-Historical Sámi orthographies
-=============================
-
-# Overview
+# Historical Sámi orthographies
+## Overview
Linguistically, there are differences within several of the orthographies listed below, the grouping here is made from an OCR point of view.
@@ -17,17 +15,15 @@ Priority: The important dictionaries we have for 1, 2, 3, 5, 6. The important te
**TODO:** Find out how many OCR models we need, and make them.
+## The Leem orthography (Norway, until 1832)
-# The Leem orthography (Norway, until 1832)
This contains Danish letters + palatalisation accents.
+## The Stockfleth/Friis orthography (Norway, 1832-2000)
+This was the dominating orthography until 1948. After 1949, its use was restricted to religious literature (''Nuorttanaste'' and related texts).
-# The Stockfleth/Friis orthography (Norway, 1832-2000)
-
-This was the dominating orthography until 1948. After 1949, its use was restricted to religious literature (''Nuorttanaste'' and related texts).
-
-## Alphabet
+### Alphabet
```
А а B b C c Č č D d Đ đ E e F f
@@ -38,26 +34,24 @@ N n Ƞ ƞ O o P p R r S s Š š T t
There are training data for this orthography in [tesstrain](https://github.com/divvungiellatekno/tesstrain). **TODO:** The Stockfleth dictionary.
+## The Nielsen orthography (Konrad Nielsens publications)
+### Alphabet
+Nothing done so far.
-# The Nielsen orthography (Konrad Nielsens publications)
-## Alphabet
-
-Nothing done so far.
-
+## The Ravila/Itkonen orthography (Finland, 1934-1978)
-# The Ravila/Itkonen orthography (Finland, 1934-1978)
-## Alphabet
+### Alphabet
Nothing done so far.
-
-# The Bergsland/Ruong orthography (Norway-Sweden, 1948-1978)
+## The Bergsland/Ruong orthography (Norway-Sweden, 1948-1978)
Most letters are the same as for Friis (`ǥ` is gone), but many glyphs are different from the 19th century. Both the dialect basis and the orthographic rules are neš, and the bigram pattern is thus new as well.
-## Alphabet
+### Alphabet
+
```
А а Á á B b C c Č č D d Đ đ E e
F f G g H h I i J j K k L l M m
@@ -67,12 +61,12 @@ N n Ƞ ƞ O o P p R r S s Š š T t
TODO: Frette.
-
-# The present 1979 orthography
+## The present 1979 orthography
The letters and the glyphs are the same as for Bergsland/Ruong, but the bigram pattern is different.
-
-## Alphabet
+
+### Alphabet
+
```
А а Á á B b C c Č č D d Đ đ E e
F f G g H h I i J j K k L l M m
diff --git a/ling/langrec.md b/ling/langrec.md
index 1c62d3bc..0706d644 100644
--- a/ling/langrec.md
+++ b/ling/langrec.md
@@ -1,7 +1,6 @@
-Language recognition
-=====
+# Language recognition
-# Usage
+## Usage
The primary use of this tool is via the corpus conversion tool
[convert2xml](corpus_conversion_tech.html). When you use convert2xml in
@@ -17,7 +16,7 @@ Typical usage as a standalone program will be something like:
`pytextcat` will return the name (the ISO code, to be exact) of the
language(s) the script believes the text to be in.
-# Adding a new recognizable language
+## Adding a new recognizable language
The pytextcat reference files are stored in
$GTHOME/tools/CorpusTools/corpustools/.
@@ -42,18 +41,18 @@ cat someinput | pytextcat compwm > lm/language-iso-code.wm
After this, the language recognition tool `pytextcat` is ready for use
with another language as shown in the previous section.
-# Source
+## Source
The home page of the original perl-based package TextCat is found at
several locations.
-- [The original page at University of
- Groningen](https://web.archive.org/web/20190111231813/http://odur.let.rug.nl/~vannoord/TextCat/), with [the
- source code](https://web.archive.org/web/20160624041646/http://odur.let.rug.nl/~vannoord/TextCat/text_cat.tgz)
- . The package is lisenced under a GPL license — see the home page
- for details — and it is developed by Gertjan van Noord
-- [The source code is also available in the Giellatekno
- repository](https://gtsvn.uit.no/langtech/trunk/tools/lang-guesser/)
+- [The original page at University of
+ Groningen](https://web.archive.org/web/20190111231813/http://odur.let.rug.nl/~vannoord/TextCat/), with [the
+ source code](https://web.archive.org/web/20160624041646/http://odur.let.rug.nl/~vannoord/TextCat/text_cat.tgz)
+ . The package is lisenced under a GPL license — see the home page
+ for details — and it is developed by Gertjan van Noord
+- [The source code is also available in the Giellatekno
+ repository](https://gtsvn.uit.no/langtech/trunk/tools/lang-guesser/)
. The Groningen home page also includes links to a background article, a
list of supported languages coming with the tools, and also a list of
diff --git a/ling/morfem-borders.md b/ling/morfem-borders.md
index 66cae2c6..e0c78539 100644
--- a/ling/morfem-borders.md
+++ b/ling/morfem-borders.md
@@ -1,29 +1,28 @@
-Morpheme border markup
-=====================
+# Morpheme border markup
We want to separate the following types of morfeme borders:
-- inflection: **< >** - *ver<kauf>en* ( that is,
- pref<root>suff)
-- derivation: **« »** - *boahti»goahtit* , *u«kast»ing*
-- word boundary: **\#** - *bil\#stol*
-- compound morfeme: - *trikk\|e\#tur*
-- clitics: **«\|** **\|»** - *gærja\|»gænnah* (these are multichar
- symbols that should be defined in the lexc and twolc alphabets)
+- inflection: **< >** - _ver<kauf>en_ ( that is,
+ pref<root>suff)
+- derivation: **« »** - _boahti»goahtit_ , _u«kast»ing_
+- word boundary: **\#** - _bil\#stol_
+- compound morfeme: - _trikk\|e\#tur_
+- clitics: **«\|** **\|»** - _gærja\|»gænnah_ (these are multichar
+ symbols that should be defined in the lexc and twolc alphabets)
When adding the markup to the lexical entries, it should be done in the
continuation lexica, not in the stem lexica.
LEXCICON Adjectives
stor cont ; ikkje stammemerke her!!!
-
+
LEXICON cont
+Def+Pl:%>e # ; ! men her
+Der:%»aktig # ; ! and here
-
+
Other examples:
stor»aktig>es
ráhkis»vuođa>id
-
+
diff --git a/ling/ordbild/GammalKravspesifikasjon.md b/ling/ordbild/GammalKravspesifikasjon.md
index a296888f..34f83d10 100644
--- a/ling/ordbild/GammalKravspesifikasjon.md
+++ b/ling/ordbild/GammalKravspesifikasjon.md
@@ -1,189 +1,148 @@
# Comments for Tino 27.4.12:
-
Her kommer vår ønskeliste:
-
## Ny ordklasse (som du selv nevner i brevet):
-
### pre- og postposisjoner, som har henholdsvis taggene Pr og Po
-
-
-
-Vi vil at det skal være mulig å skrive inn et lemma, f.eks. 'maŋŋel'
+Vi vil at det skal være mulig å skrive inn et lemma, f.eks. 'maŋŋel'
og få opp 'maŋŋel' som Pr (preposisjon) og 'maŋŋel' som Po (postposisjon)
(begge analysene fins)
-hver med presentasjon av komplementer (@>P viser til komplementet) og av
+hver med presentasjon av komplementer (@>P viser til komplementet) og av
hvilke verb de har som mor i dependenstreet (@"
- "mun" Pron Pers Sg1 Nom @SUBJ> #1->2
+ "mun" Pron Pers Sg1 Nom @SUBJ> #1->2
""
- "vuodjit" V TV Ind Prs Sg1 @FMV #2->0
+ "vuodjit" V TV Ind Prs Sg1 @FMV #2->0
""
"muorra" Plant N Sg Gen @>P #3->4 <===== komplement
""
"njeaiga" Po @2 <==== Postposisjon
"<.>"
- "." CLB #5->2
+ "." CLB #5->2
```
-Noen ord er bare preposisjoner eller bare postposisjoner,
-de får da opp bare Pr eller bare Po.
+Noen ord er bare preposisjoner eller bare postposisjoner,
+de får da opp bare Pr eller bare Po.
## Når det gjelder substantiv:
-
Det vi får i dag er ok. Men dessuten vil vi:
-
objekt @OBJ Acc
-one can @ADVL - deles inn etter kasus: Ill, Loc, Com, Ess, Gen
+one can @ADVL - deles inn etter kasus: Ill, Loc, Com, Ess, Gen
Dvs: Vi vil ha en klasse for hver kombinasjon av
+_@ADVL Ill,
+_@ADVL Loc,
+_@ADVL Com,
+_@ADVL Ess,
+\*@ADVL Gen
-*@ADVL Ill,
-*@ADVL Loc,
-*@ADVL Com,
-*@ADVL Ess,
-*@ADVL Gen
-
-
-
-
-## Når det gjelder adjektiv:
-
+## Når det gjelder adjektiv:
I dag viser du adjektiv som attributt ("gul sol")
-
Vi vil at du også (som separate kategorier) tar med
-@SUBJ of @"
"nieida" Hum N Sg Nom @SUBJ> #1->2 <=====
""
- "leat" V IV Ind Prs Sg3 @FMV #2->0
+ "leat" V IV Ind Prs Sg3 @FMV #2->0
"<šiega>"
"šiega" A Sg Nom @2 <=====
"<.>"
- "." CLB #4->2
+ "." CLB #4->2
""
- "mun" Pron Pers Sg1 Nom @SUBJ> #1->2
+ "mun" Pron Pers Sg1 Nom @SUBJ> #1->2
""
- "atnit" V TV Ind Prs Sg1 @FMV #2->0
+ "atnit" V TV Ind Prs Sg1 @FMV #2->0
""
"nieida" Hum N Sg Acc @2 <=====
""
"sápmelaš" A Ess @3 <=====
"<.>"
- "." CLB #5->2
+ "." CLB #5->2
```
-Det vi får i dag er ok. Men dessuten vil vi:
+Det vi får i dag er ok. Men dessuten vil vi:
presentere Comp og Superl hver for seg for seg, med pre-mod og pre-mod of (the adj in question)
-
Dermed blir det tre klasser av adjektiv:
-
-* adjektiv (uspesifisert, inneholder Pos (uten tag), Comp, Superl)
-* adjektiv som er Comp
-* adjektiv som er Superl
-
+- adjektiv (uspesifisert, inneholder Pos (uten tag), Comp, Superl)
+- adjektiv som er Comp
+- adjektiv som er Superl
og for hver av disse tre klassene skal 'pre-mod of' (alle disse taggene være med):
@>N, @ @, som vist ovenfor.
-
-dessuten skal 'pre-mod' (@>A) presenteres
-
-
-
-
-
-
-
+dessuten skal 'pre-mod' (@>A) presenteres
## Når det gjelder adverb:
-
Det vi får i dag er ok. Men dessuten vil vi ha:
-
-
-
adj-heads Adv @>A
-
```
""
- "mun" Pron Pers Sg1 Loc @HAB #1->2
+ "mun" Pron Pers Sg1 Loc @HAB #1->2
""
- "leat" V IV Ind Prs Sg3 @FMV #2->0
+ "leat" V IV Ind Prs Sg3 @FMV #2->0
""
"hui" Adv @>A #3->4 <===============
""
- "buorre" A Sg Nom @>N #4->5
+ "buorre" A Sg Nom @>N #4->5
""
- "girji" Txt N Sg Nom @2
+ "girji" Txt N Sg Nom @2
"<.>"
- "." CLB #6->2
+ "." CLB #6->2
```
-
## Når det gjelder verb:
-
Det vi får i dag er ok. Men dessuten vi vil ha
-* substantiv og Pron som fungerer som adverbial (@ADVL) delt inn etter kasus:
-* Ill, Loc, Com, Ess, Gen (som vist ovenfor for substantiv)
-* og dessuten Adv (ordklassen adverb) f.eks. Adv @"
- "mun" Pron Pers Sg1 Nom @SUBJ> #1->2
+ "mun" Pron Pers Sg1 Nom @SUBJ> #1->2
""
"goddit" V TV Ind Prt Sg1 @FMV #2->0 <=== aktiv
""
- "luossa" Ani N Sg Acc @2
+ "luossa" Ani N Sg Acc @2
"<.>"
- "." CLB #4->2
+ "." CLB #4->2
Eksempel (b)
""
- "luossa" Ani N Sg Nom @SUBJ> #1->2
+ "luossa" Ani N Sg Nom @SUBJ> #1->2
""
"goddit" V* TV Der/PassS V Ind Prt Sg3 @FMV #2->0 <=== Der/PassS
"<.>"
- "." CLB #3->2
+ "." CLB #3->2
Eksempel (c)
""
- "luossa" Ani N Sg Nom @SUBJ> #1->2
+ "luossa" Ani N Sg Nom @SUBJ> #1->2
""
"goddit" V* TV Der/PassL V Ind Prt Sg3 @FMV #2->0 <=== Der/PassL
"<.>"
- "." CLB #3->2
+ "." CLB #3->2
```
-
for lemmaet (goddit) skal vi dermed ha tre "avdelinger":
+
```
+---------------------------------------------
-| Verbet goddit
-|
-| ==> her kommer eksempel av typen (a)
-|
-| Der/PassS av goddit
-|
-| ==> her kommer eksempel av typen (b)
-
-
-| Der/PassL av goddit
-|
-| ==> her kommer eksempel av typen (c)
-|
-|
+| Verbet goddit
+|
+| ==> her kommer eksempel av typen (a)
+|
+| Der/PassS av goddit
+|
+| ==> her kommer eksempel av typen (b)
+
+
+| Der/PassL av goddit
+|
+| ==> her kommer eksempel av typen (c)
+|
+|
+---------------------------------------------
```
-
-
-
-
-
-
diff --git a/ling/ordbild/OversiktOverOrdbild.md b/ling/ordbild/OversiktOverOrdbild.md
index 1f553d42..83e808bd 100644
--- a/ling/ordbild/OversiktOverOrdbild.md
+++ b/ling/ordbild/OversiktOverOrdbild.md
@@ -1,31 +1,27 @@
-Oppsett av *Ordbild*, oversyn.
+# Ordbild
+Oppsett av _Ordbild_, oversyn.
```
V*, N*, osb: **Søkeordet**
| = liste over treff, ordna i liste nedover.
```
-
## Verb
-
### Verb + objekt og oblike objekt / adverbial:
-
-```
+```
OBJ ADVL ADVL ADVL ADVL
SUBJ V* ACC/Inf.. ILL LOC COM ESS
-| | | | | |
+| | | | | |
```
-
Eksempler (her ser vi at @"
"nieida" N Sem/Hum Sg Nom @SUBJ> #1->2
""
@@ -68,18 +64,14 @@ Dette burde tagges. Forslag: vente med ADVL til vi har en tilleggstagg for dette
"." CLB #4->2
```
+Note:
-
-
-
-
-Note:
-* Infinitivsobjekt blir presentert som infinitiv
-* Viss ei kolonne er tom blir ho ikkje realisert
-
+- Infinitivsobjekt blir presentert som infinitiv
+- Viss ei kolonne er tom blir ho ikkje realisert
Eksempler
-```
+
+```
""
@@ -96,31 +88,23 @@ Eksempler
```
-
-
-
-
-
Vi ventar med finitte objekt. Når vi evt. tar dei kan vi kanskje
presentere dei som (infinitiv (fin) // pres ind sg3)
-
### Verb + andre argument
-
```
ADVL ADVL Adverbial
V* ACC GEN Po+N/Pr+N/Adv
-| | |
+| | |
```
-
-Her slår vi saman alle p-posisjonar (som dei gjer i Sverige),
+Her slår vi saman alle p-posisjonar (som dei gjer i Sverige),
men modellerer dei som heilheit: "i skola" "biilla haga", ...
-
Eksempler
-```
+
+```
""
@@ -169,20 +153,14 @@ Eksempler
```
-
-
-
### Verb + p-posisjonar
+Dette er alternativ 2. Vi **ventar** med det til vi har sett på
+korleis _verb + andre argument_ ovafor ser ut.
-Dette er alternativ 2. Vi **ventar** med det til vi har sett på
-korleis *verb + andre argument* ovafor ser ut.
-
-
-Her listar vi substantiv for kvar postposisjon til kvart verb,
+Her listar vi substantiv for kvar postposisjon til kvart verb,
for dei ... 5 (?) vanlegaste p-posisjonane.
-
```
V N Po1 N Po2 ...
| |
@@ -192,8 +170,7 @@ for dei ... 5 (?) vanlegaste p-posisjonane.
| |
```
-
-```
+```
""
"nieida" N Sem/Hum Sg Nom @SUBJ> #1->2
""
@@ -206,27 +183,21 @@ for dei ... 5 (?) vanlegaste p-posisjonane.
"." CLB #5->2
```
-
-
-
## Substantiv
-
Klarer vi å pine dette inn på ei linje?
-
### Sentrale argument i NP
+
```
@>N @>N @N<
- GenN Adj N* N
+ GenN Adj N* N
| | |
-```
-
+```
Mu fiskes girji oahpahusas (mi gule bok om undervisning)
-
-```
+```
""
"mun" Pron Pers Sg1 Gen @>N #1->3
""
@@ -261,43 +232,31 @@ Mu fiskes girji oahpahusas (mi gule bok om undervisning)
```
-
-
-
### Perifere argument i NP (dvs. N PP)
-
Her listar vi Pr + N, N + Po som heilskap ("i skola", "biilla haga")
+
```
@N< @N<
- Pr N* Po
+ Pr N* Po
| |
-```
-
-
-
-
+```
-
## Preposisjonar
-
-Vi viser kva verb og substantiv som preposisjonen *står til*:
+Vi viser kva verb og substantiv som preposisjonen _står til_:
Eller vil vi det?
-
```
V N Pr* GEN
| | |
```
+- Mun vuolggán birra gávpoga.
+- Girji .
+- Ovdal bearjadaga.
-* Mun vuolggán birra gávpoga.
-* Girji .
-* Ovdal bearjadaga.
-
-
-```
+```
""
"nieida" N Sem/Hum Sg Nom @SUBJ> #1->2
""
@@ -324,29 +283,20 @@ Eller vil vi det?
```
-
-
-
-
-
## Postposisjonar
-
... som preposisjonar:
-
```
V N GEN Po*
-| | |
+| | |
```
+- Mun vuolggán dihtora haga.
+- Girji soađi birra.
+- Spábbačiekčama dihte.
-* Mun vuolggán dihtora haga.
-* Girji soađi birra.
-* Spábbačiekčama dihte.
-
-
-```
+```
""
"nieida" N Sem/Hum Sg Nom @SUBJ> #1->2
""
@@ -371,17 +321,14 @@ Eller vil vi det?
"." CLB #5->2
```
-
## Adjektiv
-
```
Adv A* N
| |
```
-
-```
+```
""
"nieida" N Sem/Hum Sg Nom @SUBJ> #1->2
""
@@ -398,24 +345,18 @@ Eller vil vi det?
```
-
-
-
## Adverb
-
```
V Adv Adv* N
-| | |
+| | |
```
+- gå mycket
+- svä
+ lyckligtvis
-* gå mycket
-* svä
-lyckligtvis
-
-
-```
+```
""
"nieida" N Sem/Hum Sg Nom @SUBJ> #1->2
""
@@ -446,30 +387,21 @@ lyckligtvis
```
-
## Andre ordklasser?
-
Talord, ...
Seinare
+## Treff
+SUBJ ...? =
-
-## Treff
-
-
-SUBJ ...? =
# pron separat
-# pron som ein kategori
-# utan pron
-
+# pron som ein kategori
+# utan pron
## Seinare utvikling
-
Ordbild med sem-taggar
-
-
diff --git a/ling/preprocessor-usage.md b/ling/preprocessor-usage.md
index 849a5512..7812811d 100644
--- a/ling/preprocessor-usage.md
+++ b/ling/preprocessor-usage.md
@@ -1,7 +1,6 @@
The preprocessor splits text into tokens (sentences and words).
-Using Hfst as a preprocessor
-============================
+# Using Hfst as a preprocessor
The command is as follows:
@@ -27,13 +26,13 @@ Here the different parts of the preprocess command (above) is explained:
```sh
cat testfile.txt \
| hfst-tokenize --giella-cg --weight-classes=1 tools/tokenisers/tokeniser-disamb-gt-desc.pmhfst \
-# tokenize and analyse, with constraint grammar as setting
+## tokenize and analyse, with constraint grammar as setting
| vislcg3 -g tools/tokenisers/mwe-dis.cg3 \
| cg-mwesplit \
-# identify, disambiguate and format multiword expressions
+## identify, disambiguate and format multiword expressions
| vislcg3 -g src/syntax/disambiguator.cg3
-# and then to normal disambiguation, and eventually further steps
-```
+## and then to normal disambiguation, and eventually further steps
+```
With this command, text is tokenised, analysed and the output is printed
in VISLCG3 format, all in one go, and everything using a single
@@ -41,7 +40,7 @@ transducer. This will handle multiword expressions properly, including
all inflections of them. This setup replaces the older, Perl-based
solution for the Xerox tools.
-# Our old preprocess method: Using the perl script preprocess
+## Our old preprocess method: Using the perl script preprocess
Below follows the description for how to use the Perl-based solution we used until we started using hfst.
diff --git a/ling/preprocessor.md b/ling/preprocessor.md
index 3d3c8a37..113ae928 100644
--- a/ling/preprocessor.md
+++ b/ling/preprocessor.md
@@ -1,12 +1,11 @@
-Preprocessor
-============
+# Preprocessor
The preprocessor file is common to all languages. It should be
parametrised, certain languages, such as Skolt Saami and Haida, have
very deviant orthographic conventions when it comes to punctuation
marks.
-# Using Hfst as a preprocessor
+## Using Hfst as a preprocessor
Using Hfst, one can tokenise («preprocess»), analyse and print the
output in VISLCG3 format, all in one go, and everything using a single
@@ -20,8 +19,7 @@ The new command is as follows:
cat testfile.txt | hfst-tokenise --giella-cg tools/tokenisers/tokeniser-disamb-gt-desc.pmhfst| vislcg3 -g tools/tokenisers/mwe-dis.cg3 | cg-mwesplit | vislcg3 -g src/syntax/disambiguator.cg3
```
-# Overview and intro
-
+## Overview and intro
This document describes how the preprocessing of the text into separate
words is implemented in the project. The document contains overall
@@ -50,7 +48,7 @@ preprocessor and the parser. To achieve this, the lexicon entries of
expressions are planned parallel with the preprocessor (see the section
[Abbreviations and the lexical files](#lex)).
-# Dividing text to sentences
+## Dividing text to sentences
The text is divided to sentences according to sentence-delimiters (?,!
and .); the question mark and exlamation mark are always
@@ -101,8 +99,7 @@ end mark sometimes adds an extra dot to the preprocessor output,
sometimes it is recognized only later in the CG, these functions should
be unified.
-# Tokens
-
+## Tokens
The string that is given to the preprocessor is divided into tokens,
which are the basic processing units used in the lexical analysis. There
@@ -117,13 +114,13 @@ string that consist only of alphabetical characters surrounded by space
is always an instance of one word token, if it does not belong to a
multiword expressions. So if two words are accidentally typed without
separating space, it is an error that's not accounted for. Word tokens
-may contain some punctuation, such as hyphen: *Davvi-Norga.*
+may contain some punctuation, such as hyphen: _Davvi-Norga._
The punctuation that occurs in the text is divided to two classes: 1.
those that are inseparable part of a word or numeric expression and 2.
those that are independent. Independent punctuation may split the input
string into two or more parts. For example the string
-*(gielddat/guovllut).* will be divided into six parts: both parentheses
+_(gielddat/guovllut)._ will be divided into six parts: both parentheses
are their own tokens as well as the dot. Both words and the slash are
also separate tokens. The preprocessor output is thus:
@@ -136,12 +133,11 @@ also separate tokens. The preprocessor output is thus:
The splitting depends on the surrounding characters. For example, a dot
rarely splits a numerical expression into two but generally splits a
-string that consist only of alphabetical characters (*1.4.2004, Minä
-lähdin.En tule takaisin.*). The treatment of [punctuation](#punctuation)
+string that consist only of alphabetical characters (_1.4.2004, Minä
+lähdin.En tule takaisin._). The treatment of [punctuation](#punctuation)
is explained in detail later in the document.
-# The preprocess script
-
+## The preprocess script
The preprocessor is a perl-script called `preprocess`; it reads `STDIN`
for input. The preprocessor is given one command line parameter, the
@@ -212,7 +208,7 @@ punctuation which belongs to the token are defined in the constant
`$CONTAIN_PUNCT`. This constant is currently the only language dependent
part of the script.
-# Punctuation
+## Punctuation
As most of the punctuation in a numerical expression is inseparable part
of it, the numerical expressions are treated as a separate class. The
@@ -221,7 +217,7 @@ but not always, depending on the punctuation mark. In the following
table, the punctuation marks are listed:
| | | |
-|---------------------------------------|------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------|
+| ------------------------------------- | ---------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| Punctuation | Numerical expressions (digits and non-word characters) | Words (word characters, no digits) |
| (){}\[\]– «»‹›“”„‘’‚ ´\`'"?!,;\\… • ¶ | Always their own tokens. | Always their own tokens. |
| : | Belongs to the expression if not followed by space (14:30, 10:s) | Belongs to the expression if not followed by space (Namdal:as) |
@@ -231,7 +227,7 @@ table, the punctuation marks are listed:
| \_\*+= | Belongs to the expression | Always its own token |
| % | Belongs to the expression, also when separated by space (50%, 50 %) | Always its own token |
-# Abbreviations
+## Abbreviations
The abbreviations are divided in three classes according to whether they
are able to end the sentence or not. The first class contains transitive
@@ -245,22 +241,22 @@ expressions are treated as abbreviations only when they are in the
middle of the sentence, that is, when followed by lowercased letter or
punctuation. In sum:
-- **TRAB:**
- abbreviations that take an object: 'Mr. Peters'
-- **ITRAB:**
- abbreviations that do not take an object: 'Lloyds Ltd.'
-- **TRNUMAB:**
- abbreviations that take NUM objects: 'Downing Str. 10') There is
- sentence boundary after TRNUMAB only for the capital+small
- combinations and all small-initial strings that consist of more than
- one letter. For all other strings (one small letter, one or several
- capital letters) there is no sentence boundary after TRNUMAB.
+- **TRAB:**
+ abbreviations that take an object: 'Mr. Peters'
+- **ITRAB:**
+ abbreviations that do not take an object: 'Lloyds Ltd.'
+- **TRNUMAB:**
+ abbreviations that take NUM objects: 'Downing Str. 10') There is
+ sentence boundary after TRNUMAB only for the capital+small
+ combinations and all small-initial strings that consist of more than
+ one letter. For all other strings (one small letter, one or several
+ capital letters) there is no sentence boundary after TRNUMAB.
-- **NOAB:**
- abbreviations such as 'du.' (dual) which is also a common noun and
- not treated as an abbreviation at the end of a sentence.
+- **NOAB:**
+ abbreviations such as 'du.' (dual) which is also a common noun and
+ not treated as an abbreviation at the end of a sentence.
To make the preprocessor handle abbreviations and multiword expressions
better than the default, one can provide an abbreviation file to it.
@@ -279,7 +275,7 @@ follows:
Now there is a new `abbr.txt` file in that dir, and you can specify it
as an option to the `preprocess` command. See further notes above.
-# Multiword expressions
+## Multiword expressions
The preprocessor takes into account multiword expressions if they are
listed in the file `abbr.txt.` For example, names such as 'De Silva' can
@@ -287,7 +283,7 @@ be listed as multiword expressions, they are not splitted but treated as
tokens. At the moment, the size of a multiword expressions may be three
words.
-# Listing abbreviations and multiword expressions
+## Listing abbreviations and multiword expressions
All the abbreviations are extracted from the lexicon files. They are
extracted from the file as a part of the compilation process `make`. The
@@ -316,7 +312,7 @@ Usage of the script `abbr-extract`:
abbr-extract --output= --abbr_lex= --lex=
-## The structure of abbreviations.lexc
+### The structure of abbreviations.lexc
The structure of the `abbreviations.lexc` is free in all the other
respects but the extracted part must have the following syntax:
@@ -344,9 +340,9 @@ exists only upper cased version of the abbreviation.
`abbr-extract`-script generates upper case versions of all the
abbreviations (since they may begin the sentence).
-# Other
+## Other
-## XML-tags
+### XML-tags
There is a primitive XML-support in the preprocessor. XML-tags are
recognized and they can be excluded or included to the output. Choose
@@ -356,7 +352,7 @@ The XML-support is called 'primitive', since the parsing of XML-tags is
not done using any library functions. Therefore, the XML-formatting must
be quite clear to function.
-## Spaces
+### Spaces
It is possible to preserve space during the tokenization with optin
--space. The space between tokens is preserved and printed in their own
@@ -385,7 +381,7 @@ Space preservation is implemented so that each token forms a complex
unit: the word is paired with the space that preceded it. The space is
preserved also when tokens are combined to multiword expressions.
-## Typos-list
+### Typos-list
In some cases, we want to fix the common typos in the texts. The list
can be provided to preprocessor with option --corr=<file>. Typos
@@ -397,7 +393,7 @@ are a tab separated list:
Each typo is replaced in the text by its correction. Typos and their
corrections may be multiword expressions.
-## Connecting expressions
+### Connecting expressions
There is a class of expressions that connect other expressions to form
complex tokens. This is for example expressions like 'fisk- og
@@ -406,7 +402,7 @@ listed in multiword expressions. At the moment, specifically this type
of complex token is handled with option --connect=<exp1,exp2,..>,
where the words that are used as connectives can be listed.
-# Possible flaws
+## Possible flaws
Abbreviations may be followed by capital letter even when not ending the
sentence
diff --git a/ling/prop-name-meeting-20051221.md b/ling/prop-name-meeting-20051221.md
index f40e5f52..64298580 100644
--- a/ling/prop-name-meeting-20051221.md
+++ b/ling/prop-name-meeting-20051221.md
@@ -1,72 +1,61 @@
# Agenda
-
-* finalize the proper name xml structure
-* prepare integration of the kvensk project, if SD accepts
-
+- finalize the proper name xml structure
+- prepare integration of the kvensk project, if SD accepts
Participants: **Børre, Linda, Sjur, Tomi, Trond**
-
Questions:
-
-* What (the content):
-** make an overview of all info we want to store
-** ... and how to organise it
-* How (the xml structure):
-** one or two files? (two actually implies three)
-** what info to split into common parts and project / language specific parts
-
+- What (the content):
+ ** make an overview of all info we want to store
+ ** ... and how to organise it
+- How (the xml structure):
+ ** one or two files? (two actually implies three)
+ ** what info to split into common parts and project / language specific parts
Views:
-
-* Iconic id better than arbitrary id.
-* Single linking, or automatically made double linking
-* pro links in common: lg-specific files are not cluttered
-* pro links in lg files: that is where the info is
-
+- Iconic id better than arbitrary id.
+- Single linking, or automatically made double linking
+- pro links in common: lg-specific files are not cluttered
+- pro links in lg files: that is where the info is
Work process:
-
-* Timbuktu: Add iconic id and semantics once, to common.
-** The machine MAKES the lg entries, based upon these assumpt:
-*** one form, inherited from iconic id
-*** one sem, inherited from common
-*** and we will decide a default declension class for each lg
-*** we may leave a tag in place saying "untouched by human hands"
-* Helsinki: - same case, but here we need heavy manual editing
-** make a tag saying "now touched (by native speaker)"
-
+- Timbuktu: Add iconic id and semantics once, to common.
+ ** The machine MAKES the lg entries, based upon these assumpt: \*** one form, inherited from iconic id
+ **_ one sem, inherited from common
+ _** and we will decide a default declension class for each lg
+ \*\*\* we may leave a tag in place saying "untouched by human hands"
+- Helsinki: - same case, but here we need heavy manual editing
+ \*\* make a tag saying "now touched (by native speaker)"
Conclusions:
-
Double linking, iconic id
Iconic id decided by the following principle:
-* Place names: pick Norwegian, Swedish, Finnish, English names.
-* Other names: pick the most common (the one which give most "identical" hits among our lgs:
- sme, smj, sma, nor/nob/nno, swe, fin, eng (sms, smn) )
+- Place names: pick Norwegian, Swedish, Finnish, English names.
+- Other names: pick the most common (the one which give most "identical" hits among our lgs:
+ sme, smj, sma, nor/nob/nno, swe, fin, eng (sms, smn) )
With the princ of inheritance (lemma inherited from common file):
-* inherit right away / at creation time (= larger files, more duplicate info)
+- inherit right away / at creation time (= larger files, more duplicate info)
```
-common | swe | fin |
-India_2 | India | Intia |
-->lg=a | ->India_2 | ->India_2 |
-(->lg=b Intia)| ->India | |
+common | swe | fin |
+India_2 | India | Intia |
+->lg=a | ->India_2 | ->India_2 |
+(->lg=b Intia)| ->India | |
sem plc
...
-Timbuktu | Timbuktu | Timbuktu |
-->lg=a id | ->Timbuktu | ->Timbuktu |
-->lg=b id | | |
+Timbuktu | Timbuktu | Timbuktu |
+->lg=a id | ->Timbuktu | ->Timbuktu |
+->lg=b id | | |
sem plc | Tmb. |
...
@@ -79,24 +68,22 @@ sem plc | ... | | |
... ... | | |
```
-
What do we store in the "common" file
- the iconic id
- the semantics + info about the world (encyclopedic info)
- links to the lg specific files
-
+the iconic id
+the semantics + info about the world (encyclopedic info)
+links to the lg specific files
What is stored in the lang-specific ones? Linguistic info:
-* inflection
-* stem
-* lemma
-* derivation class?
-* compounding?
-* senses (pointers to concepts)
-* orthographical variants (incl. (common) misspellings)
-* acronym(s) and abbreviation(s):
-** as separate entries or as part of the name entry?
+- inflection
+- stem
+- lemma
+- derivation class?
+- compounding?
+- senses (pointers to concepts)
+- orthographical variants (incl. (common) misspellings)
+- acronym(s) and abbreviation(s):
+ \*\* as separate entries or as part of the name entry?
```
NATO => OTAN
@@ -124,51 +111,42 @@ NATO NATO+N+Prop+Org+Sg+Nom
"NATO" N Prop Org Sg Nom <<< S:1285 @HNOUN
```
-
-
-
Different aspects of abbreviations and acronyms:
-
-* expansion (requires linking/common entry):
-** abbr needs to be expanded for IR and text-to-speech
-** translation systems want to transl. them to other lg abbrs (possibly requiring
- (intermediate) expansion)
-* linguistic analysis/properties:
-** the preprocessor is concerned about abbr's behaviour wrt. sentence delimitation (TRAB, ITRAB)
-** speller programs want to correct them whenever wrongly spelled (possibly
- storing misspellings of abbrs)
-** disambiguators want their underlying POS analysis (in addition to their ABBR
- tag)
-** they have inflections of their own
-*** St.dieđ. 10 / St. dieđáhus OR St. dieđáhusa... (implicit case)
-*** NRK:as (explicit case, except for Acc/Gen, who may be left unexpressed)
-** can take part in compunding, possibly derivation
-
+- expansion (requires linking/common entry):
+ ** abbr needs to be expanded for IR and text-to-speech
+ ** translation systems want to transl. them to other lg abbrs (possibly requiring
+ (intermediate) expansion)
+- linguistic analysis/properties:
+ ** the preprocessor is concerned about abbr's behaviour wrt. sentence delimitation (TRAB, ITRAB)
+ ** speller programs want to correct them whenever wrongly spelled (possibly
+ storing misspellings of abbrs)
+ ** disambiguators want their underlying POS analysis (in addition to their ABBR
+ tag)
+ ** they have inflections of their own
+ **_ St.dieđ. 10 / St. dieđáhus OR St. dieđáhusa... (implicit case)
+ _** NRK:as (explicit case, except for Acc/Gen, who may be left unexpressed)
+ \*\* can take part in compunding, possibly derivation
Lexicon conclusion:
-* store abbr. that are coming from names as separate entries?
+
+- store abbr. that are coming from names as separate entries?
(we probably have no dotted abbrs for names)
-* store accr. as separate entries in the name database, with type="acr"
-* store alternative names as separate entries
-* all linked together or to the same concept (open??? If to the concept, forces us to allow
+- store accr. as separate entries in the name database, with type="acr"
+- store alternative names as separate entries
+- all linked together or to the same concept (open??? If to the concept, forces us to allow
more than one entry/language in the common file)
-
Transducer conclusions:
-* Leave things at status quo for the abbreviations and the acr generator
-* We will return to the issue of double abbrs if they turn up (They probably don't)
-* Double acrs arelaready taken care of in the sme-dis.rle urle set (lexical acronyms
- are preferred over generated ones)
-
-
-
-
-# xml example format:
+- Leave things at status quo for the abbreviations and the acr generator
+- We will return to the issue of double abbrs if they turn up (They probably don't)
+- Double acrs arelaready taken care of in the sme-dis.rle urle set (lexical acronyms
+ are preferred over generated ones)
-## Concept center (common file):
+## xml example format:
+### Concept center (common file):
```
@@ -205,9 +183,7 @@ Transducer conclusions:
```
-
-## Language file for, say, sme:
-
+### Language file for, say, sme:
```
@@ -225,13 +201,10 @@ Transducer conclusions:
```
+### Language file for fin:
-
-
-## Language file for fin:
(numbers refer to Irene's draft, see below)
-
```
(only if different from id/headword)
@@ -259,12 +232,11 @@ Transducer conclusions:
```
+### Language file for kvensk:
-## Language file for kvensk:
(numbers refer to Irene's draft, see the [meeting memo from Nov. 28
| https://giellalt.uit.no/admin/weekly/2005/Meeting_2005-11-28.html#7.+Name+lexicon+infrastructure])
-
```
(only if different from id/headword)
@@ -304,10 +276,8 @@ In the case that stem = lemma, we have the entry:
```
-
These points from Irene's list are still open:
-
```
Print info - do they belong to the common or language-specific sections?:
12. kartprodukt
diff --git a/ling/spoken/SpokenCorpora.md b/ling/spoken/SpokenCorpora.md
index 607d5818..6da1c1ee 100644
--- a/ling/spoken/SpokenCorpora.md
+++ b/ling/spoken/SpokenCorpora.md
@@ -2,12 +2,10 @@
Giellatekno contributes to a various degree in different spoken corpus projects:
-
-* the LIA Sápmi project with UiO
-* Komi corpus work with Freiburg and Helsinki
-* Pite Sámi work with Joshua Wilbur
-* North, Inari and Skolt Saami spoken corpus in Oulu
-
+- the LIA Sápmi project with UiO
+- Komi corpus work with Freiburg and Helsinki
+- Pite Sámi work with Joshua Wilbur
+- North, Inari and Skolt Saami spoken corpus in Oulu
Common for these is that annotation is done with the help of
Giellatekno analysers. These pages document how this is done.
diff --git a/ling/tag-standardisation.md b/ling/tag-standardisation.md
index f8335974..584f19c4 100644
--- a/ling/tag-standardisation.md
+++ b/ling/tag-standardisation.md
@@ -1,36 +1,24 @@
# Tag standardisation
-
This document outlines the format and convention used for different types of tags in our linguistic descriptions.
-
-
-
-# List grep-type "tags"
-
+## List grep-type "tags"
The following tags need standardisation as they are moved into the LexC proper.
+- SUB
+- MARG
+- NOT-KJ
+- (no more writing on this list now :-)
-* SUB
-* MARG
-* NOT-KJ
-* (no more writing on this list now :-)
-
-
-# Morphosyntactic tags
-
+## Morphosyntactic tags
describe format
-
-# Syntactic functions
-
+## Syntactic functions
describe format
-
-# More categories
-
+## More categories
add more categories as necessary
diff --git a/ling/testing.md b/ling/testing.md
index 3de32270..9c5caf28 100644
--- a/ling/testing.md
+++ b/ling/testing.md
@@ -1,112 +1,79 @@
# Testing our grammatical tools
-
This document contains an overview of the testing suits we have for our different analysers.
-
Exchange xxx with the language code in question.
$TARGET is the directory of the language, e.g. gt/sme, for sme.
+## Testing morphological tools
-# Testing morphological tools
-
-
-## Morphophonology
-
+### Morphophonology
cd $TARGET/testing && make pair-test
-
-## Selftest
-
+### Selftest
Does the analyser recognise all entries in the lexicon?
-
```
cd gt/
make selftest
```
-
(Here we might have a look at the Book.)
-
-# Testing proofing tools
-
+## Testing proofing tools
When testing proofing tools, it is possible to choose between different engines,
such as the Polderland command-line speller (default speller engine), the
command-line interface to hunspell, etc. The available engines are listed below under each proofing tool section.
+### Speller testing
-## Speller testing
-
-
-```gt$ make spelltest TARGET=xxx [TESTTOOL=yy] [DATE=ddmmyyyy]```
-
+`gt$ make spelltest TARGET=xxx [TESTTOOL=yy] [DATE=ddmmyyyy]`
Available speller engines:
+- hu - hunspell
+- pl - Polderland command line (default)
+- mw - Microsoft Word through AppleScript
-* hu - hunspell
-* pl - Polderland command line (default)
-* mw - Microsoft Word through AppleScript
-
-
-### Regression test
-
-
-### Baseform test
+#### Regression test
+#### Baseform test
-### Typos test
+#### Typos test
+#### Correct-corpus test
-### Correct-corpus test
+### Hyphenator testing
+`gt$ make hyphtest TARGET=xxx`
-## Hyphenator testing
-
-
-```gt$ make hyphtest TARGET=xxx```
-
-
-### Hyphenator regression testing
-
+#### Hyphenator regression testing
To only test regressions, do:
-
```gt$ make hyphregression TARGET=xxx [TESTTOOL=yy]
-```
+```
-# Testing syntactic analysis
-
+## Testing syntactic analysis
To run all syntactic tests:
-
gt$ make cgtests TARGET=xxx
-
Note that this test cannot be run on victorio (unless you modify the gt Makefile).
-
-## Morphological analysis consistence test
-
+### Morphological analysis consistence test
-- not yet written --
-
-## Disambiguator test
-
+### Disambiguator test
gt$ make distest TARGET=xxx
-
-## Dependency test
-
+### Dependency test
make deptest
diff --git a/ling/wikipedia_as_corpus.md b/ling/wikipedia_as_corpus.md
index 4cd523b3..1334f129 100644
--- a/ling/wikipedia_as_corpus.md
+++ b/ling/wikipedia_as_corpus.md
@@ -1,65 +1,45 @@
# Wikipedia as a corpus
+For some of our languages we use Wikipedia as a corpus. This may be
+done in several ways, here is a documentation of how to make a
+text-only version of a wikipedia edition.
+## Procedure
-
-For some of our languages we use Wikipedia as a corpus. This may be
-done in several ways, here is a documentation of how to make a
-text-only version of a wikipedia edition.
-
-
-# Procedure
-
-
-We use Moksha Mordvin (ISO code mdf) as an example, replace mdf
+We use Moksha Mordvin (ISO code mdf) as an example, replace mdf
with the language of your choice.
-
-Download the wikipedia from `http://dumps.wikimedia.org/mdfGwiki/latest/`,
+Download the wikipedia from `http://dumps.wikimedia.org/mdfGwiki/latest/`,
see [Moksha Mordvin](http://dumps.wikimedia.org/mdfwiki/latest/) for
an example. The version relevant to plain-text corpus is called
-`$LANGwiki-latest-pages-articles.xml.bz2`.
-
+`$LANGwiki-latest-pages-articles.xml.bz2`.
Unzip the .bz2 file, and save the resulting .xml file somewhere.
-
The text will be split in smaller files, so make a directory, say
mdfwiki for the output.
-
Extract the xml with the WikiExtractor.py script:
-
```
cat mdfwiki-latest-pages-articles.xml | WikiExtractor.py -o mdfwiki
```
-
The result is stored in directories AA, AB, etc. with 100 files in
each directory. You may bundle the whope Wikipedia into one file by
a command like the following:
-
```
cat mdfwiki/*/wiki*|sed 's/<[^>]*>//g;' > mdfwikicorp.txt
```
-
-
-
-# Wikipedia as a source for investigating corrections
-
+## Wikipedia as a source for investigating corrections
Other versions of Wikipedia on the download site
include all corrections of all articles (just go for the biggest
file). This makes the Wikipedia an interesting source for spellchecker
development and other types of research on text production.
-
A paper describing this in some detail is the following:
-
-* [Mining Naturally-occurring Corrections and Paraphrases from Wikipedia’s Revision History](http://www.lrec-conf.org/proceedings/lrec2010/pdf/827_Paper.pdf)
-
-
+- [Mining Naturally-occurring Corrections and Paraphrases from Wikipedia’s Revision History](http://www.lrec-conf.org/proceedings/lrec2010/pdf/827_Paper.pdf)
diff --git a/localisation/LanguageSupportAndBCP47.md b/localisation/LanguageSupportAndBCP47.md
index d73511e5..722bacb7 100644
--- a/localisation/LanguageSupportAndBCP47.md
+++ b/localisation/LanguageSupportAndBCP47.md
@@ -1,69 +1,52 @@
-# BCP-47
-
+# BCP-47
BCP-47 defines the available language identification space on the net, and in practice also on systems such as Windows, macOS, Linux, iOS and more. It builds on several independent standards and registries, which are put together as follows (simplified):
-
```
nn-OOOO-PP-qqqqqq-x-abcd-efgh
```
-
where:
-
-* **nn**: ISO 639 language codes (two-letter if availablel, three-letter 639-3 codes otherwise)
-* **OOOO**: ISO 15924 Script codes
-* **PP**: ISO 3166-1 country or area code (two letters), OR UN M.49 code (three digits)
-* **qqqqq**: IANA registered variant codes (5-8 letters, or digit + 3 alphanums)
-* **x-**: separator in front of private use tags
-* **abcd, efgh**: private use tags, 1-8 letters or digits each
-
+- **nn**: ISO 639 language codes (two-letter if availablel, three-letter 639-3 codes otherwise)
+- **OOOO**: ISO 15924 Script codes
+- **PP**: ISO 3166-1 country or area code (two letters), OR UN M.49 code (three digits)
+- **qqqqq**: IANA registered variant codes (5-8 letters, or digit + 3 alphanums)
+- **x-**: separator in front of private use tags
+- **abcd, efgh**: private use tags, 1-8 letters or digits each
The relevant standards and registries are:
+- [BCP-47](https://tools.ietf.org/html/bcp47)
+- [ISO 15924 Scriopt codes, Wikipedia](https://en.wikipedia.org/wiki/ISO_15924)
+- [ISO 15924 Scriopt codes, Unicode](https://unicode.org/iso15924/codelists.html)
+- [ISO 3166-1 country codes, Wikipedia](https://en.wikipedia.org/wiki/List_of_ISO_3166_country_codes)
+- [UN M49 area codes](https://unstats.un.org/unsd/methodology/m49/)
+- [UN M49 codes in Wikipedia](https://en.wikipedia.org/wiki/UN_M49)
+- [IANA variant registry](https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry)
-* [BCP-47](https://tools.ietf.org/html/bcp47)
-* [ISO 15924 Scriopt codes, Wikipedia](https://en.wikipedia.org/wiki/ISO_15924)
-* [ISO 15924 Scriopt codes, Unicode](https://unicode.org/iso15924/codelists.html)
-* [ISO 3166-1 country codes, Wikipedia](https://en.wikipedia.org/wiki/List_of_ISO_3166_country_codes)
-* [UN M49 area codes](https://unstats.un.org/unsd/methodology/m49/)
-* [UN M49 codes in Wikipedia](https://en.wikipedia.org/wiki/UN_M49)
-* [IANA variant registry](https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry)
-
-
-# CLDR language support
-
-
-* [List of registered languages with statistical data](https://github.com/unicode-org/cldr/blob/master/common/supplemental/supplementalData.xml)
+## CLDR language support
+- [List of registered languages with statistical data](https://github.com/unicode-org/cldr/blob/master/common/supplemental/supplementalData.xml)
-# Windows language support
-
-
-* [Available language packs and interface languages](https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/available-language-packs-for-windows) (114 unique languages)
-* [full list of recognised BCP-47 codes](https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-lcid/70feba9f-294e-491e-b6eb-56532684c37f) (frequently revised and updated):
-** 140 unique languages with an LCID (for at least one locale) as of spring 2020
-** an additional 96 unique languages without an LCID (in any locale)
+## Windows language support
+- [Available language packs and interface languages](https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/available-language-packs-for-windows) (114 unique languages)
+- [full list of recognised BCP-47 codes](https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-lcid/70feba9f-294e-491e-b6eb-56532684c37f) (frequently revised and updated):
+ ** 140 unique languages with an LCID (for at least one locale) as of spring 2020
+ ** an additional 96 unique languages without an LCID (in any locale)
This means that MS Windows recognises about 3 % of the languages of the world,
many of them for several areas, and some also for multiple scripts.
-
-# macOS language support
-
+## macOS language support
TBW.
-
-# iOS language support
-
+## iOS language support
TBW.
-
-# Android language support
-
+## Android language support
TBW.
diff --git a/localisation/Linux.md b/localisation/Linux.md
index 9a7e6cbd..c0d8c4ef 100644
--- a/localisation/Linux.md
+++ b/localisation/Linux.md
@@ -2,5 +2,4 @@ If we get a decent tool chain and work flow going, it would be nice to update
the North Sámi version of [Skolelinux](https://wiki.debian.org/DebianEdu/) — it
is lagging behind a lot these days.
-
Useful links and bits of information should be added below.
diff --git a/localisation/Localisation.md b/localisation/Localisation.md
index e15b5425..07175ca6 100644
--- a/localisation/Localisation.md
+++ b/localisation/Localisation.md
@@ -1,19 +1,17 @@
# Localisation
Other than localising our own pages, Divvun and Giellatekno do not do localisation – at least not yet.
-But we have built up a number of the tools and resources that are useful for translating into the languages we work on. Notably, we have freely accessible [translation memories and glossaries](../tm/TranslationMemory.html) for sevaral languages.
+But we have built up a number of the tools and resources that are useful for translating into the languages we work on. Notably, we have freely accessible [translation memories and glossaries](../tm/TranslationMemory.html) for sevaral languages.
-Developing and integrating these resources would be something to consider in the future. We plan to collect special translation memories for localisation only, as well as developing our *machine translation* language pairs further.
+Developing and integrating these resources would be something to consider in the future. We plan to collect special translation memories for localisation only, as well as developing our _machine translation_ language pairs further.
It should
be noted that we are not without experience in localisation - one of the Divvun
team members is the main person behind the North Sámi translation of
[Skolelinux](https://wiki.debian.org/DebianEdu/).
-
The following are a couple of pages with useful links for future localisation
projects:
-
-* [Windows](TranslatingWindows.html)
-* [Linux](Linux.html)
+- [Windows](TranslatingWindows.html)
+- [Linux](Linux.html)
diff --git a/localisation/TranslatingWindows.md b/localisation/TranslatingWindows.md
index 7f0c21b9..8ea00221 100644
--- a/localisation/TranslatingWindows.md
+++ b/localisation/TranslatingWindows.md
@@ -3,12 +3,10 @@ memory, linguistic analysis etc — all our tools — to semiautomatically local
software into the Sámi languages, and at the same time provide a work bench or
ready-made setup for doing the same for other languages.
-
One of the first targets would be Windows, which is actually possible since
about year ago. Here are some informative links:
-
-* [Adding Local Experience Packs](https://techcommunity.microsoft.com/t5/windows-it-pro-blog/adding-local-experience-packs-to-your-windows-image/ba-p/254125)
-* [Local Experience Packs: What and when](https://techcommunity.microsoft.com/t5/windows-it-pro-blog/local-experience-packs-what-are-they-and-when-should-you-use/ba-p/286841)
-* [OSLocalExperiencePacks](https://osdbuilder.osdeploy.com/docs/contentpacks/multilang-content/oslocalexperiencepacks)
-* [Where & how to find the source text](https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/add-language-packs-to-windows)
+- [Adding Local Experience Packs](https://techcommunity.microsoft.com/t5/windows-it-pro-blog/adding-local-experience-packs-to-your-windows-image/ba-p/254125)
+- [Local Experience Packs: What and when](https://techcommunity.microsoft.com/t5/windows-it-pro-blog/local-experience-packs-what-are-they-and-when-should-you-use/ba-p/286841)
+- [OSLocalExperiencePacks](https://osdbuilder.osdeploy.com/docs/contentpacks/multilang-content/oslocalexperiencepacks)
+- [Where & how to find the source text](https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/add-language-packs-to-windows)
diff --git a/ml/Background.md b/ml/Background.md
index 264d141a..ab51a048 100644
--- a/ml/Background.md
+++ b/ml/Background.md
@@ -1,47 +1,33 @@
-
-Some background notes for work on machine learning
-===============
-
+# Some background notes for work on machine learning
This is for now an ad-hoc page of links and ideas
-
-# Machine learning and language technology.
-
+## Machine learning and language technology
Mainstream language technology is dominated by neural networks.
Neural networks provide powerful tools for language modeling and there are now an increasing amount of university classes, seminars and workshops about it.
+### Good to read
-## Good to read
-* [NN for NLP course ]( https://phontron.com/class/nn4nlp2019/description.html) at Carnegie Mellon University
-* [NLPL winter school ]( http://wiki.nlpl.eu/index.php/Community/training)
-* also interesting: [Recent trends in deep learning ]( https://arxiv.org/pdf/1708.02709.pdf)
-
-
-
-
-# Things we have looked at
+- [NN for NLP course](https://phontron.com/class/nn4nlp2019/description.html) at Carnegie Mellon University
+- [NLPL winter school](http://wiki.nlpl.eu/index.php/Community/training)
+- also interesting: [Recent trends in deep learning](https://arxiv.org/pdf/1708.02709.pdf)
+## Things we have looked at
-* [Grave et al 2018: Learning Word Vectors for 157 Languages](https://arxiv.org/abs/1802.06893)
-* [Modelling Natural Language, Programs, and their Intersection](https://www.aclweb.org/anthology/N18-6001)
-** [slides](naacl18tutorial.pdf)
+- [Grave et al 2018: Learning Word Vectors for 157 Languages](https://arxiv.org/abs/1802.06893)
+- [Modelling Natural Language, Programs, and their Intersection](https://www.aclweb.org/anthology/N18-6001)
+ \*\* [slides](naacl18tutorial.pdf)
+## Ideas we have
+- MT baseline, cf. [our rule-based MT](../mt/MachineTranslation.html)
+- Stavekontrollforslag, cf. [our work on proofing](../proof/index.html)
+- Missinglist-komponent
-
-# Ideas we have
-
-
-* MT baseline, cf. [our rule-based MT](../mt/MachineTranslation.html)
-* Stavekontrollforslag, cf. [our work on proofing](../proof/index.html)
-* Missinglist-komponent
-
-
-# Moglege studentoppgåver
-
+## Moglege studentoppgåver
Vi har språklege data (i Korp nå, men vi har mer):
-* sme: over 32M tokens, nesten 3M setninger
-* nob-sme: nesten 2.5M tokens, mer enn 150K setninger
+
+- sme: over 32M tokens, nesten 3M setninger
+- nob-sme: nesten 2.5M tokens, mer enn 150K setninger
diff --git a/ml/MachineLearning.md b/ml/MachineLearning.md
index 003680cf..542e9f80 100644
--- a/ml/MachineLearning.md
+++ b/ml/MachineLearning.md
@@ -1,9 +1,7 @@
-Machine learning
-========
+# Machine learning
Work on machine learning is still on an experimental state.
-# Background documents
+## Background documents
-
-- [Background](Background.html)
+- [Background](Background.html)
diff --git a/mt/ConfiguringUpdatingMTServer.md b/mt/ConfiguringUpdatingMTServer.md
index e0974303..457ed72b 100644
--- a/mt/ConfiguringUpdatingMTServer.md
+++ b/mt/ConfiguringUpdatingMTServer.md
@@ -1,17 +1,14 @@
-Configuring and updating MT Server
-===========
-
-# Introduction
+# Configuring and updating MT Server
+## Introduction
This is the documentation for how to configure and update the MT
interfaces at
-* [jorgal.uit.no](https://jorgal.uit.no/)
-* [gtweb.uit.no/tolkimine](https://gtweb.uit.no/tolkimine/)
-* [gtweb.uit.no/mt](https://gtweb.uit.no/mt/)
-* [gtweb.uit.no/mt-testing](https://gtweb.uit.no/mt-testing/)
-
+- [jorgal.uit.no](https://jorgal.uit.no/)
+- [gtweb.uit.no/tolkimine](https://gtweb.uit.no/tolkimine/)
+- [gtweb.uit.no/mt](https://gtweb.uit.no/mt/)
+- [gtweb.uit.no/mt-testing](https://gtweb.uit.no/mt-testing/)
The translation daemon/server is running under the user "apy" on the
server gtweb (gtweb.uit.no), and the html pages also reside under the
@@ -22,29 +19,23 @@ more about how to install new pairs, update the code for
apertium-apy/apertium-html-pages, or reinstall everything in case the
server explodes.
-
-
-
-# Ensuring current pairs are up-to-date
-
+## Ensuring current pairs are up-to-date
ssh into gtweb and run
+
```
sudo dnf clean metadata
sudo dnf upgrade -y
sudo systemctl restart apy
```
-
The dnf upgrade should happen every morning, but if Tino's nightly
repo needs a lot of time to compile, or the build service is down,
things may lag a bit more. Look at the bottom of
http://apertium.projectjj.com/apt/logs/apertium-sme-nob/stderr.log if
you're interested in what might have happened.
-
-# What versions are running?
-
+## What versions are running?
gtweb.uit.no/mt-testing, gtweb.uit.no/tolkimine and
gtweb.uit.no/jorgal are running the packages from Tino's nightly repo,
@@ -53,36 +44,24 @@ broken (if the version in SVN doesn't compile when Tino's machine
tries updating, the package isn't updated). There are some more
details at /home/apy/README.org on gtweb.
+(https://build.opensuse.org/package/show/home:TinoDidriksen:nightly/apertium-sme-nob shows the latest available nightly sme-nob package, while http://apertium.projectjj.com/apt/logs/apertium-sme-nob/stderr.log shows the build log of that package.)
-(https://build.opensuse.org/package/show/home:TinoDidriksen:nightly/apertium-sme-nob shows the latest available nightly sme-nob package, while http://apertium.projectjj.com/apt/logs/apertium-sme-nob/stderr.log shows the build log of that package.)
-
-
-
-
-# How do I see unknown words?
-
+## How do I see unknown words?
```
ssh gtweb /home/apy/dump-missing-words
```
+## If jorgal is down
-
-
-
-
-# If jorgal is down
-
-
-The jorgal.uit.no page may give the message `Oversettelse ikke tilgjengelig!`
-
+The jorgal.uit.no page may give the message `Oversettelse ikke tilgjengelig!`
This command should help:
-
-```
+````
ssh -t gtweb
sudo systemctl restart apy
```
+````
diff --git a/mt/DailyCompilingOfApertiumFiles.md b/mt/DailyCompilingOfApertiumFiles.md
index a4d8aaea..8e9561c0 100644
--- a/mt/DailyCompilingOfApertiumFiles.md
+++ b/mt/DailyCompilingOfApertiumFiles.md
@@ -1,23 +1,18 @@
-
-Compiling Apertium files
-========================
+# Compiling Apertium files
Apertium needs three components:
-
1. The source language
1. The target language
1. the translation program
-
-# Fetching the files from Apertium
+## Fetching the files from Apertium
We assume you have [installed the giellalt infrastructure already](https://giellalt.uit.no/infra/GettingStarted.html). The languages are found in their respective folders in `$GTHOME/langs/`.
-Fetching the **Apertium** files is done [in the same way as for the giellalt files](https://giellalt.github.io/infra/SetUpTheFiles.html). In the path to download, exchange *giellalt* with *apertium*. The language (pair) string must of course also be changed. You probably want to put the giellalt and apertium directories in different *giellalt* and *apertium* directories.
-
-Here we show how to fetch files. First, we show how to fetch *lang-sme* from giellalt, and then the *sme-sma* pair and the *nob* language model from apertium. The commands are for the ones using git with svn-style commands:
+Fetching the **Apertium** files is done [in the same way as for the giellalt files](https://giellalt.github.io/infra/SetUpTheFiles.html). In the path to download, exchange _giellalt_ with _apertium_. The language (pair) string must of course also be changed. You probably want to put the giellalt and apertium directories in different _giellalt_ and _apertium_ directories.
+Here we show how to fetch files. First, we show how to fetch _lang-sme_ from giellalt, and then the _sme-sma_ pair and the _nob_ language model from apertium. The commands are for the ones using git with svn-style commands:
```sh
svn co https://github.com/giellalt/lang-sme.git/trunk lang-sme
@@ -37,13 +32,11 @@ git clone git@github.com:apertium/apertium-nob.git
Apertium is documented on its [github page](https://github.com/apertium) and on its [wiki](https://wiki.apertium.org/wiki/Main_Page). Released apertium language pairs can be used on [apertium.org](https://apertium.org/index.eng.html#?dir=nob-nno&q=)
-# Compiling the source and target languages
-
+## Compiling the source and target languages
For each language pair you first compile each language. Note that some languages are compiled **in Apertium**, others **in the Giellalt infrastrucutre**. Norwegian Bokmål and German are e.g. compiled in Apertium. Saami and northern languages are compiled on Giellalt.
-
-## Compiling the languages in the *giellalt* infrastructure
+### Compiling the languages in the _giellalt_ infrastructure
Go to the relevant language folder, here e.g. `sme`, and set up the configuration for MT:
@@ -53,19 +46,18 @@ cd $GTLANGS/lang-sme/
```
Now, be prepared to **wait**, from 15 minutes to several hours depending upon
-the language and your computer. The compilation procedure will store the binary
+the language and your computer. The compilation procedure will store the binary
files in `tools/mt/apertium` in
each language folder and the apertium compilers will read them from that location.
While waiting, do the same for the other language(s) you want. Go to the
-folder of the other language you want to translate as well (*sma, smj, smn*),
-so that one for e.g. *sme-sma* must compile *sme* **and** *sma*.
+folder of the other language you want to translate as well (_sma, smj, smn_),
+so that one for e.g. _sme-sma_ must compile _sme_ **and** _sma_.
Remember to reset the .configure option afterwards, e.g. to
`./configure` if that is what you use for FST work.
To check that you have compiled the relevant files, file, write:
-
```sh
ls -l tools/mt/apertium/*.gz
```
@@ -74,7 +66,7 @@ If everything went well, you have new `.gz` files in the apertium folder.
**Remember that you must have compiled BOTH the languages you want to translae between.**
-## Compiling the languages in the *apertium* infrastructure.
+### Compiling the languages in the _apertium_ infrastructure.
For language pairs involving Giellalt languages, we take Norwegian Bokmål and German from Apertium. In addition to that, Apertium contains more than 100 languages (see the documentation on the Apertium github page or the Apertium wiki).
@@ -87,7 +79,7 @@ make -j
Note that all Apertium folders contain a README file.
-# Compiling the MT program itself
+## Compiling the MT program itself
All Apertium language pairs (also the giellalt ones, e.g. sme-sma) are
stored on Apertium github:
@@ -120,9 +112,9 @@ echo ja | apertium -d. sme-nob
etc.
```
-# In case of trouble
+## In case of trouble
-## cg compiler version mismatch
+### cg compiler version mismatch
You may get this type of error message:
@@ -148,6 +140,6 @@ binary files although you have updated your compeler. In that case,
Thereafter, repeat the installation procedure.
-## More
+### More
... to be written, when reported.
diff --git a/mt/MachineTranslation.md b/mt/MachineTranslation.md
index 8e9e7f37..6644f989 100644
--- a/mt/MachineTranslation.md
+++ b/mt/MachineTranslation.md
@@ -1,21 +1,21 @@
# The Machine Translation Page
These pages documents how to build translation programs using
-*giellalt* language models and the *Apertium* MT infrastructure. The
+_giellalt_ language models and the _Apertium_ MT infrastructure. The
focus is on translation from North, Lule and South Saami to Norwegian,
between North Saami and Finnish, and between Saami languages, but the
documentation is relevant for any language pair. Giellatekno and
Divvun cooperates with [Apertium](http://wiki.apertium.org) on machine
translation.
-# Publications and links to MT systems
+## Publications and links to MT systems
- [Publications on machine translation from Giellatekno and Divvun](https://giellatekno.uit.no/MTpublications.html)
- [Machine translation programs in the Giellatekno and Divvun infrastructure](https://gtweb.uit.no/mt/)
-# Linguistic documentation
+## Linguistic documentation
-## Installing and tutorials
+### Installing and tutorials
- [Installing Apertium on your machine](DailyCompilingOfApertiumFiles.html)
- Apertium tutorials
@@ -23,63 +23,63 @@ translation.
- Cf. especially [Apertium New Language Pair HOWTO](https://wiki.apertium.org/wiki/Apertium_New_Language_Pair_HOWTO)
- Sourcefiles are found in [the Giellatekno coursed folder](https://gtsvn.uit.no/langtech/trunk/courses/apertium-for-dummies/)
-## Linguistic work
+### Linguistic work
-### Commands
+#### Commands
- [Apertium commands (in North Saami)](infra/ApertiumCommands.html)
- [Regression test ja pending test](infra/Testing.html)
- [python scripts: Comparing MT and manual translation (in North
- Saami)](infra/Paralleltexts.html)
+ Saami)](infra/Paralleltexts.html)
-### Bidix work and lexical selection
+#### Bidix work and lexical selection
- [Bidix-work (in North Saami)](infra/BidixWork.html)
- [Bidix-sanity (Improving bidix)](infra/bidixsanity.html)
- [Missinglists](infra/MissingList.html)
- [Work with missinglists and bidix, without analysers on local
- computer](infra/StWorkers.html)
+ computer](infra/StWorkers.html)
- [Hash-list](infra/HashList.html)
- [Lexical selection (in North Saami)](infra/LexicalSelection.html)
-### Transfer rules
+#### Transfer rules
- [Transfer-rules – kvasicode (in North
- Saami)](infra/TransferRules.html)
+ Saami)](infra/TransferRules.html)
- [Transfer-rules – code: relevant
- examples](infra/TransferRules_examples.html)
+ examples](infra/TransferRules_examples.html)
-### Harmonising tags
+#### Harmonising tags
- [Documenting tag transfer from Giellatekno to
- Apertium](http://wiki.apertium.org/wiki/Integration_and_tagset_conversion_with_Giellatekno)
+ Apertium](http://wiki.apertium.org/wiki/Integration_and_tagset_conversion_with_Giellatekno)
- [Derivation overview](infra/DerivationOverview.html)
-## Evaluation
+### Evaluation
- [WER Word Error Rate – Comparing MT version and corrected version
- (in North Saami)](infra/WordErrorRateTesting.html)
+ (in North Saami)](infra/WordErrorRateTesting.html)
- [Classification of corrections (in Lule
- Saami)](infra/ErrorClassification.html)
+ Saami)](infra/ErrorClassification.html)
- [Coverage April 2016](courses/sjangertest.html)
- [Coverage June 2016](courses/sjangertest2.html)
-# Technical documentation
+## Technical documentation
-## Apertium
+### Apertium
- [Updating the MT versions on
- gtweb](infra/UpdatingApertiumOnGtweb.html)
+ gtweb](infra/UpdatingApertiumOnGtweb.html)
- [Maintaining jorgal and mt-testing (the web
- services)](ConfiguringUpdatingMTServer.html)
+ services)](ConfiguringUpdatingMTServer.html)
-## OmegaT
+### OmegaT
- [OmegaT](omegat/OmegaT.html)
- [Using Apertium in OmegaT](infra/ApertiumOmegaT.html)
- [OmegaT Developer Info](omegat/OmegaTTDeveloperInfo.html)
-# Former courses
+## Former courses
- [December 2015](courses/courseDecember2015.html)
- [Februar 2016](courses/courseFebruar2016.html)
@@ -87,28 +87,28 @@ translation.
- [December 2022:](courses/courseDecember2022.html) Theme:
[Apertium recursive](https://wiki.apertium.org/wiki/Apertium-recursive)
-# Links to online MT programs
+## Links to online MT programs
- [North Saami to Norwegian Bokmål](http://jorgal.uit.no/)
- [MT between North Saami and Lule and Inari
- Saami](http://gtweb.uit.no/mt/)
+ Saami](http://gtweb.uit.no/mt/)
- [Our experimental MT programs, also between North Saami and South
- Saami](http://gtweb.uit.no/mt-testing/)
+ Saami](http://gtweb.uit.no/mt-testing/)
-# Work on language pairs
+## Work on language pairs
- [North Saami - Norwegian
- Bokmål](smenob/NorthSaamiNorwegianMachineTranslation.html)
+ Bokmål](smenob/NorthSaamiNorwegianMachineTranslation.html)
- [North Saami - South
- Saami](smesma/NorthSaamiSouthSaamiMachineTranslation.html)
+ Saami](smesma/NorthSaamiSouthSaamiMachineTranslation.html)
- [North Saami - Inari
- Saami](smesmn/NorthSaamiInariSaamiMachineTranslation.html)
+ Saami](smesmn/NorthSaamiInariSaamiMachineTranslation.html)
- [Finnish - North Saami](smefin/smefin.html)
- [North Saami - Lule
- Saami](smesmj/NorthSaamiLuleSaamiMachineTranslation.html)
+ Saami](smesmj/NorthSaamiLuleSaamiMachineTranslation.html)
- [North Saami - German](smedeu/NorthSaamiGermanMachineTranslation.html)
-# Meetings 2016
+## Meetings 2016
- [12.01.2016: project planning](meetings/160112.html)
- [14.01.2016: transfer rules](meetings/160114.html)
diff --git a/mt/courses/courseApril2016.md b/mt/courses/courseApril2016.md
index cfc0927f..fe2f4959 100644
--- a/mt/courses/courseApril2016.md
+++ b/mt/courses/courseApril2016.md
@@ -1,66 +1,60 @@
-# MT-beaivvit cuoŋománus
+# MT-beaivvit cuoŋománus
Kursa 2016
-## Praktihkalaččat
-bistá 5 beaivvi, mii álgit mánnodaga dii. 9
-
+### Praktihkalaččat
+bistá 5 beaivvi, mii álgit mánnodaga dii. 9
+## Prográmmaevttohus:
-# Prográmmaevttohus:
Trond, ML ja Lene leat hupman MT-vahku sisdoalu birra. Dát lea evttohus:
+## MÁNNODAGA
-# MÁNNODAGA
-
+### 0 Bures boahtin
-
-
-## 0 Bures boahtin
- Doaibmágo? `echo ja | apertium -d. sme-smX`
- Leago HFST ok? Kevina gohččumat ođasmahttit HFST:
- - `wget http://apertium.projectjj.com/osx/install-nightly.sh`
- - `sudo bash install-nightly.sh`
+ - `wget http://apertium.projectjj.com/osx/install-nightly.sh`
+ - `sudo bash install-nightly.sh`
- Mii kompileregoahtit sme + smX + sme-smX
- - [kompileren ja configure](../infra/Compilation.html)
+ - [kompileren ja configure](../infra/Compilation.html)
- Vahku prográmma
- Jorgalanprográmmaid stáhtus
- - [Buohtastahttin](sjangertest.html)
-
-
-## 1 Evalueret sme-teavstta MT-jorgalusa
-### Jorgalit guokte oanehis teavstta: čáppagirjjálašvuođa teavstta ja čoahkkingirjji:
-- `python check_mt-otpt.py -f pseudo_tmx_data/cahceravgga_sme2smn.tmx`
-- `open otpt_dir/cahceravgga_sme2smn.html`
-- `python check_mt-otpt.py -f pseudo_tmx_data/SP-18-2012_sme2smn.tmx`
-- `open otpt_dir/SP-18-2012_sme2smn.html`
+ - [Buohtastahttin](sjangertest.html)
+### 1 Evalueret sme-teavstta MT-jorgalusa
+#### Jorgalit guokte oanehis teavstta: čáppagirjjálašvuođa teavstta ja čoahkkingirjji:
+- `python check_mt-otpt.py -f pseudo_tmx_data/cahceravgga_sme2smn.tmx`
+- `open otpt_dir/cahceravgga_sme2smn.html`
+- `python check_mt-otpt.py -f pseudo_tmx_data/SP-18-2012_sme2smn.tmx`
+- `open otpt_dir/SP-18-2012_sme2smn.html`
- geahččat násttiid ja hash
- árvvoštallat gielalaš meattáhusaid
- árvvoštallat syntávssa
- buohtastahttit dan golbma giellapára jorgalusaid kvaliteahta (WER)
- - Evalueren: [http://xixona.dlsi.ua.es/~fran/eval/]
+ - Evalueren: [http://xixona.dlsi.ua.es/~fran/eval/]
+### 2 Buoridit jorgalanfeaillaid
-## 2 Buoridit jorgalanfeaillaid
+#### Mo gávdnat jorgalanfeailla siva
-
-### Mo gávdnat jorgalanfeailla siva
Bargat ieš, oažžut bagadeami
+### Bargat násttiid ja hash
+
+#### Missing (násti)
+
+Ovdalgo sáhttá árvvoštallat cealkaga syntávssa dahje váilevaš genererema, de eai galgga leat missing-sánit cealkagis
-## Bargat násttiid ja hash
-### Missing (násti)
-Ovdalgo sáhttá árvvoštallat cealkaga syntávssa dahje váilevaš genererema, de eai galgga leat missing-sánit cealkagis
- Lasit missing sániid bidixii, ja sánit fertejit leat leksikaliserejuvvon sihke sme- ja smX-FST:s
- [bidix-bargu](../infra/BidixWork.html)
-
-### Geahččat dan maid analysáhtor ii máhte genereret (hash):
+#### Geahččat dan maid analysáhtor ii máhte genereret (hash):
`echo 'cealkka' | apertium -d. sme-smn-dgen` , omd.
@@ -68,187 +62,148 @@ Ovdalgo sáhttá árvvoštallat cealkaga syntávssa dahje váilevaš genererema,
- buohtastahte analysáhtoriin (`usmX` ja `dsmXNorm`)
-```
-usmn
-Pariisa
-Pariisa Pariisa+N+Prop+Sem/Plc+Sg+Nom
+```
+usmn
+Pariisa
+Pariisa Pariisa+N+Prop+Sem/Plc+Sg+Nom
-dsmnNorm
-Pariisa+N+Prop+Sem/Plc+Sg+Loc
+dsmnNorm
+Pariisa+N+Prop+Sem/Plc+Sg+Loc
Pariisa+N+Prop+Sem/Plc+Sg+Loc Pariisa+N+Prop+Sem/Plc+Sg+Loc +?
```
-
Ferte lasihit/divvut FST (omd. addit +Sg+Loc leksikonii masa Pariisa manná)
-
**Nubbi ovdamearka**:
- `#lávluđ
| **ISOF**