The Lemko noun земля ⟨zemlja⟩ is translatable into English as „earth”, „ground”, or „floor”, depending on the context. It translates into Polish as ziemia.
How to Pronounce and Memorize
The first syllable is pronounced like English zen, but with an ⟨m⟩ sound at the end. The second syllable is pronounced as in „la la la”. To memorize, imagine a zen monk meditating in mud outside and saying „La la la, I can’t hear you!”
Etymology
The Lemko noun земля ⟨zemlja⟩ 'earth’ comes from Proto-Slavic *zemļà (Derksen, 2008, p. 542). Cognates include Old Church Slavonic землꙗ (ⰸⰵⰿⰾⱑ) ⟨zemlja⟩ Avestan 𐬰𐬃 ⟨zā̊⟩ 'earth’ (accusative form 𐬰𐬆𐬨 ⟨zəm⟩), Sanskrit क्ष ⟨kṣá⟩ 'earth’, Persian زمین ⟨zamin⟩ 'earth’, Ancient Greek χθών ⟨khthṓn⟩ “earth”, Hittite 𒋼𒂊𒃷 ⟨tēkan⟩, (genitive 𒁖𒈾𒀸 ⟨taknas⟩), Latin humus 'ground’, and Ancient Greek χαμαί ⟨khamaí⟩ 'on the ground’ (Vasmer 1953, pp. 452–453, see also Derksen, 2008, p. 542 and Pokorny, 1959, p. 415).
Declension
Lemko земля ⟨zemlja⟩ is a soft, first-declension noun that declines as follows:
Singular
Case
Lemko
Polish
Ukrainian
Russian
Nom
зе́мля ⟨zémlja⟩
ziemia
земля́
земля́
Gen
зе́млі ⟨zémli⟩
ziemi
землі́
земли́
Dat
зе́мли ⟨zémly⟩a
ziemi
землі́
земле́
Acc
зе́млю ⟨zémlju⟩
ziemię
зе́млю
зе́млю
Ins
зе́мльом ⟨zémlʹom⟩
ziemią
земле́ю
землёй
Loc
зе́мли ⟨zémly⟩a
ziemi
землі́
земле́
Voc
зе́мльо ⟨zémlʹo⟩b
ziemio
зе́мле
—
The singular declension of the Lemko soft first declension noun земля ⟨zemlja⟩ 'earth’ compared to its Polish, Ukrainian, and Russian cognates.
a Pyrtej (2013, p. 38) gives зе́млі ⟨zémli⟩ as the dative and locative singular forms, yet Fontański and Chomiak (2000, p. 64) provide зе́мли ⟨zémly⟩.
bFontański and Chomiak (2000, p. 64) provide земле ⟨zemle⟩ as an alternative vocative singular form.
Plural
Case
Lemko
Polish
Ukrainian
Russian
Nom
зе́млі ⟨zémli⟩
ziemie
зе́млі
зе́мли
Gen
зе́мель ⟨zémlʹ⟩
ziem
земе́ль
земе́ль
Dat
зе́млям ⟨zémljam⟩
ziemiom
зе́млям
зе́млям
Acc
зе́млі ⟨zémli⟩
ziemie
зе́млі
зе́мли
Ins
земля́ми ⟨zemljámy⟩
ziemiami
зе́млями
зе́млями
Loc
зе́млях ⟨zémljax⟩
ziemiach
зе́млях
зе́млях
Voc
зе́млі ⟨zémli⟩
ziemie
зе́млі
—
The plural declension of the Lemko soft first declension noun земля ⟨zemlja⟩ 'earth’ compared to its Polish, Ukrainian, and Russian cognates.
References
^ Derksen, Rick. (2008). In Lubotsky, A. (Ed.), Leiden Indo-European Etymological Dictionary Series: Vol. 4. Etymological Dictionary of the Slavic Inherited Lexicon. Koninklijke Brill NV. https://brill.com/view/title/12607
The forms of the Lemko word рік ⟨rik⟩ listed below are translatable into English as “year” or „years”.
Mnemonic
To memorize the Lemko word рік ⟨rik⟩, English speakers might imagine something reeking at a New Year’s Eve party (Lemko rik and English reek are pronounced practically the same).
Etymology
From Proto-Slavic *rokŭ 'time’, itself a deverbal noun from *rekti 'say’, whose cognates include Old Church Slavonic рокъ (ⱃⱁⰽⱏ) ⟨rokŭ⟩ 'time, term’, as well as possibly English reckon, Sanskrit रचयति ⟨racáyati⟩ „construct, work”, Gothic 𐍂𐌰𐌷𐌽𐌾𐌰𐌽 ⟨rahnjan⟩ 'reckon’ (Pokorny 1959, p. 863, see also Vasmer, 1955, p. 532) and Welsh rhegi 'curse’ (Derksen, 2008, pp. 433, 438).
a The nominative and accusative form of Lemko рік ⟨rik⟩ 'year’ is the same as the genitive plural of ріка ⟨rika⟩ 'river’. Horoszczak (2004, p. 330) provides the nominative and accusative singular as „рик ⟨ryk⟩, рік ⟨rik⟩”.
b See Pyrtej (2013, p. 46) for the genitive and dative singular forms of Lemko рік ⟨rik⟩ 'year’. Photograph below.
Please find below the translation, etymology, full declension tables, and references for the Lemko demonstrative pronouns тот ⟨tot⟩ meaning „this” or „these”, and тамтот ⟨tamtot⟩ meaning „that” or „those”.
The Lemko demonstrative pronoun of dictionary (masculine singular) form тот ⟨tot⟩ is translatable into English as „this” in the singular and „these” in the plural. When prefixed with там ⟨tam⟩, (for example, тамтот ⟨tamtot⟩), it is translatable as „that” in the singular and „those” in the plural.
Etymology
The Lemko demonstrative pronoun of dictionary (masculine singular) form тот ⟨tot⟩ derives from reconstructed proto-Slavic *tŭ. Further afield, it is related to the English word that and Sanskrit तत् ⟨tat⟩ (Vasmer, 1958, p. 128), translatable as „this” and appearing in the famous line तत्त्वमसि ⟨tat tvam asi⟩ meaning „That thou art”.
The entry for the Muscovite Russian demonstrative pronoun тот ⟨tot⟩ in Vasmer’s Russisches Etymologisches Wörterbuch, Dritter Band: Sta–Ÿ (1958, p. 128).
Nearby („this” and „these”)
Singular („This”)
All of the following forms are translatable into English as „this”.
Masculine
Case
Lemko
Polish
Ukrainian
Russian
Nominative
тот ⟨tot⟩a
ten
цей
э́тот
Genitive
то́го ⟨tóho⟩
tego
цього́
э́того
Dative
то́му ⟨tómu⟩
temu
цьому́
э́тому
Accusative (inanimate)
тот ⟨tot⟩a
ten
цей
э́тот
Accusative (animate)
то́го ⟨tóho⟩
tego
цього́
э́того
Instrumental
тым ⟨tŷm⟩b
tym
цим
э́тим
Locative
тым ⟨tŷm⟩c
tym
цьо́му, цім
э́том
a Pyrtej (2013) gives той⟨toj⟩ as an alternative form of the Lemko masculine nominative (as well as accusative inanimate) singular demonstrative pronoun (p. 107). That form is absent in Fontański & Chomiak (2000, p. 97).
b Pyrtej (2013) gives тим⟨tym⟩ as the Lemko form of the masculine instrumental singular demonstrative pronoun (p. 107), in contrast to the form тым ⟨tŷm⟩ appearing in Fontański & Chomiak (2000, p. 97).
c Pyrtej (2013) gives тім⟨tim⟩ as the Lemko form of the masculine locative singular demonstrative pronoun (p. 107), in contrast to the form тым ⟨tŷm⟩ appearing in Fontański & Chomiak (2000, p. 97).
Feminine
Case
Lemko
Polish
Ukrainian
Russian
Nominative
то́та ⟨tóta⟩a
ta
ця
э́та
Genitive
той ⟨toj⟩
tej
ціє́ї
э́той
Dative
тій ⟨tij⟩
tej
цій
э́той
Accusative
то́ту ⟨tótu⟩b
tę
цю
э́ту
Instrumental
том ⟨tom⟩
tą
ціє́ю
э́той, э́тою
Locative
тій ⟨tij⟩
tej
цій
э́той
a Pyrtej (2013) gives та⟨ta⟩ and та́я⟨tája⟩ as alternative forms of the Lemko feminine nominative singular demonstrative pronoun (p. 107). Those forms are absent in Fontański & Chomiak (2000, p. 97).
b Pyrtej (2013) gives ту⟨tu⟩ and ту́ю⟨túju⟩ as alternative forms of the Lemko feminine accusative singular demonstrative pronoun (p. 107). Those forms are absent in Fontański & Chomiak (2000, p. 97).
Neuter
Case
Lemko
Polish
Ukrainian
Russian
Nominative
то́то ⟨tóto⟩a
to
це
э́то
Genitive
то́го ⟨tóho⟩
tego
цього́
э́того
Dative
то́му ⟨tómu⟩
temu
цьому́
э́тому
Accusative
то́то ⟨tóto⟩
to
цей
э́то
Instrumental
тым ⟨tŷm⟩b
tym
цим
э́тим
Locative
тым ⟨tŷm⟩c
tym
цьо́му, цім
э́том
a Pyrtej (2013) gives то⟨to⟩ and то́є⟨tóje⟩ as alternative forms of the Lemko neuter nominative singular demonstrative pronoun (p. 107). Those forms are absent in Fontański & Chomiak (2000, p. 97).
b Pyrtej (2013) gives тим⟨tym⟩ as the Lemko form of the neuter instrumental singular demonstrative pronoun (p. 107), in contrast to the form тым ⟨tŷm⟩ appearing in Fontański & Chomiak (2000, p. 97).
c Pyrtej (2013) gives тім⟨tim⟩ as the Lemko form of the neuter locative singular demonstrative pronoun (p. 107), in contrast to the form тым ⟨tŷm⟩ appearing in Fontański & Chomiak (2000, p. 97).
The following forms are used regardless of grammatical gender and are translatable into English as „these”.
Case
Lemko
Polish
Ukrainian
Russian
Nominative
то́ты ⟨tótŷ⟩
te/ci
ці
э́ти
Genitive
тых ⟨tŷch⟩
tych
цих
э́тих
Dative
тым ⟨tŷm⟩
tym
цим
э́тим
Accusative (inanimate)
то́ты ⟨tótŷ⟩
te
ці
э́ти
Accusative (animate)
тых ⟨tŷch⟩
tych
цих
э́тих
Instrumental
ты́ма ⟨tŷma⟩
tymi
ци́ми
э́тими
Locative
тых ⟨tŷch⟩
tych
цих
э́тих
Distant („that”, „those”)
To communicate distance from the speaker, simply prefix all of the above pronouns with Lemko там ⟨tam⟩. This is equivalent to saying „that” instead of „this” or „those” instead of „these” in English.
Singular („That”)
Case
Masculine
Feminine
Neuter
Nominative
тамтот ⟨tamtot⟩a
тамто́та ⟨tamtóta⟩d
тамто́то ⟨tamtóto⟩x
Genitive
тамто́го ⟨tamtóho⟩
тамтой ⟨tamtoj⟩
тамто́го ⟨tamtóho⟩
Dative
тамто́му ⟨tamtómu⟩
тамтій ⟨tamtij⟩
тамто́му ⟨tamtómu⟩
Accusative (inanimate)
тамтот ⟨tamtot⟩a
тамто́ту ⟨tamtótu⟩e
тамто́то ⟨tamtóto⟩
Accusative (animate)
тамто́го ⟨tamtóho⟩
тамто́ту ⟨tamtótu⟩e
тамто́то ⟨tamtóto⟩
Instrumental
тамтым ⟨tamtŷm⟩b
тамтом ⟨tamtom⟩
тамтым ⟨tamtŷm⟩b
Locative
тамтым ⟨tŷm⟩c
тамтій ⟨tamtij⟩
тамтым ⟨tamtŷm⟩c
a Pyrtej (2013) gives той⟨toj⟩ as an alternative form of the Lemko masculine nominative (as well as accusative inanimate) singular demonstrative pronoun (p. 107). That form is absent in Fontański & Chomiak (2000, p. 97).
b Pyrtej (2013) gives тим⟨tym⟩ as the Lemko form of the masculine and neuter instrumental singular demonstrative pronoun (p. 107), in contrast to the form тым ⟨tŷm⟩ appearing in Fontański & Chomiak (2000, p. 97).
c Pyrtej (2013) gives тім⟨tim⟩ as the Lemko form of the masculine and neuter locative singular demonstrative pronoun (p. 107), in contrast to the form тым ⟨tŷm⟩ appearing in Fontański & Chomiak (2000, p. 97).
d Pyrtej (2013) gives та⟨ta⟩ and та́я⟨tája⟩ as alternative forms of the Lemko feminine nominative singular demonstrative pronoun (p. 107). Those forms are absent in Fontański & Chomiak (2000, p. 97).
e Pyrtej (2013) gives ту⟨tu⟩ and ту́ю⟨túju⟩ as alternative forms of the Lemko feminine accusative singular demonstrative pronoun (p. 107). Those forms are absent in Fontański & Chomiak (2000, p. 97).
f Pyrtej (2013) gives то⟨to⟩ and то́є⟨tóje⟩ as alternative forms of the Lemko neuter nominative singular demonstrative pronoun (p. 107). Those forms are absent in Fontański & Chomiak (2000, p. 97).
Artificial-intelligence powered neural machine translation might soon resuscitate endangered languages by empowering new speakers to communicate in real time using sentences quantifiably closer to the literary norm than those of native speakers, and starting from day one of their language reclamation journey. While Silicon Valley has been investing enormous resources into neural translation technology capable of superhuman speed and accuracy for the world’s most widely used languages, 98% have been left behind, for want of corpora: neural machine translation models train on millions of words of bilingual text, which simply do not exist for most languages, and cost upwards of a hundred thousand United States dollars per tongue to assemble.
For low-resource languages, there is a more resourceful approach, if not a more effective one: transfer learning, which enables lower-resource languages to benefit from achievements among higher-resource ones. In this experiment, Google’s English-Polish neural translation service was coupled with my classical, rule-based engine to translate from English into the endangered, low-resource, East Slavic language of Lemko. The system achieved a bilingual evaluation understudy (BLEU) quality score of 6.28, several times better than Google Translate’s English to Standard Ukrainian (BLEU 2.17), Russian (BLEU 1.10), and Polish (BLEU 1.70) services. Finally, the fruit of this experiment, the world’s first English to Lemko translation service, was made available at the web address www.LemkoTran.com to empower new speakers to revitalize their language.
New speakers are key to language revitalization, and the power to “say it right” in Lemko is now at their fingertips.
Keywords: Human-Centered AI, Language Revitalization, Lemko.
Please cite as: Orynycz, P. (2022). Say It Right: AI Neural Machine Translation Empowers New Speakers to Revitalize Lemko. In: Degen, H., Ntoa, S. (eds) Artificial Intelligence in HCI. HCII 2022. Lecture Notes in Computer Science(), vol 13336. Springer, Cham. https://doi.org/10.1007/978-3-031-05643-7_37
This experiment aims to contribute at the local level to the global challenge of language loss, which may be occurring at the rate of one per day, with as few as one tongue in ten set to survive [1, p. 1329]. At press time, SIL International’s Ethnologue uses Lewis and Simons’ 2010 Expanded Graded Intergenerational Disruption Scale to estimate that 3,018 languages are endangered [2], which is 43% of the 7,001 individual living ones tallied at press time in International Organization for Standardization standard ISO 639-3 [3]. Meanwhile, Google Translate only serves 108 [4], and Facebook, 112 [5], which is a start. Nevertheless, one less language is now underserved, as the fruit of this experiment has been deployed to a web server as a public translation service.
New, artificial intelligence technologies beckon with the promise of an aid that instantly compensates for language loss via human-computer interaction. In my previous experiment, next-generation neural engines achieved higher quality scores translating from Russian and Polish into English than the human control [6, p. 9]. Meanwhile, Facebook and Google1 have invested enormous resources into delivering better-than-human automatic translation systems at zero cost to consumer.
1 Disclosure: I work as a paid Russian, Polish, and Ukrainian linguist and translation quality control specialist for the Google Translate project; headquarters are in San Francisco.
Superhuman artificial intelligence does not come cheap: training neural language models requires bilingual corpora with wordcounts in the hundreds of thousands, and ideally, millions, which would cost hundreds of thousands of dollars to translate, sums beyond the means of most low-resource language communities. Fortunately, this experiment shows that there are more resourceful and effective ways to respond to the challenge of creating translation aids for revitalizing endangered languages in low-resource settings.
1.2 Work So Far
I built the world’s first Lemko to English machine translation system and have made it available to the public. Its objective translation quality scores have been improving: the engine achieved a bilingual evaluation understudy (BLEU) score of 14.57 in the summer of 2021, as presented to professionals at the National Defense Industrial Association’s Interservice/Industry Training, Simulation and Education Conference and published in its proceedings [6]. For reference, I scored BLEU 28.66 as a human translator working in field conditions, cut off from the outside world. By the autumn of 2021, the engine had reached BLEU 15.74, as reported to linguists, academics, and the wider community at an unveiling event hosted by the University of Pittsburgh.2
2 Disclosure: the event was sponsored by the Carpatho-Rusyn Society (Pennsylvania), and I was paid by the University of Pittsburgh for my presentation.
1.3 System Under Study
Lemko is a definitively to severely endangered [6, p. 3, 7, pp. 177-178], low-resource [8], officially recognized minority language [9] presumably indigenous to transborder highlands south of the Cracow, Tarnów, and Rzeszów metropolitan areas; historical demarcating isoglosses will hopefully be the topic of a future paper. Poland’s census bureau tallied 6,279 residents for whom Lemko was a language “usually used at home” (even if in addition to Polish) in 2011 [10, p. 3], a 12% increase from the 5,605 for whom Lemko was a “language spoken most often at home” in 2002 [11, p. 6, 12, p. 7]. At press time, the results of a fresh count are being tabulated.
Lemko is classifiable as an East Slavic language as it fits the customary genetic structural feature criteria, the most significant of which is pleophony [13, p. 20], whereby a vowel is assumed to have arisen in proto-Slavic sequences of consonant C followed by mid or low vowel V (*e, or *o, with which *a had merged [14, p. 366]), followed by liquid R (that is, *l or *r), followed by another consonant C, that is, CVRC > CVRVC. To illustrate, compare the Old English word for “melt”, meltan (CVRC) [15, p. 718] to its putative Lemko cognate mołódyj [16, p. 92, 17, p. 150] (CVRC), meaning “young”. Other East Slavic cognates include Ukrainian mołodýj and Russian mołodój[17], both exhibiting a vowel after the liquid (CVRVC). Meanwhile, West Slavic languages lack a vowel before the liquid; compare Polish młody and Slovak mladý (both CRVC) [17]. Further afield, kinship has been posited for other words translatable as “mild”, including Sanskrit mṛdú (CRC) [18, p. 830] and Latin mollis (CVRC if from *moldvis) [15, 17, 19, p. 323].
How well Lemko meets customary, modern Ukrainian genetic structural feature criteria was not evaluated in this experiment. However, similarity between Lemko and Standard Ukrainian was quantified, for the first time in print of which I am aware. Below, my Lemko engine scored BLEU 6.28, nearly three times the score of Google Translate’s Ukrainian at BLEU 2.17. Further experiments could be performed for the purposes of quantification of similarity between Lemko, Standard Ukrainian, Polish, and Rusyn as codified in Slovakia, as well as a fresh take on the typological classification of Lemko.
The quantity and quality of resources have been improving, as has resourcefulness empowered by technology. All known bilingual corpora, comprising fewer than seventy thousand Lemko words, were mustered for this experiment. I have been cleaning a bilingual corpus of transcriptions of interviews conducted with native speakers in Poland and my translations into English, which a United States client paid me to perform and permitted me to use. I am also compiling monolingual corpora, which total 534,512 words at press time.
1.4 Hypothesis
Based on my subjective impression as a professional translator that Lemko native speakers interviewed in Poland were more likely to use words with obvious Polish cognates than Standard Ukrainian ones, I hypothesized that, all else being equal, a machine could be configured to translate into Lemko from English and achieve BLEU objective quality scores higher than those of Google Translate’s Ukrainian and Russian services.
1.5 Predictions
Lemko Translation System. I predicted that the aforementioned translation system would achieve a BLEU score of 15 translating into Lemko from English against the bilingual corpus.
Google Translate.
English to Ukrainian service. I predicted that Google Translate’s English to Ukrainian service would achieve a BLEU score of 10 against the bilingual corpus.
English to Russian service. I predicted that Google Translate’s English to Russian service would achieve a BLEU score of 1 against the bilingual corpus.
1.6 Methods and Justification
In the interest of speed, resource conversation, and ruggedizability, a laptop computer discarded as obsolete by my employer was configured to translate into Lemko and make calls to the Google Cloud Platform Google Translate service, as well as configured to evaluate said translations using the industry standard BLEU metric.
1.7 Principal Results
The English to Lemko translation system achieved a cumulative BLEU score of 6.28431824990417. Meanwhile, Google Translate’s Ukrainian service scored BLEU 2.16830846776652, its Russian service BLEU 1.10424105952048, and the control of Polish transliterated into the Cyrillic alphabet BLEU 1.70036447680114.
2 Materials and Methods
The above hypothesis was tested by calculating BLEU quality scores for each translation system set up in the manner detailed below.
2.1 Setup
Hardware. The experiment was conducted on an HP Elitebook 850 G2 laptop with a Core i7-5600U 2.6GHz processor, and 16 gigabytes of random-access memory. It had been discarded by my employer as obsolete and listed for sale at USD 450 at time of press.
Configuration. In the basic input/output system (BIOS) menu, the device was configured to enable Virtualization Technology (VTx).
Operating System. Windows 10 Professional 64 bit had been installed on bare metal. It was ensured that Virtual Machine Platform and Windows Subsystem for Linux Windows features were enabled. Next, the WSL2 Linux kernel update for x64 machines (wsl_update_x64.msi) available from Microsoft at https://aka.ms/wsl2kernel was installed.
Software. The Docker Desktop for Windows version 4.4.3 (73365) installer was downloaded from https://www.docker.com/get-started and run with the option to Install required Windows components for WSL 2 selected.
Packages. The experiment depended on the below packages from the Python Package Index.
SacreBLEU. Version 2.0.0 was installed using the Python package documented at the following universal resource locator (URL): https://pypi.org/project/sacrebleu/2.0.0/
The above dependencies were specified in the requirements file as follows: google-cloud-translate==2.0.1 sacrebleu==2.0.0
Container.
Build. The experiment was run in a Docker container featuring the latest version of the Python programming language, which was version 3.10.2 at the time, running on the Debian Bullseye 11 Linux operating system of AMD64 architecture, of Secure Hash Algorithm 2 shortened digest bcb158d5ddb6, obtainable via the following command: docker pull python@sha256:bcb158d5ddb636fa3aa567c987e7fcf61113307820d466813527ca90d60fedc7
Runtime. The container was configured to save raw experiment data files to a local bind mounted volume.
Translation Quality Scoring. Translation quality scores were calculated according to the BLEU metric using version 2.0.0 of the SacreBLEU tool invented by Post [20].
Case sensitivity. The evaluation was performed in a case-sensitive manner.
Tokenization. Segments were tokenized using version 13a of the Workshop on Statistical Machine Translation standard scoring script metric internal tokenization procedure.
Smoothing Method. The smoothing technique developed at the National Institute of Standards and Technology by United States Federal Government employees for their Multimodal Information Group BLEU toolkit, being the third technique described by Chen and Cherry [21, p. 363], was employed by default.
Signature. The above settings produced the following signature: nrefs:1|case:mixed|eff:no|tok:13a|smooth:exp|version:2.0.0
Calibration. Configured as above, the machine produces the following output:
Explanation. The hypothesis was different from the reference by two characters. The machine mistranslated the particle negating the verb, using the word for “no” (ni) instead of the expected word for “not” (ne). This has since been largely fixed. The machine also added a comma after pamjatam, which means “I remember”. That dropped the score from what would have been a perfect score of 100 to 43.47.
Control. As the corpus is based on interviews conducted in Poland, translations into Polish were used as a control. They were transliterated into the Cyrillic alphabet by reversing the rules for transliterating Lemko names established by Poland’s Ministry of the Interior and Administration [22, p. 6564]. Polish nasal vowels were decomposed into a vowel plus a nasal stop, except before approximants, where they were directly denasalized. Word finally, the front nasal vowel /ę/ was simply denasalized, and the back one /ą/ was transliterated as if followed by a dental stop.
3 Results
The engine available to the public at www.LemkoTran.com took first place with a cumulative translation quality score of BLEU 6.28, nearly three times that of the runner-up, Google Translate’s English-Ukrainian service (BLEU 2.17). Next was its English-Polish service (BLEU 1.70), with its English-Russian service in last place (BLEU 1.10).
Table 1. English to Lemko Translation Quality: LemkoTran.com versus Google Translate
3.1 Results by machine translation service
Control. When transliterated into the Cyrillic alphabet, Google Translate’s translations into Standard Polish achieved a corpus-level BLEU score of 1.70. Samples of its performances are as follows:
Segment 2174.
English source
We had still been in Izby, right.
Lemko reference and transliteration
То мы іщы были в Ізбах, так.
To mŷ iščŷ bŷly v Izbach, tak.
Polish hypothesis and transliteration
Билісьми єще в Ізбах, так.
Byliśmy jeszcze w Izbach, tak.
Score
BLEU = 46.20
Segment 854.
English source
And that's what it's all about.
Lemko reference and transliteration
І о то ходит.
I o to chodyt.
Polish hypothesis and transliteration
І о то власьнє ходзі.
I o to właśnie chodzi.
Score
BLEU = 32.47
Segment 217.
English source
And that's what it's all about.
Lemko reference and transliteration
Так мі повіл.
Tak mi povil.
Polish hypothesis and transliteration
Так мі повєдзял.
Tak mi powiedział.
Score
BLEU = 35.36
Hybrid English-Lemko Engine. The engine freely available to the public at the URL www.LemkoTran.com achieved a corpus-level BLEU score of 6.28.
Segment 1031.
English source
Everything was there.
Lemko reference and transliteration
Вшытко там было.
Všŷtko tam bŷlo.
Lemkotran.com hypothesis and transliteration
Вшытко там было.
Všŷtko tam bŷlo.
Score
BLEU = 100.00
Segment 1445.
English source
But that officer took that medal and said,
Lemko reference and transliteration
Але тот офіцер взял тот медаль і повідат:
Ale tot oficer vzial tot medal' i povidat:
Lemkotran.com hypothesis and transliteration
Але тот офіцер взял тот медаль і повіл:
Ale tot oficer vzial tot medal' i povil:
Score
BLEU = 75.06
Segment 217.
English source
That's what he said to me.
Lemko reference and transliteration
Так мі повіл.
Tak mi povil.
Lemkotran.com hypothesis and transliteration
Так мі повіл.
Tak mi povil.
Score
BLEU = 100.00
Ukrainian. Google Translate’s translations into Standard Ukrainian achieved a corpus-level BLEU score of 2.35.
Segment 2419.
English source
Where and when?
Lemko reference and transliteration
Де і коли?
De i koly?
Ukrainian hypothesis and transliteration
Де і коли?
De i koly?
Score
BLEU = 100.00
Segment 1096.
English source
We were there for three months.
Lemko reference and transliteration
Там зме были три місяці.
Tam zme bŷly try misiaci.
Ukrainian hypothesis and transliteration
Ми були там три місяці.
My buly tam try misjaci.
Score
BLEU = 30.21
Segment 2513.
English source
Well, here to the west.
Lemko reference and transliteration
Но то ту на захід.
No to tu na zachid.
Ukrainian hypothesis and transliteration
Ну, тут на захід.
Nu, tut na zachid.
Score
BLEU = 30.21
Russian. Google Translate’s English to Russian service achieved a corpus-level BLEU score of 1.10.
Segment 432.
English source
Nobody knew.
Lemko reference and transliteration
Нихто не знал.
Nychto ne znal.
Russian hypothesis and transliteration
Никто не знал.
Nikto ne znal.
Score
BLEU = 59.46
Segment 2751.
English source
What did they expel us for?
Lemko reference and transliteration
За што нас выгнали?
Za što nas vŷhnaly?
Russian hypothesis and transliteration
За что нас выгнали?
Za čto nas vygnali?
Score
BLEU = 42.73
Segment 2164.
English source
Brother went off to war.
Lemko reference and transliteration
Брат пішол на войну.
Brat pišol na vojnu.
Russian hypothesis and transliteration
Брат ушел на войну.
Brat ušel na vojnu.
Score
BLEU = 42.73
4 Discussion
The Lemko translation system corpus-level BLEU score of 6.28 indicates that while there is much still to be done, things are on track. The Standard Russian score of BLEU 1.10 indicates that Lemko is less similar to Russian than Polish (BLEU 1.70). Perhaps using pre-revolutionary orthography could boost Russian’s score, but that would be an expensive experiment with little obvious benefit.
The transliterated Standard Polish control similarity score of BLEU 1.70 indicates less interference from the dominant language in Poland than might be expected. It would be interesting to redesign the experiment where a handful of computationally inexpensive and obvious sound correspondences (for example, denasalization of *ę to /ja/ and *ǫ to /u/, retraction of *i to /y/, and change of *g to /h/ [23]) were applied to Polish to see if it then scored higher than Standard Ukrainian.
In summary, Lemko has been synthesized in the lab and the power to produce it placed in the hands of speakers both new and native. After a thorough engine overhaul and glossary ramp-up, the next step is to objectively measure, and if feasible, have speakers subjectively rate, the quality of synthetic Lemko versus that produced by native speakers. The day when new speakers of low-resource languages can use machine translation to start communicating in their language overnight is closer, as is the day the Lemko language joins the ranks of those previously endangered, but now revitalized.
Acknowledgements. I would like to thank my colleague Ming Qian of Peraton Labs for inspiring me to conduct this experiment, and Brian Stensrud of Soar Technology, Inc. for introducing us, as well as his encouragement.
I would also like to thank my friend Corinna Caudill for her encouragement and personal interest in the project, as well as for introducing me to Carpatho-Rusyn Society President Maryann Sivak of the University of Pittsburgh, whom I would like to thank for the opportunity to present my work.
I would also like to thank Maria Silvestri of the John and Helen Timo Foundation for conducting interviews with Lemko native speakers and donating the transcripts and my translations of them to research and development.
I would like to Achim Rabus of the University of Freiburg and Yves Scherrer of the University of Helsinki for their interest in the project and ideas.
I would also like to thank Myhal’ Lŷžečko of the minority-language technology blog InterFyisa for his early interest in the project and community outreach.
I would also like to thank fellow son of Zahoczewie Marko Łyszyk for his interest in the project and community outreach.
Finally, I would like to thank my co-author and Antech Systems Inc. colleague Tom Dobry for his encouragement and guidance.
2. ^ Eberhard, D. M., Simons, G. F., & Fennig, C. D.: Ethnologue: Languages of the World, SIL International. Twenty-fourth edition. SIL International, Dallas (2021). Online version: How many languages are endangered?, https://www.ethnologue.com/guides/how-many-languages-endangered, last accessed 2022/02/11.
6. ^^ Orynycz, P., Dobry, T., Jackson, A., & Litzenberg, K.: Yes I Speak… AI Neural Machine Translation in Multi-Lingual Training. In: Proceedings of the Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) 2021, Paper no. 21176. National Training and Simulation Association, Orlando (2021). https://www.xcdsystem.com/iitsec/proceedings/index.cfm?Year=2021&AbID=96953&CID=862
7. ^ Duć-Fajfer, O.: Literatura a proces rozwoju i rewitalizacja tożsamości językowej na przykładzie literatury łemkowskiej. In: Olko, J., Wicherkiewicz, T., Borges, R. (eds.), Integral Strategies for Language Revitalization, pp. 175–200. First edition. Faculty of „Artes Liberales”, University of Warsaw, Warsaw (2016).
8. ^ Scherrer, Y., Rabus, A.: Neural morphosyntactic tagging for Rusyn. In: Mitkov, R., Tait, J., Boguraev, B. (eds.), Natural Language Engineering, 25(5), 633–650. Cambridge University Press, Cambridge (2019). https://doi.org/10.1017/S1351324919000287
17. ^^^^ Vasmer, M. Russisches etymologisches Wörterbuch. Zweiter Band. Carl Winter, Universitätsverlag, Heidelberg (1955).
18. ^ Monier-Williams, M.: A Sanskrit-English Dictionary Etymologically and Philologically Arranged with Special Reference to Cognate Indo-European Languages, The Clarendon Press, Oxford (1899).
19. ^ Derksen, R.: Etymological Dictionary of the Slavic Inherited Lexicon. In: Lubotsky, A. (ed.) Leiden Indo-European Etymological Dictionary Series, vol. 4, Koninklijke Brill, Leiden (2008).
20. ^ Post, M.: A Call for Clarity in Reporting BLEU Scores. In: Proceedings of the Third Conference on Machine Translation (WMT), vol. 1, pp. 186–191. Association for Computational Linguistics, Brussels (2018). https://aclanthology.org/W18-63
21. ^ Chen B., Cherry, C.: A Systematic Comparison of Smoothing Techniques for Sentence-Level BLEU. In: Proceedings of the Ninth Workshop on Statistical Machine Translation, pp. 362–367. Association for Computational Linguistics, Baltimore (2014). http://dx.doi.org/10.3115/v1/W14-33
22. ^ Ministerstwo Spraw Wewnętrznych i Administracji: Rozporządzenie Ministra Spraw Wewnętrznych i Administracji z dnia 30 maja 2005 r. w sprawie sposobu transliteracji imion i nazwisk osób należących do mniejszości narodowych i etnicznych zapisanych w alfabecie innym niż alfabet łaciński. In: Dziennik Ustaw Nr 102, pp. 6560–6573. Rządowe Centrum Legislacji, Warsaw (2005).
23. ^ Shevelov, G.: On the Chronology of H and the New G in Ukrainian. In: Harvard Ukrainian Studies, vol. 1, no. 2, pp. 137–152. Harvard Ukrainian Research Institute, Cambridge (1977). https://www.jstor.org/stable/40999942
To be or not to be? Быти або не быти? That is the question, and now you can conjugate the infinitives made famous by the opening line of Hamlet’s soliloquy in Lemko using the automatic translation service LemkoTran, or craft your own copulae using this handy DIY guide.
The Lemko verb быти (scientific transliteration: ⟨bŷty⟩) means”to be” in English, być in Polish, бути ⟨buty⟩ in Standard Ukrainian, and быть ⟨byt’⟩ in Muscovite Russian.
English
Lemko
Polish
Ukrainian
Russian
be
быти ⟨bŷty⟩
być
бути
быть
Translations of the Lemko verb быти into English, Polish, Ukrainian, and Russian.
Etymology
The Lemko infinitive быти ⟨bŷty⟩, meaning „to be”, comes from the Proto-Slavic athematic verb *byti, and is related to Sanskrit भूति ⟨bhūtíṣ⟩ „wellbeing” (Vasmer 1953, p. 159; Pokorny 1959 147), Persian بودن ⟨būdan⟩ „be” (Pokorny, p. 147), Latin futūrus „future” (Vasmer, p. 159, Pokorny, p. 149), and via Old English bēon, English be (Pokorny, p. 149).
Attestation
Hamlet’s famous opening line „To be or not to be, that is the question” is alluded to in the following published pieces found in the wild:
Для дакотрых орґанізаций є то быти або не быти, значыт, без тых грошів не сут в силі нич зреализувати.” (LEM.fm 2021)
Transcription
dl'a dakotrŷch organizacyj je to bŷty abo ne bŷty, značŷt, bez tŷch hrošiv ne sut v syl'i nyč zrealyzuvaty.
Translation
For some organizations, it's to be or not to be, meaning they will not be able to achieve anything without those funds.
От нашых діл и нашой віры буде рішатися вопрос: ци нам лемкам быти, ци не быти?….” (Цисляк 1964, p. 162)
Transliteration
Ot našŷch dil y našoj virŷ bude rišatysia vopros: cy nam lemkam bŷty, cy ne bŷty?…
Translation
Our affairs and our faith will be decide the question of whether we Lemkos are to be or not to be…
Inflection
Future Tense
Root: буд– ⟨bud-⟩
The future tense of the Lemko verb for to be, быти ⟨bŷty⟩, is formed by adding personal endings to the root bud-, equivalent to will in English.
Etymology
Lemko bud- comes from the Proto-Slavic root *bǫd-. Compare the suffix -bund in English moribund from Latin moribundus (Pokorny, p. 150, Vasmer, p. 136).
Conjugation Table
English
Lemko
Polish
Ukrainian
Russian
I will
буду ⟨búdu⟩
będę
буду
буду
you will
будеш⟨búdeš⟩
będziesz
будеш
будешь
(s)he will
буде ⟨búdet⟩
będzie
буде
будет
we will
будеме ⟨budéme⟩
będziemy
будемо
будем
you all will
будете ⟨budéte⟩
będziecie
будете
будете
they will
будут ⟨búdut⟩
będą
будуть
будут
Forms of the future tense conjugation of Lemko verb быти ⟨bŷty⟩ translated into English, Polish, Standard Ukrainian, and Russian.
In Lemko, the present tense of the verb to be is formed in the singular from the root є- ⟨je-⟩, and in the plural from the root с- ⟨s-⟩.
Etymology
All the below forms trace back to the ancestor of the Proto-Slavic root *es-, to which personal endings were affixed. Compare to English is, German ist, Latin est, Ancient Greek ἐστί ⟨estí⟩, Persian است ⟨ast⟩, and Sanskrit अस्ति ⟨ásti⟩ (Pokorny, pp. 340-341; Vasmer, p. 405).
Conjugation Table
English
Lemko
Polish
Ukrainian
Russian
I am
єм ⟨jem⟩
jestem
є
есть
you are
єс ⟨jes⟩
jesteś
є
есть
(s)he is
єст ⟨jest⟩a
jest
є
есть
we are
сме ⟨sme⟩b
jesteśmy
є
есть
you all are
сте ⟨ste⟩c
jesteście
є
есть
they are
сут ⟨sut⟩
są
є
есть
Forms of the present tense conjugation of the Lemko verb быти ⟨bŷty⟩ translated into English, Polish, Standard Ukrainian, and Russian.
a The Lemko third-person singular form єст ⟨jest⟩ is now being replaced by є ⟨je⟩, though this is still rare (Fontański & Chomiak 2000, p. 109).
bFontański & Chomiak (2000, p. 109) give the Lemko first-person plural form as (єсме)сме/зме ⟨(jesme)sme/zme⟩.
cFontański & Chomiak (2000, p. 109) give the Lemko second-person plural form as (єсте)сте ⟨(jeste)ste⟩.
The past tense of the verb „to be” is formed in Lemko by adding any appropriate gender and plural markers to the stem был- ⟨bŷl-⟩, translatable into English as was or were.
Etymology
Lemko был⟨bŷl⟩ is undoubtedly the continuation of Proto-Slavic resultative participle *bylŭ. Compare to Ancient Greek φῦλον ⟨phylon⟩ (Vasmer, p. 159), whence English phylum.
Conjugation Tables
Masculine
Use the following to refer to males or mixed parties of males and females, as well as objects of grammatically masculine gender. Male virility is not a grammatical category in Lemko, unlike Polish.
English
Lemko
Polish
Ukrainian
Russian
I was
я былa ⟨ja bŷl⟩
byłem
я був
я был
you were
ты былb ⟨tý bŷl⟩
byłeś
ти був
ты был
he was
він был ⟨vin bŷl⟩
był
він був
он был
we were
мы былиc ⟨mŷ bŷly⟩
byliśmy
ми були
мы были
you guys were
вы былиd ⟨vŷ bŷly⟩
byliście
ви були
вы были
those guys were
они были ⟨ony bŷly⟩
byli
вони були
они были
Forms of the masculine past tense conjugation of the Lemko verb быти ⟨bŷty⟩ translated into English, Polish, Standard Ukrainian, and Russian.
aFontański & Chomiak (2000, p. 109) cite был єм ⟨bŷl em⟩ as an alternative masculine first person singular form of the past of the verb „to be”.
bFontański & Chomiak (2000, p. 109) cite был єс ⟨bŷl es⟩ as an alternative masculine second person singular form of the past of the verb „to be”.
cFontański & Chomiak (2000, p. 109) cite были сме ⟨bŷly sme⟩ as an alternative first person plural form of the past of the verb „to be”.
dFontański & Chomiak (2000, p. 109) cite были сте ⟨bŷly ste⟩ as an alternative second person plural form of the past of the verb „to be”.
Use the below to refer to females and objects of grammatically feminine gender.
English
Lemko
Polish
Ukrainian
Russian
I was
я былаa ⟨ja bŷla⟩
byłam
я була
я была
you were
ты былаb ⟨tý bŷla⟩
byłaś
ти була
ты была
she was
она была ⟨ona bŷla⟩
była
вона була
он была
we were
мы былиc ⟨mŷ bŷly⟩
byłyśmy
ми були
мы были
you gals were
вы былиd ⟨wŷ bŷly⟩
byłyście
ви були
вы были
those gals were
они были ⟨ony bŷly⟩
były
вони були
они были
Forms of the feminine past tense conjugation of the Lemko verb быти ⟨bŷty⟩ translated into English, Polish, Standard Ukrainian, and Russian.
aFontański & Chomiak (2000, p. 109) cite была єм ⟨bŷla em⟩ and былам ⟨bŷlam⟩ as alternative feminine first person singular forms of the past of the verb „to be”.
bFontański & Chomiak (2000, p. 109) cite была єс ⟨bŷla es⟩ and былас ⟨bŷlas⟩ as alternative feminine second person singular forms of the past of the verb „to be”.
cFontański & Chomiak (2000, p. 109) cite были сме ⟨bŷly sme⟩ as an alternative first person plural form of the past of the verb „to be”.
dFontański & Chomiak (2000, p. 109) cite были сте ⟨bŷly ste⟩ as an alternative second person plural form of the past of the verb „to be”.
I will be conducting an experiment this month to see if machines can be made to translate into Lemko better than Google Translate or humans.
Hypothesis
A machine can be configured to translate from English into the endangered Slavic language of Lemko and achieve quality scores higher than those of Google Translate’s Ukrainian service, but not yet higher than those of humans.
Predictions
My English to Lemko rule-based machine translation (RBMT) engine will achieve a bilingual evaluation understudy (BLEU) score of 15 against a clean bilingual corpus.
The above engine will achieve a BLEU score that is a third higher (e.g. 20) when coupled with an improvised dictionary-based machine translation (DBMT) created from Lemko-Polish unit-test assertion pairs.
Google Translate’s English to Ukrainian translation service will achieve a BLEU score of 10 against the above corpus.
I, a human, will achieve a higher BLEU score than all the above machines against the above corpus.
The experiments will be conducted over the next week or so, for subsequent publication.
ORLANDO, Dec 2 (Orynycz.com) – It was an honor to present the breakthroughs in our paper Yes I Speak… AI Neural Machine Translation in Multilingual Training at the National Defense Industrial Association’s (NDIA) I/ITSEC 2021 conference, the world’s largest modeling, simulation, and training event, with 13,000 in-person attendees from 47 countries representing governments, universities, corporations and militaries, including including United States Marine Corps Commandant General David H. Berger and Chief of Naval Operations Admiral Michael Gilday.
Special thanks to Emerging Concepts and Innovative Technologies (ECIT) Session 7 Chair Brian Stensrud, Ph.D. and Session Deputy Neil Stagner of United States Marine Corps Systems Command for all the support that made it possible.
Breakthroughs
On low-end, air-gapped laptops in secure, field conditions, our translation engines achieved:
Translation quality BLEU scores 59% better than those of professional linguists for the Russian to English language pair
The world’s first usable Lemko to English machine translations
Engineer Petro Orynycz unveils AI technology that empowers endangered language (Lemko) new speakers to read their language immediately. Watch and follow along with this interactive seminar.
In a hands-on demo attended by over 50 worldwide, Natural Language Processing Engineer Petro Orynycz and Carpatho-Rusyn Society President Maryann Sivak unveil hybrid artificial intelligence technology that empowers new speakers of Lemko to read in the language immediately. Implications for endangered, low-resource language revitalization are discussed.