Showing posts with label TLG. Show all posts
Showing posts with label TLG. Show all posts

2010-05-12

TLG Updates, May 2010

The TLG has just released the latest updates to its text collection. This is what has been added, from the oldest to the most recent texts, with Early Modern Greek texts separate:
Philodemus (i BC): On Anger (ed. Indelli, 1988)
Philodemus is a Hellenistic philosopher, who we know about mainly thanks to Mt Vesuvius, carbonising Pompeii and Herculaneum and everyone in it—as well as a library full of papyri. Papyri that have made it to our day in the optimal conditions of Egypt are still more bitty and fragmentary than we'd like; and carbonised papyri are not preserved in optimal condition. Their preservation is especially suboptimal because they were first discovered and read under 18th century technology rather than 21st century—which meant scalpels rather than X-rays, and the leaves of papyrus turning to dust as they were being copied out.
There is an ongoing project to reedit Philodemus, including the papyri that were copied back then (and for which the 18th century copies are sometimes better witnesses than the modern-day remnant), and the papyri that couldn't be (and that modern technology can now get to). The major texts recently published have been On Piety and On Poems. This text, On Anger, presents a papyrus in a poorer state: something like 7,000 words' worth, some of which are even legible.
Hesychius Illustrius (vi AD): Life of Aristotle (Vita Menagiana) (ed. Düring, 1957)
Hesychius of Miletus was an historian much of whose work was lost, though his biographical dictionary has substantially contributed to the Suda. The Life of Aristotle first published by Gilles Ménage in 1663 is anonymous; Düring associated it with Hesychius.
Constantine Manasses (xii AD): Hodoiporikon (Guidebook) (ed. Horna, 1904)
An account of the historian's mission to Jerusalem and back, pursuing diplomacy through marriage between Byzantium and the crusader states. For more on the text, see Aerts, W.J. 2003. A Byzantine Traveller to one of the Crusader States. In Ciggaar, K.N. & Teule, H.G.B. (eds), East and West in the Crusader States. Peeters. 165–222.

Constantine Harmenopulus (xiv AD): Hexabiblos (ed. Heimbach, 1851)
Harmenopulus (or Armenopoulos) wrote the Hexabiblos (The Six Volumes), which was the final major compendium of Byzantine Law, which itself was a continuation of Roman Law—as witnessed by the undigested Latin expressions that pervaded it. (Yes, even more so than in English-language law.) As the final major compendium, it was much more accessible than its forebears, and has served as a starting point for many historians since. It also served as a foundation of Ottoman law—and of the civil code in the Modern Greek State until 1946.
Anonymous works on Church Music (xiv–xvi AD): Anonymous questions and answers on the interval signs (Ἀκρίβεια) (ed. Schartau, 1998)
This is a treatise on Byzantine musical notation (a topic I have already written an 11 MB PDF on), and the notation it talks about is recognisably the notation used now. The treatise includes examples of the notation as used; but like other early treatises, it isn't really a Teach Yourself manual. To this day textbooks defer on the subtler inflections of Byzantine music to the instruction of your local cantor.
Andronicus Callistus (xv AD): Monody on Unfortunate Constantinople (ed. Pertusi, 1976)
"Yet all this is gone, and now, alas, the queen has become a slave. O, how can one narrate her suffering? It was daytime, but darkness and gloom for the City, and sudden war flowed into the City from land and brine; and the Impious One throws projectiles against the walls with machines, and it falls to the ground in many places, and he rushes against the City bearing a heavy hand."
George of Trebizond (xv AD): Letter to Mehmed II on the Christian faith (ed. Pertusi, 1976)
"For these reasons have I dared write to Your Majesty concerning your eternal and wondrous glory. And you, most golden emir, do receive these letters gladly, which I have written in a simpler tongue, so that they may be more easily translated by your people into the sunlike and most bright dialect of the Turks, whose sweet-sounding and succinct words, whose boldness and virility of pronunciation is attested by all who have tasted it. For I believe that this speech translated may be of some benefit."
The circumstances of the letter are discussed in Monfasani's 1976 biography of George—with an ingenious explanation of the millenarian thinking that made George attempt to make a Christian of Mehmed (the "Impious One" of Andronicus Callistus).



Belthandros and Chrysantza (xiv AD?) (ed. Egea, 1998)
This vernacular romance (aka Velthandros and Chrysantza) has the distinction of being the only example of its kind to have its own English Wikipedia page (for now).
Libistros and Rhodamne (xiv AD?): Vatican manuscript redaction (ed. Lendari, 2007)
This is the second of three surviving redactions of this romance (and we know that there were other copies which have disappeared). Redaction α, representing most surviving manuscripts, was included in the preceding update.
Father Synadinos (xvii AD): Chronicle of Serres (ed. Odorico, 1996)
This is an important text—although I obtained it too late to use for my thesis, and therefore to get properly familiar with it. It is a vernacular chronicle of a provincial Macedonian town, and is one of the few extensive pieces of vernacular we have from the mainland at all for that time. It is also an important early witness of the dialect of Greek Macedonia. In fact the blog I've linked to—though emphatically on the Hellenic side of the fence—notes that the surnames in the Chronicle are all Greek, yet in recent history Synadinos' native village had become Slavic-speaking, so populations had kept moving during the Ottoman Empire.
The chronicle has another reason for mattering to me: the linguistic commentary to Odorico's edition was written by the late Tassos Karanastassis.
Laments on the Fall of Constantinople (xv, xix AD) (eds Pertusi, 1976; Lampros, 1908; Zoras, 1955)
The final collection in this update is of anonymous vernacular laments written about the Fall of Constantinople, taken from an edition by Pertusi; it is supplemented by two vernacular laments published by other scholars.
The laments were written within the timeframe of 1453; but the end of Rhomania kept casting a shadow over the Christians who identified with it, and it remained a topic of folk song. These folk songs were important in the nation-building of Modern Greece (as Herzfeld has analysed in Ours Once More), and every Greek schoolchild has read the song:
Σημαίνει ὁ Θιός, σημαίνει ἡ γής, σημαίνουν τὰ ἐπουράνια,
God strikes, the Earth strikes, and the Heavens strike.
Hagia Sophia strikes, the great cathedral,
with two and sixty bells, four hundred prayer blocks.

The better-read will also know the laments that were recorded in Pontic folk song:
Μὴ κλαίς, μὴ κλαίς, ἅϊ Γιάννε μου, καὶ δερνοκοπισκᾶσαι.
Ἡ Ρωμανία ’πέρασεν, ἡ Ρωμανία ’πάρθεν.
Ἡ Ρωμανία κι ἂν ’πέρασεν, ἀνθεῖ καὶ φέρει κι ἄλλο
Cry not, St John, and beat your breast not so.
Rhomania's time has passed; Rhomania's fallen.
Rhomania's passed; it buds, and bears a new one.

Τὴν Πόλην ὅνταν ὥριζεν ὁ Ἕλλην Κωσταντῖνον,
εἶχε πορτάρους δίκλωπους, ἀφέντους φοβετσάρους,
εἶχεν ἀφέντην σερασκέρ’ τὸν μέγαν Ἰωάννην.
When Constantine the Hellene ruled the City,
his lords were scared, his gatekeepers were two-faced,
and John the Mighty was his general.

Ὁ ἥλιον πάει ’ς σὴν μάνναν ἀτ’ μαυρομελανιασμένον.
Φέρει καὶ τραπεζώνει ἀτον γάλαν καὶ παξιμάτιν,
ὁ πρόσωπος ἀτ’ ’κὶ γελᾷ, ἀπολογιὰν ’κ’ ἐδῶκεν.
The Sun goes to his mother, black and blue.
She brings him milk and serves him rusks to dine on.
His face won't smile, and he gives no reply.

It turns out that Pertusi has included a selection of these folk songs in his collection. Which means the TLG now includes Pontic. (And I have to worry about morphologically analysing it.)

...Read more

2010-03-02

What are the longest words of Greek?

Everyone knows (or should know) about the longest word of Greek ever—the word that broke the title bar of Wikipedia, Aristophanes' fantastical dish of 17 ingredients at the end of the Ecclesiazusae, that lopado-temacho-thing:

λοπαδοτεμαχοσελαχογαλεοκρανιολειψανοδριμυποτριμματοσιλφιολιπαρομελιτοκατακεχυμενοκιχλεπικοσσυφοφαττοπεριστεραλεκτρυονοπτοπιφαλλιδοκιγκλοπελειολαγῳοσιραιοβαφητραγανοπτερυγών (172 chars)

Ah. It breaks blogspot too. :-)

λοπαδοτεμαχοσελαχογαλεοκρανιολειψανοδριμυποτριμματοσιλφιο­λιπαρομελιτοκατακεχυμενοκιχλεπικοσσυφοφαττοπεριστεραλεκτρυονοπτο­πιφαλλιδοκιγκλοπελειολαγῳοσιραιοβαφητραγανοπτερυγών (172 chars)

Have you ever wondered what the next longest words of Greek are? No? Well, you'll find out anyway. With the necessary provisos as usual:

  • I'm basing this on the word forms in the current TLG, and the lexica I have access to—LSJ, LSJ Supplement, Lampe, Trapp, Kriaras.
  • This doesn't prove much of anything, since the point is that Greek compounding is agglutinative and productive, so you can keep stringing words up. We're just happening to go through the long words turning up in Ancient and Mediaeval Greek literature.
  • Do not even think of coming away with the impression that Greek literature makes the longest words in the world—although few if any seem to get anywhere near Aristophanes' monster. The Guinness Book of Records did unearth a word in the Varadambika Parinaya by Tirumalamba in Sanskrit adding up to 428 letters in Roman transliteration. I can't find it (I think this is the Google Books snippet); which is a shame, I wanted to break Blogspot some more...
  • For other languages, have a Facebook thread and a Wikipedia laundry list.
  • I'm sure there's plenty of compounding happening in Modern Greek; I cited in the Quadrupeds (p. 150) the Hellas-L coinage πολυμαθουφοχριστιανοπεοκρουστόπαιδο "polymath UFO Christian penis-stroking lad" (35 chars), which maker poster Constantine Thomas coiner of the sixth longest real Greek word I know of. That… doesn't prove that much either.

What it shows is, there was a Byzantine fad for long compounds, in a couple of genres, though possibly ultimately indebted to Aristophanes; and Aristophanes himself went there more than once.

I'm going to count down the top 40-odd words in length I can find, down to a length of 29 characters, and I'll give their dictionary definitions (where available) and who has used them.

The longest words in English, long enough to be cheating, are protein names. The longest words in Greek outside of Aristophanes, long enough to be cheating, are fractions. They're so boring, dictionaries don't bother to list them at all. So I'm putting them separately:
  1. τριακοντατρισμυριοστοχιλιοστοεπτακοσιοστοεβδομηκοστόεκτα (56 chars), "1/331776", Scholia on Diophantus, 135
  2. μυριοστοεπτακισχιλιοστοπεντακοσιοστοτεσσαρακοστόπεμπτα (54 chars), Trapp: "1/17545", Nicholas Artabasdos "Rhabdas", Letters 34.
  3. μυριοστοτετρακισχιλιοστοεξακοσιοστοτεσσαρακοστόπρωτα (52 chars), Trapp: "1/14641", Scholia on Diophantus, 128
  4. τρισκαιδεκακισμυριοστοτριακοσιοστοεικοστοπρώτων (47 chars), τρισκαιδεκακισμυριοστοτριακοσιοστοεικοστόπρωτα (46 chars), "1/130321", Scholia on Diophantus, 130
  5. τετρακισμυριοστοχιλιοστοεξακοσιοστοεξκαιδέκατον (47 chars), "1/41616", Isaac Argyrus, Περὶ εὑρέσεως τῶν τετραγωνικῶν πλευρῶν τῶν μὴ ῥητῶν τετραγώνων ἀριθμῶν, p. 23
  6. τετρακισμυριοστοτετρακισχιλιοστοοκτακοσιοστόν (45 chars), "1/44800", John Pediasimus (aka John Galenus), Geometry p. 19
  7. τρισχιλιοστοδιακοσιοστοτεσσαρακοστοέννατον (42 chars), "1/3249", Scholia on Diophantus 140
  8. δισμυριοστοδισχιλιοστοτετρακοσιοστόν (36 chars), "1/22400", John Pediasimus, Geometry p. 19
  9. ὀκτακισχιλιοστοεκατοστοεικοστόγδοον (35 chars), "1/8128", Pseudo-David & Pseudo-Elias, Lectures on Porphyry's isagoge, praxis 8 p. 3
  10. ἐννεακαιεικοσικαιεπτακοσιοπλασιάκις (35 chars), LSJ: "seven-hundred-and-twenty-nine times", Plato, Republic 587e.
  11. τετταρακοντακαιπεντακισχιλιοστόν (32 chars), LSJ: "forty-five thousandth" (or: 1/5040), Plato, Laws 877d
  12. τετρακισχιλιοστοεξηκοστοτέταρτον (32 chars), "1/4064", Pseudo-David & Pseudo-Elias, Lectures on Porphyry's isagoge, praxis 8 p. 3
  13. ἐπιτετρακοσιοστοκαιτεσσαρακοστῷ (31 chars), "ratio of 441:440", George Pachymeres, Quadrivium 2.45
  14. χιλιοκτακοσιογδοηκονταπλασίονα (30 chars), LSJ: "eighteen hundred and eighty times as great", Theon of Smyrna De utilitate mathematicae p. 197, citing Hipparchus
  15. τετρακισχιλιοστοενενηκοστόεκτα (30 chars), "1/4096", Scholia on Diophantus 138
  16. διακοσιοστοτεσσαρακοστοτρίτοις (30 chars), Trapp: "1/243", Proclus, Commentary on Plato's Timaeus, Vol. 2 p. 180

And now, the "real" words. Words not (yet) in the TLG are asterisked.
  1. κεραυνομεγακλονοζηνπερατοκοσμολαμπροβελοπλουτοδότα (50 chars), (Luis Muñoz Delgado, Léxico de Magia y Religiòn en los Papiros Mágicos Griegos, 2001) "making great noise with a thunderclap, bounded by the sky, shining rays in the cosmos, and giver of wealth", Magical Papyri 12. The magical papyri have even longer strings of nonsense characters, but this really does count as a word. It's worth taking it apart:
    • κεραυνο "thunder"
    • μεγα "big"
    • κλονο "turmoil"
    • ζην "Zeus"
    • περατο "boundary"
    • κοσμο "cosmos"
    • λαμπρο "shining"
    • βελο "ray"
    • πλουτο "wealth"
    • δότα "giver"
    It's positively Sanskrit...
  2. *ἀκτινοχρυσοφαιδροβροντολαμπροφεγγοφωτοστόλιστος (47 chars), Trapp: "dressed in golden-shining, thundering and incandescent clothes", Codices graeci Chisiani e Borgiani (ed. P. Franchi de' Cavalieri), 124; Bibliotheca Coisliana (ed. Montfaucon), 59. What astonishes me is that someone chose to use this word a second time…
    • ἀκτινο "ray"
    • χρυσο "gold"
    • φαιδρο "beaming"
    • βροντο "thunder"
    • λαμπρο "bright"
    • φεγγο "shining"
    • φωτο "light"
    • στόλιστος "dressed"
  3. *Ἡρακλειανοκυροσεργιοπυρροπαυλοπετρῖται (38 chars), Trapp: "followers of Heracleus, Cyrus, Sergius, Pyrrhus, Paul and Peter", Scripta saeculi VII vitam Maximi Confessoris illustrantia (ed. P. Allen & B. Neil) 223,378. (Hey, I know them! And the "Heracleus" is in brackets, so some scribe may have decided five leaders were already enough.)
  4. *παναξιοκτηνοπτηναστροφωστηροκοσμοποιία (38 chars), Trapp: "quite worthy creation of animals, birds, stars, celestial bodies and the world", Dioscorus of Aphrodito (ed. Fournier), 40,6
  5. ὀρθροφοιτοσυκοφαντοδικοταλαιπώρων (33 chars), LSJ: "early-prowling base-informing sad-litigious plaguy", Aristophanes Wasps 505. Misquoted by Suda (omicron 581, epsilon iota 68) as ὀρθοφοιτοσυκοφαντοδικοταλαιπώρων (32 chars), "upright-prowling..."
  6. ἀστραποβροντοχαλαζορειθροδαμάστου (33 chars), Lampe: "overcome by lightning, thunder, hail, and flood", Basil of Caesarea, Letters 365
  7. στρογγυλοφιλοσοφογραμματογράφου (31 chars), (not yet in a dictionary) "writer of round philosophical letters", Theodore II Ducas Lascaris, 218 Letters, 128.
  8. σπερμαγοραιολεκιθολαχανοπώλιδες (31 chars), LSJ: "green-grocery-market-woman", Aristophanes Lysistrata 457
  9. *πανυπερπρωτοπανσεβαστοϋπέρτατος (31 chars), Trapp: "above all else and first of all, most highly honoured", Gregory hegumen of Oxia 225,126.
  10. Βρυσωνοθρασυμαχειοληψικερμάτων (30 chars), LSJ: "taking coin like Bryso and Thrasymachus", Ephippus of Athens fr. 14 Kock, cited in Athenaeus 11.120
  11. ἀκτινολαμπροφεγγοφωτοστόλιστος (30 chars), Trapp: "sparkling with shining light beams", Synaxarium Ecclesiae Constantinopoleos, May 9 section 2
  12. τοξαισελαιοξανθεπιπαγκαπύρωτος (30 chars), "bow-???-oil-blond-on-all-airdried" (ref. to a kind of cake), Philoxenus of Cythera fr. e, cited in Athenaeus 14.50; the edition of Philoxenus by Page marks the word as corrupt
  13. κομπορηματοχρηματομετεωροφέναξ (30 chars), Trapp: "boasting cheat puffed up with words and treasures", John Tzetzes, Letters 4 p. 6
  14. *πανσεβαστοκοσμοποθοπροσκύνητος (30 chars), Trapp: "highly honored and welcomed by the world with longing", Catalogus codicum manuscriptorum Graecorum (ed. Beniševič), I 332
  15. ἀκτιστοσυμπλαστουργοσύνθρονον (29 chars), Lampe: "uncreated fellow worker sharing the throne", Gregory Pardus, Exegesis in canonem iambicum de festo die Spiritus Sancti, 25; ἀκτιστοσυμπλαστουργοσύνθρονε (28 chars), Vita of Nicephorus the Doctor 22.
  16. σκοροδοπανδοκευτριαρτοπώλιδες (29 chars), LSJ: "garlic-bread-selling hostess", Aristophanes Lysistrata 458
  17. λογαριαστοπνευματικοοικονόμος (29 chars), Trapp: "invoice controller and religious manager", John Apocaucus, Letters (ed. Bees) 60

Special separate mention goes to Constantine of Rhodes, who wrote a couple of pages of train-carriage–length epithets; I have cited an excerpt elsewhere:
  1. *πρεσβευτοκερδοσυγχυτοσπονδοφθόρος (33 chars), Trapp: "who interferes with the profits of intermediaries and destroys contracts", Constantine of Rhodes (ed. Matranga), 625,25.
  2. *πλαστιγγοζυγοκαμπανοσφαιρωστάτης (32 chars), Trapp: "one who sets the round weight on the steelyard balance with the gold bearing scales and balance beam", ibid. 624,8.
  3. *βαρβιτοναβλοπλινθοκυμβαλοκτύπος (31 chars), Trapp: "playing the barbiton, the nabla, the plinthos (hydraulic organ?), and the cymbals", ibid. 625,21.
  4. *Ἑλληνοθρησοχριστοβλασφημότροπος (31 chars), Trapp: "a character who worshiped the pagan and sinned against Christ", ibid. 624,15.
  5. *κασαλβοπορνομαχλοπρωκτεπεμβάτης (31 chars), Trapp: "one who mounts the arses of whores, prostitutes and courtesans", ibid. 624,11.
  6. *λακτεντοχοιροκριοβουτραγοσφάγος (31 chars), Trapp: "butcher of piglets, pigs, sheep, cattle, and goats", ibid. 624,7.
  7. *ὀλεθριοβιβλοφαλσογραμματοφθόρος (31 chars), Trapp: "ruinous counterfitter of books and destroyer of writing", ibid., 624,12.
  8. *λαρυγγογλασκοξεστοχανδοεκπότης (30 chars), Trapp: "one who gulps down bottles and glasses with an open throat", ibid. 624,10.
  9. *κορνουτοπαρθενοτριβοψυχοφθόρος (30 chars), Trapp: "horned abuser of virgins and destroyer of souls",ibid. 624,10.
  10. *ἀλλαντοχορδοκοιλιεντεροπλύτης (29 chars), Trapp: "washer of entrails [sausage, intestines, bellies, entrails]", ibid., 624,5.
  11. *κλεπτοτυμβονυκτεροσκοτεργάτης (29 chars), Trapp: "one who operates in the dark of night as a grave robber", ibid. 626,25.
  12. *μοιχοπαιδοδουλοσκανδαλεργάτης (29 chars), Trapp: "one who arouses a scandal through lewdness with underage slaves", ibid. 625,24.

...Read more

2010-02-15

New TLG words in DGE VII

As I posted last month, the new volume of DGE (Diccionario Griego–Español) has appeared, spanning ἐκπελλεύω–ἔξαυος. As with any lexicographic work of an older language, some philology and textual emendation has been involved; this paper by Eugenio Luján Martínez gives four such instances, in Epicurus, Aretaeus, Nicander, and Galen.

I have gone through this volume and the TLG texts dated from before i AD, to find words not given in the other dictionaries out there (LSJ, LSJ Supplement, Bauer, Lampe, Trapp). I'm posting them here for interest. Note that the list may be small, but that's because the major gaps are in papyri (which are not in the TLG)—not ancient literature, which is already (nominally) well-ploughed land. (There should be quite a few more words from TLG AD texts.) DGE's entries are still much more detailed than LSJ's, and its coverage of antiquity broader. I'm also omitting proper names, which I have been treating differently in lemmatisation.

I'm leaving out the glosses, because the DGE is a commercial product and all. Most of them you can guess the meaning of, if you know your way around Greek vocabulary...

  • ἐκπυρώδης, ες.
  • ἐλάφινος, η, ον. (But Trapp has the variant ἐλαφινός.)
  • ἐλέκεβρα, ας, ἡ. (ἠλεκέβρα, ἰλλεκέβρα, ἐκλεκέβρα).
  • ἐλέσσω.
  • ἑλωρεύς, έως, ὁ.
  • ἕμα, ματος, τό.
  • ἐμβρωσί, τό.
  • ἐμπιεστός, ή, όν.
  • ἐμπύρευσις, εως, ἡ.
  • ἔνδεινος, ον.
  • ἐνναγώνιον, ου, τό.
  • ἔνοψ, πος, ὁ.
  • ἐναντιοεργός, όν.
  • ἐνεδρευτός, ή, όν.
  • ἐνιζυγίς, ίδος, ἡ.
  • ἐντεροκοιλιακός, ή, όν.
  • ἐντραπής, ές.
  • ἐντυπάδιον, ου, τό.
  • ἐξάρακτος, ον.

...Read more

2010-02-06

Comparison, TLG BC and AD: log-likelihood

Helma Dik left a comment on my post on comparing TLG AD and BC through Wordle, suggesting I use Dunning's Log-Likelihood measure of differential word frequencies in corpora, as Wordled by Martin Mueller. That lets you work out what the real shifts in frequency are, rather than trying to eyeball them through the aggregate word counts.

Here for instance is his comparison of the Iliad to the Odyssey—which words are more frequent in the one, or the other:
Wordle: Odyssey_plusCorrected
Wordle: Iliad_plus
I looked up Ted Dunning's paper, failed to understand it :-( , and used instead the walkthrough of the computation on the user manual of the Wordhoard corpus software package.

And this is the more statistically sound Wordle comparison. Words more frequent BC are in red, words more frequent AD are in black. I'm leaving in stop words this time, and not cleaning up the ambiguity, because this says some interesting things about the changes in Greek grammar between Classical and Late Greek. Do click:
Wordle: TLG AD vs BC comparison, using Dunning's Log-Likelihood metric

Here's my impressionistic notes, that haven't already been covered in the previous post (where I was working through rankings):
  • Both corpora talk about θεός God, but the big jump, of course, is Χριστός Christ. The second biggest jump is in ἅγιος holy, displacing ἱερός. (Was ἱερός too pagan-sounding?)
  • But the biggest discrepancy between BC and AD Greek is the avoidance of δέ but, on the other hand, followed by avoidance of μέν on the one hand. That tells you that AD Greek used different sentence structures, such as a lot more ἀλλά but. Tucked away, there's also more καί and (i.e. more coordinating constructions) and a lot less τε and (a very archaic phrase-second construction).
  • There are a lot more ἤγουν and τουτέστι that is, and a lot less ἐάν if and ἄρα therefore; I'm tempted to think that says something about changing rhetoric in the genres popular in the respective periods—less logic, more exemplification. It's foolhardy, but not impossible.
  • There is a lot more τίς who? being reported, and that's an error in ambiguity, but it's an illuminating error. τοῦ in Attic (though not Late Greek) is ambiguous between "whose?", and the genitive definite article. And there are a lot more definite articles in Late Greek, as you can see by the black ὁ. (My friend Io Manolessou actually wrote her PhD on that shift; nice to see it visually confirmed.)
  • There's also more ἵνα in order to, which suggests Late Greek was already moving towards more subjunctive constructions rather than participles and infinitives, even before Early Modern Greek made the switch completely.
  • Clearly less ὦ Ο!—A very Classical way of addressing people.
  • Some of the odder looking words more prevalent in BC Greek are there because there are a lot more geometric texts in the BC corpus: Ἄβ is actually mistakenly picking up the line ΑΒ, and you can also see in smaller print ΑΒΓ, ΒΔ, ΓΔ, ΕΖ, ΞΖ.

Hm. Yes, that was somewhat more illuminating. Thanks, Helma!
...Read more

2010-02-02

TLG updates

The TLG has just released a new update to its corpus. As of tonight, the automatic recognition of lemmata in the TLG which I've been working on has just reached 95% of all wordforms. With these two milestones, I'll be posting a few things about the current corpus; I've already put up some Wordles, as you will have seen.

First, about the new texts.
  • Early Modern Greek is well represented in the update. Readers of this blog will have noticed as much, because I've spent some time dealing with the peculiarities of those texts. Romances are the genre that attracts the most scholarly interest—they make the most sense as literature to contemporary readers. Accordingly, this update includes redaction α of Livistros and Rodamni as recently published by Panagiotis Agapitos of U Cyprus (who also writes Byzantine detective novels); the War of Troy (on which I've already posted); and the four redactions of the Tale of Belisarius.
  • Jumping forward a couple of centuries, the update also includes Cretan Renaissance drama: George Chortatzes' tragedy Erophile, and the intermedios from Erophile, retelling Tasso's Jerusalem Delivered.
    1595 isn't the latest date of TLG texts; the Chapbook of Alexander the Great (h/t Diver of Sinks) was published in 1750, and while the collections of monastic documents are normally late Byzantine and Ottoman, one collection goes up to 1968. But Chortatzes is the first representative of Cretan dialect, and I was quite happy to tweak the lemmatiser to accomodate it.
    I'll pause to note something which shouldn't be an oddity, but is. Chortatzes writes his stage directions in the same Cretan dialect as his dialogue. Of course he would; our stage directions are in the same language as the rest of the text. But now that there is a Standard Greek, and Cretan isn't it, that comes across as quizzical. If anyone now writes drama in dialect (not Cretan, but possibly Pontic, and certainly Cypriot), the metalanguage of the drama is going to be Standard Greek.
    That's because the metalanguage is the dramatist's own voice; and while hillbilly dialect might be good enough for the dramatists' characters, it will not do for their own directions. I'd imagine it's the same for other Dachsprache languages (language variants under the "roof" of a prestige language), like say German or Italian dialect.
  • A few texts are recent re-editions. Moeris' Atticist dictionary has been updated. By telling us which colloquial forms not to use, Moeris and the other Atticists (Phrynichus is the main one) tell us a lot about what their colloquial language actually looked like.
    There is also a new edition of Cyril of Alexandria's Paschal Letters, and Eudocia's Homeric Centos. The centos are rearrangements of verses from an old poem, to tell a new story—in this case, Homer rearranged into an account of the Passion of Christ. This sounds like a very postmodern thing to do. But then again, there's nothing new in postmodernism, apart from its bankruptcy of an intellectual programme, as it magpies any flotsam that gets it out of having to tell a story.
    Not that you need to hear about my cultural conservatism.
  • The TLG also has an updated edition of the fragments of John of Antioch. There has been a controversy about whether the fragments attributed to John belong to one or two authors (there are two different linguistic registers in what we have); as a result, we're in the odd situation by Classical standards of having two new editions of the author, within three years, from the same publisher: Roberto 2005 and Mariev 2008. (The Bryn Mawr Classical Review gives context in its review of Mariev.) The TLG has gone with Roberto, which considers both registers to belong to the same author; that has the added advantage of not throwing half the fragments out of the corpus.
  • In gathering up bits left out from Middle Byzantine authors, the update also includes Michael Psellus' commentary on Aristotle's Physics, and Constantine Manasses' Moral Poem.
  • Finally, the update includes the irascible John Tzetzes' retelling of the Theogony.
    We already have the original Theogony from Hesiod; so Tzetzes' retelling might have some interest for cultural history, but is not telling us much we don't already know. Tzetzes' retelling has attracted attention for different reasons. Tzetzes was Georgian on his mother's side; and he sees fit to show off his command of foreign languages, in the epilogue to his poem. That makes his Theogony the earliest attestation of Ossetian. (And in his loose Greek translation, as linked, it may well be one of the first attestations of Greek μουνίν: er, "pudendum muliebre".)

...Read more

2010-02-01

Comparison, TLG BC and AD

In the previous post, I used Wordle to illustrate stop words in Greek (and, by the by, the exponential distribution of function words following Zipf's Law). After getting rid of a whole bunch of stop words, I ended up with a Wordle of the lemmata of the TLG:

But I stopped short of making sense of the Wordle, because the TLG contains both Ancient and Mediaeval texts, and they talk about different things. I promised Wordles of the texts in the TLG from BC and AD, which will give at least a rough sense of the difference.

So here they are:





Images created by the Wordle.net web application are licensed under a Creative Commons Attribution 3.0 United States License.


The Wordle images are hyperlinked to the Wordle applets hosted there, so you can play with the applets by eliminating words. The stopwords are as before, but I also got rid of πολύς "much", which was crowding the BC texts a bit much.

A few things jump out quickly: there's a lot more God AD, as you'd expect (θεός), slightly more talk of "people" than of "men" (ἄνθρωπος, ἀνήρ), less talk of the City and more talk of power (πόλις, δύναμις).

But I'm not really a visual person, so I'm going to use more quantitative ways of working out the changes in vocabulary.

To begin with, the two Wordles show the 150 most frequent lemmata for each period, not counting stop words. These are the differences between the two—words in the top 150 of one period, but not the other.
Ancients talked more about...and less about...
Ἕλλην, Ἀθηναῖος, Ζεύς, ἀμφότερος, διαφέρω, ἑκάτερος, ἐλάσσων, εἶμι, εὐ, ἤλιος, ἡγέομαι, ἱερός, κεῖμαι, κύκλος, ναῦς, νέος, νομίζω, ὀρθός, οἶκος, πλέως, πλεῖστος, πλῆθος, πόλεμος, πολέμιος, ποταμός, θάλασσα, θεά, σημεῖον, ταχύς, ὔστερος, χρῆμα, χώρα, ζῷον Χριστός, ἅγιος, ἁπλόος, ἄξιος, ἀδελφός, ἀλήθεια, βασιλεία, δέχομαι, δηλόω, δόξα, ἐκκλησία, ἐνέργεια, εἶδος, φωνή, κίνησις, κόσμος, νόος, οἰκεῖος, οὐρανός, οὐσία, πάθος, πίστις, πνεῦμα, πρόσωπον, θάνατος, θεῖος, σάρξ, τέλος, τρίτος, χάρις, ζητέω, ζωή
Greek, Athenian, Zeus, both, to differ, either, less, go, good, sun, to lead, dawn, holy, to lie, circle, ship, new, to think, right, house, full, most, crowd, war, enemy, river, sea, goddess, point, fast, last, need, land, animal Christ, holy, simple, worthy, brother, truth, kingdom, to accept, to declare, glory, church, activity, form, voice, movement, world, mind, own, heaven, substance, passion, faith, spirit, face, death, divine, flesh, end, third, grace, to ask, life

The effect of Christianity on vocabulary use is pretty obvious. A few other changes are worth noting:
  • Byzantines nominalised a lot more than Ancients did. That's at last some of the reason for ἀλήθεια "truth" (instead of the more Attic τὸ ἀληθές "the true"), and it may relate to other nominalisations like κίνησις "movement" and ἐνέργεια "activity". (βασιλεία "kingdom" has a Biblical pedigree—but that is also because the Bible was not written in Attic.)
  • Many of the differences are a matter of language change, rather than different ideology. For all that most Byzantines did not write in the vernacular, their language was usually more akin to Koine than to Attic. That explains the absence of εἶμι, εὐ, ναῦς, πλέως, ἐλάσσων, ἱερός, πολέμιος (replaced by στέλλω, καλός, πλοῖον, πλήρης, μικρότερος/ὀλιγότερος, ἄγιος, ἐχθρός) "send, good, ship, full, less, holy, enemy", and presumably also the avoidance of ἀμφότερος and ἑκάτερος "both, either".


I've left out from those lists words that show up in the top 150 only because they're ambiguous with other legitimate words. (Yes, I should have pruned the Wordles.)
  • BC: δίκαιον, δοκεύς, ἠώς, θέα: rights, beam, dawn, view
  • AD: ἅγιον, βασίλειος, ἴδιον, κενόω, πρόσωπος, ζωός: sanctuary, royal, particularity, make void, face, alive

There's one further comparison I'll attempt: the words whose frequency changed the most between the two periods. To track this, I'm going to use the 2000 most frequent lemmata for each period—including both normal words and stop words; that constraint means we're only looking at words that are likely to matter. I'll go through the lemmata in those lists whose ranking changed by the greatest amount (e.g. from #1537 to #10342).

Because it's a pretty heterogeneous list—and different kinds of words tells us different things, I'll split them up into categories. (And I will do some silent suppressing of ill-recognised ambiguous words.)

These are the biggest shifts in proper names:
Ancients talked more about...Rank Shift
ἜφοροςEphorus-8530
ΠοσειδώνιοςPosidonus-8397
ΠελοποννήσιοςPeloponnesian-6655
ΑἰτωλόςAetolian-5399
ἙκαταῖοςHecataeus-5157
ΘεόπομποςTheopomus-5046
ἈπολλόδωροςApollodorus-4948
ΦωκεύςPhocian-4786
ΤυρρηνικόςTyrrhenian-4587
ΧρύσιπποςChrysippus-4043

Two things are going on here. First, some ancient authorities—primarily historians, if I read the names right—were of interest to several ancient writers, but of less interest to the Byzantines. They tend to be the historians whose texts didn't survive, which is related to them being of less interest to the Byzantines. (I don't know offhand whether that's cause or effect.)

Second, Greece was very important to Ancient Greeks, and so were the various regions of Greece. To the Byzantines though, Greece was a backwater, and the old regions did not survive into the Byzantine system of themes. So there was no reason to talk about Aetolia or Phocia outside of Ancient History; and less reason to talk about the Peloponnese than you might think, even while the name survived. The same goes for Tyrrhenians: it wasn't Etruscans that the Byzantines were having to deal with in Italy, but Lombards.

Ancients talked less about...Rank Shift
ΚύριλλοςCyril+214,509
ΚωνσταντινούπολιςConstantinople+214,399
ΓρηγόριοςGregory+214,391
ἈθανάσιοςAthanasius+214,154
ΓεώργιοςGeorge+85,856
ΚωνσταντῖνοςConstantine+47,064
ΠέτροςPeter+40,947
ΧριστιανόςChristian+36,162
ΒασίλειοςBasil+28,217
ΧριστόςChrist+23,988

The only surprise is that Christians turn in BC texts at all; there's only 5 instances though, and the dating of texts in the corpus is porous (late citations can appear as testimonia of earlier authors).

These are the biggest shifts in common nominals:
Ancients talked more about...Rank Shift
εὔδοξοςreputable-8805
κύλινδροςcylinder-6569
ἀσύμμετροςasymmetrical-5939
δημοκρατίαdemocracy-5389
πυραμίςpyramid-4714
ναυμαχίαsea battle-4274
κῶνοςcone-4205
παραλληλόγραμμοςparallelogram-3837
παρεμβολήinterpolation; encampment-3668
ψήφισμαdecree passed by vote-3194

If the AD texts have more theology, they clearly have a lot less geometry, and a lot less to do with representational systems of government. The drop in εὔδοξος is surprising, given it's in Plato; I wonder if the change of -δοξ- in compound from "reputations" to "glory" made the adjective confusing for later writers.

Ancients talked less about...Rank Shift
ἀποστολικόςapostolic+214,282
θεοτόκοςGod-bearing (Theotokos)+85,966
βάπτισμαbaptism+85,945
θεότηςdivinity+59,016
μόδιοςbushel+58,616
μοναστήριονmonastery+57,696
σεβάσμιοςreverend+57,691
αἱρετικόςheretic+35,602
χάρισμα(spiritual) gift+35,588
πατριάρχηςpatriarch+27,001

No surprises again; the only non-religious term is μόδιος "bushel", both as a vessel and a measure.

These are the biggest shifts in verbs:
Ancients talked more about...Rank Shift
διαπορεύωpass across-5939
βλώσκωgo-5710
εἰσοράωlook upon-4134
ἄημιblow (wind)-4113
κλύωhear-3392
ἐπιζεύγνυμιjoin to-3039
ἀμφισβητέωdoubt-2436
ἐφάπτωhang on-2122
ἱκνέομαιcome-2088
μεταπέμπωsend for-1974

Many of the missing verbs are poetic and/or dialectal, and would not have a natural place in Byzantine prose; that includes βλώσκω, εἰσοράω, ἄημι, κλύω, ἱκνέομαι. The surprise here is the vanishing of doubt in the Middle Ages.

... Yes, yes, the jokes just write themselves, I know...

Ancients talked less about...Rank Shift
ἐνάγωpersuade+10,366
βαπτίζωbaptise+8529
ψάλλωchant+5021
φανερόωreveal+3988
καταδικάζωcondemn+3483
φωτίζωilluminate+3308
περισπάωtake a circumflex+2948
βαστάζωcarry+2911
ἀνέρχομαιgo up+2809
προλαμβάνωanticipate+2769

I admit to being less sure about some the shifts here, such as ἐνάγω and προλαμβάνω. The Christian influence is clear in βαπτίζω, ψάλλω, φανερόω and φωτίζω. Language change accounts for βαστάζω and ἀνέρχομαι replacing φέρω and ἄνειμι, and I assume καταδικάζω for "condemn" replaced what came to look like more generic verbs, in καθαιρέω or καταγιγνώσκω. And unlike the Ancients, the Byzantines had to learn about polytonic orthography; so what word took a circumflex and what word took an acute was a matter much ink was spilled about.

Finally, these are the biggest shifts in function words:
Ancients talked more about...Rank Shift
τοτέat times-5796
αὖτεagain-4707
δισχίλιοιtwo thousand-4676
αἴalas-3334
πεντακόσιοιfive hundred-2859
ἠέor-2844
νή[I swear] by [deity]-2663
διακόσιοιtwo hundred-2597
μάyea-2470
πωyet, at all-2466

There is some Epic dialect here, in αὖτε and ἠέ; some strictly Attic rather than Koine words in τοτέ, πω, and δισχίλιοι; and a rather different approach to exclamations, with the old oaths by the Gods dispensed with, and the ai!'s of tragedy avoided in theological discourse. (There are 2100 instances AD of φεῦ "alas"; maybe αἴ was too specific to tragedy? *shrug*) Not sure why the written-out 500 and 200 were less popular. Maybe the armies just got bigger, so historians talked in the thousands instead of 300...

Ancients talked less about...Rank Shift
ἀμήνamen+19,195
νάto (Modern Greek)+18,984
ἀλλαχοῦelsewhere+7689
δηλαδήthat is+7587
ἤγουνthat is+6367
ιζ΄XVII+4541
καθόinsofar as+4524
ιϛʹXVI+4001
ιηʹXVIII+3727
ιδʹXIV+3196

It's obvious why amen is there; it's also obvious why να, the Modern Greek equivalent of the ancient infinitive inflection, is there. ἀλλαχοῦ for "elsewhere" is attested in Sophocles and Xenophon, but it became prevalent much later, and LSJ reports that Moeris proscribed it as vernacular, in favour of ἄλλοθι. The other conjunctions are run-in phrases, which Byzantine texts in general are rather more sympathetic to treating as single words than are ancient texts: δῆλα δή "so [they are] obvious", ἤ γε οὖν "or indeed then", καθ’ ὅ "according to what".

Finally, the numerals aren't there because the Byzantines were more numerate than the Ancients. After all, the Byzantines had given up on geometry, from what the counts tell us. (And that's a silly enough thing to conclude that you should not take much of this too seriously.) No, the reason there's a whole lot of XVII's and XIV's in the AD corpus is that there are a lot more chapter headings in the theologians...
...Read more

2010-01-31

Wordle and Greek stop words

Some of you may be familiar with Wordle, an online tool which displays the words in a text with different sizes, depending on their frequency. Wordle is a convenient tool for seeing what the frequently mentioned concepts are in a text, so it gets a fair amount of use in blogs. It's the same concept as Word Clouds; but done with much more typographical finesse. This, for instance, is Wordle run over the English text of Plato's Republic:


And courtesy of The Crazy Australian, this is the ESV New Testament:

(As The Crazy Australian noted, you can learn one thing immediately from that: the Third Person of the Trinity doesn't get as much stage presence as the Other Two in Holy Writ. Not really a surprise, but the point of Wordle is as much to visualise the obvious as it is to discover the not as obvious.)

Wordle works quite well with English, because most words don't have a lot of inflection, to multiply the instances of the concept you're looking for. In a language like Greek, on the other hand, lemmatisation—or as it's more often called in search engines, stemming—is essential. Otherwise, you get not one instance of "Jesus" or "state", but four or five, with no material difference.

Funnily enough, I do lemmatising. So what happens when you put the TLG through Wordle?


Images created by the Wordle.net web application are licensed under a Creative Commons Attribution 3.0 United States License.


Well, what you get is this:

I've highlighted the top seven verbs in green, and the top seven nouns in green. You can see the nouns, right?

Of course you can't, because there's a whopping great big ὁ and another rather outsize καί there, crowding everything else out. And being told that Greek texts have a whole lot of instances of the and and is unlikely to be what most people are curious to know.

What we have here is the notion of stop words: grammatical words that don't convey a lot of content, and which search engines traditionally ignore. Wordle also ignores them, which is why you don't see a lot of the and and in English-language Wordles. But Wordle doesn't happen to be configured for Classical Greek.

So what happens if we whittle away at the stop words? Let's do this slowly. We'll start by getting rid of ὁ and καί.

Woah. Where did all that come from? You can see something now: θεός, λόγος, and if you really squint, ἄνθρωπος. But that's still making life too difficult, because there are more stop words to dismiss. I've highlighted the next batch in red: τίς, δέ, αὐτός, εἰμί, who?, but, he/himself, be. Of these, τίς "who?" is inflated through ambiguity with τις "someone"; because the lemmatisation is not disambiguated by context, a few word counts are more sizeable than they should be.

With those four out of the way, we have:

An improvement; you can see ἄνθρωπος now, and maybe even πατήρ "father" next to θεός "god". But we still can do better. We have eight more stop words that we don't really need to hear about: ἐγώ "I", ὡς "as, that", ὅς "who, that", τις "someone", οὐ "not", γάρ "because", ἐν "in", and οὗτος "this".

With them left out, we have:

Still better: you can make out ἔχις "viper" now, at the bottom left hand edge. Not that Greeks spent a lot of time talking about vipers; they just spent a lot of time using the verb ἔχει "has", which happens to be ambiguous with the dative of ἔχις. It's automated lemmatisation, this kind of thing can happen.

We have sixteen more stop words, and as you may have worked out, the easiest criterion is to bundle up all function words—prepositions, adverbs, conjunctions, interjections, pronouns. With some of the ambiguity inherent in the venture—is πᾶς "every" a pronoun or an adjective?—but we can keep slicing nonetheless:

And again:

We're not making as much of a difference now; but notice that the screen is being crowded out by verbs: λέγω "say" (and "pick", as a synonym that used to be the same verb—just like "count" and "recount" in English); γίγνομαι "become", ἔχω "have". These are verbs, and are properly considered content words. But I already got rid of εἰμί "to be" (which as a copula is not a content word; and I'm happy to also throw out "have", "become" (close to a copula itself), and verbs for "say". (There is a lot of "he said she said" in the TLG, because there is a lot of narrative.)

If we get rid of those verbs?

And tidying up getting rid of the next hundred and fifty function words, which are a distraction as you squint for content:

You could argue there's still some guff there: ποιέω "do" doesn't tell you much more than ἔχω "have", and πολύς "much" doesn't really deserve its disproportionate size. But we have enough cleaned up that we can say now something about what the texts talk about. It's certainly a sight better than this:

So what do the TLG texts talk about? You may well be starting to come up with ideas if you can read Greek. But before you do, remember that there a whole lot of Christian texts in the TLG, and they quantitatively crowd the ancient texts out. The texts of John Chrysostom alone in the TLG are almost as sizeable as all surviving Ancient literature between Homer and Aristotle.

So yes, the TLG as a whole talks about God and logos a fair bit. But we'd expect that of John Chrysostom; it doesn't mean its what Plato or Homer talk about.

What'd be useful is to split up the corpus, say BC and AD, and see how they differ. Sounds like the next blog post to me...

Btw, I've been stamping out stop words, but stop words are of interest if you're looking at grammar; and Nikos Sarantakos did ask me to pony up the word counts that I was tossing out. So, for the TLG and the lemmatiser as of last night, these are the twenty five most frequent lemmata of Greek, with their textual frequency:
πᾶς534,845every
ἕ547,255he
ἀλλά548,203but
διά561,813for
ἐπί566,238on
πρός566,476towards
κατά643,767by
εἰς694,035to
τῷ732,938therefore (ambiguous with "to the")
μέν762,890on the one hand
ἐγώ767,104I
ὡς771,416as, that
ὅς801,401who, that
λέγω811,330say
τις834,155someone
οὐ926,059not
γάρ951,810because
ἐν1,128,716in
οὗτος1,228,627this
αὐτός1,646,014he, himself
εἰμί1,704,651be
δέ2,265,028but
τίς2,624,172who?
καί5,765,491and
ὁ14,335,717the

Of the lemmata we have not thrown out, θεός "god" is the 39th most frequent, with 388,933 instances.
...Read more

2009-10-13

The 23 to 29 Apolloniuses of Classical Literature

I'm parking this posting here for lack of somewhere else to park it. (It's not strictly language-related, but I'm realising philology posts are probably better pitched here than in The Other Place.)

In my day-job capacity, I'm posting on the fluidity of identity in repositories—how, particularly if you're relying on computer deduplication of identity, there will always be some tentativeness about who is identified as the same person. Repositories have to deal with that tentativeness, rather than hardcoding identity. This is an issue Wikipedia often comes up against, having to split off one article subject from another.

And I was reminded of the morass of Apolloniuses in Classical literature, particularly among medical authors. There are no less than potentially 13 medical authors we know of called Apollonius, according to the TLG Canon of Greek Authors and Works (3rd ed.). Potentially 13, potentially just 9; f.i.q. "possibly the same person as", appears several times in the listing:
(Note: links are to the Catalan Wikipedia.)

Our problem in working out who is who is that almost all of them are cited in passing in other medical authors, so we have very little to go on. The Online TLG Canon only represents works published under a distinct author's name, even if only as Testimonia. So it only refers to two of them, Of Citium and Mys. The other 11 authors also have Testimonia, in Galen and Oribasius and Alexander of Tralles and Aëtius, but they haven't been published independently.
So the 2009 Online Canon has more limited coverage of types of author than the 1990 Print Canon (though a longer time range). The Print Canon includes all ancient authors we know about. The Online Canon only includes those authors with texts represented in the corpus, and that is determined by an editor publishing text under that author's name. So if an Apollonius has been cited in Galen, and an editor publishes that citation as a Fragment of Apollonius, Apollonius will have an entry in the Online Canon, and will link to the published text in the corpus. If we only have secondary source material on the Apollonius from Galen, and an editor publishes that material as Testimonia on Apollonius, the Online Canon will still have an entry, because the point of the author entry is to navigate the corpus.

If OTOH an editor has never combed Galen for Apollonius, the Print Canon will still mention the Apollonius as cited in Galen (as a cross-reference), but the Online Canon will not have an entry for him, because it doesn't have a discrete text for him. That places the Online Canon notion of authors at the mercy of their editorial history; but the Online Canon is documenting edited texts.


The 9–13 medical Apolloniuses in the TLG Canon are joined there by 16 other Apolloniuses in Ancient literature:


The English Wikipedia knows of 16 non-medical Apolloniuses (but five of them are not in the list above), and no less than 21 physicians called Apollonius, since they're not restricted to medical authors. And Wikipedia is just as aware of the f.i.q. issue. The German Wikipedia's list has 24 Apolloniuses, and they don't seem to overlap completely with the English list. The Catalan Wikipedia's list wins, with 18 medical and 39 non-medical Apolloniuses. And its list is even less clean.

Even adding in places of birth, nicknames, and the genres they wrote in, there is difficulty in differentiating these Apolloniuses. If we had enough metadata on them, after all, instead of passing mentions in Galen, we wouldn't be seeing all those f.i.q. For 0741 Apollonius and 0739 Apollonius, we're reduced to distinguishing them by who else they might be confused with. And the medical Apolloniuses may all be obscure (only Of Citium gets his own English Wikipedia page); but the other Apolloniuses include a major Late Epic poet (0001), the founder of Western grammar (0082), a major figure in Roman religious history (0619), a primary source in Homeric scholarship (1168), and an important contributor to the development of 3D geometry (0550). If it wasn't for places of birth and nicknames, we would not know who we were talking about.

Things are slightly better now with the invention of surnames, and recording years of birth in the Library of Congress record. But only slightly: confusion is certainly still possible. There is now a profusion of identities that people write under in cyberspace; if anything, that's now making things even worse. But that's a topic for my day-job blog...
...Read more

2009-07-21

Lerna VIIc: Variants

The various counts of lemmata that I've been putting out for the last while have made little mention of the difficulty in deciding whether two forms belong to variants of the same lemma, or distinct lemmata. The judgement call is difficult enough within a homogeneous language, with slight variations in derivational morphology. It's even worse with a large linguistic span like we've been dealing with, with lots of dialectal variation, phonological change across time, and spelling mutability.

So confronted with two similar nominative singulars in the vocabulary, or two 1st person present indicatives, you need to decide whether you'll count them as the same lexeme or not. And how liberal you are in your counting will decide how many lemmata you count, and how many you dismiss as variants.

To illustrate with English: you won't count color and colour as distinct lemmata, nor publicise and publicize. You won't count recieve as a distinct lemma from receive: misspellings happen, and you still need to count those misspellings as something. Though imbed is not a misspelling of embed, but a different derivation, you'd still want to conflate them too as the one headword.

OTOH, some people draw a distinction between racist and racialist. You may not, but if enough people do, you have to treat them as distinct. (The fact that the OED has decided they're now the same thing does not mean the entire language community has.) That's a judgement call in itself: people are uncomfortable with morphological variation as much as with any other kind, and people swore to the OED that there's a meaning distinction between gray and grey too. So there's no right answers. But there are arbitrary decisions.

Dictionaries normally make those arbitrary decisions for you, because they cross-reference variants to main headwords, and you can rely on their judgement. But dictionaries' judgement can be as arbitrary as any other's: the distinctions can be fine—particularly with variant suffixes, as we'll see. And some dictionaries conflate variants more effectively than others. Kriaras has a lot of phonetic variability, and hides a lot more spelling variability, because of its normalised modern spelling. But it does a reasonable job of indicating which forms are just variants.

By contrast, LSJ's more discursive entries include derived lemmata and similar lemmata, as well as simple variants, in the one entry. So it's harder for automated processing of dictionaries, such as the TLG lemmatiser does, to trust that two words cited in the one entry are in fact the same lexeme. Because the automated processing errs on the side of caution, it will distinguish variants as lemmata more than it should, which inflates the lemma count. The TLG lemmatiser currently seens two or more lemmata in some 12,000 LSJ entries; eyeballing, I'd say maybe a fifth of those could arguably be conflated. There are also a number of lemmata in later dictionaries (Trapp and Kriaras most notably) which could also arguably be conflated with lemmata in LSJ—but haven't been yet, because I haven't been through all 70,000-odd of their lemmata manually. (I'm not bold enough to guess how many.)

So there is a margin of overcounting lemmata; OTOH, there are many more instances where one could argue lemmata are undercounted, because the lemmatiser conflates variants. *I* won't argue that, because I've been responsible for a lot of the conflating. But this has been driven by a particular take on the vocabulary of Greek: if you're searching for words through a search engine, you're likelier to care about meaning than inflection, and you'll want the search to retrieve any word instance that looks enough like your word to match. You won't want to skip matches just because the spelling is slightly off. So for that purpose, less lemmata, meaning more search hits per lemma, is a good thing.

This has meant that, where there was doubt about whether a form is a distinct lemma or not, e.g. in LSJ cross-references, or variation between LSJ and Trapp, I've usually conflated those forms that I've been through manually. That's not everyone's purpose. If you're trying to inflate the count of distinct words of Greek, it's definitely not your purpose. Even if that isn't what you're trying to do, there will be disagreements on how much to conflate.

Part of those disagreements are tied to the dead tree: paper dictionaries have been understandably more reluctant to conflate variants that are alphabetically distant from each other. But there have been times that the choice to conflate hasn't been clear; and times when I've decided not to conflate. (The TLG search engine does display cross-referencing to the user, so that decision is not fatal.)

So what sorts of things may or may not be conflated as the same lemma in Greek? This laundry list will cover at least some of it:
Dialectal phonological differences
For some of the ancient dialects some of the time, these differences are so predictable, they're not even mentioned in LSJ entries. The big example is the different treatment of Proto-Greek */aː/ in Doric and Aeolic (stays α), in Ionic (goes to η), and in Attic (η except after ε, ι, ρ). The second example is Aeolic being accented as far back in the word as humanly possible, something which in the rest of Greek regularly happens for just verbs and compounds. There's more, and together they all mean that you shouldn't count Attic βοηθέω, Ionic βωθέω, and Doric βοαθοέω as different verbs for "help". Nor for that matter Aeolic βαθόημι: the different inflection is normal for Aeolic, and α is a plausible fate for /oaː/. (Just like the Ionic ω is.)
Classical spelling variations
If the odd inscription spells βοηθέω as βοιηθέω, that still isn't reason enough to call it a different lemma. Some spelling variation is endemic to the Classical language, because the pronunciation of those words was in flux—as occurs at any stage of any language. Usually, dictionaries will sweep this spelling variation under the carpet too, in generic cross-references like "κρεω-: see κρεο-". Once LSJ has worked out that λιπ- was the original pronunciation of compounds meaning "lacking in", every compound that can be spelled with λειπ- is listed under λιπ- .
... With the inconvenient exception of λειπογνώμων "without an inspector (or tariff, or distinguishing mark)", for which no ancient authority gives a λιπ- spelling, because the pronunciation was already changing. LSJ does dismiss the mediaeval spellings of the word as λιπογνώμων; but really, who could blame the mediaevals?
Late spelling variations
λιπογνώμων is very, *very* far from the only instance when mediaeval or even Hellenistic writers could not cope with the increasingly historical spelling of Greek, and spelled words more creatively than LSJ allows. LSJ is an historical dictionary, so its business is to go back to the original phonology of the word where possible, and dismiss everything outside its ambit. But if the manuscripts and papyri spell the word differently, that does not mean it's a different word. I've spent several entertaining months helping the lemmatiser cope with λλ as an alternate spelling of λ, or ι as an alternate spelling of ει, or σζ as an... interesting spelling of σ.
Hypotheticals
In making sense of the derivation of words, grammarians would often suggest what they thought the real underlying form of a word was. For example, in discussing ἀζηχής "continuous", they would suggest the form is underlyingly ἀδιηχής "unseparated" or ἀδιεχθής "unhostile". I have on occasion conflated these hypothetical derivations with the word they're explaining, when there was not anything else to be done with them.
Hypercorrections
With written Greek increasingly remote from the spoken language, Byzantine writers did increasingly oddball things to make their words sound high-falutin'. Theodore Metochites' Homer-Through-The-Looking-Glass is one egregious instance, the result of too much Classical learning ("if Homer said νοῦσος for νόσος, then I'll say σουφία for σοφία"). Theodore Studites' is another, and the result of not enough Classical learning: I'll never quite get over him working out that the imperfect of περισσεύω is περι-έσσευε. Throughout the period, there's a persistent tendency to stress words on the "wrong" syllable. I assume the authorities don't discuss it because it was beneath their notice; and I assume the Byzantines did it Because They Could.
This means that Byzantines have made up a bunch of variants for lemmata which did not exist, quite artificially. However artificial they are, they turn up in the corpus, so they need to be accounted for; but they shouldn't be counted as new words. They're old words in clown outfits.
Language change
But Classical words don't only turn up in the corpus in either chlamys or clown outfits. They also turn up in denim. Hm, I don't know if that analogy is going to work...
Variation in words is also going to result from natural language change. (In fact, differentiating natural and artificial language change is harder than it looks.) As we saw in earlier episodes, the *anr- stem of Proto-Greek, "man", turns up as ἀνήρ, ἀνδρός in Classical Greek, as ἄνδρας, ἄνδρα in Byzantine Greek, and as ἄντρας, ἄντρα in Modern Greek. The third declension of the original has been done away with, and the Ancient /ndr/ cluster is now spelled differently, leaving the eye-pronunciation /nðr/ to learned forms.
For the purposes that the TLG lemmatiser is put to—searching words in a diachronic corpus—that's not enough reason to call them different lemmata. ἄντρας is the natural development of ἀνήρ, it means the same thing, so they count as the same thing. But morphologically ἄντρας has already moved on from ἀνήρ, and counting them as the same is a diachronic artifice. It's only because we're trying to cover three thousand years in the one corpus, that we're conflating words three thousand years apart.
Variations in compounding
Putting stem A and stem B together forms you a new compound AB. That compound should for the most part count as the same word, regardless of slight differences in how the compound is put together. So ὑδατοφόρος "water-bearing" does not show up in the dictionaries, but ὑδροφόρος "water-bearing" does; they should count as variants of the same lemma, since hydro and hydato are allomorphs of the same noun, /húdɔːr/ gen. /húdatos/ "water". This, already, is a conflation lexicographers will not be equally eager to embrace.
Similarly, Classical Greek has δαφνηφόρος "carrying laurels" (as a ceremonial act), using -η- as a combining vowel; Late Greek no longer used -η- as a combining vowel, so the word turns up there as the more regular δαφνοφόρος. I'm disinclined to call these distinct lemmata. Others may not be.
Variations in inflection
Here things really do start getting murky: if the inflection of two forms is slightly different, though their stem is the same, should they count as the same lemma? I have allowed them to some times, especially if there is a distinction between an earlier and a later, more transparent or commonplace form. So when Theodore Studites, with his shaky command of classical morphology, uses εὐέλπης, εὐέλπες instead of εὔελπις, εὔελπι, or when he forms the aorist passive participle of εὐκρινέω as εὐκριθέν, implying a citation form εὐκρίνω, I smile benignly, and allow that he's gotten the Classical form wrong—not that he's come up with a brand new lemma. When the variants are contemporary, I'm more reluctant to make that conflation. So I have let ἀθλεύω and ἀθλέω remain distinct verbs. That's not to say I've never done such a conflation—especially when LSJ has said "= sq."; but I've had less of a motivation to.


So I have come up with a count of lemmata that conflates some variants and deflates others. In conflating variants, I am coming up with a lower count of lemmata than others might. Which means there is a question mark over the count of 173,000 lemmata I've claimed.

Well, good. Like I keep saying, there's a question mark over any count of words of any sort. But given that the lemmata contain multitudes, it's worth uncovering how many of those multitudes there are. So I'm going to count up the variants.

The first warning about these counts is that this is still not how you're going to beat English in counts of words. The OED counts some 250,000 lemmata (and its coverage of Old and Middle English is *not* exhaustive); with variants, that gets up to 615,000. There will be a lot more than 173,000 variants to count in the Greek corpus; but there won't be 615,000.

The second is to explain what I'm counting.
  • I allow any variation in the lexical rot, recorded in the lexical database, to count as a distinct variant. So ἀνήρ, ἄνδρας, and ἄντρας count as three variants, and Doric ἀγέννατος and Attic ἀγέννητος as two.
  • I do *not* count dialectal or diachronic change in the same inflection paradigm (including tense stems) as different. So I don't count both Ionic χώρ-η and Attic χώρ-α, or Doric δωρ-ίσδω and Attic δωρ-ίζω, as different.
  • I don't count dialectal variants in preverbs, because they are not part of the root; so ξυμμαχέω is not counted separately from συμμαχέω.
  • I don't count uppercase and lowercase variants of the same lemma. (But I do count them when they are distinct lemmata.)
  • I don't count active and passive variants of the same root verb as distinct.
  • And I don't count the adhoc respellings that the lemmatiser can do on the fly, to recognised deviations particularly in diplomatic editions.

So. I had 214,381 lemmata in the corpus. Without proper names and Milesian numbers, that came down to 172,646. How many variants does that translate to?
  • Variants: 362,947
  • Without numbers: 352,895
  • Without names and numbers: 286,652.

(I'd guessed around 350,000 variants a couple of postings ago. That's pretty good. It would be even better, if my guess hadn't excluded proper names...)

This amounts to 1.7 variants per lemma. I'll admit to some surprise that the OED ratio of variants to lemmata is more like 2.5: Greek historical spelling should allow for comparable confusion. My suspicion is it does, and I'm discounting adhoc misspellings which the OED doesn't.

Names are slightly more variable than normal words: 66,243 variants for 37,306 names, which is a ratio of 1.8:1. Foreign names in particular get mangled in several ways, including creative hellenisations: that's why there are 75 different variants of "Muhammad", and 43 variants of "Lombard". (Those examples aren't fair, since "Muhammad" includes the Turkish "Mehmed", and "Lombard" also includes the earlier "Longibard"—so again, I'm conflating variants more than some might.)

That count can be whittled down further of course.
  • If we discount all variants noted as hypothetical—which were made up by grammarians, and were not used in the actual language, we come down to 274,650.
  • If we ignore variation in accentuation, which is mostly a Byzantine hypercorrection, we're down to 265,233.
  • If we ignore the uncertainty between ει and ι, which bedevilled the Koine, we're down to 260,362.
  • Ignore double consonants: 253,891.
  • Ignore the distinction between η and α, as a brute-force levelling of Doric and Attic, and we're down to 247,294.
  • Ignore the distinction between smooth and rough breathing (which occasionally tripped scribes up): 245,672.

Even with these common causes of variation excluded, that's still some 73,000 added variants (41%) that I have not counted as distinct lemmata. That means that one could argue some of them should be counted as separate—although I have trouble seeing how a consistent criterion could be devised, especially over such a large timespan.

So the lemma count may be an underestimate, because of different judgements on what counts as distinct; but at its most inflated, the lemma count will no more than double. In reality, I think the debatable instances are closer to 20% than to 70%. So no, not even this way are we getting to 5,000,000 lemmata.
...Read more

2009-07-15

Lerna VIIb: Lemma counts and proportion of text recognised

We can keep dredging lemmata up to move towards a target of 300,000. But of course for a living language, as Modern Greek now is and as Ancient Greek once was, there is no ceiling in lemmata: people can always make up new words, and do. And because dictionaries will never exhaust what words people come up with, even if they work off a limited corpus, the constructive thing to do is not to say how many lemmata are in a language.

The constructive thing rather is to say, if I know n lemmata in a language, how many word instances in a corpus will I understand? If I know n lemmata, how much of the text I'm confronted with can I make sense of? If a vocabulary of 500 words lets you understand just 70% of all the text you'll see, you're in some trouble. If a vocabulary of 50,000 lets you understand 99.7% of a text, on the other hand, that's one word instance out of three hundred that you'll draw a blank on. Assuming 500 words a page, that's around three unknown words every couple of pages. That's still a lot: if you're having to run to the dictionary once a page, you've got catching up to do in the Word-A-Day club. One word every ten pages—say 99.98% of all word instances: that's probably more reasonable.

That gives you a statistic of how many word instances are recognised; but when you're listing the words you don't know yet, you tend to list unfamiliar word forms, not word instances. So if you come away from your reading of The Superior Person's Little Book of Words with a list of words you need to look up in the dictionary, you won't count contrafibularity three times and floccipaucinihilation seven times. You only have to look up contrafibularity once to understand it, so you'll list it as a single unknown word form.

The proportion of recognised word forms is going to be much lower than the proportion of recognised word instances. The word instances will give you credit for knowing words like and and the and of: hey presto, with those three words, you already understand 20% of all printed words of English! The words you won't know will tend to be one-offs, occurring just once or twice in a text: it's rare words, which people don't come across a lot in texts, that they won't have needed to learn. But with word forms, and and the and of don't count as 20% of all printed words of English: they only count as three word forms. The unfamiliar one-offs will make a much larger dent in the size of your vocabulary, than in the proportion of a page you can grok.

Again, the point of this is to say, not that there are n words in a language, which is deeply problematic in ways I've gone into great length on. It's to say that, if you know n lemmata, you will understand n1% of the vocabulary of a corpus (its word forms), and n2% of all the text in a corpus (its word instances). The value of n can go up or down, and the proportions of words you understand can go up or down with it. This means two things which are more useful to keep in mind than any grand How Many Words statement.

First, it's not about how many words there are ever, so much as how many words are *useful* to know. If there are fifty words which were made up for Joe Blow's autobiography, and Jow Blow's autobiography has never been published or indeed sighted outside his kitchen, then those fifty words will not form part of your corpus, so they need not count. Or, if there are three hundred words of phrenology which noone has used for the past century, and they only got used once in a blue moon, then even if those three hundred words do show up in your corpus, they will be marginal enough to cut out most of the time. Tying vocabulary size to recognition allows you to limit the lexicon to what you will actually use, and how frequently you will use it.

The second realisation is the admission this makes, that the size of a vocabulary is asymptotic. People can keep making up words, or using words in increasingly niche and esoteric contexts. If n words let you understand 99% of the vocabulary, then you may well be able to come up with 5n words to recognise 99.9% of the vocabularly, and 20n to recognise 99.99% of the vocabulary, and even 100n to recognise 99.999% of the vocabulary. But by the time you're up to 99.99% recognised words, you can reasonably ask whether it's worth spending an extra five years building up your vocabulary, just to deal with the remaining 0.01%.

The answer is no. Dictionaries do not wait forever before they decide they're done: they have a large corpus, and dip into it fairly eclectically, but they do miss stuff (not just "sausage" in that Blackadder episode on Johnson's Dictionary). And that's OK, if the word is obscure enough for the dictionary's purposes. Dictionaries employ some subjectivity in leaving out words until they think they're worth taking seriously; but the way a corpus is put together usually filters the obscurities out for you already. If you're relying on printed text to prove a word is worth describing, you're leaving out all the made up and nonce words and speech errors that were never written down. Of course, that's an elitist way of viewing language, and print is nowhere near the barrier it used to be. But it does cut your corpus down to something manageable.

If you're working on a Classical language, the cruelty of Time (and Bastard Crusader scum), the indifference of scribes, and the snootiness of schoolmasters do plenty of filtering for you as well. That's why the PHI #7 corpus, which was not subject to the same filters as the literary corpus, has so much distinct vocabulary.

(For an example of why the bones of the Bastard Fourth-Crusader scum should boil in pitch in eternity, see the fate of the text of Ctesias and the other manuscripts unfortunate enough to be in Constantinople in 1204.)

So what sort of recognitions do the figures I've been quoting represent? I'm using the five corpora as before, and I'm also differentiating between all word forms, and just lowercase word forms—because proper name recognition lags behind recognition of common words in general. Lowercase word forms is a somewhat crude metric for leaving out proper names, and there are a few TLG editions which follow the e.e.cummings stylings of their manuscripts, leaving names lowercase. But it's all about the indicative figures, always.
% Instances RecognisedRecognised Instances Ratio% Lowercase Instances RecognisedRecognised Lowercase Instances Ratio
TLG + PHI #799.66%1:29499.861:740
TLG99.84%1:62499.9151:1170
LSJ99.37%1:15899.83%1:585
Mostly Pagan99.964%1:275999.979%1:4750
Strictly Classical99.967%1:299399.975%1:4019

% Forms RecognisedRecognised Forms Ratio% Lowercase Forms RecognisedRecognised Lowercase Forms Ratio
TLG + PHI #789.56%1:9.694.33%1:17.6
TLG93.91%1:16.495.59%1:22.7
LSJ89.51%1:9.595.77%1:23.6
Mostly Pagan99.16%1:11899.42%1:172
Strictly Classical99.56%1:22699.62%1:263

Let's go through this slowly.

The lemmatiser understands the Strictly Classical corpus—literary Greek up to iv BC—quite well. It only fails to pick up 1 in every 226 distinct word forms, which mean you have go through on average 2993 word instances—say six pages of text—before you hit a word it does not understand. But you can ignore capitalised words, because they're typically proper names, and we don't expect to have those in our vocabulary anyway. You can make sense of "Alcidamophron slaughtered the servant of Tlesipator" more readily than you can "Jack fnocilurphed the smorchnepot of Jill". If we do ignore capitalised words, the lemmatiser fails to understand just 1 in 263 word forms, and over eight pages of text on average before it finds a problem word. As machine understanding of morphology goes, that's not bad at all.

So the 55,000 lemmata that the lemmatiser knows of for the Strictly Classical corpus get you through eight pages of Greek on average as smooth sailing. And that is the real meaning of "55,000" lemmata here. Of course, that's an eight page average across a corpus that is still not terribly homogeneous; and some bits of the corpus are going to be understood a lot better than others. The lemmatiser understands all 199,000 word instances in Homer, for instance: 400 pages by our reckoning, not just 8. On the other hand, the Strictly Classical corpus also includes Aeschylus, whose transmission has been corrupted frequently, and where the lemmatiser falls over 63 word instances of 74,000—once every couple of pages.

With the Mostly Pagan corpus, which sticks to literary texts up to IV AD, the lemmatiser understands the corpus almost as well: 76,000 lemmata give you all but 1 in 172 word forms, and in fact because the later texts are slightly more homogeneous linguistically, almost 10 pages on average of text before there is a problem word. So 76,000 lemmata for Mostly Pagan is about as meaningful a claim as 55,000 lemmata for Strictly Classical: it lets you understand almost the same proportion of text in the corpus. There's bound to be more lemmata than that in the corpus, that the dictionaries have not officially recorded; but it's not going to be overwhelmingly more. I'd guessed maybe 500 lemmata underestimated for the Strictly Classical corpus, with 1,500 unrecognised word forms. The Mostly Pagan corpus has 5,000 unrecognised word forms, so I'll guess maybe 2,000 underestimated lemmata.

The LSJ corpus is much less well understood, partly because it includes technical writing, but mostly because it includes the more unruly texts from the inscriptions and papyri, with their distinct vocabulary and grammars, and confusing spellings. We claimed 124,000 lemmata here, but that only gets you one word form unrecognised per 23; including potential proper names, it's as bad as one word unrecognised in ten. And you'll be stumbling over one word per a page and a bit. Our unrecognised word forms are now up to 35,000 lowercase forms. That does not necessarily mean 10,000 more lemmata unaccounted for, given the problems in spelling and grammar; so I'm reluctant to guess how many more lemmata you need to get to the same level of recognition as with the Strictly Classical corpus. But there are clearly more lemmata to go.

You can see the trouble the papyri and inscriptions bring more clearly in the last two counts, which include and exclude them. Without them, the TLG corpus has one lowercase word form in 23 unrecognised, and a little over a word per two pages unrecognised. That's not that bad for the claimed 162,000 lemmata, given the bewildering diversity of texts in the corpus. Let the inscriptions and papyri back in, and you now miss a word form for every 18, and a word every one and a half pages. And that's for increasing the size of the corpus by just a twentieth.

So the lemma counts are more and less reliable for different periods of Greek: we can tell how much text they allow you to recognise in different corpora, and we can allow that there are cut-offs for how many lemmata it is useful to know in a corpus. The lemma count is still not open-ended, so long as the corpus is finite. (That's the thing about langue instead of parole: the corpus size of *potential* text, using language as a theoretical system, is infinite.) And the word form coverage of the lemmatiser will keep improving, as an ongoing project; as I'd already mentioned before, TLG word form recognition has gone up from 90% to 94% in the past two years. But the lemma count does peter off.

So let me give one last batch of numbers to illustrate the relativity of lemma counts: how much less of a corpus do we understand, if we cut down on the number of lemmata. I'll do that using the word instances per lemma count for the TLG. Because there is a fair bit of ambiguity in Greek morphology, many word forms are ambiguous between two lemmata (and a few between more than two); so there is some double counting of instances to be had. As a result, the 202,000 lemmata recognised in the TLG corpus—proper names and not—account for 112 million word instances, though the corpus really contains only 95 million.

So if we take 112 million as our baseline, how many instances are accounted for by admitting less lemmata?

LemmataWord Instances
100 61,166,25354.44%
500 78,932,45170.25%
1,000 86,575,28677.06%
2,000 94,016,671 83.68%
5,000 102,370,24391.11%
10,000 106,926,32495.17%
20,000 109,884,24897.80%
50,000 111,727,54499.44%
100,000112,191,09599.85%
120,000112,251,18199.91%
150,000112,302,89599.95%
180,000112,332,89599.98%
190,000112,342,89599.98%
202,000112,354,703100%


There's your Zipf's Law in action. The table neatly parallels what Wikipedia says for vocabulary size, quoting a 1989 paper presumably on English: "We need to understand about 95% of a text in order to gain close to full understanding and it looks like one needs to know more than 10,000 words for that."

The difference between 100,000 and 200,000 lemmata accounts for just 163,000 word instances out of 112 million, around one word in 700. The difference between 180,000 and 200,000 accounts for less than a word every ten pages. So there's a very very long tail of increasingly rare words: the last 60,000 lemmata each occur just once in the 95 million word corpus, and the last 25,000 lemmata before that occur just twice. There's a *lot* of these one-offs, which is why all together they account for an unknown word every four pages. And we need dictionaries for words we don't come across every day, not words we do.

Still, they are one-offs (hapaxes). They're not useless—they were clearly useful to whoever used them that one time in the 2,500 year span of the corpus. But noone needs all 60,000 of them at once. And by the time you're down to lemmata that happen just once or twice in a roomfull of books (a small room admittedly), you can appreciate why real human beings walk around with close to 20,000 lemmata in their skulls, and not 200,000. For the rest, we have guessing from context (and related words); and we have dictionaries. And once Classical Greek became a bookish language, the Byzantines used dictionaries too.
...Read more