StyloMetrix

Zakład Inżynierii Lingwistycznej i Analizy Tekstu, NASK PIB

📌 Quick

💡 Stylometry tool in beta version for English, German, Polish, Russian and Ukrainian language, distributed as a Python package

💡 Tutorial notebook

💡 Notebook with example usage for classification and clustering with StyloMetrix

💡 List of built-in metrics for Polish, English, German, Ukrainian, Russian

🔖 Citation

Please cite this article when referring to StyloMetrix:

Okulska, I., Stetsenko, D., Kołos, A., Karlińska, A., Głąbińska, K., & Nowakowski, A. (2023). StyloMetrix: An Open-Source Multilingual Tool for Representing Stylometric Vectors. arXiv preprint arXiv:2309.12810.

🔔 About

StyloMetrix is a tool for creating text representations as StyloMetrix vectors. Each metric in vector quantifies a linguistic feature in text. Therefore a detailed information of the style of text can be translated to numeric values and used for - whatever you want!

The metrics are:

interpretable - each metric represents an aspect of linguistic knowledge
normalized - metrics express number of ocurrences of given feature per number of tokens in text, which lets us escape scaling effect in texts of different lengths
reproducible - values of metrics can be recalculated or even counted manually giving always the same output. The representation doesn't depend on any random factor or seeding
customizable - if your needs exceed the scope of built-in metrics, create your own! Don't forget to share your work and contribute to the community of StyloMetrix!

A StyloMetrix vector can be used as:

stylometric signature that encodes the writing style of the author and the genre
input for classifiers of supervised or unsupervised learning, for example Random Forest classifier or feature selection algorithms
values for statistical analyses in science
set of linguistic data for manual reference

The tool offers customization of vectors by selecting from built-in metrics or creating new metrics according to user's needs. We provide a user-friendly interface to support these tasks. See instructions below! ⬇

Currently StyloMetrix is available for English, German, Polish, Russian and Ukrainian language.

📢 Release

Our most recent release is:

v0.1.0

Changing the structure of StyloMetrix
Works much faster!
New metrics and categories in Polish and English language
German language in beta version
Russian language in beta version
Ukrainian language in beta version
Possibility to define metrics to use / categories of metrics as list of strings containing names.
Possibility to save intermediate steps so even if something crashes, you still have some of work saved.

Please notice that support for Russian and Ukrainian languages will no longer be available.

Previous releases ⌛

v0.0.6

Add categories Syntactic and Lexical for English

v0.0.4

Add English beta with built-in metrics in category Grammatical Forms

v0.0.3

Add StyloMetrix structure
Add tutorial
Add 6 built-in metrics categories for Polish beta: Grammatical Forms, Inflection, Lexical, Psycholinguistic, Syntactic, Word Formation
Specify license & citation

🔨 Installation

1. Install spaCy

Install spacy according to spaCy install instructions

2. Install model

▶ For Polish:

Download and install model pl_nask v0.0.7

📍 pl_nask is the new HerBERT based model from IPI PAN, requires spacy==3.3

python -m pip install <PATH_TO_MODEL/pl_nask-0.0.7.tar.gz>

▶ For other languages:

For English install en_core_web_trf from spaCy install instructions
For German install de_core_news_lg from spaCy install instructions
For Russian install ru_core_news_lg from spaCy install instructions
For Ukrainian install uk_core_web_trf from spaCy install instructions

3. Install StyloMetrix

pip install stylo_metrix

🪁 How to use

Get your texts and import StyloMetrix:

import stylo_metrix as sm

texts = ['Panno święta, co Jasnej bronisz Częstochowy I w Ostrej świecisz Bramie!',
        'Ofiarowany, martwą podniosłem powiekę; I zaraz mogłem pieszo, do Twych świątyń progu...',
        'W ludziach straty nie było. Ale wszystkie ławy Miały zwichnione nogi;']

Use StyloMetrix object for this texts:

stylo = sm.StyloMetrix('pl')
metrics = stylo.transform(texts)
print(metrics)

Your results is now in metrics object.

That's it! Find out about more usages and customization options in notebook tutorial.

Find out about using StyloMetrix in classification or in clustering in example notebook

📈 Metrics

We have put care into creating a set of powerful built-in metrics. See the list below ⬇. However, since flexibility is strength, we provide an easy way to create new metrics.

Polish (see full list)

English (see full list)

German (see full list)

Russian (see full list)

Ukrainian (see full list)