Cross-lingual version of WEAT, which includes the tests in English, German, Spanish, Croation, Italian, Russian, and Turkish.
Example usage: This example would test monolingual bias in seven languages.
for similarity_type in "cosine" "euclidean" ; do
for language in "en" "de" "es" "hr" "it" "ru" "tr" ; do
for test_number in 3 4 5 6 10; do
python weat.py \
--test_number $test_number \
--permutation_number 1000000 \
--output_file ./results/w2v_wiki_${language}_${similarity_type}_${test_number}_cased.res \
--lower False \
--use_glove False \
--is_vec_format True \
--lang $language \
--embeddings \
<PATH TO YOUR EMBEDDINGS>/cbow.wiki.${language}.300w5.vec \
--similarity_type $similarity_type |& tee ./results/w2v_wiki_${language}_${similarity_type}_${test_number}_cased.out
done
done
done
This example tests bias in cross-lingual embedding spaces. Here, Spanish is tested in bilingual spaces with each of the other six languages.
for similarity_type in "cosine" ; do
targets_language="es"
for attributes_language in "en" "de" "hr" "it" "ru" "tr" ; do
for test_number in 6 7 8 9 10 1 2 ; do
xspace=${targets_language}-${attributes_language}
embedding_dir="/smith/fasttext/${xspace}" # be aware that you need to have trained the bilingual embedding space first
python xweat.py \
--test_number $test_number \
--permutation_number 1000000 \
--output_file ./results/ft_xling_space-${xspace}_ta-${targets_language}-${attributes_language}_${similarity_type}_${test_number}.res \
--lower False \
--use_glove False \
--targets_lang $targets_language \
--attributes_lang $attributes_language \
--targets_embedding_vocab \
${embedding_dir}/${targets_language}.vocab \
--targets_embedding_vectors \
${embedding_dir}/${targets_language}.vectors \
--attributes_embedding_vocab \
${embedding_dir}/${attributes_language}.vocab \
--attributes_embedding_vectors \
${embedding_dir}/${attributes_language}.vectors \
--similarity_type $similarity_type |& tee ./results/ft_xling_space-${xspace}_ta-${targets_language}-${attributes_language}_${similarity_type}_${test_number}.out
done
done
done
For more information on our approach, experiments, and results we refer to our paper:
Lauscher A. and Glavas G. (2019). Are We Consistently Biased? Multidimensional Analysis of Biases in Distributional Word Vectors. To appear at *SEM 2019.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent