fbk-NEUTR-evAL

This repository includes solutions related to the evALuation of gender NEUTRality, specifically focusing on metrics designed to assess whether a given text contains inclusive language with respect to gender, in both monolingual and cross-lingual contexts.
The repository is maintained by the Machine Translation unit of FBK.

Installation

To clone and install our repository, execute the following commands:

git clone https://github.com/hlt-mt/fbk-NEUTR-evAL.git
cd fbk-NEUTR-evAL
pip install -e .

Available Solutions

You can find a dedicated README for each available evaluation solution in the solutions directory, along with Bibtex citations referencing the respective papers:

• Neutral Classifier for GeNTE v1 from [EMNLP 2023] Hi Guys or Hi Folks? Benchmarking Gender-Neutral Machine Translation with the GeNTE Corpus
• Neutral Classifier for GeNTE v2 from [EACL 2024] A Prompt Response to the Demand for Automatic Gender-Neutral Translation
• Evaluation Script for Neo-GATE from [EAMT 2024] Enhancing Gender-Inclusive Machine Translation with Neomorphemes and Large Language Models
• Scripts and Data for LLM-based neutrality evaluation from [GITT 2025] An LLM-as-a-judge Approach for Scalable Gender-Neutral Translation Evaluation

License

This project is licensed under the Apache License 2.0.