Taxonomy Enhancer

From Open Food Facts wiki
Jump to navigation Jump to search


Taxonomy Enhancer has been introduced in 2025 (https://github.com/openfoodfacts/openfoodfacts-server/pull/11267) to help to find improvements for the taxonomies and the Open Food Facts Server code by comparing ingredients analysis between languages.

Reminders

  • Ingredient analysis is done on a unique language, the main language

For example, for this product hereafter Italian is the main language

Taxonomy-enhancer-main-language



  • When a contributor extracts the text from the image, the whole text is extracted using stopwords defined in the Open Food Facts Server code (lib/ProductOpener/Ingredients.pm)


Selected image:

taxonomy-enhancer-ingredients-taxonomies-stopword

Location of the code in Open Food Facts server:

taxonomy-enhancer-ingredients-taxonomies-repo-lib

Stopwords before in the code:

taxonomy-enhancer-ingredients-taxonomies-stopwordbefore

Stopwords after in the code:

taxonomy-enhancer-ingredients-taxonomies-stopwordafter

Extracted ingredients:

taxonomy-enhancer-ingredients



  • The ingredients extracted for Italian are used for the ingredients analysis:

Results of the analysis as shown on the Product page:

taxonomy-enhancer-ingredients-analysis

Ingredients are known if they are in the taxonomies

Location of the taxonomies files in Open Food Facts server:

taxonomy-enhancer-ingredients-taxonomies

First ingredient as seen in the taxonomy:

taxonomy-enhancer-ingredients-taxonomies-ingredients


  • Sometimes the ingredients list for other languages are different than the one of the main language:

taxonomy-enhancer-ingredients-taxonomies-other-lang


Detections

  • Detect stopwords before ingredients (example: Possible-stop-word-before-bg-съставки)
  • Detect stopwords after ingredients (example: Possible-stop-word-after-fr-tenir-au-sec)
  • Detect new translations (example: Ingredients-sl-morska-sol-is-new-translation-for-en-sea-salt)
  • Detect typos (example: Ingredients-hr-spinat-is-possible-typo-for-hr-špinat)
  • Detect ingredients that should have the same id (example: Ingredients-taxonomy-between-coconut-id-en-coconut-oil-and-kokos-id-en-coconut-should-be-same-id)


How to find all errors?

https://hr.openfoodfacts.org/facets/taxonomies-enhancer

Replace hr by a different country code

How to fix these errors?

If you can only make changes on the website

  1. For Possible-stop-word-before- if it is not a missing stopwords, it can be fixed by editing the product:
    1. a dust on the word
    2. the word in split in two (in- gredient)
    3. package is folded
    4. package is blurry
  1. Many errors are due to reuser App auto-translating in English and can be fixed by removing the ingredients for English and unselecting the image if the ingredients does not exist in English for this product.
  1. Many errors for Possible-stop-word-before- appear because the image contains the whole product or more than a single ingredients list and can be fixed by reselecting the ingredients list and reextracting the text.
  1. Many errors for Possible-stop-word-after- appear because a previous contributor removed "can contains traces of" from the list and can be fixed by reextracting the text.
  1. Not all, but some errors for -is-possible-typo-for- are really typo and can be fixed manually (for example: "Ingredients-bg-регулатор-на-киселиността-is-possible-typo-for-bg-регулатор-на-киселинността")

If you know how to add ingredients using the Taxonomy Editor

Taxonomy Editor: https://github.com/openfoodfacts/taxonomy-editor?tab=readme-ov-file#getting-started

  1. Review Ingredients-<language>-<ingredient>-is-new-translation-for-en-<ingredient-id>, if it is a new ingredient, add it to the taxonomy
  1. Not all, but some errors for -is-possible-typo-for- are not typo but synonyms and can be added to the taxonomy

If you know how to make a pull request in Github

  • Review Ingredients-<language>-<ingredient>-is-new-translation-for-en-<ingredient-id>, if it is a new ingredient, add it to the taxonomy
  • Not all, but some errors for -is-possible-typo-for- are not typo but synonyms and can be added to the taxonomy
  • Add missing stopwords (Possible-stop-word-after-, Possible-stop-word-before-) in Ingredients.pm (phrases_before_ingredients_list, phrases_before_ingredients_list_uppercase, phrases_after_ingredients_list)
  • Add missing regex words in Ingredients.pm for "country of origin" (origin_of_the_regexp_in_lc), phrases to ignore (ignore_regexps, ignore_phrases, ignore_strings_after_percent), "can contains traces of" (may_contain_regexps, contains_regexps), eventually also abbreviations, of, from, and, and_of, and_or, the, per, of_finished_product, prepared_with, min_regexp, max_regexp, minimum_or_total, content_of_ingredient, ingredient_content.

Feedbacks, analysis

Benoit (benbenben) - review for Croatia slides of the presentation (pdf) File:Openfoodfacts-taxonomy-enhancer benbenben.pdf