Difference between revisions of "Eco-Score - Agribalyse coverage"

From Open Food Facts wiki
Jump to navigation Jump to search
 
(9 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
[[Category:Environment]]
 
= Introduction =
 
= Introduction =
  
Line 15: Line 16:
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
! [https://fr.openfoodfacts.org/categories-properties FR products] !! 2020-07-22 !!
+
! [https://fr.openfoodfacts.org/categories-properties FR products] !! 2020-07-22 !! 2020-08-28 !! 2020-10-21
 
|-
 
|-
| All-products || 697812 - 100%||  
+
| All-products || 697812 - 100% || 708863 || 724653
 
|-
 
|-
| Categories-known || 362464 - 52% ||  
+
| Categories-known || 362464 - 52% || 365522 - 52% || 373380
 
|-
 
|-
| Agribalyse-food-code-known || 106549 - 15% ||  
+
| Agribalyse-food-code-known || 106549 - 15% || 113053 - 16% || 121291
 
|-
 
|-
| Agribalyse-proxy-food-code-known || 1431 - 0% ||  
+
| Agribalyse-proxy-food-code-known || 1431 - 0% || 14327 - 2% || 57720
 
|-
 
|-
| Agribalyse-known || 107913 - 15% ||  
+
| Agribalyse-known || 107913 - 15% || 124573 - 18% || 167746
 +
|}
 +
 
 +
{| class="wikitable"
 +
|-
 +
! [https://fr.openfoodfacts.org/popularity/top-10000-fr-scans-2019/categories-properties Top 10k FR products] !! 2020-07-22 !! 2020-07-26 !! 2020-08-28 !! 2020-10-21
 +
|-
 +
| All-products || 9798 - 100% || 9813 - 100% || 9813 - 100% || 9821
 +
|-
 +
| Categories-known || 9759 - 100% || 9774 - 100% || 9788 - 100% || 9800
 +
|-
 +
| Agribalyse-food-code-known || 3696 - 38% || 3701 - 38% || 3985 - 41% || 4415
 +
|-
 +
| Agribalyse-proxy-food-code-known || 110 - 1% || 386 - 4% || 622 - 6% || 2669
 +
|-
 +
| Agribalyse-known || 3801 - 39% || 4041 - 41% || 4487 - 46% || 6402
 
|}
 
|}
  
Line 40: Line 56:
 
Unfortunately, in many cases, the Agribalyse categories differ from the OFF categories, in which case the Agribalyse categories have been added to the OFF taxonomy. In that case, the categories are empty and do not have products yet.
 
Unfortunately, in many cases, the Agribalyse categories differ from the OFF categories, in which case the Agribalyse categories have been added to the OFF taxonomy. In that case, the categories are empty and do not have products yet.
  
== Metrics - May-July 2020 ==
+
== Implementation of products coverage metrics - May-July 2020 ==
  
 
In order to measure the coverage (% of products with an associated Agribalyse category), we are computing a new /categories-properties facets that shows which products have an associated Agribalyse category.
 
In order to measure the coverage (% of products with an associated Agribalyse category), we are computing a new /categories-properties facets that shows which products have an associated Agribalyse category.
Line 70: Line 86:
 
</pre>
 
</pre>
  
=== How to identify potential Agribalyse proxy categories ===
+
=== How to identify and add potential Agribalyse proxy categories ===
 +
 
 +
'''Detailed guide in French: [https://docs.google.com/document/d/1MimfhVuFw1Xd1VRvqyduaHMiqr6eoxlOaWR_86yLr4Q/edit#heading=h.ihogua6y37an Guide d’ajout de correspondances Agribalyse - OFF]
 +
'''
  
Ask Pierre or Stéphane for the Agribalyse list of categories.
+
Ask Pierre or Stéphane for the [https://drive.google.com/file/d/15KrNf_RETm90lrpM-bnzaDDJbaH4gMWM/view?usp=sharing Agribalyse list of categories]
  
 
* Search for groups of Agribalyse categories that correspond to different versions (e.g. flavoured, sugared etc.) of similar products
 
* Search for groups of Agribalyse categories that correspond to different versions (e.g. flavoured, sugared etc.) of similar products
 +
** It can be useful to look at the current list of [https://fr.openfoodfacts.org/categories-properties/agribalyse-unknown/categories categories of products for which we don't have an Agribalyse association] in order to see which of those categories could have good potential Agribalyse categories matches
 +
*** [https://fr.openfoodfacts.org/popularity/top-10000-fr-scans-2019/categories-properties/agribalyse-unknown/categories unmatched categories for Top 10k products]
 
* Check if the environmental impact for the categories are the same or almost the same
 
* Check if the environmental impact for the categories are the same or almost the same
 
* If the environmental impact is the same, look in the OFF categories taxonomy to see if we have a parent category that matches well the group of Agribalyse taxonomies
 
* If the environmental impact is the same, look in the OFF categories taxonomy to see if we have a parent category that matches well the group of Agribalyse taxonomies
Line 83: Line 104:
 
If a product does not have a category, we cannot match it to the Agribalyse categories. So it is very important to add categories to products. The more specific the category, the better chance we have that it will match either an Agribalyse category or an Agribalyse proxy category.
 
If a product does not have a category, we cannot match it to the Agribalyse categories. So it is very important to add categories to products. The more specific the category, the better chance we have that it will match either an Agribalyse category or an Agribalyse proxy category.
  
(add links to Hunger Games)
+
=== Tools to easily add categories to hundreds of products ===
  
 +
* [https://hunger.openfoodfacts.org/questions?type=category&countries=en%3AFrance Hunger Games] shows you category predictions that you can very easily and quickly validate if they are correct.
 +
** To specifically validate categories for which we already have an Agribalyse category, you can use the links from this file: [https://docs.google.com/spreadsheets/d/1nldTEVoVVZ_c-VgiwhzdawThq4L4f0U4E6UxYMLG0e0/edit?usp=sharing Hunger Games links to validate Agribalyse categories]</big>
  
 
= Tags rules to automatically add more specific categories =
 
= Tags rules to automatically add more specific categories =
  
 
(not coded yet)
 
(not coded yet)

Latest revision as of 14:31, 23 February 2023

Introduction

The main factor for computing the Eco-Score of a product is the measure of its environmental impact in the Agribalyse database from ADEME (French national environmental agency).

Agribalyse contains data for 2500 categories of products (CIQUAL categories, different from the Open Food Facts categories).

This page:

  • tracks the % of products in the Open Food Facts database that we can associate with an Agribalyse category
  • defines a plan to improve that %

Metrics

The /categories-properties shows the products that have associated Agribalyse properties.

FR products 2020-07-22 2020-08-28 2020-10-21
All-products 697812 - 100% 708863 724653
Categories-known 362464 - 52% 365522 - 52% 373380
Agribalyse-food-code-known 106549 - 15% 113053 - 16% 121291
Agribalyse-proxy-food-code-known 1431 - 0% 14327 - 2% 57720
Agribalyse-known 107913 - 15% 124573 - 18% 167746
Top 10k FR products 2020-07-22 2020-07-26 2020-08-28 2020-10-21
All-products 9798 - 100% 9813 - 100% 9813 - 100% 9821
Categories-known 9759 - 100% 9774 - 100% 9788 - 100% 9800
Agribalyse-food-code-known 3696 - 38% 3701 - 38% 3985 - 41% 4415
Agribalyse-proxy-food-code-known 110 - 1% 386 - 4% 622 - 6% 2669
Agribalyse-known 3801 - 39% 4041 - 41% 4487 - 46% 6402

Facets can be combined, for instance to see the categories of products for which we don't have an Agribalyse association.

Plan to improve the Agribalyse coverage

Initial matching of Agribalyse categories to OFF categories - April-June 2020

As part of the Data4Good program, volunteers have added all the Agribalyse categories inside the OFF categories taxonomy.

When there was an exact match (the Agribalyse category exists in OFF), the agribalyse_food_code:en property was added to the existing category.

Unfortunately, in many cases, the Agribalyse categories differ from the OFF categories, in which case the Agribalyse categories have been added to the OFF taxonomy. In that case, the categories are empty and do not have products yet.

Implementation of products coverage metrics - May-July 2020

In order to measure the coverage (% of products with an associated Agribalyse category), we are computing a new /categories-properties facets that shows which products have an associated Agribalyse category.

Adding Agribalyse proxy categories in the OFF taxonomy

There are many cases where the Agribalyse categories are much more specific than the OFF categories, while that specificity is not needed to estimate environmental impacts.

This is because the Agribalyse categories are in fact the CIQUAL categories (the French national database for nutrition). For instance, many categories specify if something is flavoured, sugared, un-sugared, or with artificial sweeteners (e.g. yogurts). That distinction is very important for nutrition, but has often little or no impact on environmental impacts.

For example, CIQUAL define 4 categories for drinkable yogurts:

Boisson lactée, lait fermenté ou yaourt à boire, aromatisé, sucré
Boisson lactée, lait fermenté ou yaourt à boire, nature, sucré
Boisson lactée, lait fermenté ou yaourt à boire, nature, sucré, au L Casei
Boisson lactée, lait fermenté ou yaourt à boire, aux fruits, sucré

But those 4 categories in Agribalyse have the same or mostly the same environmental impact.

In that case, we can associate one of the specific Agribalyse category to the broader "Drinkable yogurts" category on OFF. We call it a "proxy" Agribalyse category.

en:Drinkable yogurts
agribalyse_proxy_food_code:en:19508
agribalyse_proxy_food_name:en:Dairy drink or fermented milk or yogurt, flavoured, with sugar
agribalyse_proxy_food_name:fr:Boisson lactée, lait fermenté ou yaourt à boire, aromatisé, sucré

How to identify and add potential Agribalyse proxy categories

Detailed guide in French: Guide d’ajout de correspondances Agribalyse - OFF

Ask Pierre or Stéphane for the Agribalyse list of categories

  • Search for groups of Agribalyse categories that correspond to different versions (e.g. flavoured, sugared etc.) of similar products
  • Check if the environmental impact for the categories are the same or almost the same
  • If the environmental impact is the same, look in the OFF categories taxonomy to see if we have a parent category that matches well the group of Agribalyse taxonomies
  • Add the agribalyse_proxy_food_code:en: property on the parent OFF category

Adding categories or more specific categories to products

If a product does not have a category, we cannot match it to the Agribalyse categories. So it is very important to add categories to products. The more specific the category, the better chance we have that it will match either an Agribalyse category or an Agribalyse proxy category.

Tools to easily add categories to hundreds of products

Tags rules to automatically add more specific categories

(not coded yet)