Difference between revisions of "Ingredients Analysis Quality"

From Open Food Facts wiki
Jump to navigation Jump to search
Tag: Removed redirect
 
(4 intermediate revisions by 3 users not shown)
Line 1: Line 1:
  
This page explains how we measure and monitor the quality of ingredients analysis.
+
This page explains how we measure and monitor the quality of [[Ingredients Extraction and Analysis]].
 +
[[Category:Metrics]]
 +
== Metrics for ingredients analysis ==
  
= Metrics for ingredients analysis =
+
=== Ingredients parsing and recognition metrics ===
 
 
== Ingredients parsing and recognition metrics ==
 
  
 
For each product, the ingredients list is parsed to separate each ingredient. Ingredients that we can match to our multilingual ingredients taxonomy as marked as "known", others as "unknown".
 
For each product, the ingredients list is parsed to separate each ingredient. Ingredients that we can match to our multilingual ingredients taxonomy as marked as "known", others as "unknown".
Line 15: Line 15:
 
* The ingredient is not yet present in our taxonomy (or not with the right synonym or in the right language)
 
* The ingredient is not yet present in our taxonomy (or not with the right synonym or in the right language)
  
=== Percentage of known ingredients ===
+
==== Percentage of known ingredients ====
  
 
The metric we use to assess the quality of ingredients parsing and recognition is the % of known ingredients over all ingredients.
 
The metric we use to assess the quality of ingredients parsing and recognition is the % of known ingredients over all ingredients.
Line 27: Line 27:
 
*** This is the most representative metric for ingredients parsing and recognition
 
*** This is the most representative metric for ingredients parsing and recognition
  
=== How to see the percentage of known ingredients ===
+
==== How to see the percentage of known ingredients ====
  
 
For one specific product, on the product page on the Open Food Facts web site, you can click on "Details of the ingredients analysis" to see how many ingredients have been recognized.
 
For one specific product, on the product page on the Open Food Facts web site, you can click on "Details of the ingredients analysis" to see how many ingredients have been recognized.
Line 48: Line 48:
  
  
== Ingredients analysis at the product level ==
+
=== Ingredients analysis at the product level ===
  
 
For each product, once we have parsed the ingredients list to separate each ingredient and match them to our taxonomy, we can analyze if the products are vegan, vegetarian, without palm oil etc.
 
For each product, once we have parsed the ingredients list to separate each ingredient and match them to our taxonomy, we can analyze if the products are vegan, vegetarian, without palm oil etc.
Line 54: Line 54:
 
To mark a product as non-vegan etc. we only need to find one non-vegan ingredient. But to mark a product as vegan, we need to be sure that we have been able to correctly recognize all ingredients so that we can verify that all of them are vegan.
 
To mark a product as non-vegan etc. we only need to find one non-vegan ingredient. But to mark a product as vegan, we need to be sure that we have been able to correctly recognize all ingredients so that we can verify that all of them are vegan.
  
=== Percentage of known values for each property ===
+
==== Percentage of known values for each property ====
  
 
For a property (e.g. is the product vegetarian, vegan or palm oil free), we can have different values, for instance for the vegetarian property:
 
For a property (e.g. is the product vegetarian, vegan or palm oil free), we can have different values, for instance for the vegetarian property:
Line 66: Line 66:
 
The metric we use is the percentage of known values over all values.
 
The metric we use is the percentage of known values over all values.
  
=== How to see the percentage of known values ===
+
==== How to see the percentage of known values ====
  
 
For one specific product, the product page shows the results of the ingredients analysis with icons for vegetarian products, vegan products and palm oil free products (red, orange or green depending on the value). If the property has an unknown value, the icon is not displayed.
 
For one specific product, the product page shows the results of the ingredients analysis with icons for vegetarian products, vegan products and palm oil free products (red, orange or green depending on the value). If the property has an unknown value, the icon is not displayed.
Line 88: Line 88:
 
</pre>
 
</pre>
  
= Ingredients Analysis Quality Evaluation - March 2020 =
+
== Ingredients Analysis Quality Evaluation - March 2020 ==
  
 
An initial evaluation of the quality of ingredients analysis for major European languages was done in March 2020 as part of the first task [[Ingredients analysis and search features extraction]] of the [[Project:Personalized_Search]]
 
An initial evaluation of the quality of ingredients analysis for major European languages was done in March 2020 as part of the first task [[Ingredients analysis and search features extraction]] of the [[Project:Personalized_Search]]
Line 95: Line 95:
  
 
[[Category:Project:Personalized_Search]]
 
[[Category:Project:Personalized_Search]]
 +
[[Category:Data quality]]
 +
[[Category:Ingredients]]

Latest revision as of 10:44, 7 August 2024

This page explains how we measure and monitor the quality of Ingredients Extraction and Analysis.

Metrics for ingredients analysis

Ingredients parsing and recognition metrics

For each product, the ingredients list is parsed to separate each ingredient. Ingredients that we can match to our multilingual ingredients taxonomy as marked as "known", others as "unknown".

There are different reasons an ingredient can be marked as unknown:

  • The input ingredient list is incorrect (mispellings etc.)
  • The ingredients list contains other things than ingredients and we have not been able to split it at the right place.
  • We have not been able to parse a particular sentence structure or formating (e.g. "a drop of delicious honey with a shower of powder sugar")
  • The ingredient is not yet present in our taxonomy (or not with the right synonym or in the right language)

Percentage of known ingredients

The metric we use to assess the quality of ingredients parsing and recognition is the % of known ingredients over all ingredients.

The percentage of known ingredients can be evaluated:

  • For one specific product.
  • For all products in the database or in a subset of the database (e.g. all products sold in one country)
    • For unique ingredients accross all products
    • Weighted by the number of occurences of each ingredient. e.g. if a specific ingredient appears in 5 products, it is counted 5 times.
      • This is the most representative metric for ingredients parsing and recognition

How to see the percentage of known ingredients

For one specific product, on the product page on the Open Food Facts web site, you can click on "Details of the ingredients analysis" to see how many ingredients have been recognized.

For a given set of products returned on the Open Food Facts web site (e.g. https://us.openfoodfacts.org/ for products sold in the United States), you can add /ingredients?stats=1 to get the percentage of known ingredients for this set of products (e.g. https://us.openfoodfacts.org/ingredients?stats=1 or https://us.openfoodfacts.org/popularity/top-10000-us-scans-2019/ingredients?stats=1 ).

Type Unique tags Occurrences
known 2217 (1.78%) 6451232 (84.62%)
unknown 122064 (98.22%) 1172697 (15.38%)
all 124281 (100.00%) 7623929 (100.00%)

In the table above, the first column of numbers corresponds to the number of unique ingredients accross all products. And the second column of number corresponds to the number of occurrences of those ingredients.


Ingredients analysis at the product level

For each product, once we have parsed the ingredients list to separate each ingredient and match them to our taxonomy, we can analyze if the products are vegan, vegetarian, without palm oil etc.

To mark a product as non-vegan etc. we only need to find one non-vegan ingredient. But to mark a product as vegan, we need to be sure that we have been able to correctly recognize all ingredients so that we can verify that all of them are vegan.

Percentage of known values for each property

For a property (e.g. is the product vegetarian, vegan or palm oil free), we can have different values, for instance for the vegetarian property:

  • Unknown value: we have not been able to make a determination
  • Known values
    • Yes: the product is vegetarian (we have recognized all ingredients, and we know all of them are vegetarian)
    • No: the product is not vegetarian (at least one ingredient that we recognized is not vegetarian)
    • Maybe: the product may be vegetarian (we have recognized all ingredients, but at least one of them can be vegetarian or not)

The metric we use is the percentage of known values over all values.

How to see the percentage of known values

For one specific product, the product page shows the results of the ingredients analysis with icons for vegetarian products, vegan products and palm oil free products (red, orange or green depending on the value). If the property has an unknown value, the icon is not displayed.

For a given set of products returned on the Open Food Facts web site, you can add /ingredients-analysis to the URL (e.g. https://us.openfoodfacts.org/ingredients-analysis)

Ingredients analysis	Products	*
Vegetarian status unknown	230236	
Palm oil content unknown	208375	
Vegan status unknown	142640	
Non-vegan	134162	
Palm oil free	52268	
Non-vegetarian	39834	
Vegetarian	36098	
Palm oil	34461	
Vegan	32785	
May contain palm oil	21314	
Maybe vegetarian	10442	
Maybe vegan	7020		

Ingredients Analysis Quality Evaluation - March 2020

An initial evaluation of the quality of ingredients analysis for major European languages was done in March 2020 as part of the first task Ingredients analysis and search features extraction of the Project:Personalized_Search

Ingredients Analysis Quality Evaluation - March 2020