Difference between revisions of "Ingredients Analysis Quality Evaluation - March 2020 - May 2020"

From Open Food Facts wiki
Jump to navigation Jump to search
Line 171: Line 171:
 
=== EN - English - UK ===
 
=== EN - English - UK ===
  
[https://uk.openfoodfacts.org/ingredients?stats=1 All products:]
+
[https://uk.openfoodfacts.org/ingredients-analysis All products:]
  
 
<pre>
 
<pre>
Line 177: Line 177:
 
</pre>
 
</pre>
  
[https://uk.openfoodfacts.org/popularity/top-10000-gb-scans-2019/ingredients?stats=1 Top 10k most scanned products:]
+
[https://uk.openfoodfacts.org/popularity/top-10000-gb-scans-2019/ingredients-analysis Top 10k most scanned products:]
  
 
<pre>
 
<pre>
Line 185: Line 185:
 
=== DE - German - Germany ===
 
=== DE - German - Germany ===
  
[https://de.openfoodfacts.org/ingredients?stats=1 All products:]
+
[https://de.openfoodfacts.org/ingredients-analysis All products:]
  
 
<pre>
 
<pre>
Line 191: Line 191:
 
</pre>
 
</pre>
  
[https://de.openfoodfacts.org/popularity/top-10000-de-scans-2019/ingredients?stats=1 Top 10k most scanned products:]
+
[https://de.openfoodfacts.org/popularity/top-10000-de-scans-2019/ingredients-analysis Top 10k most scanned products:]
  
 
<pre>
 
<pre>
Line 199: Line 199:
 
=== ES - Spanish - Spain ===
 
=== ES - Spanish - Spain ===
  
[https://es.openfoodfacts.org/ingredients?stats=1 All products:]
+
[https://es.openfoodfacts.org/ingredients-analysis All products:]
  
 
<pre>
 
<pre>
Line 205: Line 205:
 
</pre>
 
</pre>
  
[https://es.openfoodfacts.org/popularity/top-10000-es-scans-2019/ingredients?stats=1 Top 10k most scanned products:]
+
[https://es.openfoodfacts.org/popularity/top-10000-es-scans-2019/ingredients-analysis Top 10k most scanned products:]
  
 
<pre>
 
<pre>
Line 213: Line 213:
 
=== FI - Finnish - FI ===
 
=== FI - Finnish - FI ===
  
[https://fi.openfoodfacts.org/ingredients?stats=1 All products:]
+
[https://fi.openfoodfacts.org/ingredients-analysis All products:]
  
 
<pre>
 
<pre>
Line 219: Line 219:
 
</pre>
 
</pre>
  
[https://fi.openfoodfacts.org/popularity/top-10000-fi-scans-2019/ingredients?stats=1 Top 10k most scanned products:]
+
[https://fi.openfoodfacts.org/popularity/top-10000-fi-scans-2019/ingredients-analysis Top 10k most scanned products:]
  
 
<pre>
 
<pre>
Line 227: Line 227:
 
=== FR - French - France ===
 
=== FR - French - France ===
  
[https://fr.openfoodfacts.org/ingredients?stats=1 All products:]
+
[https://fr.openfoodfacts.org/ingredients-analysis All products:]
  
 
<pre>
 
<pre>
Line 233: Line 233:
 
</pre>
 
</pre>
  
[https://fr.openfoodfacts.org/popularity/top-10000-fr-scans-2019/ingredients?stats=1 Top 10k most scanned products:]
+
[https://fr.openfoodfacts.org/popularity/top-10000-fr-scans-2019/ingredients-analysis Top 10k most scanned products:]
  
 
<pre>
 
<pre>
Line 241: Line 241:
 
=== IT - Italian - Italy ===
 
=== IT - Italian - Italy ===
  
[https://it.openfoodfacts.org/ingredients?stats=1 All products:]
+
[https://it.openfoodfacts.org/ingredients-analysis All products:]
  
 
<pre>
 
<pre>
Line 247: Line 247:
 
</pre>
 
</pre>
  
[https://it.openfoodfacts.org/popularity/top-10000-it-scans-2019/ingredients?stats=1 Top 10k most scanned products:]
+
[https://it.openfoodfacts.org/popularity/top-10000-it-scans-2019/ingredients-analysis Top 10k most scanned products:]
  
 
<pre>
 
<pre>
Line 255: Line 255:
 
=== NL - Dutch - The Netherlands ===
 
=== NL - Dutch - The Netherlands ===
  
[https://nl.openfoodfacts.org/ingredients?stats=1 All products:]
+
[https://nl.openfoodfacts.org/ingredients-analysis All products:]
  
 
<pre>
 
<pre>
Line 261: Line 261:
 
</pre>
 
</pre>
  
[https://nl.openfoodfacts.org/popularity/top-10000-nl-scans-2019/ingredients?stats=1 Top 10k most scanned products:]
+
[https://nl.openfoodfacts.org/popularity/top-10000-nl-scans-2019/ingredients-analysis Top 10k most scanned products:]
  
 
<pre>
 
<pre>
Line 269: Line 269:
 
=== PT - Portuguese - Portugal ===
 
=== PT - Portuguese - Portugal ===
  
[https://pt.openfoodfacts.org/ingredients?stats=1 All products:]
+
[https://pt.openfoodfacts.org/ingredients-analysis All products:]
  
 
<pre>
 
<pre>
Line 275: Line 275:
 
</pre>
 
</pre>
  
[https://pt.openfoodfacts.org/popularity/top-10000-pt-scans-2019/ingredients?stats=1 Top 10k most scanned products:]
+
[https://pt.openfoodfacts.org/popularity/top-10000-pt-scans-2019/ingredients-analysis Top 10k most scanned products:]
  
 
<pre>
 
<pre>
  
 
</pre>
 
</pre>

Revision as of 15:08, 5 March 2020


Ingredients parsing and recognition metrics

For each product, the ingredients list is parsed to separate each ingredient. Ingredients that we can match to our multilingual ingredients taxonomy as marked as "known", others as "unknown".

There are different reasons an ingredient can be marked as unknown:

  • The input ingredient list is incorrect (mispellings etc.)
  • The ingredients list contains other things than ingredients and we have not been able to split it at the right place.
  • We have not been able to parse a particular sentence structure or formating (e.g. "a drop of delicious honey with a shower of powder sugar")
  • The ingredient is not yet present in our taxonomy (or not with the right synonym or in the right language)

Retrieved on 2020-03-05:

EN - English - UK

All products:

known	1702 (8.77%)	104818 (79.85%)
unknown	17706 (91.23%)	26448 (20.15%)
all	19409 (100.00%)	131266 (100.00%)

Top 10k most scanned products:

known	1012 (20.67%)	19061 (79.38%)
unknown	3884 (79.31%)	4952 (20.62%)
all	4897 (100.00%)	24013 (100.00%)

DE - German - Germany

All products:

known	2139 (5.36%)	363967 (84.75%)
unknown	37790 (94.64%)	65503 (15.25%)
all	39930 (100.00%)	429470 (100.00%)

Top 10k most scanned products:

known	1435 (14.73%)	69066 (85.53%)
unknown	8309 (85.26%)	11689 (14.47%)
all	9745 (100.00%)	80755 (100.00%)

ES - Spanish - Spain

All products:

known	1970 (4.25%)	176101 (71.95%)
unknown	44352 (95.75%)	68659 (28.05%)
all	46323 (100.00%)	244760 (100.00%)

Top 10k most scanned products:

known	1262 (10.47%)	48797 (75.99%)
unknown	10793 (89.52%)	15417 (24.01%)
all	12056 (100.00%)	64214 (100.00%)

FI - Finnish - FI

All products:

known	829 (43.96%)	11069 (89.15%)
unknown	1056 (55.99%)	1347 (10.85%)
all	1886 (100.00%)	12416 (100.00%)

Top 10k most scanned products:

known	566 (61.66%)	3465 (89.07%)
unknown	351 (38.24%)	425 (10.93%)
all	918 (100.00%)	3890 (100.00%)

FR - French - France

All products:

known	3825 (0.85%)	3708820 (83.40%)
unknown	446838 (99.15%)	738291 (16.60%)
all	450664 (100.00%)	4447111 (100.00%)

Top 10k most scanned products:

known	2208 (22.34%)	184218 (94.52%)
unknown	7673 (77.65%)	10686 (5.48%)
all	9882 (100.00%)	194904 (100.00%)

IT - Italian - Italy

All products:

known	1319 (9.43%)	53345 (74.58%)
unknown	12669 (90.56%)	18186 (25.42%)
all	13989 (100.00%)	71531 (100.00%)

Top 10k most scanned products:

known	911 (13.11%)	20838 (72.21%)
unknown	6039 (86.88%)	8018 (27.79%)
all	6951 (100.00%)	28856 (100.00%)

NL - Dutch - The Netherlands

All products:

known	1304 (17.03%)	31491 (78.84%)
unknown	6353 (82.96%)	8453 (21.16%)
all	7658 (100.00%)	39944 (100.00%)

Top 10k most scanned products:

known	755 (24.25%)	8069 (73.71%)
unknown	2358 (75.72%)	2878 (26.29%)
all	3114 (100.00%)	10947 (100.00%)

PT - Portuguese - Portugal

All products:

known	918 (16.05%)	15816 (70.50%)
unknown	4800 (83.93%)	6617 (29.50%)
all	5719 (100.00%)	22433 (100.00%)

Top 10k most scanned products:

known	578 (19.37%)	6261 (68.40%)
unknown	2405 (80.60%)	2892 (31.60%)
all	2984 (100.00%)	9153 (100.00%)



Ingredients analysis

For each product, once we have parsed the ingredients list to separate each ingredient and match them to our taxonomy, we can analyze if the products are vegan, vegetarian, without palm oil etc.

To mark a product as non-vegan etc. we only need to find one non-vegan ingredient. But to mark a product as vegan, we need to be sure that we have been able to correctly recognize all ingredients so that we can verify that all of them are vegan.

Retrieved on 2020-03-05:

EN - English - UK

All products:


Top 10k most scanned products:


DE - German - Germany

All products:


Top 10k most scanned products:


ES - Spanish - Spain

All products:


Top 10k most scanned products:


FI - Finnish - FI

All products:


Top 10k most scanned products:


FR - French - France

All products:


Top 10k most scanned products:


IT - Italian - Italy

All products:


Top 10k most scanned products:


NL - Dutch - The Netherlands

All products:


Top 10k most scanned products:


PT - Portuguese - Portugal

All products:


Top 10k most scanned products: