Difference between revisions of "Ingredients Analysis Quality Evaluation - March 2020 - May 2020"

From Open Food Facts wiki
Jump to navigation Jump to search
Line 318: Line 318:
  
 
<pre>
 
<pre>
 
+
Palm oil content unknown 128788
 +
Vegetarian status unknown 127223
 +
Non-vegan 105765
 +
Vegan status unknown 82754
 +
Palm oil free 66322
 +
Non-vegetarian 42135
 +
Vegetarian 36283
 +
Vegan 27356
 +
Maybe vegetarian 16327
 +
Palm oil 14315
 +
May contain palm oil 12543
 +
Maybe vegan 6093
 
</pre>
 
</pre>
  
Line 324: Line 335:
  
 
<pre>
 
<pre>
 
+
Non-vegan 4629
 +
Vegetarian status unknown 4489
 +
Palm oil free 4277
 +
Palm oil content unknown 3643
 +
Vegan status unknown 2542
 +
Vegetarian 2420
 +
Vegan 1723
 +
Maybe vegetarian 1460
 +
Non-vegetarian 1089
 +
Palm oil 897
 +
May contain palm oil 641
 +
Maybe vegan 564
 
</pre>
 
</pre>
  

Revision as of 15:16, 5 March 2020


Ingredients parsing and recognition metrics

For each product, the ingredients list is parsed to separate each ingredient. Ingredients that we can match to our multilingual ingredients taxonomy as marked as "known", others as "unknown".

There are different reasons an ingredient can be marked as unknown:

  • The input ingredient list is incorrect (mispellings etc.)
  • The ingredients list contains other things than ingredients and we have not been able to split it at the right place.
  • We have not been able to parse a particular sentence structure or formating (e.g. "a drop of delicious honey with a shower of powder sugar")
  • The ingredient is not yet present in our taxonomy (or not with the right synonym or in the right language)

Retrieved on 2020-03-05:

EN - English - UK

All products:

known	1702 (8.77%)	104818 (79.85%)
unknown	17706 (91.23%)	26448 (20.15%)
all	19409 (100.00%)	131266 (100.00%)

Top 10k most scanned products:

known	1012 (20.67%)	19061 (79.38%)
unknown	3884 (79.31%)	4952 (20.62%)
all	4897 (100.00%)	24013 (100.00%)

DE - German - Germany

All products:

known	2139 (5.36%)	363967 (84.75%)
unknown	37790 (94.64%)	65503 (15.25%)
all	39930 (100.00%)	429470 (100.00%)

Top 10k most scanned products:

known	1435 (14.73%)	69066 (85.53%)
unknown	8309 (85.26%)	11689 (14.47%)
all	9745 (100.00%)	80755 (100.00%)

ES - Spanish - Spain

All products:

known	1970 (4.25%)	176101 (71.95%)
unknown	44352 (95.75%)	68659 (28.05%)
all	46323 (100.00%)	244760 (100.00%)

Top 10k most scanned products:

known	1262 (10.47%)	48797 (75.99%)
unknown	10793 (89.52%)	15417 (24.01%)
all	12056 (100.00%)	64214 (100.00%)

FI - Finnish - FI

All products:

known	829 (43.96%)	11069 (89.15%)
unknown	1056 (55.99%)	1347 (10.85%)
all	1886 (100.00%)	12416 (100.00%)

Top 10k most scanned products:

known	566 (61.66%)	3465 (89.07%)
unknown	351 (38.24%)	425 (10.93%)
all	918 (100.00%)	3890 (100.00%)

FR - French - France

All products:

known	3825 (0.85%)	3708820 (83.40%)
unknown	446838 (99.15%)	738291 (16.60%)
all	450664 (100.00%)	4447111 (100.00%)

Top 10k most scanned products:

known	2208 (22.34%)	184218 (94.52%)
unknown	7673 (77.65%)	10686 (5.48%)
all	9882 (100.00%)	194904 (100.00%)

IT - Italian - Italy

All products:

known	1319 (9.43%)	53345 (74.58%)
unknown	12669 (90.56%)	18186 (25.42%)
all	13989 (100.00%)	71531 (100.00%)

Top 10k most scanned products:

known	911 (13.11%)	20838 (72.21%)
unknown	6039 (86.88%)	8018 (27.79%)
all	6951 (100.00%)	28856 (100.00%)

NL - Dutch - The Netherlands

All products:

known	1304 (17.03%)	31491 (78.84%)
unknown	6353 (82.96%)	8453 (21.16%)
all	7658 (100.00%)	39944 (100.00%)

Top 10k most scanned products:

known	755 (24.25%)	8069 (73.71%)
unknown	2358 (75.72%)	2878 (26.29%)
all	3114 (100.00%)	10947 (100.00%)

PT - Portuguese - Portugal

All products:

known	918 (16.05%)	15816 (70.50%)
unknown	4800 (83.93%)	6617 (29.50%)
all	5719 (100.00%)	22433 (100.00%)

Top 10k most scanned products:

known	578 (19.37%)	6261 (68.40%)
unknown	2405 (80.60%)	2892 (31.60%)
all	2984 (100.00%)	9153 (100.00%)



Ingredients analysis

For each product, once we have parsed the ingredients list to separate each ingredient and match them to our taxonomy, we can analyze if the products are vegan, vegetarian, without palm oil etc.

To mark a product as non-vegan etc. we only need to find one non-vegan ingredient. But to mark a product as vegan, we need to be sure that we have been able to correctly recognize all ingredients so that we can verify that all of them are vegan.

Retrieved on 2020-03-05:

EN - English - UK

All products:

Vegetarian status unknown	4814	
Palm oil content unknown	4138	
Vegan status unknown	3281	
Non-vegan	2525	
Palm oil free	1286	
Vegetarian	903	
Palm oil	875	
Vegan	725	
Non-vegetarian	719	
May contain palm oil	314	
Maybe vegetarian	177	
Maybe vegan	82	

Top 10k most scanned products:

Vegetarian status unknown	1016	
Palm oil content unknown	817	
Vegan status unknown	711	
Non-vegan	439	
Palm oil free	222	
Palm oil	192	
Vegetarian	171	
Vegan	139	
Non-vegetarian	81	
May contain palm oil	73	
Maybe vegetarian	36	
Maybe vegan	15	

DE - German - Germany

All products:

Vegetarian status unknown	15605	
Palm oil content unknown	14599	
Vegan status unknown	10377	
Non-vegan	10151	
Palm oil free	6199	
Vegetarian	4394	
Vegan	3367	
Non-vegetarian	2805	
Palm oil	2357	
Maybe vegetarian	1486	
May contain palm oil	1135	
Maybe vegan	395	

Top 10k most scanned products:

Vegetarian status unknown	2925	
Palm oil content unknown	2449	
Non-vegan	1903	
Vegan status unknown	1874	
Palm oil free	1255	
Vegetarian	849	
Vegan	634	
Palm oil	567	
Maybe vegetarian	381	
Non-vegetarian	362	
May contain palm oil	246	
Maybe vegan	106	

ES - Spanish - Spain

All products:

Vegetarian status unknown	12076	
Palm oil content unknown	11192	
Vegan status unknown	9812	
Non-vegan	3877	
Palm oil free	2707	
Vegetarian	1906	
Vegan	1689	
Non-vegetarian	1145	
May contain palm oil	978	
Palm oil	809	
Maybe vegetarian	559	
Maybe vegan	308	

Top 10k most scanned products:

Vegetarian status unknown	2733	
Palm oil content unknown	2387	
Vegan status unknown	2081	
Non-vegan	1105	
Palm oil free	733	
Vegetarian	484	
Vegan	394	
May contain palm oil	308	
Non-vegetarian	272	
Palm oil	267	
Maybe vegetarian	206	
Maybe vegan	115	

FI - Finnish - FI

All products:

Vegetarian status unknown	375	
Palm oil content unknown	312	
Non-vegan	278	
Vegan status unknown	245	
Palm oil free	191	
Maybe vegetarian	94	
Non-vegetarian	91	
Palm oil	85	
Vegetarian	83	
Vegan	76	
May contain palm oil	55	
Maybe vegan	44	

Top 10k most scanned products:

Vegetarian status unknown	101	
Palm oil content unknown	93	
Non-vegan	77	
Vegan status unknown	70	
Palm oil free	59	
Non-vegetarian	31	
Vegetarian	27	
Maybe vegetarian	25	
Vegan	24	
Palm oil	19	
Maybe vegan	13	
May contain palm oil	13	

FR - French - France

All products:

Palm oil content unknown	128788	
Vegetarian status unknown	127223	
Non-vegan	105765	
Vegan status unknown	82754	
Palm oil free	66322	
Non-vegetarian	42135	
Vegetarian	36283	
Vegan	27356	
Maybe vegetarian	16327	
Palm oil	14315	
May contain palm oil	12543	
Maybe vegan	6093	

Top 10k most scanned products:

Non-vegan	4629	
Vegetarian status unknown	4489	
Palm oil free	4277	
Palm oil content unknown	3643	
Vegan status unknown	2542	
Vegetarian	2420	
Vegan	1723	
Maybe vegetarian	1460	
Non-vegetarian	1089	
Palm oil	897	
May contain palm oil	641	
Maybe vegan	564	

IT - Italian - Italy

All products:


Top 10k most scanned products:


NL - Dutch - The Netherlands

All products:


Top 10k most scanned products:


PT - Portuguese - Portugal

All products:


Top 10k most scanned products: