Ingredients Analysis Quality Evaluation - March 2020 - May 2020

From Open Food Facts wiki
Revision as of 15:14, 5 March 2020 by Stephane (talk | contribs)
Jump to navigation Jump to search


Ingredients parsing and recognition metrics

For each product, the ingredients list is parsed to separate each ingredient. Ingredients that we can match to our multilingual ingredients taxonomy as marked as "known", others as "unknown".

There are different reasons an ingredient can be marked as unknown:

  • The input ingredient list is incorrect (mispellings etc.)
  • The ingredients list contains other things than ingredients and we have not been able to split it at the right place.
  • We have not been able to parse a particular sentence structure or formating (e.g. "a drop of delicious honey with a shower of powder sugar")
  • The ingredient is not yet present in our taxonomy (or not with the right synonym or in the right language)

Retrieved on 2020-03-05:

EN - English - UK

All products:

known	1702 (8.77%)	104818 (79.85%)
unknown	17706 (91.23%)	26448 (20.15%)
all	19409 (100.00%)	131266 (100.00%)

Top 10k most scanned products:

known	1012 (20.67%)	19061 (79.38%)
unknown	3884 (79.31%)	4952 (20.62%)
all	4897 (100.00%)	24013 (100.00%)

DE - German - Germany

All products:

known	2139 (5.36%)	363967 (84.75%)
unknown	37790 (94.64%)	65503 (15.25%)
all	39930 (100.00%)	429470 (100.00%)

Top 10k most scanned products:

known	1435 (14.73%)	69066 (85.53%)
unknown	8309 (85.26%)	11689 (14.47%)
all	9745 (100.00%)	80755 (100.00%)

ES - Spanish - Spain

All products:

known	1970 (4.25%)	176101 (71.95%)
unknown	44352 (95.75%)	68659 (28.05%)
all	46323 (100.00%)	244760 (100.00%)

Top 10k most scanned products:

known	1262 (10.47%)	48797 (75.99%)
unknown	10793 (89.52%)	15417 (24.01%)
all	12056 (100.00%)	64214 (100.00%)

FI - Finnish - FI

All products:

known	829 (43.96%)	11069 (89.15%)
unknown	1056 (55.99%)	1347 (10.85%)
all	1886 (100.00%)	12416 (100.00%)

Top 10k most scanned products:

known	566 (61.66%)	3465 (89.07%)
unknown	351 (38.24%)	425 (10.93%)
all	918 (100.00%)	3890 (100.00%)

FR - French - France

All products:

known	3825 (0.85%)	3708820 (83.40%)
unknown	446838 (99.15%)	738291 (16.60%)
all	450664 (100.00%)	4447111 (100.00%)

Top 10k most scanned products:

known	2208 (22.34%)	184218 (94.52%)
unknown	7673 (77.65%)	10686 (5.48%)
all	9882 (100.00%)	194904 (100.00%)

IT - Italian - Italy

All products:

known	1319 (9.43%)	53345 (74.58%)
unknown	12669 (90.56%)	18186 (25.42%)
all	13989 (100.00%)	71531 (100.00%)

Top 10k most scanned products:

known	911 (13.11%)	20838 (72.21%)
unknown	6039 (86.88%)	8018 (27.79%)
all	6951 (100.00%)	28856 (100.00%)

NL - Dutch - The Netherlands

All products:

known	1304 (17.03%)	31491 (78.84%)
unknown	6353 (82.96%)	8453 (21.16%)
all	7658 (100.00%)	39944 (100.00%)

Top 10k most scanned products:

known	755 (24.25%)	8069 (73.71%)
unknown	2358 (75.72%)	2878 (26.29%)
all	3114 (100.00%)	10947 (100.00%)

PT - Portuguese - Portugal

All products:

known	918 (16.05%)	15816 (70.50%)
unknown	4800 (83.93%)	6617 (29.50%)
all	5719 (100.00%)	22433 (100.00%)

Top 10k most scanned products:

known	578 (19.37%)	6261 (68.40%)
unknown	2405 (80.60%)	2892 (31.60%)
all	2984 (100.00%)	9153 (100.00%)



Ingredients analysis

For each product, once we have parsed the ingredients list to separate each ingredient and match them to our taxonomy, we can analyze if the products are vegan, vegetarian, without palm oil etc.

To mark a product as non-vegan etc. we only need to find one non-vegan ingredient. But to mark a product as vegan, we need to be sure that we have been able to correctly recognize all ingredients so that we can verify that all of them are vegan.

Retrieved on 2020-03-05:

EN - English - UK

All products:

Vegetarian status unknown	4814	
Palm oil content unknown	4138	
Vegan status unknown	3281	
Non-vegan	2525	
Palm oil free	1286	
Vegetarian	903	
Palm oil	875	
Vegan	725	
Non-vegetarian	719	
May contain palm oil	314	
Maybe vegetarian	177	
Maybe vegan	82	

Top 10k most scanned products:

Vegetarian status unknown	1016	
Palm oil content unknown	817	
Vegan status unknown	711	
Non-vegan	439	
Palm oil free	222	
Palm oil	192	
Vegetarian	171	
Vegan	139	
Non-vegetarian	81	
May contain palm oil	73	
Maybe vegetarian	36	
Maybe vegan	15	

DE - German - Germany

All products:

Vegetarian status unknown	15605	
Palm oil content unknown	14599	
Vegan status unknown	10377	
Non-vegan	10151	
Palm oil free	6199	
Vegetarian	4394	
Vegan	3367	
Non-vegetarian	2805	
Palm oil	2357	
Maybe vegetarian	1486	
May contain palm oil	1135	
Maybe vegan	395	

Top 10k most scanned products:

Vegetarian status unknown	2925	
Palm oil content unknown	2449	
Non-vegan	1903	
Vegan status unknown	1874	
Palm oil free	1255	
Vegetarian	849	
Vegan	634	
Palm oil	567	
Maybe vegetarian	381	
Non-vegetarian	362	
May contain palm oil	246	
Maybe vegan	106	

ES - Spanish - Spain

All products:


Top 10k most scanned products:


FI - Finnish - FI

All products:


Top 10k most scanned products:


FR - French - France

All products:


Top 10k most scanned products:


IT - Italian - Italy

All products:


Top 10k most scanned products:


NL - Dutch - The Netherlands

All products:


Top 10k most scanned products:


PT - Portuguese - Portugal

All products:


Top 10k most scanned products: