|
|
| (One intermediate revision by one other user not shown) |
| Line 1: |
Line 1: |
| | Evaluation of the quality of [[Ingredients Extraction and Analysis]] so that we can measure the improvements. | | Evaluation of the quality of [[Ingredients Extraction and Analysis]] so that we can measure the improvements. |
| − | | + | [[Category:Metrics]] |
| | == Ingredients parsing and recognition == | | == Ingredients parsing and recognition == |
| | | | |
| Line 417: |
Line 417: |
| | [[Category:Project:Personalized_Search]] | | [[Category:Project:Personalized_Search]] |
| | [[Category:Data quality]] | | [[Category:Data quality]] |
| | + | [[Category:Ingredients]] |
Latest revision as of 10:44, 7 August 2024
Evaluation of the quality of Ingredients Extraction and Analysis so that we can measure the improvements.
Ingredients parsing and recognition
For each product, the ingredients list is parsed to separate each ingredient. Ingredients that we can match to our multilingual ingredients taxonomy as marked as "known", others as "unknown".
There are different reasons an ingredient can be marked as unknown:
- The input ingredient list is incorrect (misspellings etc.)
- The ingredients list contains other things than ingredients and we have not been able to split it at the right place.
- We have not been able to parse a particular sentence structure or formatting (e.g. "a drop of delicious honey with a shower of powder sugar")
- The ingredient is not yet present in our taxonomy (or not with the right synonym or in the right language)
In the table below,
- the first column of numbers corresponds to the number of unique ingredients across all products.
- And the second column of number corresponds to the number of occurrences of those ingredients. e.g. if a specific ingredient appears in 5 products, it is counted 5 times.
EN - English - India
|
2022-06-23 |
2022-06-27
|
| All products: |
known 776 (23.07%) 8444 (69.62%)
unknown 2588 (76.93%) 3685 (30.38%)
all 3364 (100.00%) 12129 (100.00%)
|
known 780 (23.10%) 9062 (71.49%)
unknown 2596 (76.90%) 3614 (28.51%)
all 3376 (100.00%) 12676 (100.00%)
|
| Top 10k most scanned products: |
known 407 (35.83%) 2149 (69.23%)
unknown 729 (64.17%) 955 (30.77%)
all 1136 (100.00%) 3104 (100.00%)
|
known 407 (36.02%) 2284 (71.38%)
unknown 723 (63.98%) 916 (28.62%)
all 1130 (100.00%) 3200 (100.00%)
|
EN - English - UK
|
2022-06-23 |
|
| All products: |
known 2591 (6.22%) 406797 (88.32%)
unknown 39034 (93.78%) 53807 (11.68%)
all 41625 (100.00%) 460604 (100.00%)
|
|
| Top 10k most scanned products: |
known 1664 (21.69%) 77617 (91.14%)
unknown 6008 (78.31%) 7546 (8.86%)
all 7672 (100.00%) 85163 (100.00%)
|
|
EN - English - US
|
2022-06-23 |
|
| All products: |
known 3027 (1.77%) 8121359 (91.22%)
unknown 167758 (98.23%) 781910 (8.78%)
all 170785 (100.00%) 8903269 (100.00%)
|
|
| Top 10k most scanned products: |
known 1619 (18.24%) 97755 (88.84%)
unknown 7255 (81.76%) 12283 (11.16%)
all 8874 (100.00%) 110038 (100.00%)
|
|
FR - French
|
2022-07-13 |
|
| All products: |
known 4440 (1.05%) 5069577 (88.03%)
unknown 419374 (98.95%) 689143 (11.97%)
all 423814 (100.00%) 5758720 (100.00%)
|
|
| Top 10k most scanned products: |
known 2436 (21.84%) 210027 (94.37%)
unknown 8718 (78.16%) 12527 (5.63%)
all 11154 (100.00%) 222554 (100.00%)
|
|
HR - Croatia
|
2022-09-20 |
|
| All products: |
known 680 (8.64%) 8476 (46.93%)
unknown 7187 (91.36%) 9584 (53.07%)
all 7867 (100.00%) 18060 (100.00%)
|
|
| Top 10k most scanned products: |
known 369 (12.05%) 2394 (42.61%)
unknown 2693 (87.95%) 3224 (57.39%)
all 3062 (100.00%) 5618 (100.00%)
|
|
PL - Polish - PN
|
2022-07-08 |
|
| All products: |
known 1215 (11.38%) 68425 (78.48%)
unknown 9457 (88.62%) 18758 (21.52%)
all 10672 (100.00%) 87183 (100.00%)
|
|
| Top 10k most scanned products: |
known 742 (16.75%) 21685 (77.06%)
unknown 3689 (83.25%) 6455 (22.94%)
all 4431 (100.00%) 28140 (100.00%)
|
|
Ingredients analysis
For each product, once we have parsed the ingredients list to separate each ingredient and match them to our taxonomy, we can analyze if the products are vegan, vegetarian, without palm oil etc.
To mark a product as non-vegan etc. we only need to find one non-vegan ingredient. But to mark a product as vegan, we need to be sure that we have been able to correctly recognize all ingredients so that we can verify that all of them are vegan.
EN - English - IN
|
2022-06-23 |
|
| All products: |
Vegetarian status unknown 477
Vegan status unknown 389
Palm oil content unknown 370
Non-vegan 184
Palm oil free 160
Vegetarian 160
Vegan 80
Palm oil 74
May contain palm oil 47
Non-vegetarian 20
Maybe vegan 8
Maybe vegetarian 7
|
|
| Top 10k most scanned products:
|
Vegetarian status unknown 127
Vegan status unknown 100
Palm oil content unknown 93
Non-vegan 43
Palm oil free 37
Vegetarian 27
Palm oil 22
Vegan 13
May contain palm oil 5
Non-vegetarian 3
Maybe vegan 2
Maybe vegetarian 2
|
|
EN - English - UK
|
2022-06-23 |
|
| All products: |
Vegetarian status unknown 8631
Palm oil content unknown 8379
Non-vegan 7699
Vegetarian 7295
Vegan status unknown 6193
Palm oil free 5730
Vegan 5023
Non-vegetarian 2756
Palm oil 2663
May contain palm oil 1651
Maybe vegetarian 912
Maybe vegan 559
|
|
| Top 10k most scanned products:
|
Vegetarian 1742
Palm oil content unknown 1474
Vegetarian status unknown 1413
Palm oil free 1406
Non-vegan 1359
Vegan 1139
Vegan status unknown 1121
Palm oil 511
Non-vegetarian 434
May contain palm oil 270
Maybe vegetarian 159
Maybe vegan 112
|
|
EN - English - US
|
2022-06-23 |
|
| All products: |
Vegetarian status unknown 212874
Palm oil content unknown 176193
Non-vegan 146466
Vegan status unknown 126232
Palm oil free 85546
Vegetarian 51118
Non-vegetarian 45192
Vegan 44944
Palm oil 37920
May contain palm oil 30916
Maybe vegetarian 21744
Maybe vegan 13281
|
|
| Top 10k most scanned products:
|
Vegetarian status unknown 2914
Palm oil content unknown 2486
Vegan status unknown 1954
Non-vegan 1552
Palm oil free 1226
Vegetarian 944
Vegan 847
Palm oil 405
Non-vegetarian 391
May contain palm oil 360
Maybe vegetarian 248
Maybe vegan 143
|
|
HR - Croatia
|
2022-09-20 |
|
| All products: |
en:Vegetarian status unknown 706
en:Palm oil content unknown 656
en:Vegan status unknown 556
en:Non-vegan 206
en:Vegetarian 189
en:Vegan 165
en:Palm oil free 152
en:May contain palm oil 50
en:Palm oil 50
en:Non-vegetarian 29
en:Maybe vegetarian 8
en:Maybe vegan 5
|
|
| Top 10k most scanned products:
|
en:Vegetarian status unknown 267
en:Palm oil content unknown 248
en:Vegan status unknown 215
en:Non-vegan 65
en:Vegetarian 50
en:Palm oil free 45
en:Vegan 43
en:May contain palm oil 18
en:Palm oil 13
en:Non-vegetarian 6
en:Maybe vegetarian 2
en:Maybe vegan 2
|
|
PL - Polish - PL
|
2022-07-08 |
|
| All products: |
en:Vegetarian status unknown 3442
en:Palm oil content unknown 2571
en:Vegan status unknown 2297
en:Non-vegan 1756
Bez oleju palmowego 1336
en:Vegetarian 699
en:Vegan 581
en:May contain palm oil 435
en:Non-vegetarian 413
Olej palmowy 311
en:Maybe vegetarian 137
en:Maybe vegan 51
|
|
| Top 10k most scanned products:
|
en:Vegetarian status unknown 1382
en:Palm oil content unknown 1032
en:Vegan status unknown 964
en:Non-vegan 599
Bez oleju palmowego 491
en:Vegetarian 244
en:Vegan 187
en:May contain palm oil 142
en:Non-vegetarian 109
Olej palmowy 92
en:Maybe vegetarian 32
en:Maybe vegan 16
|
|
Observations