1,083
edits
Changes
Jump to navigation
Jump to search
Ingredients Analysis Quality Evaluation - August 2023 (view source)
Revision as of 07:19, 31 July 2023
, 07:19, 31 July 2023Created page with "Evaluation of the quality of Ingredients Extraction and Analysis so that we can measure the improvements. == Ingredients parsing and recognition == For each product, the..."
Evaluation of the quality of [[Ingredients Extraction and Analysis]] so that we can measure the improvements.
== Ingredients parsing and recognition ==
For each product, the ingredients list is parsed to separate each ingredient. Ingredients that we can match to our multilingual ingredients taxonomy as marked as "known", others as "unknown".
There are different reasons an ingredient can be marked as unknown:
* The input ingredient list is incorrect (misspellings etc.)
* The ingredients list contains other things than ingredients and we have not been able to split it at the right place.
* We have not been able to parse a particular sentence structure or formatting (e.g. "a drop of delicious honey with a shower of powder sugar")
* The ingredient is not yet present in our taxonomy (or not with the right synonym or in the right language)
In the table below,
* the first column of numbers corresponds to the '''number of unique ingredients''' across all products.
* And the second column of number corresponds to the '''number of occurrences''' of those ingredients. e.g. if a specific ingredient appears in 5 products, it is counted 5 times.
=== JP - Japanese ===
{| class="wikitable"
|-
! !! 2023-07-30 || 2022-07-31
|-
| [https://jp.openfoodfacts.org/ingredients?stats=1&no_cache=1 All products:] ||
<pre>
2822 原材料:
Type Unique tags Occurrences
known 726 (25.73%) 7191 (69.84%)
unknown 2096 (74.27%) 3105 (30.16%)
all 2822 (100.00%) 10296 (100.00%)
</pre>
||
<pre>
</pre>
|-
| [https://jp.openfoodfacts.org/popularity/top-10000-in-scans-2022/ingredients?stats=1&no_cache=1 Top 10k most scanned products:] ||
<pre>
16 原材料:
Type Unique tags Occurrences
known 15 (93.75%) 15 (93.75%)
unknown 1 (6.25%) 1 (6.25%)
all 16 (100.00%) 16 (100.00%)
</pre>
||
<pre>
</pre>
|}
=== HR - Croatian ===
{| class="wikitable"
|-
! !! 2023-07-30 || 2022-07-31
|-
| [https://hr.openfoodfacts.org/ingredients?stats=1&no_cache=1 All products:] ||
<pre>
5700 sastojci:
Type Unique tags Occurrences
known 1304 (22.88%) 25105 (83.02%)
unknown 4396 (77.12%) 5134 (16.98%)
all 5700 (100.00%) 30239 (100.00%)
</pre>
||
<pre>
</pre>
|-
| [https://hr.openfoodfacts.org/popularity/top-10000-in-scans-2022//ingredients?stats=1&no_cache=1 Top 10k most scanned products:] ||
<pre>
45 sastojci:
Type Unique tags Occurrences
known 36 (80.00%) 46 (83.64%)
unknown 9 (20.00%) 9 (16.36%)
all 45 (100.00%) 55 (100.00%)
</pre>
||
|}
=== Observations ===
[[Category:Project:Personalized_Search]]
[[Category:Data quality]]
== Ingredients parsing and recognition ==
For each product, the ingredients list is parsed to separate each ingredient. Ingredients that we can match to our multilingual ingredients taxonomy as marked as "known", others as "unknown".
There are different reasons an ingredient can be marked as unknown:
* The input ingredient list is incorrect (misspellings etc.)
* The ingredients list contains other things than ingredients and we have not been able to split it at the right place.
* We have not been able to parse a particular sentence structure or formatting (e.g. "a drop of delicious honey with a shower of powder sugar")
* The ingredient is not yet present in our taxonomy (or not with the right synonym or in the right language)
In the table below,
* the first column of numbers corresponds to the '''number of unique ingredients''' across all products.
* And the second column of number corresponds to the '''number of occurrences''' of those ingredients. e.g. if a specific ingredient appears in 5 products, it is counted 5 times.
=== JP - Japanese ===
{| class="wikitable"
|-
! !! 2023-07-30 || 2022-07-31
|-
| [https://jp.openfoodfacts.org/ingredients?stats=1&no_cache=1 All products:] ||
<pre>
2822 原材料:
Type Unique tags Occurrences
known 726 (25.73%) 7191 (69.84%)
unknown 2096 (74.27%) 3105 (30.16%)
all 2822 (100.00%) 10296 (100.00%)
</pre>
||
<pre>
</pre>
|-
| [https://jp.openfoodfacts.org/popularity/top-10000-in-scans-2022/ingredients?stats=1&no_cache=1 Top 10k most scanned products:] ||
<pre>
16 原材料:
Type Unique tags Occurrences
known 15 (93.75%) 15 (93.75%)
unknown 1 (6.25%) 1 (6.25%)
all 16 (100.00%) 16 (100.00%)
</pre>
||
<pre>
</pre>
|}
=== HR - Croatian ===
{| class="wikitable"
|-
! !! 2023-07-30 || 2022-07-31
|-
| [https://hr.openfoodfacts.org/ingredients?stats=1&no_cache=1 All products:] ||
<pre>
5700 sastojci:
Type Unique tags Occurrences
known 1304 (22.88%) 25105 (83.02%)
unknown 4396 (77.12%) 5134 (16.98%)
all 5700 (100.00%) 30239 (100.00%)
</pre>
||
<pre>
</pre>
|-
| [https://hr.openfoodfacts.org/popularity/top-10000-in-scans-2022//ingredients?stats=1&no_cache=1 Top 10k most scanned products:] ||
<pre>
45 sastojci:
Type Unique tags Occurrences
known 36 (80.00%) 46 (83.64%)
unknown 9 (20.00%) 9 (16.36%)
all 45 (100.00%) 55 (100.00%)
</pre>
||
|}
=== Observations ===
[[Category:Project:Personalized_Search]]
[[Category:Data quality]]