Difference between revisions of "Data quality missions"

From Open Food Facts wiki
Jump to navigation Jump to search
 
(25 intermediate revisions by 5 users not shown)
Line 5: Line 5:
 
*[https://world.openfoodfacts.org/data-quality/nutrition-value-over-105-carbohydrates Carbohydrate is higher than 100g per 100g]
 
*[https://world.openfoodfacts.org/data-quality/nutrition-value-over-105-carbohydrates Carbohydrate is higher than 100g per 100g]
 
*[https://world.openfoodfacts.org/data-quality/nutrition-value-over-105-fat Fat is higher than 100g per 100g]
 
*[https://world.openfoodfacts.org/data-quality/nutrition-value-over-105-fat Fat is higher than 100g per 100g]
 +
 
===Nutri-Score quality===
 
===Nutri-Score quality===
 
Some products now have Nutri-Score printed on the front of pack. Some differs from our Nutri-Score calculation. We should take care about that:
 
Some products now have Nutri-Score printed on the front of pack. Some differs from our Nutri-Score calculation. We should take care about that:
−
*[https://world.openfoodfacts.org/nutrition-grade/e/label/nutriscore-grade-a?sort_by=popularity Nutri-Score printed A but calculated E]
+
*[https://world.openfoodfacts.org/nutrition-grade/e/label/nutriscore-grade-a?sort_by=popularity Nutri-Score printed A but calculated E] - June 3rd 2024: 79 products
−
*[https://world.openfoodfacts.org/nutrition-grade/d/label/nutriscore-grade-a?sort_by=popularity Nutri-Score printed A but calculated D]
+
*[https://world.openfoodfacts.org/nutrition-grade/d/label/nutriscore-grade-a?sort_by=popularity Nutri-Score printed A but calculated D] - June 3rd 2024: 90 products
−
*[https://world.openfoodfacts.org/nutrition-grade/c/label/nutriscore-grade-a?sort_by=popularity Nutri-Score printed A but calculated C]
+
*[https://world.openfoodfacts.org/nutrition-grade/c/label/nutriscore-grade-a?sort_by=popularity Nutri-Score printed A but calculated C] - June 3rd 2024: 611 products
−
*[https://world.openfoodfacts.org/nutrition-grade/a/label/nutriscore-grade-e?sort_by=popularity Nutri-Score printed E but calculated A]
+
*[https://world.openfoodfacts.org/nutrition-grade/a/label/nutriscore-grade-e?sort_by=popularity Nutri-Score printed E but calculated A] - June 3rd 2024: 9 products
−
*[https://world.openfoodfacts.org/nutrition-grade/b/label/nutriscore-grade-e?sort_by=popularity Nutri-Score printed E but calculated B]
+
*[https://world.openfoodfacts.org/nutrition-grade/b/label/nutriscore-grade-e?sort_by=popularity Nutri-Score printed E but calculated B] - June 3rd 2024: 34 products
−
*[https://world.openfoodfacts.org/nutrition-grade/c/label/nutriscore-grade-e?sort_by=popularity Nutri-Score printed E but calculated C]
+
*[https://world.openfoodfacts.org/nutrition-grade/c/label/nutriscore-grade-e?sort_by=popularity Nutri-Score printed E but calculated C] - June 3rd 2024: 40 products
−
*[https://world.openfoodfacts.org/nutrition-grade/a/label/nutriscore-grade-d?sort_by=popularity Nutri-Score printed D but calculated A]
+
*[https://world.openfoodfacts.org/nutrition-grade/a/label/nutriscore-grade-d?sort_by=popularity Nutri-Score printed D but calculated A] - June 3rd 2024: 4 products
−
*[https://world.openfoodfacts.org/nutrition-grade/b/label/nutriscore-grade-d?sort_by=popularity Nutri-Score printed D but calculated B]
+
*[https://world.openfoodfacts.org/nutrition-grade/b/label/nutriscore-grade-d?sort_by=popularity Nutri-Score printed D but calculated B] - June 3rd 2024: 46 products
−
*[https://world.openfoodfacts.org/nutrition-grade/c/label/nutriscore-grade-d?sort_by=popularity Nutri-Score printed D but calculated C]
+
*[https://world.openfoodfacts.org/nutrition-grade/c/label/nutriscore-grade-d?sort_by=popularity Nutri-Score printed D but calculated C] - June 3rd 2024: 308 products
−
*[https://world.openfoodfacts.org/nutrition-grade/e/label/nutriscore-grade-b?sort_by=popularity Nutri-Score printed B but calculated E]
+
*[https://world.openfoodfacts.org/nutrition-grade/e/label/nutriscore-grade-b?sort_by=popularity Nutri-Score printed B but calculated E] - June 3rd 2024: 153 products
−
*[https://world.openfoodfacts.org/nutrition-grade/d/label/nutriscore-grade-b?sort_by=popularity Nutri-Score printed B but calculated D]
+
*[https://world.openfoodfacts.org/nutrition-grade/d/label/nutriscore-grade-b?sort_by=popularity Nutri-Score printed B but calculated D] - June 3rd 2024: 166 products
−
*[https://world.openfoodfacts.org/nutrition-grade/c/label/nutriscore-grade-b?sort_by=popularity Nutri-Score printed B but calculated C]
+
*[https://world.openfoodfacts.org/nutrition-grade/c/label/nutriscore-grade-b?sort_by=popularity Nutri-Score printed B but calculated C] - June 3rd 2024: 1169 products
 +
 
 
There are many reasons why it can differ:
 
There are many reasons why it can differ:
 
*the label in Open Food Facts does not represent the label printed on the package (easy to solve)
 
*the label in Open Food Facts does not represent the label printed on the package (easy to solve)
Line 33: Line 35:
 
|The Nutri-Score displayed by the producer is different from the Nutri-Score Open Food Facts computes
 
|The Nutri-Score displayed by the producer is different from the Nutri-Score Open Food Facts computes
 
|The label in Open Food Facts does not represent the label printed on the package
 
|The label in Open Food Facts does not represent the label printed on the package
−
|Change the label
+
|Change the Nutri-Score label or labels to what's actually on pack. If there is a special 2 Nutri-Score logo, please use en:Nutri-Score A/B
 
|-
 
|-
−
|idem
+
|The Nutri-Score displayed by the producer is different from the Nutri-Score Open Food Facts computes
 
|The category is wrong
 
|The category is wrong
 
|Change the category and see if it's modifying the Nutri-Score
 
|Change the category and see if it's modifying the Nutri-Score
 
|-
 
|-
−
|idem
+
|The Nutri-Score displayed by the producer is different from the Nutri-Score Open Food Facts computes
 
|The nutrition facts are wrong
 
|The nutrition facts are wrong
 
|
 
|
−
*Update it
+
*Update the nutrition facts. Pay special attention to the fibers if they are mentionned, or any indications on fruits/nuts. Pay attention to values with commas.
 
*If the product's nutrition facts has evolved from a previous version, you can add it to the [[Verification/Noteworthy Products#Products with a Nutri-Score that has improved over time|products whose recipe has changed]] or even better, if it's the case, the [[Verification/Noteworthy Products#Products with a Nutri-Score that has improved over time|products with a Nutri-Score that has improved over time]].
 
*If the product's nutrition facts has evolved from a previous version, you can add it to the [[Verification/Noteworthy Products#Products with a Nutri-Score that has improved over time|products whose recipe has changed]] or even better, if it's the case, the [[Verification/Noteworthy Products#Products with a Nutri-Score that has improved over time|products with a Nutri-Score that has improved over time]].
 +
|-
 +
|The Nutri-Score displayed by the producer is different from the Nutri-Score Open Food Facts computes
 +
|The ingredients are not filled, or filled with mistakes, or not properly detected
 +
|Extract and save the ingredients. Fix any mistakes in the ingredients list. Ask in the #taxonomy channel that someone taxonomizes the ingredient, or do it yourself.
 +
|-
 
|}
 
|}
  
 +
=== Two different Nutriscores as labels ===
 +
These are easy to fix: just watch the last pictures and delete the Nutri-Score that don't fit.
 +
*[https://world.openfoodfacts.org/label/nutriscore-grade-e/label/nutriscore-grade-a?sort_by=popularity Users have filled both Nutriscore A and E] > 1 product
 +
*[https://world.openfoodfacts.org/label/nutriscore-grade-d/label/nutriscore-grade-a?sort_by=popularity Users have filled both Nutriscore A and D] 0 product
 +
*[https://world.openfoodfacts.org/label/nutriscore-grade-c/label/nutriscore-grade-a?sort_by=popularity Users have filled both Nutriscore A and C] 1 product
 +
*[https://world.openfoodfacts.org/label/nutriscore-grade-b/label/nutriscore-grade-a?sort_by=popularity Users have filled both Nutriscore A and B] -- 2024-06-31: 347 products =>
 +
*[https://world.openfoodfacts.org/label/nutriscore-grade-e/label/nutriscore-grade-b?sort_by=popularity Users have filled both Nutriscore B and E] 2 products
 +
*[https://world.openfoodfacts.org/label/nutriscore-grade-d/label/nutriscore-grade-b?sort_by=popularity Users have filled both Nutriscore B and D] -- 2024-05-31: 14 products => 0
 +
*[https://world.openfoodfacts.org/label/nutriscore-grade-c/label/nutriscore-grade-b?sort_by=popularity Users have filled both Nutriscore B and C] -- 2024-06-31: 157 products =>
 +
*[https://world.openfoodfacts.org/label/nutriscore-grade-e/label/nutriscore-grade-c?sort_by=popularity Users have filled both Nutriscore C and E] -- 2024-05-31: 21 products => 0
 +
*[https://world.openfoodfacts.org/label/nutriscore-grade-d/label/nutriscore-grade-c?sort_by=popularity Users have filled both Nutriscore C and D] -- 2024-06-31: 70 products =>
 +
*[https://world.openfoodfacts.org/label/nutriscore-grade-e/label/nutriscore-grade-d?sort_by=popularity Users have filled both Nutriscore D and E] -- 2024-05-31: 56 products => 0
 +
 +
==== Special cases ====
 +
* The Nutri-Score actually changed. Please copy paste the URL below or in Folksonomy Engine if you have access
 +
* There is a 2 Nutri-Score logo. Please use our special values:
 +
<pre>
 +
en:Nutri-Score A/B
 +
en:Nutri-Score A/C
 +
en:Nutri-Score A/D
 +
en:Nutri-Score A/E
 +
en:Nutri-Score B/C
 +
en:Nutri-Score B/D
 +
en:Nutri-Score B/E
 +
en:Nutri-Score C/D
 +
en:Nutri-Score C/E
 +
en:Nutri-Score D/E
 +
</pre>
  
 
=== Intersections to check regularly ===
 
=== Intersections to check regularly ===
−
* https://world.openfoodfacts.org/category/beverages/category/meals
+
==== Categories ====
−
* https://world.openfoodfacts.org/category/sweet%20snacks/category/salted%20snacks
+
Every product has an exclusive entry in tha categories taxonomy.
−
* https://world.openfoodfacts.org/category/non-alcoholic-beverages/category/alcoholic-beverages
+
===== Leaf exclusives =====
−
* https://world.openfoodfacts.org/category/fruits/category/vegetables
+
Each product should belong to a single leaf, This allows us to assign a single Nutriscore and Ecoscore to a product. Two leaves can have the same parent node (are on the same branch). These are then part of the same set and in some cases opposites.
−
* https://world.openfoodfacts.org/category/seeds/category/dairies
 
−
* https://world.openfoodfacts.org/category/meats/category/seafood
 
−
* https://world.openfoodfacts.org/category/poultries/category/fishes
 
−
* https://world.openfoodfacts.org/category/baked-goods/category/confectioneries
 
−
* https://world.openfoodfacts.org/category/Spices/category/herbs
 
−
* https://world.openfoodfacts.org/category/oils/category/vinegars
 
 
* https://world.openfoodfacts.org/category/white-peppers/category/black-peppers
 
* https://world.openfoodfacts.org/category/white-peppers/category/black-peppers
−
* https://world.openfoodfacts.org/category/short-grain-rices/category/long-grain-rices
+
===== Branch exclusives =====
−
* https://world.openfoodfacts.org/category/white-breads/category/whole-wheat-bread
+
The taxonomy is designed in such a way that each branch is unique and contains its own specific products. This hold for each node.
−
* https://world.openfoodfacts.org/category/whole-milks/category/Skimmed-milks
+
* top nodes
 +
** https://world.openfoodfacts.org/category/beverages/category/meals
 +
** https://world.openfoodfacts.org/category/dairy-substitutes/category/dairies
 +
** https://world.openfoodfacts.org/category/meat-analogues/category/meats
 +
* intermediary nodes
 +
** https://world.openfoodfacts.org/category/sweet%20snacks/category/salted%20snacks
 +
** https://world.openfoodfacts.org/category/non-alcoholic-beverages/category/alcoholic-beverages
 +
** https://world.openfoodfacts.org/category/fruits/category/vegetables
 +
** https://world.openfoodfacts.org/category/seeds/category/dairies
 +
** https://world.openfoodfacts.org/category/meats/category/seafood
 +
** https://world.openfoodfacts.org/category/poultries/category/fishes
 +
** https://world.openfoodfacts.org/category/baked-goods/category/confectioneries
 +
** https://world.openfoodfacts.org/category/Spices/category/herbs
 +
** https://world.openfoodfacts.org/category/oils/category/vinegars
 +
** https://world.openfoodfacts.org/category/short-grain-rices/category/long-grain-rices
 +
** https://world.openfoodfacts.org/category/white-breads/category/whole-wheat-bread
 +
** https://world.openfoodfacts.org/category/whole-milks/category/Skimmed-milks
 +
** https://world.openfoodfacts.org/category/Hard%20cheeses/category/Soft%20cheeses
 +
** https://world.openfoodfacts.org/category/pasteurized-cheeses/category/unpasteurised-cheeses
  
−
=== 0 ===
+
==== Labels & Categories ====
 +
* https://world.openfoodfacts.org/label/organic/category/fishes
 +
* https://world.openfoodfacts.org/label/pgi/origin/united-states
 
* https://world.openfoodfacts.org/category/balsamic-vinegars/category/red-wine-vinegars
 
* https://world.openfoodfacts.org/category/balsamic-vinegars/category/red-wine-vinegars
 +
* Fish should never be organic (to be converted in a request)
 +
 +
===Dealing with the consequences of an old Android bug===
 +
"Loading…" (or the equivalent in other languages) replaced the product name. 99% of phones have been updated with the fix, but we still have some unfixed products.
 +
*[https://world.openfoodfacts.org/cgi/search.pl?action=process&search_terms=Loading%E2%80%A6 English]
 +
*[https://world.openfoodfacts.org/cgi/search.pl?action=process&search_terms=chargement%E2%80%A6 French]
 +
*[https://world.openfoodfacts.org/cgi/search.pl?search_terms=laden%E2%80%A6 German]
 +
*[https://world.openfoodfacts.org/cgi/search.pl?action=process&search_terms=Caricamento%E2%80%A6 Italian]
 +
*[https://world.openfoodfacts.org/cgi/search.pl?action=process&search_terms=Carregar%E2%80%A6 Portuguese]
 +
It should be noted that it's pointless to "fix" it by normal editing of the product, as it will just get put back the way it was, as seen [https://pt.openfoodfacts.org/produto/20621483/a-carregar-crownfield?rev=196 here]. A Slack report exists, [https://github.com/openfoodfacts/openfoodfacts-server/issues/4362 here].
 +
=== Removing "Null values" ===
 +
* https://world.openfoodfacts.org/brand/null (853 product)
 +
* https://world.openfoodfacts.org/category/null (1 product)
 +
* https://world.openfoodfacts.org/label/null (0 product)
 +
* https://world.openfoodfacts.org/origin/null (0 product)
  
−
===Cleaning up the consequences of an old Android bug===
 
−
The word "Loading…" replaced the correct product name. 99% of phones have been updated with the fix, but we still have some unfixed products.
 
−
*[https://world.openfoodfacts.org/cgi/search.pl?action=process&search_terms=Caricamento%E2%80%A6&sort_by=unique_scans_n&page_size=24&page=1&sort_by=unique_scans_n Italian]
 
−
*[https://world.openfoodfacts.org/cgi/search.pl?action=process&search_terms=Loading%E2%80%A6&sort_by=unique_scans_n&page_size=24&page=1&sort_by=unique_scans_n English]
 
−
*[https://world.openfoodfacts.org/cgi/search.pl?action=process&search_terms=chargement%E2%80%A6&sort_by=unique_scans_n&page_size=24&page=1&sort_by=unique_scans_n French]
 
−
*[https://world.openfoodfacts.org/cgi/search.pl?search_terms=laden%E2%80%A6&search_simple=1&action=process German]
 
 
===Non-Food Products===
 
===Non-Food Products===
 
Some people are adding products which are not food: beauty products, books, pet food, etc. These products have to be moved to Open Food Facts side projects. Our AI (artificial intelligence) already identifies many cases. These cases are published in the [https://app.slack.com/client/T02KVRT1Q/CT2N423PA/thread/GCUD53J5R-1586349162.333800?cdn_fallback=2 #bot-image-alerts] channel on our slack space.
 
Some people are adding products which are not food: beauty products, books, pet food, etc. These products have to be moved to Open Food Facts side projects. Our AI (artificial intelligence) already identifies many cases. These cases are published in the [https://app.slack.com/client/T02KVRT1Q/CT2N423PA/thread/GCUD53J5R-1586349162.333800?cdn_fallback=2 #bot-image-alerts] channel on our slack space.
Line 84: Line 142:
 
*in [https://app.slack.com/client/T02KVRT1Q/CT2N423PA/thread/GCUD53J5R-1586349162.333800?cdn_fallback=2 #bot-image-alerts] channel, annotate the product with a "checked" icon to tell others that the product has been moved
 
*in [https://app.slack.com/client/T02KVRT1Q/CT2N423PA/thread/GCUD53J5R-1586349162.333800?cdn_fallback=2 #bot-image-alerts] channel, annotate the product with a "checked" icon to tell others that the product has been moved
 
[[Category:Data quality]]
 
[[Category:Data quality]]
 +
[[Category:Nutri-Score]]

Latest revision as of 12:03, 13 September 2024

Nutrition values issues

Open Food Facts identifies some issues related to nutrition values. Some of them are very easy to solve:

Nutri-Score quality

Some products now have Nutri-Score printed on the front of pack. Some differs from our Nutri-Score calculation. We should take care about that:

There are many reasons why it can differ:

  • the label in Open Food Facts does not represent the label printed on the package (easy to solve)
  • the label is correct, but our calculation doesn't provide the same result:
    • check the category,
    • then check the nutrition facts: the issue is sometimes the lack of "fibers" information or the lack of "Fruits, vegetables, nuts and rapeseed, walnut and olive oils" percentage.
  • it can be a software issue (quite rare but possible).
Issue Rationale How to fix
The Nutri-Score displayed by the producer is different from the Nutri-Score Open Food Facts computes The label in Open Food Facts does not represent the label printed on the package Change the Nutri-Score label or labels to what's actually on pack. If there is a special 2 Nutri-Score logo, please use en:Nutri-Score A/B
The Nutri-Score displayed by the producer is different from the Nutri-Score Open Food Facts computes The category is wrong Change the category and see if it's modifying the Nutri-Score
The Nutri-Score displayed by the producer is different from the Nutri-Score Open Food Facts computes The nutrition facts are wrong
The Nutri-Score displayed by the producer is different from the Nutri-Score Open Food Facts computes The ingredients are not filled, or filled with mistakes, or not properly detected Extract and save the ingredients. Fix any mistakes in the ingredients list. Ask in the #taxonomy channel that someone taxonomizes the ingredient, or do it yourself.

Two different Nutriscores as labels

These are easy to fix: just watch the last pictures and delete the Nutri-Score that don't fit.

Special cases

  • The Nutri-Score actually changed. Please copy paste the URL below or in Folksonomy Engine if you have access
  • There is a 2 Nutri-Score logo. Please use our special values:
en:Nutri-Score A/B
en:Nutri-Score A/C
en:Nutri-Score A/D
en:Nutri-Score A/E
en:Nutri-Score B/C
en:Nutri-Score B/D
en:Nutri-Score B/E
en:Nutri-Score C/D
en:Nutri-Score C/E
en:Nutri-Score D/E

Intersections to check regularly

Categories

Every product has an exclusive entry in tha categories taxonomy.

Leaf exclusives

Each product should belong to a single leaf, This allows us to assign a single Nutriscore and Ecoscore to a product. Two leaves can have the same parent node (are on the same branch). These are then part of the same set and in some cases opposites.

Branch exclusives

The taxonomy is designed in such a way that each branch is unique and contains its own specific products. This hold for each node.

Labels & Categories

Dealing with the consequences of an old Android bug

"Loading…" (or the equivalent in other languages) replaced the product name. 99% of phones have been updated with the fix, but we still have some unfixed products.

It should be noted that it's pointless to "fix" it by normal editing of the product, as it will just get put back the way it was, as seen here. A Slack report exists, here.

Removing "Null values"

Non-Food Products

Some people are adding products which are not food: beauty products, books, pet food, etc. These products have to be moved to Open Food Facts side projects. Our AI (artificial intelligence) already identifies many cases. These cases are published in the #bot-image-alerts channel on our slack space.

How to move these products?

  • identify a product in the #bot-image-alerts channel
  • clic on the link after "edit:"
  • if you have the rights to so, you will see "If the barcode is not correct, please correct it here"
    • enter "obf" to move beauty products to Open Beauty Facts
    • enter "opff" to move products to Open Pet Food Facts
    • enter "opf" to move products to Open Product Facts
  • save (if "A product already exists with the new code" message appear, move it manually, and delete it)
  • in #bot-image-alerts channel, annotate the product with a "checked" icon to tell others that the product has been moved
Retrieved from "https://wiki.openfoodfacts.org/index.php?title=Data_quality_missions&oldid=24144"