Difference between revisions of "Data quality missions"

From Open Food Facts wiki
Jump to navigation Jump to search
(count products)
 
(10 intermediate revisions by 5 users not shown)
Line 8: Line 8:
 
===Nutri-Score quality===
 
===Nutri-Score quality===
 
Some products now have Nutri-Score printed on the front of pack. Some differs from our Nutri-Score calculation. We should take care about that:
 
Some products now have Nutri-Score printed on the front of pack. Some differs from our Nutri-Score calculation. We should take care about that:
−
*[https://world.openfoodfacts.org/nutrition-grade/e/label/nutriscore-grade-a?sort_by=popularity Nutri-Score printed A but calculated E]
+
*[https://world.openfoodfacts.org/nutrition-grade/e/label/nutriscore-grade-a?sort_by=popularity Nutri-Score printed A but calculated E] - June 3rd 2024: 79 products
−
*[https://world.openfoodfacts.org/nutrition-grade/d/label/nutriscore-grade-a?sort_by=popularity Nutri-Score printed A but calculated D]
+
*[https://world.openfoodfacts.org/nutrition-grade/d/label/nutriscore-grade-a?sort_by=popularity Nutri-Score printed A but calculated D] - June 3rd 2024: 90 products
−
*[https://world.openfoodfacts.org/nutrition-grade/c/label/nutriscore-grade-a?sort_by=popularity Nutri-Score printed A but calculated C]
+
*[https://world.openfoodfacts.org/nutrition-grade/c/label/nutriscore-grade-a?sort_by=popularity Nutri-Score printed A but calculated C] - June 3rd 2024: 611 products
−
*[https://world.openfoodfacts.org/nutrition-grade/a/label/nutriscore-grade-e?sort_by=popularity Nutri-Score printed E but calculated A]
+
*[https://world.openfoodfacts.org/nutrition-grade/a/label/nutriscore-grade-e?sort_by=popularity Nutri-Score printed E but calculated A] - June 3rd 2024: 9 products
−
*[https://world.openfoodfacts.org/nutrition-grade/b/label/nutriscore-grade-e?sort_by=popularity Nutri-Score printed E but calculated B]
+
*[https://world.openfoodfacts.org/nutrition-grade/b/label/nutriscore-grade-e?sort_by=popularity Nutri-Score printed E but calculated B] - June 3rd 2024: 34 products
−
*[https://world.openfoodfacts.org/nutrition-grade/c/label/nutriscore-grade-e?sort_by=popularity Nutri-Score printed E but calculated C]
+
*[https://world.openfoodfacts.org/nutrition-grade/c/label/nutriscore-grade-e?sort_by=popularity Nutri-Score printed E but calculated C] - June 3rd 2024: 40 products
−
*[https://world.openfoodfacts.org/nutrition-grade/a/label/nutriscore-grade-d?sort_by=popularity Nutri-Score printed D but calculated A]
+
*[https://world.openfoodfacts.org/nutrition-grade/a/label/nutriscore-grade-d?sort_by=popularity Nutri-Score printed D but calculated A] - June 3rd 2024: 4 products
−
*[https://world.openfoodfacts.org/nutrition-grade/b/label/nutriscore-grade-d?sort_by=popularity Nutri-Score printed D but calculated B]
+
*[https://world.openfoodfacts.org/nutrition-grade/b/label/nutriscore-grade-d?sort_by=popularity Nutri-Score printed D but calculated B] - June 3rd 2024: 46 products
−
*[https://world.openfoodfacts.org/nutrition-grade/c/label/nutriscore-grade-d?sort_by=popularity Nutri-Score printed D but calculated C]
+
*[https://world.openfoodfacts.org/nutrition-grade/c/label/nutriscore-grade-d?sort_by=popularity Nutri-Score printed D but calculated C] - June 3rd 2024: 308 products
−
*[https://world.openfoodfacts.org/nutrition-grade/e/label/nutriscore-grade-b?sort_by=popularity Nutri-Score printed B but calculated E]
+
*[https://world.openfoodfacts.org/nutrition-grade/e/label/nutriscore-grade-b?sort_by=popularity Nutri-Score printed B but calculated E] - June 3rd 2024: 153 products
−
*[https://world.openfoodfacts.org/nutrition-grade/d/label/nutriscore-grade-b?sort_by=popularity Nutri-Score printed B but calculated D]
+
*[https://world.openfoodfacts.org/nutrition-grade/d/label/nutriscore-grade-b?sort_by=popularity Nutri-Score printed B but calculated D] - June 3rd 2024: 166 products
−
*[https://world.openfoodfacts.org/nutrition-grade/c/label/nutriscore-grade-b?sort_by=popularity Nutri-Score printed B but calculated C]
+
*[https://world.openfoodfacts.org/nutrition-grade/c/label/nutriscore-grade-b?sort_by=popularity Nutri-Score printed B but calculated C] - June 3rd 2024: 1169 products
  
 
There are many reasons why it can differ:
 
There are many reasons why it can differ:
Line 55: Line 55:
 
=== Two different Nutriscores as labels ===
 
=== Two different Nutriscores as labels ===
 
These are easy to fix: just watch the last pictures and delete the Nutri-Score that don't fit.
 
These are easy to fix: just watch the last pictures and delete the Nutri-Score that don't fit.
−
*[https://world.openfoodfacts.org/label/nutriscore-grade-e/label/nutriscore-grade-a?sort_by=popularity Users have filled both Nutriscore A and E]
+
*[https://world.openfoodfacts.org/label/nutriscore-grade-e/label/nutriscore-grade-a?sort_by=popularity Users have filled both Nutriscore A and E] > 1 product
−
*[https://world.openfoodfacts.org/label/nutriscore-grade-d/label/nutriscore-grade-a?sort_by=popularity Users have filled both Nutriscore A and D]
+
*[https://world.openfoodfacts.org/label/nutriscore-grade-d/label/nutriscore-grade-a?sort_by=popularity Users have filled both Nutriscore A and D] 0 product
−
*[https://world.openfoodfacts.org/label/nutriscore-grade-c/label/nutriscore-grade-a?sort_by=popularity Users have filled both Nutriscore A and C]
+
*[https://world.openfoodfacts.org/label/nutriscore-grade-c/label/nutriscore-grade-a?sort_by=popularity Users have filled both Nutriscore A and C] 1 product
−
*[https://world.openfoodfacts.org/label/nutriscore-grade-b/label/nutriscore-grade-a?sort_by=popularity Users have filled both Nutriscore A and B]
+
*[https://world.openfoodfacts.org/label/nutriscore-grade-b/label/nutriscore-grade-a?sort_by=popularity Users have filled both Nutriscore A and B] -- 2024-06-31: 347 products =>
−
*[https://world.openfoodfacts.org/label/nutriscore-grade-e/label/nutriscore-grade-b?sort_by=popularity Users have filled both Nutriscore B and E]
+
*[https://world.openfoodfacts.org/label/nutriscore-grade-e/label/nutriscore-grade-b?sort_by=popularity Users have filled both Nutriscore B and E] 2 products
 
*[https://world.openfoodfacts.org/label/nutriscore-grade-d/label/nutriscore-grade-b?sort_by=popularity Users have filled both Nutriscore B and D] -- 2024-05-31: 14 products => 0
 
*[https://world.openfoodfacts.org/label/nutriscore-grade-d/label/nutriscore-grade-b?sort_by=popularity Users have filled both Nutriscore B and D] -- 2024-05-31: 14 products => 0
−
*[https://world.openfoodfacts.org/label/nutriscore-grade-c/label/nutriscore-grade-b?sort_by=popularity Users have filled both Nutriscore B and C]
+
*[https://world.openfoodfacts.org/label/nutriscore-grade-c/label/nutriscore-grade-b?sort_by=popularity Users have filled both Nutriscore B and C] -- 2024-06-31: 157 products =>
 
*[https://world.openfoodfacts.org/label/nutriscore-grade-e/label/nutriscore-grade-c?sort_by=popularity Users have filled both Nutriscore C and E] -- 2024-05-31: 21 products => 0
 
*[https://world.openfoodfacts.org/label/nutriscore-grade-e/label/nutriscore-grade-c?sort_by=popularity Users have filled both Nutriscore C and E] -- 2024-05-31: 21 products => 0
−
*[https://world.openfoodfacts.org/label/nutriscore-grade-d/label/nutriscore-grade-c?sort_by=popularity Users have filled both Nutriscore C and D]
+
*[https://world.openfoodfacts.org/label/nutriscore-grade-d/label/nutriscore-grade-c?sort_by=popularity Users have filled both Nutriscore C and D] -- 2024-06-31: 70 products =>
−
*[https://world.openfoodfacts.org/label/nutriscore-grade-e/label/nutriscore-grade-d?sort_by=popularity Users have filled both Nutriscore D and E] -- 2024-05-31: 56 products => 6
+
*[https://world.openfoodfacts.org/label/nutriscore-grade-e/label/nutriscore-grade-d?sort_by=popularity Users have filled both Nutriscore D and E] -- 2024-05-31: 56 products => 0
  
 
==== Special cases ====
 
==== Special cases ====
Line 84: Line 84:
 
=== Intersections to check regularly ===
 
=== Intersections to check regularly ===
 
==== Categories ====
 
==== Categories ====
−
* https://world.openfoodfacts.org/category/beverages/category/meals
+
Every product has an exclusive entry in tha categories taxonomy.
−
* https://world.openfoodfacts.org/category/sweet%20snacks/category/salted%20snacks
+
===== Leaf exclusives =====
−
* https://world.openfoodfacts.org/category/non-alcoholic-beverages/category/alcoholic-beverages
+
Each product should belong to a single leaf, This allows us to assign a single Nutriscore and Ecoscore to a product. Two leaves can have the same parent node (are on the same branch). These are then part of the same set and in some cases opposites.
−
* https://world.openfoodfacts.org/category/fruits/category/vegetables
 
−
* https://world.openfoodfacts.org/category/seeds/category/dairies
 
−
* https://world.openfoodfacts.org/category/meats/category/seafood
 
−
* https://world.openfoodfacts.org/category/poultries/category/fishes
 
−
* https://world.openfoodfacts.org/category/baked-goods/category/confectioneries
 
−
* https://world.openfoodfacts.org/category/Spices/category/herbs
 
−
* https://world.openfoodfacts.org/category/oils/category/vinegars
 
 
* https://world.openfoodfacts.org/category/white-peppers/category/black-peppers
 
* https://world.openfoodfacts.org/category/white-peppers/category/black-peppers
−
* https://world.openfoodfacts.org/category/short-grain-rices/category/long-grain-rices
+
===== Branch exclusives =====
−
* https://world.openfoodfacts.org/category/white-breads/category/whole-wheat-bread
+
The taxonomy is designed in such a way that each branch is unique and contains its own specific products. This hold for each node.
−
* https://world.openfoodfacts.org/category/whole-milks/category/Skimmed-milks
+
* top nodes
−
* https://world.openfoodfacts.org/category/Hard%20cheeses/category/Soft%20cheeses
+
** https://world.openfoodfacts.org/category/beverages/category/meals
−
* https://world.openfoodfacts.org/category/pasteurized-cheeses/category/unpasteurised-cheeses
+
** https://world.openfoodfacts.org/category/dairy-substitutes/category/dairies
−
* https://world.openfoodfacts.org/category/dairy-substitutes/category/dairies
+
** https://world.openfoodfacts.org/category/meat-analogues/category/meats
−
* https://world.openfoodfacts.org/category/meat-analogues/category/meats
+
* intermediary nodes
 +
** https://world.openfoodfacts.org/category/sweet%20snacks/category/salted%20snacks
 +
** https://world.openfoodfacts.org/category/non-alcoholic-beverages/category/alcoholic-beverages
 +
** https://world.openfoodfacts.org/category/fruits/category/vegetables
 +
** https://world.openfoodfacts.org/category/seeds/category/dairies
 +
** https://world.openfoodfacts.org/category/meats/category/seafood
 +
** https://world.openfoodfacts.org/category/poultries/category/fishes
 +
** https://world.openfoodfacts.org/category/baked-goods/category/confectioneries
 +
** https://world.openfoodfacts.org/category/Spices/category/herbs
 +
** https://world.openfoodfacts.org/category/oils/category/vinegars
 +
** https://world.openfoodfacts.org/category/short-grain-rices/category/long-grain-rices
 +
** https://world.openfoodfacts.org/category/white-breads/category/whole-wheat-bread
 +
** https://world.openfoodfacts.org/category/whole-milks/category/Skimmed-milks
 +
** https://world.openfoodfacts.org/category/Hard%20cheeses/category/Soft%20cheeses
 +
** https://world.openfoodfacts.org/category/pasteurized-cheeses/category/unpasteurised-cheeses
  
 
==== Labels & Categories ====
 
==== Labels & Categories ====
 
* https://world.openfoodfacts.org/label/organic/category/fishes
 
* https://world.openfoodfacts.org/label/organic/category/fishes
 
* https://world.openfoodfacts.org/label/pgi/origin/united-states
 
* https://world.openfoodfacts.org/label/pgi/origin/united-states
 +
* https://world.openfoodfacts.org/category/balsamic-vinegars/category/red-wine-vinegars
 +
* Fish should never be organic (to be converted in a request)
  
−
=== 0 ===
+
===Dealing with the consequences of an old Android bug===
−
* https://world.openfoodfacts.org/category/balsamic-vinegars/category/red-wine-vinegars
+
"Loading…" (or the equivalent in other languages) replaced the product name. 99% of phones have been updated with the fix, but we still have some unfixed products.
 +
*[https://world.openfoodfacts.org/cgi/search.pl?action=process&search_terms=Loading%E2%80%A6 English]
 +
*[https://world.openfoodfacts.org/cgi/search.pl?action=process&search_terms=chargement%E2%80%A6 French]
 +
*[https://world.openfoodfacts.org/cgi/search.pl?search_terms=laden%E2%80%A6 German]
 +
*[https://world.openfoodfacts.org/cgi/search.pl?action=process&search_terms=Caricamento%E2%80%A6 Italian]
 +
*[https://world.openfoodfacts.org/cgi/search.pl?action=process&search_terms=Carregar%E2%80%A6 Portuguese]
 +
It should be noted that it's pointless to "fix" it by normal editing of the product, as it will just get put back the way it was, as seen [https://pt.openfoodfacts.org/produto/20621483/a-carregar-crownfield?rev=196 here]. A Slack report exists, [https://github.com/openfoodfacts/openfoodfacts-server/issues/4362 here].
 +
=== Removing "Null values" ===
 +
* https://world.openfoodfacts.org/brand/null (853 product)
 +
* https://world.openfoodfacts.org/category/null (1 product)
 +
* https://world.openfoodfacts.org/label/null (0 product)
 +
* https://world.openfoodfacts.org/origin/null (0 product)
  
−
===Cleaning up the consequences of an old Android bug===
 
−
The word "Loading…" replaced the correct product name. 99% of phones have been updated with the fix, but we still have some unfixed products.
 
−
*[https://world.openfoodfacts.org/cgi/search.pl?action=process&search_terms=Caricamento%E2%80%A6&sort_by=unique_scans_n&page_size=24&page=1&sort_by=unique_scans_n Italian]
 
−
*[https://world.openfoodfacts.org/cgi/search.pl?action=process&search_terms=Loading%E2%80%A6&sort_by=unique_scans_n&page_size=24&page=1&sort_by=unique_scans_n English]
 
−
*[https://world.openfoodfacts.org/cgi/search.pl?action=process&search_terms=chargement%E2%80%A6&sort_by=unique_scans_n&page_size=24&page=1&sort_by=unique_scans_n French]
 
−
*[https://world.openfoodfacts.org/cgi/search.pl?search_terms=laden%E2%80%A6&search_simple=1&action=process German]
 
 
===Non-Food Products===
 
===Non-Food Products===
 
Some people are adding products which are not food: beauty products, books, pet food, etc. These products have to be moved to Open Food Facts side projects. Our AI (artificial intelligence) already identifies many cases. These cases are published in the [https://app.slack.com/client/T02KVRT1Q/CT2N423PA/thread/GCUD53J5R-1586349162.333800?cdn_fallback=2 #bot-image-alerts] channel on our slack space.
 
Some people are adding products which are not food: beauty products, books, pet food, etc. These products have to be moved to Open Food Facts side projects. Our AI (artificial intelligence) already identifies many cases. These cases are published in the [https://app.slack.com/client/T02KVRT1Q/CT2N423PA/thread/GCUD53J5R-1586349162.333800?cdn_fallback=2 #bot-image-alerts] channel on our slack space.
Line 128: Line 142:
 
*in [https://app.slack.com/client/T02KVRT1Q/CT2N423PA/thread/GCUD53J5R-1586349162.333800?cdn_fallback=2 #bot-image-alerts] channel, annotate the product with a "checked" icon to tell others that the product has been moved
 
*in [https://app.slack.com/client/T02KVRT1Q/CT2N423PA/thread/GCUD53J5R-1586349162.333800?cdn_fallback=2 #bot-image-alerts] channel, annotate the product with a "checked" icon to tell others that the product has been moved
 
[[Category:Data quality]]
 
[[Category:Data quality]]
 +
[[Category:Nutri-Score]]

Latest revision as of 12:03, 13 September 2024

Nutrition values issues

Open Food Facts identifies some issues related to nutrition values. Some of them are very easy to solve:

Nutri-Score quality

Some products now have Nutri-Score printed on the front of pack. Some differs from our Nutri-Score calculation. We should take care about that:

There are many reasons why it can differ:

  • the label in Open Food Facts does not represent the label printed on the package (easy to solve)
  • the label is correct, but our calculation doesn't provide the same result:
    • check the category,
    • then check the nutrition facts: the issue is sometimes the lack of "fibers" information or the lack of "Fruits, vegetables, nuts and rapeseed, walnut and olive oils" percentage.
  • it can be a software issue (quite rare but possible).
Issue Rationale How to fix
The Nutri-Score displayed by the producer is different from the Nutri-Score Open Food Facts computes The label in Open Food Facts does not represent the label printed on the package Change the Nutri-Score label or labels to what's actually on pack. If there is a special 2 Nutri-Score logo, please use en:Nutri-Score A/B
The Nutri-Score displayed by the producer is different from the Nutri-Score Open Food Facts computes The category is wrong Change the category and see if it's modifying the Nutri-Score
The Nutri-Score displayed by the producer is different from the Nutri-Score Open Food Facts computes The nutrition facts are wrong
The Nutri-Score displayed by the producer is different from the Nutri-Score Open Food Facts computes The ingredients are not filled, or filled with mistakes, or not properly detected Extract and save the ingredients. Fix any mistakes in the ingredients list. Ask in the #taxonomy channel that someone taxonomizes the ingredient, or do it yourself.

Two different Nutriscores as labels

These are easy to fix: just watch the last pictures and delete the Nutri-Score that don't fit.

Special cases

  • The Nutri-Score actually changed. Please copy paste the URL below or in Folksonomy Engine if you have access
  • There is a 2 Nutri-Score logo. Please use our special values:
en:Nutri-Score A/B
en:Nutri-Score A/C
en:Nutri-Score A/D
en:Nutri-Score A/E
en:Nutri-Score B/C
en:Nutri-Score B/D
en:Nutri-Score B/E
en:Nutri-Score C/D
en:Nutri-Score C/E
en:Nutri-Score D/E

Intersections to check regularly

Categories

Every product has an exclusive entry in tha categories taxonomy.

Leaf exclusives

Each product should belong to a single leaf, This allows us to assign a single Nutriscore and Ecoscore to a product. Two leaves can have the same parent node (are on the same branch). These are then part of the same set and in some cases opposites.

Branch exclusives

The taxonomy is designed in such a way that each branch is unique and contains its own specific products. This hold for each node.

Labels & Categories

Dealing with the consequences of an old Android bug

"Loading…" (or the equivalent in other languages) replaced the product name. 99% of phones have been updated with the fix, but we still have some unfixed products.

It should be noted that it's pointless to "fix" it by normal editing of the product, as it will just get put back the way it was, as seen here. A Slack report exists, here.

Removing "Null values"

Non-Food Products

Some people are adding products which are not food: beauty products, books, pet food, etc. These products have to be moved to Open Food Facts side projects. Our AI (artificial intelligence) already identifies many cases. These cases are published in the #bot-image-alerts channel on our slack space.

How to move these products?

  • identify a product in the #bot-image-alerts channel
  • clic on the link after "edit:"
  • if you have the rights to so, you will see "If the barcode is not correct, please correct it here"
    • enter "obf" to move beauty products to Open Beauty Facts
    • enter "opff" to move products to Open Pet Food Facts
    • enter "opf" to move products to Open Product Facts
  • save (if "A product already exists with the new code" message appear, move it manually, and delete it)
  • in #bot-image-alerts channel, annotate the product with a "checked" icon to tell others that the product has been moved
Retrieved from "https://wiki.openfoodfacts.org/index.php?title=Data_quality_missions&oldid=24144"