Difference between revisions of "Showstoppers for Ingredients Extraction and Analysis"
Jump to navigation
Jump to search
Line 14: | Line 14: | ||
=== Step 3 - Fix errors of ingredients cutting === | === Step 3 - Fix errors of ingredients cutting === | ||
− | * Crop/Recrop more tightly: [https://github.com/openfoodfacts/openfoodfacts-ai/issues/40 Autocrop ingredient lists] | + | * Crop/Recrop more tightly: [https://github.com/openfoodfacts/openfoodfacts-ai/issues/40 Autocrop ingredient lists] - [https://github.com/openfoodfacts/openfoodfacts-server/issues/2900 Image recognition & suggested auto-crop] |
* Add more cutters in more languages | * Add more cutters in more languages | ||
− | * Add more cutters in French | + | ** Add more cutters in French |
+ | *** https://github.com/openfoodfacts/openfoodfacts-server/blob/main/lib/ProductOpener/Ingredients.pm#L5137 | ||
* Move it to AI/LLM: [https://github.com/openfoodfacts/openfoodfacts-ai/issues/242 Extract automatically ingredient list from OCR] | * Move it to AI/LLM: [https://github.com/openfoodfacts/openfoodfacts-ai/issues/242 Extract automatically ingredient list from OCR] | ||
=== Step 4 - Fix errors with Bad OCRs === | === Step 4 - Fix errors with Bad OCRs === | ||
Line 26: | Line 27: | ||
=== Step 6 - Bad ingredient properties recognition === | === Step 6 - Bad ingredient properties recognition === | ||
* Processing, Origins, Other properties | * Processing, Origins, Other properties | ||
+ | ** https://github.com/openfoodfacts/openfoodfacts-server/blob/main/taxonomies/ingredients_processing.txt | ||
=== Step 7 - Missing translations or synonyms === | === Step 7 - Missing translations or synonyms === | ||
* Translate the taxonomies using LLM recipes | * Translate the taxonomies using LLM recipes | ||
* Add synonyms | * Add synonyms | ||
+ | * https://github.com/openfoodfacts/openfoodfacts-server/blob/main/taxonomies/food/ingredients.txt | ||
=== Step 8 - Truly untaxonomized ingredients === | === Step 8 - Truly untaxonomized ingredients === | ||
− | * Add those to the taxonomy | + | * Add those to the taxonomy - https://github.com/openfoodfacts/openfoodfacts-server/blob/main/taxonomies/food/ingredients.txt |
=== Step 9 - Missing or Bad CIQUAL associations === | === Step 9 - Missing or Bad CIQUAL associations === | ||
* Add CIQUAL matches | * Add CIQUAL matches | ||
* Add matches from other Nutrition databases | * Add matches from other Nutrition databases | ||
+ | ** https://www.fao.org/infoods/infoods/tables-and-databases/europe/en/ | ||
+ | ** https://www.wikidata.org/wiki/Wikidata:WikiProject_Food/Properties#National_nutrition_profiles | ||
=== Step 10 - Bad or insufficient categorization === | === Step 10 - Bad or insufficient categorization === | ||
* Identify problematic categories | * Identify problematic categories | ||
− | * Play Hunger Games on problematic categories | + | * Play Hunger Games on problematic categories (evaporation categories for instance) |
+ | ** https://hunger.openfoodfacts.org/questions?type=category&sorted=false&value_tag=en%3Atomato-sauces | ||
=== Step 11 - Bad nutrition === | === Step 11 - Bad nutrition === |
Revision as of 15:45, 17 January 2025
Step 1 - Remove pet food & Cosmetics
- AI: identify them, especially with ingredients, and move them to OBF, OPFF, OPF
Step 2 - Fix ingredients stored in the wrong language
- AI: automatic image deselection & reselection
- AI: drastic cleanup of bad lists
- AI: Data quality facets about ingredients become errors
Step 3 - Fix errors of ingredients cutting
- Crop/Recrop more tightly: Autocrop ingredient lists - Image recognition & suggested auto-crop
- Add more cutters in more languages
- Add more cutters in French
- Move it to AI/LLM: Extract automatically ingredient list from OCR
Step 4 - Fix errors with Bad OCRs
- Reprocess OCRs
- Perform spellcheck
Step 5 - Bad splitting into individual ingredients
- Add a multilingual no cut list of words like U.H.T Cat 1…
Step 6 - Bad ingredient properties recognition
- Processing, Origins, Other properties
Step 7 - Missing translations or synonyms
- Translate the taxonomies using LLM recipes
- Add synonyms
- https://github.com/openfoodfacts/openfoodfacts-server/blob/main/taxonomies/food/ingredients.txt
Step 8 - Truly untaxonomized ingredients
- Add those to the taxonomy - https://github.com/openfoodfacts/openfoodfacts-server/blob/main/taxonomies/food/ingredients.txt
Step 9 - Missing or Bad CIQUAL associations
- Add CIQUAL matches
- Add matches from other Nutrition databases
Step 10 - Bad or insufficient categorization
- Identify problematic categories
- Play Hunger Games on problematic categories (evaporation categories for instance)
Step 11 - Bad nutrition
- Clean nutrition
Step 12 - Additional data not the ingredient list
- Origins, -25% de sel, …