Line 1: |
Line 1: |
− | How to process the 1.4 million JSON files we've generated | + | How to process the 400k JSON files we've generated |
| + | [[Category:OCR]] |
| + | == Example == |
| + | === Image === |
| + | http://static.openfoodfacts.ovh/images/products/40193045/ingredients.12.400.jpg |
| + | === Output === |
| + | http://static.openfoodfacts.ovh/images/products/40193045/ingredients.12.full.json |
| + | <pre> |
| + | { |
| + | u'responses':[ |
| + | { |
| + | u'textAnnotations':[ |
| + | { |
| + | u'locale':u'de', |
| + | u'description': u'Doppelrahmstufe\nAT Kasezubereitung 60% F.LT\nzutaten:Frischkase, |
| + | 2 gefriergetrockne\nter Gorgonzola, |
| + | Salz, |
| + | Verdickungsmittet\nJohannisbrotkernmehl und Guarkemmehlt\nAroma, |
| + | Stickstoff zum Aufschlagen\nNach dem Offnen alsbald verzehren\nUngeoffnet gekuhlt mindestens\nhaltbar bis:siehe Boden\nBY 706\nIn Deutschland hergestellt\n', |
| + | u'boundingPoly':{ |
| + | u'vertices':[ |
| + | { |
| + | u'y':232, |
| + | u'x':190 |
| + | }, |
| + | { |
| + | u'y':232, |
| + | u'x':1500 |
| + | }, |
| + | { |
| + | u'y':1410, |
| + | u'x':1500 |
| + | }, |
| + | { |
| + | u'y':1410, |
| + | u'x':190 |
| + | } |
| + | ] |
| + | } |
| + | } |
| + | ] |
| + | } |
| + | ] |
| + | } |
| + | </pre> |
| + | |
| + | === Comments=== |
| + | |
| + | == Instructions == |
| + | === How to process images to generate JSONs === |
| + | |
| + | Needs to be executed in home/off/html/ |
| + | |
| + | <pre> |
| + | sudo nohup sh -c 'for f in $(find . -type f -regex ".*/[0-9]+.jpg*"); do sudo python /home/teolemon/ocr/cloudvision4.py $f;done' |
| + | sudo nohup sh -c 'for f in $(find . -name 'nutrition.*.full.jpg' -type f); do sudo python /home/teolemon/ocr/cloudvision4.py $f;done' |
| + | sudo nohup sh -c 'for f in $(find . -name 'front.*.full.jpg' -type f); do sudo python /home/teolemon/ocr/cloudvision4.py $f;done' |
| + | sudo nohup sh -c 'for f in $(find . -name 'ingredients.*.full.jpg' -type f); do sudo python /home/teolemon/ocr/cloudvision4.py $f;done' |
| + | </pre> |
| + | |
| + | <pre> |
| + | sudo nohup sh -c 'for f in $(find . -type f -regex ".*/[1-1].jpg*"); do sudo python /home/teolemon/ocr/cloudvision4.py $f;done' |
| + | sudo nohup sh -c 'for f in $(find . -type f -regex ".*/[2-2].jpg*"); do sudo python /home/teolemon/ocr/cloudvision4.py $f;done' |
| + | sudo nohup sh -c 'for f in $(find . -type f -regex ".*/[3-3].jpg*"); do sudo python /home/teolemon/ocr/cloudvision4.py $f;done' |
| + | sudo nohup sh -c 'for f in $(find . -type f -regex ".*/[4-4].jpg*"); do sudo python /home/teolemon/ocr/cloudvision4.py $f;done' |
| + | sudo nohup sh -c 'for f in $(find . -type f -regex ".*/[5-5].jpg*"); do sudo python /home/teolemon/ocr/cloudvision4.py $f;done' |
| + | sudo nohup sh -c 'for f in $(find . -type f -regex ".*/[6-6].jpg*"); do sudo python /home/teolemon/ocr/cloudvision4.py $f;done' |
| + | sudo nohup sh -c 'for f in $(find . -type f -regex ".*/[7-7].jpg*"); do sudo python /home/teolemon/ocr/cloudvision4.py $f;done' |
| + | </pre> |
| + | |
| + | === How to check the results === |
| + | ==== Number of images ==== |
| + | 46715+128832+59961+180420 = 415928 (edited) |
| + | <pre> |
| + | ls -lR /srv/off/html/images/products/ | find . -name 'nutrition.*.full.jpg' -type f | wc -l |
| + | 46715 |
| + | ls -lR /srv/off/html/images/products/ | find . -name 'front.*.full.jpg' -type f | wc -l |
| + | 128382 |
| + | ls -lR /srv/off/html/images/products/ | find . -name 'ingredients.*.full.jpg' -type f | wc -l |
| + | 59961 |
| + | ls -lR /srv/off/html/images/products/ | find . -type f -regex ".*/[0-9].jpg*" | wc -l |
| + | 180420 |
| + | </pre> |
| + | |
| + | <pre> |
| + | Nov 1 - 13:22 - Paris Time |
| + | ls -lR /srv/off/html/images/products | find . -type f -regex ".*/[0-9].jpg*" | wc -l |
| + | 287119 |
| + | </pre> |
| + | |
| + | ==== Number of JSONs==== |
| + | <pre> |
| + | ls -lR /srv/off/html/images/products/ | find . -type f -regex ".*/[0-9].json*" | wc -l |
| + | 415308 |
| + | </pre> |
| + | |
| + | |
| + | <pre> |
| + | Nov 1 2016 - 13:30 |
| + | teolemon@ns3362784:/srv/off/html/images/products$ ls -lR /srv/off/html/images/products | find . -type f -regex ".*/[0-9].json*" | wc -l |
| + | 253322 |
| + | </pre> |
| + | |
| + | ==== Number of files in the directory ==== |
| + | |
| + | <pre> |
| + | teolemon@ns3362784:/home/off/html/images/products$ find . -type f | sed 's/.*\.//' | sort | uniq -c |
| + | 2514 front |
| + | 35 gif |
| + | 27 ingredients |
| + | 2890282 jpg |
| + | 478982 json |
| + | 1 kxs2sl |
| + | 1 out |
| + | 3311 png |
| + | 210 tif |
| + | 153 txt |
| + | |
| + | </pre> |
| + | |
| + | ==== Search for a string in the generated JSONs ==== |
| + | grep --include=*.json -rnwl '/srv/off/html/images/products/' -e "halloween" |
| + | |
| + | == Status == |
| + | === TODO === |
| + | * remove json with error messages |
| + | |
| + | === Open Food Facts === |
| + | * raw images |
| + | ** existing |
| + | ** processed |
| + | * front |
| + | ** existing |
| + | ** processed |
| + | * ingredients |
| + | ** existing |
| + | ** processed |
| + | * nutrition |
| + | ** existing |
| + | ** processed |
| + | |
| + | === Open Beauty Facts === |
| + | * raw images |
| + | ** existing |
| + | ** processed |
| + | * front |
| + | ** existing |
| + | ** processed |
| + | * ingredients |
| + | ** existing |
| + | ** processed |
| + | * nutrition |
| + | ** existing |
| + | ** processed |
| + | |
| + | == How we can leverage the data == |
| + | |
| + | === Perform checks on products that are already complete === |
| + | * Check how similar ingredients lists are to what is detected |
| + | ** Calculate percentage of ingredients similarity |
| + | *** List of parsed in ingredients list |
| + | *** Ingredients found in the JSON (from the list of ingredients) |
| + | |
| + | * Check for ingredients not found in the current ingredients list (if not empty) |
| + | ** Needs ingredients taxonomisation to avoid false positives |
| + | |
| + | * Check if brands are suggested that are not here (eg Umbrella brands) |
| + | ** Taxonomize brands |
| + | ** Propose candidates |
| + | |
| + | * Check nutrition values on new images |
| + | ** Marcel Salathé seems to be working on a machine learning solution to extract nutrition values from raw images |
| + | |
| + | * Look for new languages |
| + | ** Generate a list of languages in each image/product |
| + | *** Use language detection libs to detect all languages |
| + | *** Use language value in JSON |
| + | ** Check if the product has images and values for all of them |
| + | *** values that change are (front/ingredients/nutrition images) & (front/ingredients text) |
| + | ** Make a page with a list of products with new languages to add |
| | | |
− | === Cropped front images ===
| |
− | === Cropped ingredient images ===
| |
− | === Cropped nutrition images ===
| |
− | === Raw images ===
| |
− | === Versions of the same product ===
| |
− | === Check complete products ===
| |
| === Suggest values for incomplete products === | | === Suggest values for incomplete products === |
| + | |
| + | * Categories candidates, based on extracted text |
| + | |
| + | * Brands candidates based on: |
| + | ** the Brand JSON field |
| + | ** Raw text extraction |
| + | ** URL regex |
| + | *** Based on Wikidata urls |
| + | *** Based on urls extracted in other products, or set as Product urls |
| + | |
| + | * Packaging candidates |
| + | ** Taxonomy match |
| + | ** Existing values match |
| + | |
| + | * Location candidates |
| + | ** OpenStreetMap/Wikidata match |
| + | ** Taxonomy match |
| + | ** Existing values match |
| + | |
| + | * Packager codes candidates |
| + | ** EMB code |
| + | ** EU codes |
| + | |
| + | * Quantity candidates |
| + | ** Regex match |
| + | |
| + | * Labels, certifications, awards |
| + | ** Taxonomy match |
| + | ** Existing values match |
| + | |
| + | * Origin of ingredients |
| + | ** Taxonomy match + Dropdown |
| + | ** Existing values match + Dropdown |
| + | |
| + | * Manufacturing or processing places |
| + | ** OpenStreetMap/Wikidata match |
| + | ** Taxonomy match |
| + | ** Existing values match |
| + | |
| + | * Link to the product page on the official site of the producer |
| + | ** Regex match |
| + | ** Google Search with name+brand |
| + | |
| + | * Best before date |
| + | ** Regex match |
| + | |
| + | * City, state and country where purchased |
| + | ** OpenStreetMap/Wikidata match |
| + | ** Taxonomy match |
| + | ** Existing values match |
| + | |
| + | * Stores |
| + | ** Taxonomy match |
| + | ** Existing values match |
| + | |
| + | * Countries where sold |
| + | ** OpenStreetMap/Wikidata match |
| + | ** Taxonomy match |
| + | ** Existing values match |
| + | |
| + | * Ingredients list |
| + | ** Regex match |
| + | ** Taxonomy match |
| + | ** Existing values match |
| + | |
| + | * Traces |
| + | ** Regex match |
| + | ** Taxonomy match |
| + | ** Existing values match |
| + | |
| + | * Serving size |
| + | ** Regex match |
| + | |
| + | * Nutrients volume |
| + | ** Regex match |
| + | |
| + | * Ecological footprint |
| + | ** Regex match |
| + | |
| + | === Make uncomplete products searchable === |
| + | * Add raw JSON output as hidden by default text to enable search engines to index incomplete products (it could be collapsed and shown after the click of a button) |
| + | * Add hidden field to enable the internal search engine to find incomplete products |
| + | |
| + | === Expand Open Food Facts coverage by adding additional information in new fields === |
| + | * Description |
| + | ** some products are very talkative, non-essential marketing information (eg on wines) |
| + | ** it can be a first step before we add the other fields |
| + | * Cooking recommandation |
| + | * Storage recommandation |
| + | * Postal address of the producer |
| + | * Postal address of the distributor |
| + | * Email Address of the producer |
| + | * Email Address of the distributor |
| + | * Phone number of the producer |
| + | * Phone number of the distributor |
| + | * Website of the brand |
| + | * Ingredient percentage |
| + | |
| + | == Kind of images processed == |
| + | ==== Cropped front images ==== |
| + | ==== Cropped ingredient images ==== |
| + | ==== Cropped nutrition images ==== |
| + | ==== Raw images ==== |