Difference between revisions of "OCR/Results"

From Open Food Facts wiki
Jump to navigation Jump to search
Line 51: Line 51:
 
* remove json with error messages
 
* remove json with error messages
  
=== Process ===
+
=== How to process images to generate JSONs ===
 
<pre>
 
<pre>
 
sudo nohup sh -c 'for f in $(find . -type f -regex ".*/[0-9].jpg*"); do sudo python /home/xxxxxxxx.py $f;done'
 
sudo nohup sh -c 'for f in $(find . -type f -regex ".*/[0-9].jpg*"); do sudo python /home/xxxxxxxx.py $f;done'
Line 59: Line 59:
 
</pre>
 
</pre>
  
=== Results ===
+
=== How to check the results ===
 
==== Number of images ====
 
==== Number of images ====
46715+128832+59961+180420 = 415928(edited)
+
46715+128832+59961+180420 = 415928 (edited)
 
<pre>
 
<pre>
 
teolemon@ks3095298:/home/off/html/images/products$ ls -lR /home/off/html/images/products/ | find . -name 'nutrition.*.full.jpg' -type f | wc -l
 
teolemon@ks3095298:/home/off/html/images/products$ ls -lR /home/off/html/images/products/ | find . -name 'nutrition.*.full.jpg' -type f | wc -l
Line 78: Line 78:
 
</pre>
 
</pre>
  
==== Number of files ====
+
==== Number of files in the directory ====
  
 
<pre>
 
<pre>
Line 94: Line 94:
  
 
</pre>
 
</pre>
 +
 +
  
 
=== Versions of the same product ===
 
=== Versions of the same product ===
 
* All versions of images selected have been processed
 
* All versions of images selected have been processed
* All images ever uploaded have been processed
+
* All images ever uploaded have been processed
=== Check complete products ===
+
 
 +
 
 +
== How we can leverage the data ==
 +
 
 +
=== Perform checks on products that are already complete ===
 
* Check how similar ingredients lists are to what is detected
 
* Check how similar ingredients lists are to what is detected
 
* Check if brands are suggested that are not here (eg Umbrella brands)
 
* Check if brands are suggested that are not here (eg Umbrella brands)
* Extract addresses
+
* Extract cooking instructions
 +
* Extract storage information
 +
* Extract non-essential marketing information (eg on wines)
 +
* Extract postal addresses
 
* Extract URLs
 
* Extract URLs
 +
  
 
=== Suggest values for incomplete products ===
 
=== Suggest values for incomplete products ===
* Try to suggest possible categories based on extracted text
 
* Try to suggest brands, both based on the Brand JSON field and on Raw text extraction
 
  
=== Search engine (internal/external) considerations ===
+
* Categories candidates, based on extracted text
* Add raw output as hidden text to enable search engine to index incomplete products
+
 
 +
* Brands candidates based on:
 +
** the Brand JSON field
 +
** Raw text extraction
 +
** URL regex
 +
*** Based on Wikidata urls
 +
*** Based on urls extracted in other products, or set as Product urls
 +
 
 +
* Packaging candidates
 +
** Taxonomy match
 +
** Existing values match
 +
 
 +
* Location candidates
 +
** OpenStreetMap/Wikidata match
 +
** Taxonomy match
 +
** Existing values match
 +
 
 +
* Packager codes candidates
 +
** EMB code
 +
** EU codes
 +
 
 +
* Quantity candidates
 +
** Regex match
 +
 
 +
* Labels, certifications, awards
 +
** Taxonomy match
 +
** Existing values match
 +
 
 +
* Origin of ingredients
 +
** Taxonomy match + Dropdown
 +
** Existing values match + Dropdown
 +
 
 +
* Manufacturing or processing places
 +
** OpenStreetMap/Wikidata match
 +
** Taxonomy match
 +
** Existing values match
 +
 
 +
* Link to the product page on the official site of the producer
 +
** Regex match
 +
** Google Search with name+brand
 +
 
 +
* Best before date
 +
** Regex match
 +
 
 +
* City, state and country where purchased
 +
** OpenStreetMap/Wikidata match
 +
** Taxonomy match
 +
** Existing values match
 +
 
 +
* Stores
 +
** Taxonomy match
 +
** Existing values match
 +
 
 +
* Countries where sold
 +
** OpenStreetMap/Wikidata match
 +
** Taxonomy match
 +
** Existing values match
 +
 
 +
* Ingredients list
 +
** Regex match
 +
** Taxonomy match
 +
** Existing values match
 +
 
 +
* Traces
 +
** Regex match
 +
** Taxonomy match
 +
** Existing values match
 +
 
 +
* Serving size
 +
** Regex match
 +
 
 +
* Nutrients volume
 +
** Regex match
 +
 
 +
* Ecological footprint
 +
** Regex match
 +
 
 +
=== Make uncomplete products searchable ===
 +
* Add raw JSON output as hidden by default text to enable search engines to index incomplete products (it could be collapsed and shown after the click of a button)
 
* Add hidden field to enable the internal search engine to find incomplete products
 
* Add hidden field to enable the internal search engine to find incomplete products
=== New fields to add ? ===
+
 
 +
=== Expand Open Food Facts coverage by adding additional information in new fields ===
 +
* Description
 +
** some products are very talkative
 +
** it can be a first step before we add the other fields
 
* Cooking recommandation
 
* Cooking recommandation
 
* Storage recommandation
 
* Storage recommandation
* Description (some products are very talkative)
 
 
* Postal address of the producer
 
* Postal address of the producer
 
* Postal address of the distributor
 
* Postal address of the distributor
* Email Address
+
* Email Address of the producer
* Phone number
+
* Email Address of the distributor
* Website of the brand (vs product)
+
* Phone number of the producer
 +
* Phone number of the distributor
 +
* Website of the brand
 
* Ingredient percentage
 
* Ingredient percentage
  

Revision as of 15:39, 19 October 2016

How to process the 400k JSON files we've generated

Example

Image

ingredients.12.400.jpg

Output

http://static.openfoodfacts.ovh/images/products/40193045/ingredients.12.full.json

{  
   u'responses':[  
      {  
         u'textAnnotations':[  
            {  
               u'locale':u'de',
               u'description':               u'Doppelrahmstufe\nAT Kasezubereitung 60% F.LT\nzutaten:Frischkase,
               2               gefriergetrockne\nter Gorgonzola,
               Salz,
               Verdickungsmittet\nJohannisbrotkernmehl und Guarkemmehlt\nAroma,
               Stickstoff zum Aufschlagen\nNach dem Offnen alsbald verzehren\nUngeoffnet gekuhlt mindestens\nhaltbar bis:siehe Boden\nBY 706\nIn Deutschland hergestellt\n',
               u'boundingPoly':{  
                  u'vertices':[  
                     {  
                        u'y':232,
                        u'x':190
                     },
                     {  
                        u'y':232,
                        u'x':1500
                     },
                     {  
                        u'y':1410,
                        u'x':1500
                     },
                     {  
                        u'y':1410,
                        u'x':190
                     }
                  ]
               }
            }
         ]
      }
   ]
}

Comments

What to do

TODO

  • remove json with error messages

How to process images to generate JSONs

sudo nohup sh -c 'for f in $(find . -type f -regex ".*/[0-9].jpg*"); do sudo python /home/xxxxxxxx.py $f;done'
sudo nohup sh -c 'for f in $(find . -name 'nutrition.*.full.jpg' -type f); do sudo python /home/xxxxxxxx.py $f;done'
sudo nohup sh -c 'for f in $(find . -name 'front.*.full.jpg' -type f); do sudo python /home/xxxxxxxx.py $f;done'
sudo nohup sh -c 'for f in $(find . -name 'ingredients.*.full.jpg' -type f); do sudo python /home/xxxxxxxx.py $f;done'

How to check the results

Number of images

46715+128832+59961+180420 = 415928 (edited)

teolemon@ks3095298:/home/off/html/images/products$ ls -lR /home/off/html/images/products/ | find . -name 'nutrition.*.full.jpg' -type f | wc -l
46715
teolemon@ks3095298:/home/off/html/images/products$ ls -lR /home/off/html/images/products/ | find . -name 'front.*.full.jpg' -type f | wc -l
128382
teolemon@ks3095298:/home/off/html/images/products$ ls -lR /home/off/html/images/products/ | find . -name 'ingredients.*.full.jpg' -type f | wc -l
59961
teolemon@ks3095298:/home/off/html/images/products$ ls -lR /home/off/html/images/products/ | find . -type f -regex ".​​*/[0-9].jpg*​​" | wc -l
180420

Number of JSONs

ls -lR /home/off/html/images/products/ | find . -type f -regex ".​​*/[0-9].json*​​" | wc -l
415308

Number of files in the directory

teolemon@ns3362784:/home/off/html/images/products$ find . -type f | sed 's/.*\.//' | sort | uniq -c
   2514 front
     35 gif
     27 ingredients
2890282 jpg
 478982 json
      1 kxs2sl
      1 out
   3311 png
    210 tif
    153 txt


Versions of the same product

  • All versions of images selected have been processed
  • All images ever uploaded have been processed


How we can leverage the data

Perform checks on products that are already complete

  • Check how similar ingredients lists are to what is detected
  • Check if brands are suggested that are not here (eg Umbrella brands)
  • Extract cooking instructions
  • Extract storage information
  • Extract non-essential marketing information (eg on wines)
  • Extract postal addresses
  • Extract URLs


Suggest values for incomplete products

  • Categories candidates, based on extracted text
  • Brands candidates based on:
    • the Brand JSON field
    • Raw text extraction
    • URL regex
      • Based on Wikidata urls
      • Based on urls extracted in other products, or set as Product urls
  • Packaging candidates
    • Taxonomy match
    • Existing values match
  • Location candidates
    • OpenStreetMap/Wikidata match
    • Taxonomy match
    • Existing values match
  • Packager codes candidates
    • EMB code
    • EU codes
  • Quantity candidates
    • Regex match
  • Labels, certifications, awards
    • Taxonomy match
    • Existing values match
  • Origin of ingredients
    • Taxonomy match + Dropdown
    • Existing values match + Dropdown
  • Manufacturing or processing places
    • OpenStreetMap/Wikidata match
    • Taxonomy match
    • Existing values match
  • Link to the product page on the official site of the producer
    • Regex match
    • Google Search with name+brand
  • Best before date
    • Regex match
  • City, state and country where purchased
    • OpenStreetMap/Wikidata match
    • Taxonomy match
    • Existing values match
  • Stores
    • Taxonomy match
    • Existing values match
  • Countries where sold
    • OpenStreetMap/Wikidata match
    • Taxonomy match
    • Existing values match
  • Ingredients list
    • Regex match
    • Taxonomy match
    • Existing values match
  • Traces
    • Regex match
    • Taxonomy match
    • Existing values match
  • Serving size
    • Regex match
  • Nutrients volume
    • Regex match
  • Ecological footprint
    • Regex match

Make uncomplete products searchable

  • Add raw JSON output as hidden by default text to enable search engines to index incomplete products (it could be collapsed and shown after the click of a button)
  • Add hidden field to enable the internal search engine to find incomplete products

Expand Open Food Facts coverage by adding additional information in new fields

  • Description
    • some products are very talkative
    • it can be a first step before we add the other fields
  • Cooking recommandation
  • Storage recommandation
  • Postal address of the producer
  • Postal address of the distributor
  • Email Address of the producer
  • Email Address of the distributor
  • Phone number of the producer
  • Phone number of the distributor
  • Website of the brand
  • Ingredient percentage

Kind of images processed

Cropped front images

Cropped ingredient images

Cropped nutrition images

Raw images