Difference between revisions of "OCR/Roadmap"

From Open Food Facts wiki
Jump to navigation Jump to search
m (Teolemon moved page Project:OCR to OCR/Roadmap)
Line 1: Line 1:
 
Currently, all products are edited manually. This project is about automatic or semi-automatic detection of a number of things using OCR and Computer vision.
 
Currently, all products are edited manually. This project is about automatic or semi-automatic detection of a number of things using OCR and Computer vision.
  
== Current state ==
 
* OCR extraction of Ingredients using Tesseract 2 (production) and 3 (.net)
 
* Uses the French dictionary for all languages
 
<pre>
 
-- /home/off-fr/cgi# grep get_ocr *
 
Ingredients.pm:use Image::OCR::Tesseract 'get_ocr';
 
Ingredients.pm: $text =  decode utf8=>get_ocr($image,undef,'fra');
 
</pre>
 
* Has a small custom dictionary for French ( /usr/share/tesseract-ocr/tessdata/fra.user-words)
 
**https://code.google.com/p/tesseract-ocr/wiki/FAQ#How_do_I_provide_my_own_dictionary
 
 
== Short term goals ==
 
== Short term goals ==
 
* Use the right standard dict for each language
 
* Use the right standard dict for each language

Revision as of 12:03, 24 January 2016

Currently, all products are edited manually. This project is about automatic or semi-automatic detection of a number of things using OCR and Computer vision.

Short term goals

Dictionaries

Testing

Create a golden set of products that are complete

  • Product
    • Category: "Ingredients complete" "Ingredient images selected"
    • Get the ingredients image
    • Get the canonical (typed by contributors) ingredient list
    • Get the ingredients list generated with the current OCR system
    • Generate the ingredient list on your laptop based on the image, and the custom dictionary above
    • Compare the result with the canonical/golden test and report some accuracy measures
  • Draft Script: https://lite6.framapad.org/p/OFF_OCR_Script

Easy wins

  • Process all images and make products searchable, even if not filled yet

Long-term goals

  • Get dictionaries translations from Wikidata
  • Investigate Ocropus for complex layout extractions
  • Investigate Open CV for detection of patterns, logos…

Targets

  • Logos of brands (Getting them from POD ?)
  • Logos of Labels (standardized)
  • Text (distorted - bottle case, diagonally - with low light, bright light)
  • Standardized layouts (US Nutrition labels)
    • Store in separate image for further reference
  • Standardized text (quantities, EU Packaging codes)
    • Store in separate image for further reference
  • Barcodes (extraction in uploaded images)
    • Store in separate image for further reference
  • Image orientation: check that the text is properly oriented to guess if the image is properly oriented.
  • Deep Learning
    • Product photo on packaging - guess category based on product picture
    • Container: guess whether it's a bottle, cardboard…

Extracting areas is already great work: if we can extract logos or patterns, it will be faster for humans to double check and turn that into text.