Line 110: |
Line 110: |
| ** Test sets need to be created | | ** Test sets need to be created |
| *** Run spellcheckers on actual ingredients lists from OFF, review corrections | | *** Run spellcheckers on actual ingredients lists from OFF, review corrections |
| + | |
| + | |
| + | == Ingredients analysis == |
| + | |
| + | === Steps for ingredients analysis === |
| + | |
| + | ==== Ingredients pre-parsing ==== |
| + | |
| + | * The ingredients list is transformed to make parsing easier |
| + | ** Remove / normalize strange characters |
| + | ** De-abbreviate abbreviations |
| + | ** Split enumerations |
| + | *** e.g. "Vitamins A, B et C" -> Vitamine A, Vitamine B, Vitamine C |
| + | ** Additives E-numbers normalization (E330, e330, e-330, INS 330, SIN330 etc.) |
| + | ** Additives classes + additive splits |
| + | *** e.g. "Colour caramel" -> Colour: Caramel |
| + | ** Split some "A of B, C and D" (but not all...) |
| + | *** e.g. "Huile de palme, colza et tournesol" -> Huile de palme, huile de colza, huile de tournesol |
| + | * Current solution |
| + | ** Perl code and regular expressions |
| + | *** lib/ProductOpener/Ingredients.pm - preparse_ingredients_text() |