Changes

== ⭐ Guidelines ==

== ⭐ Guidelines ==

The impact of the Ingredient-Spellcheck needs to be controlled to avoid altering contributions and/or add new errors to "healthy" ingredient names. From the different types of errors observed across products, we came up with these spellcheck guidelines:

The impact of the Ingredient-Spellcheck needs to be controlled to avoid altering contributions and/or add new errors to "healthy" ingredient names. The text needs to be kept as unchanged as possible. From the different types of errors observed across products, we came up with these spellcheck guidelines:

* Whitespaces between words and percentages shouldn't be corrected. The text needs to be kept as unchanged as possible (Example: `Ingredient 0,2   %`).

* Whitespaces between words and percentages shouldn't be corrected.

* If the percentage is ambiguous, we don't correct it. (*ex: "15 0%" - "396"*)

** Example: "salt 0,2 %"

* The only case when a whitespace involving a percentage should be modified is if the *digit* is stuck in the previous word (*ex: cheese1.9% -> cheese 1.9%*)

* If a percentage is ambiguous, it shouldn't be corrected.

** Examples: "15 0%" ; "396"

* A whitespace involving a percentage shouldn't be modified, except if the digit is stuck in the previous word.

** Example: "cheese1.9%" → "cheese 1.9%"

* Some ingredients are enclosed with `_`, such as `_milk_` or `_cacahuetes_`, to detect allergens. Should remain unchanged. However, in the case it is not an ingredient, such as `_Cacahuetes_ con cáscara tostado. _Trazas de frutos de cáscara_.`, it needs to be modified into `_Cacahuetes_ con cáscara tostado. Trazas de frutos de cáscara.`;

* Some ingredients are enclosed with `_`, such as `_milk_` or `_cacahuetes_`, to detect allergens. Should remain unchanged. However, in the case it is not an ingredient, such as `_Cacahuetes_ con cáscara tostado. _Trazas de frutos de cáscara_.`, it needs to be modified into `_Cacahuetes_ con cáscara tostado. Trazas de frutos de cáscara.`;

* Some percentages were badly parsed by the OCR. Since we cannot be sure about what is the right value, it is preferable to keep it as it is.

* Some percentages were badly parsed by the OCR. Since we cannot be sure about what is the right value, it is preferable to keep it as it is.

* Accents and other language specific punctuations:

* Accents and other language specific punctuations:

* In Romanian, the characters ["ş" (351), "ţ" (355)] (ASCII id)  should be retrieved by the Spellcheck when necessary,

* In Romanian, the characters ["ş" (351), "ţ" (355)] (ASCII id)  should be retrieved by the Spellcheck when necessary,

* Uppercase letters should remain unchanged => "ECOSSE" -> "ECOSSE"; "ÉCOSSE" -> "ÉCOSSE"

* Uppercase letters shouldn't be modified.

** Examples: "ECOSSE" ; "ÉCOSSE"

* If lowercase, accent should be added if missing.

* If lowercase, accent should be added if missing.

* `*` should remain in the corrected text as much as possible (*ex: Schweinefleisch\* -> Schweinefleisch\**)

* '*' shouldn't be removed.

* Whitespaces shouldn't been modified except for these cases:

** Example: "Schweinefleisch*"

* When two words are stuck to each other: *"rizbrun -> riz brun*

* Whitespaces shouldn't been modified, except in the following cases:

* Regarding uppercases and lowercases, since the spellcheck should modify at least as possible lists of ingredient, we don't modify uppercases or lowercases except for two reasons: 1. After a period: `orange.trace de...` ->  `orange. Trace de...`; 2. If it's a proper noun: `france`-> `France`

** when two words are stuck to each other.

* In French, the character `oe` or `œ` should remain unchanged after correction (*ex: œuf, bœuf). If it is missing, should be replaced by default by `œ`.

*** Example: "rizbrun" → "riz brun"

* Commas and other word separators (, -, .) should be added to distinct ingredients. **We don't add a period** or modify the existing punctuation at the end of the list of ingredients - Example: *"citric acid electrolytes (salt, magnesium and calcium chlorides, mono-potassion phosphate)"* -> *"citric acid, electrolytes (salt, magnesium and calcium chlorides, mono-potassion phosphate)"*

* Regarding uppercases and lowercases, since the spellcheck should modify at least as possible lists of ingredient, we don't modify uppercases or lowercases except for the following reasons:

* If ":" is missing to, such as `conservateur nitrite de sodium`, we add it:  `conservateur: nitrite de sodium`

** after a period: "orange.trace de..." → "orange. Trace de..."

** if it's a proper noun: "france" → "France"

* In French, the character "oe" or 'œ' should remain unchanged after correction. If it is missing, it should be replaced by default by 'œ'.

* Commas and other word separators (, -, .) should be added to distinct ingredients.

** Example: "citric acid electrolytes (salt, magnesium and calcium chlorides, mono-potassion phosphate)" → "citric acid, electrolytes (salt, magnesium and calcium chlorides, mono-potassion phosphate)"

* We don't add a period or modify the existing punctuation at the end of the list of ingredients.

* A ':' should be added in the following cases:

** it is missing between the additive category and the additive name

*** Example: "conservateur nitrite de sodium" → "conservateur: nitrite de sodium"