Changes

Jump to navigation Jump to search
2,785 bytes added ,  07:17, 23 October 2023
Line 1: Line 1:  +
[[Category:Taxonomies]]
 
== Introduction ==
 
== Introduction ==
 
== Why? ==  
 
== Why? ==  
Line 130: Line 131:  
== Setup ==
 
== Setup ==
 
Adding a new language for ingredients to OFF, starts with the basics: adding a lot of translations. Without this, your language will not be recognised, handles and analysed. There are two ways to add these translations.
 
Adding a new language for ingredients to OFF, starts with the basics: adding a lot of translations. Without this, your language will not be recognised, handles and analysed. There are two ways to add these translations.
=== Adding translations ====
+
=== Adding translations ===
 
The basic translation work can be done by anyone with a web-browser. You can use the available web-interface for this. For example to translate the dutch ingredients, use this link: [https://world-nl.openfoodfacts.org/ingredients?translate=add]. In this url ''world'' stands for all products in the database and ''nl'' stands for those with dutch as a main language. And the go through the provided list of ingredients. Be careful, as you might easily introduce wrong translations, which are not actually used on ingredient lists.
 
The basic translation work can be done by anyone with a web-browser. You can use the available web-interface for this. For example to translate the dutch ingredients, use this link: [https://world-nl.openfoodfacts.org/ingredients?translate=add]. In this url ''world'' stands for all products in the database and ''nl'' stands for those with dutch as a main language. And the go through the provided list of ingredients. Be careful, as you might easily introduce wrong translations, which are not actually used on ingredient lists.
   −
=== Editing translations ====
+
=== Editing translations ===
If you have experience with Github, you can edit the necessary files as well on your desktop. The ingredients are split over multiple files:
+
If you have experience with Github, you can edit the necessary files as well on your desktop. The ingredients are split over multiple files, which allows for specific handling:
 
* [https://github.com/openfoodfacts/openfoodfacts-server/blob/master/taxonomies/ingredients.txt Ingredients taxonomy] - this file contains all the normal ingredients;
 
* [https://github.com/openfoodfacts/openfoodfacts-server/blob/master/taxonomies/ingredients.txt Ingredients taxonomy] - this file contains all the normal ingredients;
 
* [https://github.com/openfoodfacts/openfoodfacts-server/blob/master/taxonomies/additives.txt Additives] - the file with all the additives;
 
* [https://github.com/openfoodfacts/openfoodfacts-server/blob/master/taxonomies/additives.txt Additives] - the file with all the additives;
 +
* [https://github.com/openfoodfacts/openfoodfacts-server/blob/master/taxonomies/allergens.txt Allergens] - the allergens required by legislation;
 +
* [https://github.com/openfoodfacts/openfoodfacts-server/blob/master/taxonomies/amino_acids.txt Amino-acids] - the amino acids, required for infant products;
 +
* [https://github.com/openfoodfacts/openfoodfacts-server/blob/master/taxonomies/minerals.txt Minerals] - the minerals required by legislation;
 +
* [https://github.com/openfoodfacts/openfoodfacts-server/blob/master/taxonomies/nucleotides.txt Nucleotides] - the nucleotides required by legislation;
 +
* [https://github.com/openfoodfacts/openfoodfacts-server/blob/master/taxonomies/other_nutritional_substances.txt Other nutritional substances] - also legislation;
 +
* [https://github.com/openfoodfacts/openfoodfacts-server/blob/master/taxonomies/vitamins.txt Vitamins] - legislation issue;
 +
You will noticed that many ingredients are combined with an adjective either in front or after the ingredient. As many adjectives will occur often, it is not necessary to add all variants to the ingredient taxonomy. This will make the taxonomy smaller and makes processing faster. Several different types of adjectives are identified:
 +
* [https://github.com/openfoodfacts/openfoodfacts-server/blob/master/taxonomies/ingredients_processing.txt Processes] - describe what has be done to an ingredient (roasted, rehydrated, etc.);
 +
* [https://github.com/openfoodfacts/openfoodfacts-server/blob/master/taxonomies/labels.txt Labels] - describe the nature (?) of the ingredient, i.e. organic, fair trade, etc;
 +
* [https://github.com/openfoodfacts/openfoodfacts-server/blob/master/taxonomies/additives_classes.txt Additive classes] - each additive can be assigned to one or more additive class, which describes the role of the additive  in the product;
 +
 +
== Maintenance ==
 +
Once the bulk of the ingredients have been added and translated, the maintenance phase begins. You will quickly see that the system identifies words and phrases as ingredients, when it should not do that. Time for finetuning and maintenance.
 +
 +
=== Detection ===
 +
Finetuning and maintenance starts with detection, detection that an ingredient is not or falsely detected.
 +
* Unknowns - start by creating a list of unknown ingredients with the url: https://nl.openfoodfacts.org/ingredients?status=unknown . This will show all products sold in the Netherlands (nl), with unknown ingredients.
 +
 +
You might find unknown ingredients (prefixed with a language code) in other languages if the main language is not the requested language.
    
=== Finetuning ===
 
=== Finetuning ===
Before starting with the maintenance action, it is first necessary to get the ingredients your language going. The next steps can be taken:
+
Finetuing allows you to tell the system, where the ingredients start and end, which words should not be taken into account, etc. You might regularly return to this task, as the system analyses more and more ingredient lists.
 
* Adapt file [https://github.com/openfoodfacts/openfoodfacts-server/blob/master/lib/ProductOpener/Ingredients.pm ingredients.pm] - this file contains various elements, which are used to analyse the ingredients of a product (in the order they appear in the file):
 
* Adapt file [https://github.com/openfoodfacts/openfoodfacts-server/blob/master/lib/ProductOpener/Ingredients.pm ingredients.pm] - this file contains various elements, which are used to analyse the ingredients of a product (in the order they appear in the file):
 
** Traces (''%may_contain_regexps'') - is the start of a phrase that introduces traces;
 
** Traces (''%may_contain_regexps'') - is the start of a phrase that introduces traces;
Line 157: Line 177:  
You might require special handling, which in turn requires specific code. Please contact the OFF'ers on Slack for that (#ingredients).
 
You might require special handling, which in turn requires specific code. Please contact the OFF'ers on Slack for that (#ingredients).
   −
== Maintenance ==
+
=== Adding ===
In this section the actions and procedures for the maintenance of the taxonomies are described. Th
  −
 
  −
=== Actions ===
   
- Finding unknown ingredients
 
- Finding unknown ingredients
 
- Adding processes
 
- Adding processes
Line 166: Line 183:  
- Adding black listed ingredients
 
- Adding black listed ingredients
 
- Language transposes
 
- Language transposes
 +
 +
== Related pages==
 +
There is more documentation that is related ingredients and the corresponding taxonomy:
 +
* [[Ingredients Events list]]
 +
[[Category:Taxonomies]]
 +
[[Category:Ingredients]]
3,393

edits

Navigation menu