Changes

Jump to navigation Jump to search
2,422 bytes added ,  13:34, 26 September 2018
Line 90: Line 90:     
==== Guiding principles ====
 
==== Guiding principles ====
* Primary language  
+
* ''Ingredient list guides'' - the entries as found on ingredient lists are the basis and guide the taxonomy. So no entries from other places. (an exception is translation bootstrapping)
 +
* '''Primary language''' - the first language of an ingredient should be in english, if it is available in that language. Otherwise use the language where the ingredient occurs the most (at the moment that will probably be french);
 +
* '''No assumption''' - if in doubt about a translation or an assignment or whatever, keep the ingredient separate. Make no assumption about an ingredient. If it specified vaguely, keep it vaguely;
 
==== Steps ====
 
==== Steps ====
 
To build the taxonomy from raw data, the follow steps need to be taken
 
To build the taxonomy from raw data, the follow steps need to be taken
* '''Gathering''' - the first step is gathering the ingredients from the ingredient lists. This results in a list of  ''raw ingredients''. This process needs to be described @stephane does this;
+
* '''Gather''' - the first step is gathering the ingredients from the ingredient lists. This results in a list of  ''raw ingredients''. This process needs to be described @stephane does this;
* '''Assigning''' - the raw ingredients can be mapped unto the existing taxonomy, in order to find ingredients that are new and not yet part of the taxonomy (the ''new raw ingredients'');
+
* '''Assign''' - the raw ingredients can be mapped unto the existing taxonomy, in order to find ingredients that are new and not yet part of the taxonomy (the ''new raw ingredients'');
* '''Merging''' - each new raw ingredient must be looked at and if possible merged with an ingredient in the existing taxonomy;
+
* '''Merge''' - each new raw ingredient must be looked at and if possible merged with an ingredient in the existing taxonomy. This also involves merging translations;
* '''Keying''' - if a new raw ingredient can be mapped to the taxonomy, it will probably be a new synonym of an existing entry. Ideally one should check which of the synonyms occurs the most and set that one as main ingredient name;
+
* '''Key''' - if a new raw ingredient can be mapped to the taxonomy, it will probably be a new synonym of an existing entry. Ideally one should check which of the synonyms occurs the most and set that one as main ingredient name;
 +
* '''Create''' - if the ingredient is totally new, a new entry can be created. It should be decided what the super ingredient is (if any). It can then be entered in that part of the taxonomy.
 +
* ''Translate'' - try to find the translations of the ingredient. Either by searching translations in OFF. For instance [https://world.openfoodfacts.org/ingredient/fr:polydextrose/languages fr:polydextrose] finds the languages available and try to use the translations offered;
 +
* '''Define''' - add a definition from [http://wikipedia.org wikipedia] and [http://wikidata.org wikidata] if available.
 +
* '''OFF''' - add a link to the corresponding ingredients page. For instance for [https://world.openfoodfacts.org/ingredient/fr:polydextrose/ fr:polydextrose].
 +
* '''Occurences''' - note how often the ingredient occurs, in how many language and at what date that was determined. This might require an advanced search ([https://world.openfoodfacts.org/cgi/search.pl?action=process&tagtype_0=ingredients&tag_contains_0=contains&tag_0=polydextrose&sort_by=unique_scans_n&page_size=20&axis_x=energy&axis_y=products_n&action=display polydextrose]). This might allow us to track if changes are necessary;
 +
==== Watch out ====
 +
It is possible to make mistakes in these steps.
 +
* '''Wrong translation''' - the ingredient lists in different languages available on products, do not always offer a translations.
 +
* '''Translation bootstrapping''' - in order to get up to speed with the translations, we use the Wikipedia and Wikidata to find translations. This can result in the wrong translations as it can be something different than what is found on the package. So be careful for wikipedia disambiguation strings, latin species names, etc. In the long run these should be superseded by what is fond on the products.
    
== Ontology building ==
 
== Ontology building ==
3,390

edits

Navigation menu