Line 1: |
Line 1: |
− | [[Category:OpenWineFacts]] | + | [[Project:ProductOpener]] > [[Product Opener documentation]] |
− | <pre>
| + | [[Category:ProductOpener]] |
− | <en:Alcoholic beverages
| + | == Introduction == |
− | en:Wines
| |
− | es:Vinos
| |
− | fr:Vins, vin
| |
| | | |
− | <en:Beverages
| + | Global taxonomies are used in Product Opener to "tag" products. For instance Open Food Facts uses taxonomies for categories, countries, labels, additives etc. |
− | en:Non-Alcoholic beverages
| |
− | fr:Boissons sans alcool, boisson sans alcool
| |
| | | |
− | <en:Wines
| + | The taxonomies are multilingual (each tag value can have translations in different languages) and hierarchical (tags have parents, and applying a tag to a product also applies the parent tags). |
− | <en:Non-Alcoholic beverages
| |
− | en:Non-Alcoholic wines, Alcohol-free wines
| |
− | fr:Vins sans alcool, vins déalcoolisés
| |
| | | |
− | <en:Wines
| + | == Taxonomy files == |
− | en:Bordeaux, Bordeaux wines
| |
− | fr:Bordeaux
| |
| | | |
− | <en:Wines
| + | Taxonomies are kept in the taxonomies directory: |
− | en:French wines, Wines from France
| |
− | fr:Vins français, vin français
| |
| | | |
− | <en:French wines | + | <pre> |
− | fr:Vins de pays, vin de pays
| + | stephane@ks3095298:~/product-opener/taxonomies$ ls |
| + | additives.result.sto categories.result.txt countries.txt |
| + | additives.result.txt categories.txt labels.result.sto |
| + | additives.txt countries.result.sto labels.result.txt |
| + | categories.result.sto countries.result.txt labels.txt |
| + | </pre> |
| | | |
− | <en:Wines
| + | * categories.txt is the source definition of the categories taxonomy |
− | en:Red wines
| + | * categories.result.txt is the resulting normalized definition after compilation in the same format as the source categories.txt |
− | fr:vins rouges, vin rouge
| + | ** entries are normalized (e.g. the en:value is preferred to specify parents) |
| + | ** entries can be sorted in a different order |
| + | * categories.result.sto in the resulting binary representation of the taxonomy, in Perl's native format |
| | | |
− | <en:Wines
| + | == Taxonomy source format == |
− | en:White wines
| |
− | fr:vins blancs, vin blanc
| |
| | | |
− | <en:Wines
| + | See [[Global taxonomies]] |
− | en:Wines from Italy, Italian wines
| |
− | fr:Vins italiens, vin italien, vins d'Italie, vin d'Italie
| |
| | | |
− | <en:Wines
| + | == Taxonomy source files == |
− | en:Wines from Portugal, Portuguese wines
| |
− | fr:Vins portugais, Vins du Portugal, Vin portugais, Vin du Portugal
| |
| | | |
− | <en:Wines from Portugal
| + | Open Food Facts taxonomy source files are on the wiki so that contributors can improve them (adding tags, adding translations, reorganizing tags): http://en.wiki.openfoodfacts.org/Global_taxonomies |
− | pt:Denominação de Origem Controlada
| |
| | | |
− | <pt:Denominação de Origem Controlada
| + | == Compilation == |
− | pt:Vinho verde (DOC)
| |
| | | |
− | <pt:Denominação de Origem Controlada
| + | Before taxonomies can be used in Product Opener, they need to be compiled in a binary format. The compilation has two goals: |
− | pt:Porto (DOC)
| |
| | | |
− | <pt:Denominação de Origem Controlada
| + | # Generate as many synonyms as possible for each tag |
− | pt:Douro (DOC)
| + | #* This is so that end users can enter any form of a tag and still have it matched to the canonical tag |
| + | #* e.g. "Yaourts fraise" and "Yaourt aux fraises" |
| + | # Compute the hierarchy |
| + | #* for each tag, knowing which tags are parents and childrens |
| + | |
| + | The compilation code is in the build_tags_taxonomy() function of Tags.pm. This function is called by script/build_tags_taxonomy.pl |
| | | |
− | <pt:Denominação de Origem Controlada
| + | === Compilation steps === |
− | pt:Távora Varosa (DOC)
| |
| | | |
− | <pt:Denominação de Origem Controlada
| + | The compilation steps are: |
− | pt:Dão (DOC)
| + | * read translations and synonyms |
| + | * compute synonyms for each tag (using synonyms for some of the words) |
| + | * add more synonyms (remove stopwords, generate simple singular/plural forms) |
| + | * compute the hierarchy |
| | | |
− | <pt:Denominação de Origem Controlada
| + | Note: if the taxonomy is big and/or contain many synoym |
− | pt:Bairrada (DOC)
| |
| | | |
− | <pt:Denominação de Origem Controlada
| + | === Compilation script === |
− | pt:Beira Interior (DOC)
| |
| | | |
− | <pt:Denominação de Origem Controlada | + | ==== Testing ==== |
− | pt:Encostas de Aire (DOC)
| + | <pre> |
| + | stephane@ks3095298:~/product-opener/scripts$ ./build_tags_taxonomy.pl categories |
| + | </pre> |
| + | This will compile the categories but generate only categories.results.txt from categories.txt |
| | | |
− | <pt:Denominação de Origem Controlada
| + | You should review the resulting txt to make sure unwanted synonyms or relations were not created. |
− | pt:Alentejo (DOC)
| |
| | | |
− | <pt:Denominação de Origem Controlada
| + | ===== empty names : ===== |
− | pt:Ribatejo (DOC)
| |
| | | |
− | <pt:Denominação de Origem Controlada
| + | An empty name " : " indicates there is an error in the taxonomy source file, usually a non-existing entry was entered as the parent. |
− | pt:Lourinhã (DOC)
| |
| | | |
− | <pt:Denominação de Origem Controlada | + | <pre> |
− | pt:Alenquer (DOC)
| + | < : |
| + | < en:Wheat flours |
| + | en:Wheat malt flours |
| + | es:Harinas de trigo malteadas, Harinas de malta de trigo |
| + | fr:Farines de malt de blé |
| + | </pre> |
| | | |
− | <pt:Denominação de Origem Controlada
| + | Check the source file to see which entry is wrong: |
− | pt:Torres Vedras (DOC)
| |
| | | |
− | <pt:Denominação de Origem Controlada | + | <pre> |
− | pt:Arruda (DOC)
| + | <en:Wheat flours |
| + | <fr:Malt flours |
| + | en:Wheat malt flours |
| + | es:Harinas de trigo malteadas, Harinas de malta de trigo |
| + | fr:Farines de malt de blé |
| + | </pre> |
| | | |
− | <pt:Denominação de Origem Controlada
| + | Here "fr:Malt flours" is non existing (and should be "en:Malt flours"). |
− | pt:Bucelas (DOC)
| |
| | | |
− | <pt:Denominação de Origem Controlada
| + | Note: the build_tags_taxonomy.pl script has been updated to report those errors. |
− | pt:Colares (DOC)
| |
| | | |
− | <pt:Denominação de Origem Controlada
| + | ==== Publishing ==== |
− | pt:Carcavelos (DOC)
| |
| | | |
− | <pt:Denominação de Origem Controlada | + | <pre> |
− | pt:Palmela (DOC)
| + | stephane@ks3095298:~/product-opener/scripts$ ./build_tags_taxonomy.pl categories publish |
| + | </pre> |
| + | This will also store categories.result.sto |
| | | |
− | <pt:Denominação de Origem Controlada
| + | == Test == |
− | pt:Muscat de Setúbal (DOC)
| |
| | | |
− | <pt:Denominação de Origem Controlada
| + | Taxonomies are very powerful tools, but with great powers comes great risks. The synonyms, stopwords etc. can create unwanted synonyms. |
− | pt:Lagos (DOC)
| |
| | | |
− | <pt:Denominação de Origem Controlada
| + | === Testing tools === |
− | pt:Portimão (DOC)
| |
| | | |
− | <pt:Denominação de Origem Controlada
| + | We need testing tools. Please create some. :-) |
− | pt:Lagoa (DOC)
| |
| | | |
− | <pt:Denominação de Origem Controlada
| + | == Restart Apache == |
− | pt:Tavira (DOC)
| |
| | | |
− | <pt:Denominação de Origem Controlada | + | <pre> |
− | pt:Madère (DOC)
| + | </pre> |
| | | |
| + | After restarting Apache, editing a product will recompute all tags using the newest published taxonomies. |
| | | |
− | <en:Wines
| |
− | en:Wines from Spain, Spanish wines
| |
− | fr:Vins espagnols, Vin espagnol, Vin d'Espagne, Vins d'Espagne
| |
| | | |
− | <en:Wines
| + | == Updating products == |
− | fr:Vins américains
| |
| | | |
− | <fr:Vins américains
| + | The scripts directory contains scripts that will recompute tags for all products in the database. |
− | en:Wines from California, Californian Wines, California Wines
| |
− | fr:Vins californiens, vins de Californie, vin californien, vin de Californie
| |
− | wikidata:en:Q1134713
| |
− | | |
− | <en:Wines
| |
− | fr:Vins effervescents
| |
− | en:Sparkling wines
| |
− | wikidata:en:Q321263
| |
− | | |
− | <fr:Vins effervescents
| |
− | fr:Blanquette de Limoux
| |
− | wikidata:en:Q881685
| |
− | | |
− | <fr:Vins effervescents
| |
− | fr:Champagne, champagnes, vins de Champagne
| |
− | en:Champagne, champagnes
| |
− | | |
− | <fr:Vins effervescents
| |
− | fr:Clairette de Die, Clairettes de Die
| |
− | wikidata:en:Q1094848
| |
− | | |
− | <fr:Vins effervescents
| |
− | fr:vins mousseux, vin mousseux, mousseux
| |
− | | |
− | <fr:Vins effervescents
| |
− | fr:vins perlant, vin perlant, perlé
| |
− | | |
− | <fr:Vins effervescents
| |
− | fr:vins pétillant, vin pétillant, pétillant
| |
− | | |
− | <en:Wines
| |
− | fr:Vins mutés, vin muté
| |
− | en:Fortified wines
| |
− | wikidata:en:Q722338
| |
− | | |
− | <en:Wines from Portugal
| |
− | <fr:Vins mutés de type vin doux naturel
| |
− | fr:Porto, vins de Porto, vin de Porto
| |
− | | |
− | <en:Red wines
| |
− | <fr:Porto
| |
− | en:red ports
| |
− | fr:porto rouges, porto rouge
| |
− | | |
− | <en:Red ports
| |
− | fr:porto réductifs, porto réductif
| |
− | | |
− | <fr:Porto réductifs
| |
− | fr:Porto Crusted, Crusted
| |
− | | |
− | <fr:Porto réductifs
| |
− | fr:Porto Late Bottled Vintage, LBV
| |
− | | |
− | <fr:Porto réductifs
| |
− | fr:Porto Ruby
| |
− | | |
− | <fr:Porto réductifs
| |
− | fr:Porto Single Quinta Vintage
| |
− | | |
− | <fr:Porto réductifs
| |
− | fr:Porto Vintage, porto millésimé
| |
− | | |
− | <en:Red ports
| |
− | fr:portos oxydatifs, porto oxydatif
| |
− | | |
− | <fr:Portos oxydatifs
| |
− | fr:Porto Tawny, tawny
| |
− | | |
− | <en:White wines
| |
− | <fr:Porto
| |
− | en:White Ports
| |
− | fr:porto blancs, porto blanc
| |
− | | |
− | en:Sweet wines, natural sweet wines
| |
− | es:Vinos dulces
| |
− | fr:Vins doux, Vin doux, Vins doux naturels, Vin doux naturel
| |
− | | |
− | <en:Sweet wines
| |
− | <fr:Vins mutés
| |
− | fr:vins mutés de type vin doux naturel, VDN
| |
− | | |
− | <fr:Vins mutés de type vin doux naturel
| |
− | fr:Banyuls
| |
− | | |
− | <fr:Vins mutés de type vin doux naturel
| |
− | fr:Maury
| |
− | | |
− | <fr:Vins mutés de type vin doux naturel
| |
− | fr:Muscats, Muscat
| |
− | | |
− | <fr:Muscats
| |
− | fr:Muscat de Beaumes-de-Venise, Muscats de Beaumes-de-Venise
| |
− | | |
− | <fr:Muscats
| |
− | fr:Muscat de Frontignan, Muscats de Frontignan
| |
− | | |
− | <fr:Muscats
| |
− | fr:Muscat de Lunel, Muscats de Lunel
| |
− | | |
− | <fr:Muscats
| |
− | fr:Muscat de Mireval, Muscats de Mireval
| |
− | | |
− | <fr:Muscats
| |
− | fr:Muscat de Rivesaltes, Muscats de Rivesaltes
| |
− | | |
− | <fr:Muscats
| |
− | fr:Muscat de Saint-Jean-de-Minervois, Muscats de Saint-Jean-de-Minervois
| |
− | | |
− | <fr:Muscats
| |
− | fr:Muscat du Cap-Corse, Muscats du Cap-Corse
| |
− | | |
− | <fr:Vins mutés de type vin doux naturel
| |
− | fr:Rasteau
| |
− | | |
− | <fr:Vins mutés de type vin doux naturel
| |
− | fr:Rivesaltes
| |
− | | |
− | <fr:Vins mutés
| |
− | fr:mistelle, vins mutés de types mistelle, vins de liqueur, vin de liqueur
| |
− | | |
− | <fr:Mistelle
| |
− | fr:Pineau des Charentes, Floc de Gascogne, Macvin du Jura
| |
− | | |
− | <fr:Vins mutés
| |
− | fr:vins vinés, vin viné
| |
− | | |
− | <en:Wines
| |
− | fr:Vins rosés, vin rosé
| |
| | | |
| + | <pre> |
| + | stephane@ks3095298:~/product-opener/scripts$ ./update_all_products_categories.pl |
| </pre> | | </pre> |