Changes

Jump to navigation Jump to search
no edit summary
Line 92: Line 92:  
*Estampille sanitaire est délivrée par autorité du pays.
 
*Estampille sanitaire est délivrée par autorité du pays.
 
*Les entreprises obtenant une estampille sanitaire sont organisées par Sections (entreposage, abattage volaille ...). Un travail de normalisation de ces sections sur l'ensemble des pays européens est en cours par les équipes de OFF.
 
*Les entreprises obtenant une estampille sanitaire sont organisées par Sections (entreposage, abattage volaille ...). Un travail de normalisation de ces sections sur l'ensemble des pays européens est en cours par les équipes de OFF.
==Traçabilité des fruits et légumes==
+
 
*France: http://www.fruits-et-legumes.net/revue_en_ligne/point_sur/fich_pdf/PStra%C3%A7abilite.PDF
+
==Methodology to extract data==
*US:http://www.motherjones.com/environment/2014/09/fruit-veggies-produce-origins-trade-secret
+
===General===
===PLU Code===
+
====Get Data Files====
*Système volontaire (non obligatoire) codifiant le type de produit (ex banane cavendish) mais ne donne aucune information sur la provenance
+
* All data (url list and data files) can be found here https://github.com/openfoodfacts/eu-food-data
*Lookup database: http://plucodes.com/
+
**There is a general csv file with the general link to the data repository for each country: https://github.com/openfoodfacts/eu-food-data/blob/master/list-eu-and-partner-countries.csv
*Liste mondiale et régionale des codes PLU: http://www.plucodes.com/docs/PLU_List_Commodity_Varietal_Bilingual.pdf
+
**There are individual folders for each target country (specific url-list for each European Agreement Section and data files)
*Table des codes PLU Français/Anglais: https://slack-files.com/T02KVRT1Q-F02M8KZKV-9d592d
+
* A google sheet document is used to map all files available in the target countries. It also map the section name for every country in its own language (or translation in English) and the related European Section, which is used as a general taxonomy. This Google Sheet can be found here: https://docs.google.com/spreadsheets/d/1egdo58Ds8PNi5G_4F2UtWOWC1V0k3tXBgPhZXs5FRqM/edit?usp=sharing
*Collection d'étiquettes de fruits et légumes:http://legufrulabelofolie.fr/index.php
+
*We can usually find txt or csv file but for some countries the data is only available in PDF. Those need OCR treatment before data extraction.
 +
====Build CSV files====
 +
*Several formats are used in EU countries. A specific approach is needed for each of them. Refer below for details for each country.
 +
====Geocode====
 +
*Script geocoding + google maps
 +
 
 +
===France===
 +
*I just build a script that takes all the french agreement info from Agriculture Ministry and concatenate them in one file. Next step is to do the same for UK. The step after that is to cleverly agregate the duplicates (some companies have several health agreements under the same agreement number)
 +
https://github.com/openfoodfacts/eu-food-data/blob/master/scripts/FR-script.py
 +
This script use this file to get the list of URL to retrieve https://github.com/openfoodfacts/eu-food-data/blob/master/fr/urls-fr.txt
 +
*First work performed on
 +
===UK===
 +
*As UK is divided in 4 regions (Ireland, England, Wales and Scotland) and because they have different file format, we use a 3-file script
 +
https://github.com/openfoodfacts/eu-food-data/blob/master/scripts/UK-urls.txt => all UK urls
 +
https://github.com/openfoodfacts/eu-food-data/blob/master/scripts/UK-methods.txt => list which method to use depending on the file type
 +
https://github.com/openfoodfacts/eu-food-data/blob/master/scripts/UK-script.py => the script itself
 +
===DE===
 +
*@vince has per
 +
==Inspiration==
 +
*http://free.sourcemap.com/
 +
 
     

Navigation menu