75
edits
Changes
Jump to navigation
Jump to search
Line 226:
Line 226: − + + +
→Build CSV files
* A google sheet document is used to map all files available in the target countries. It also map the section name for every country in its own language (or translation in English) and the related European Section, which is used as a general taxonomy. This Google Sheet can be found here: https://docs.google.com/spreadsheets/d/1egdo58Ds8PNi5G_4F2UtWOWC1V0k3tXBgPhZXs5FRqM/edit?usp=sharing
* A google sheet document is used to map all files available in the target countries. It also map the section name for every country in its own language (or translation in English) and the related European Section, which is used as a general taxonomy. This Google Sheet can be found here: https://docs.google.com/spreadsheets/d/1egdo58Ds8PNi5G_4F2UtWOWC1V0k3tXBgPhZXs5FRqM/edit?usp=sharing
*We can usually find txt or csv file but for some countries the data is only available in PDF. Those need OCR treatment before data extraction.
*We can usually find txt or csv file but for some countries the data is only available in PDF. Those need OCR treatment before data extraction.
===Build CSV files===
===Build CSV/TXT files===
*Several formats are used in EU countries. A specific approach is needed for each of them. Refer below for details for each country.
*Several formats are used in EU countries. A specific approach is needed for each of them. Refer below for details for each country.
*For PDF files, use Tesseract and this Python script https://github.com/virantha/pypdfocr
===Geocode===
===Geocode===
*Script geocoding + google maps
*Script geocoding + google maps