Difference between revisions of "Reusing Open Food Facts Data"
(First version) |
(How to reuse? section) |
||
| Line 22: | Line 22: | ||
It represents a subset of the database but it is generally fitted to the majority of usages. It's a 2.3GB file (as of April 2020), so it can't be opened by Libre Office or Excel with an 8GB machine. | It represents a subset of the database but it is generally fitted to the majority of usages. It's a 2.3GB file (as of April 2020), so it can't be opened by Libre Office or Excel with an 8GB machine. | ||
| − | ===== csvkit tips | + | == How to reuse? == |
| + | |||
| + | === CSV daily export === | ||
| + | |||
| + | ==== csvkit tips ==== | ||
[https://csvkit.readthedocs.io/en/latest/ csvkit] is a very efficient tool to manipulate huge amounts of CSV data. Here are some useful tips to manipulate Open Food Facts CSV export. | [https://csvkit.readthedocs.io/en/latest/ csvkit] is a very efficient tool to manipulate huge amounts of CSV data. Here are some useful tips to manipulate Open Food Facts CSV export. | ||
| Line 31: | Line 35: | ||
</code> | </code> | ||
| − | + | ==== Import CSV in PostGRE SQL ==== | |
See this article: https://blog-postgresql.verite.pro/2018/12/21/import-openfoodfacts.html (in french, but should be understandable with Google Translator). | See this article: https://blog-postgresql.verite.pro/2018/12/21/import-openfoodfacts.html (in french, but should be understandable with Google Translator). | ||
Revision as of 12:01, 16 April 2020
Open Food Facts data is released as Open Data: it can be reused freely by anyone, under the Open Database License (ODBL).
Where is the data
You'll find different kind of ways to get the data.
Searching for a selection of product?
The use the advanced search. The Open Food Facts advanced search feature allows to download selections of the data. See: https://world.openfoodfacts.org/cgi/search.pl
When you search is done, you will be able to download the selection, just give a try!
Searching for the whole database?
The whole database can be downloaded at https://world.openfoodfacts.org/data
It's very big. Open Food Facts hosts more than 1,200,000 products (as of April 2020). So you will probably need skills to reuse the data.
You'll be able to find there different kinds of data.
The MongoDB daily export
It represents the most complete data; it's very big and you have to know how to deal with MongoDB.
The CSV daily export
It represents a subset of the database but it is generally fitted to the majority of usages. It's a 2.3GB file (as of April 2020), so it can't be opened by Libre Office or Excel with an 8GB machine.
How to reuse?
CSV daily export
csvkit tips
csvkit is a very efficient tool to manipulate huge amounts of CSV data. Here are some useful tips to manipulate Open Food Facts CSV export.
Selecting 2 columns. Selecting two or three columns can be useful for some usages. Extracting two columns produce a smaller CSV file which can be opened by common softwares such as Libre Office or Excel. The following command creates a CSV file (brands.csv) containing two columns from Open Food Facts (code and brands). (It generally takes more than 2 minutes, depending on your computer.)
$ csvcut -t -c code,brands en.openfoodfacts.org.products.csv > brands.csv
Import CSV in PostGRE SQL
See this article: https://blog-postgresql.verite.pro/2018/12/21/import-openfoodfacts.html (in french, but should be understandable with Google Translator).