984
edits
Changes
Jump to navigation
Jump to search
Line 85:
Line 85: + + + + + + + + +
Reusing Open Food Facts Data (view source)
Revision as of 14:58, 30 November 2021
, 14:58, 30 November 2021Merging delta exports documentation
Beware, each line is a product and some variable need to be unnest: tidyverser::unnest_wider()
Beware, each line is a product and some variable need to be unnest: tidyverser::unnest_wider()
=== JSONL delta exports ===
Every day, Open Food Facts exports all the products created during the last 24 hours. The documentation of this export can be found in the /data page.
If you don't have MongoDB and just want to use these delta exports to build an up-to-date database, you can merge each export with the help of <code>[https://stedolan.github.io/jq/manual/v1.6/ jq]</code> tool.
$ gunzip products_1638076899_1638162314.json.gz # will decompress the file
$ wc -l products_1638076899_1638162314.json # will count the number of products in this export (in JSONL each line is a JSON object)
$ jq -c '. + .' 2021-11-30.json products_1638162314_1638248379.json > 2021-12-01.json # merge the delta with previous complete data
=== JSONL export ===
=== JSONL export ===