Changes

Jump to navigation Jump to search
Merging delta exports documentation
Line 85: Line 85:     
Beware, each line is a product and some variable need to be unnest: tidyverser::unnest_wider()
 
Beware, each line is a product and some variable need to be unnest: tidyverser::unnest_wider()
 +
 +
=== JSONL delta exports ===
 +
Every day, Open Food Facts exports all the products created during the last 24 hours. The documentation of this export can be found in the /data page.
 +
 +
If you don't have MongoDB and just want to use these delta exports to build an up-to-date database, you can merge each export with the help of <code>[https://stedolan.github.io/jq/manual/v1.6/ jq]</code> tool.
 +
 +
$ gunzip products_1638076899_1638162314.json.gz # will decompress the file
 +
$ wc -l products_1638076899_1638162314.json # will count the number of products in this export (in JSONL each line is a JSON object)
 +
$ jq -c '. + .' 2021-11-30.json products_1638162314_1638248379.json > 2021-12-01.json # merge the delta with previous complete data
    
=== JSONL export ===
 
=== JSONL export ===

Navigation menu