Changes

Jump to navigation Jump to search
m
Link
Line 136: Line 136:     
=== MongoDB dump ===
 
=== MongoDB dump ===
The MongoDB dump needs to be reused with MongoDB. It allows building a full replication of the Open Food Facts database and use MongoDB for selecting, filtering and exporting data. Using MongoDB allows faster manipulations compared to the other methods.
+
The [https://world.openfoodfacts.org/data MongoDB dump] needs to be reused with MongoDB. It allows building a full replication of the Open Food Facts database and use MongoDB for selecting, filtering and exporting data. Using MongoDB allows faster manipulations compared to the other methods.
    
First, you '''need a running MongoDB installation'''. Open Food Facts is using MongoDB 4.4. It has been reported that prior version should not work for Open Food Facts dump.
 
First, you '''need a running MongoDB installation'''. Open Food Facts is using MongoDB 4.4. It has been reported that prior version should not work for Open Food Facts dump.
Line 208: Line 208:  
# Open Food Facts database contains hundreds of fields.
 
# Open Food Facts database contains hundreds of fields.
   −
# An easy way to list them all is to use "variety" Schema Analyzer
+
# An easy way to list them all is to use "variety" Schema Analyzer:
 +
# https://github.com/variety/variety
    
# 1. Install "variety"
 
# 1. Install "variety"
Line 222: Line 223:  
time mongo off --eval "var collection = 'products', limit = 10000" variety.js > off_schema_10000.txt
 
time mongo off --eval "var collection = 'products', limit = 10000" variety.js > off_schema_10000.txt
 
# (3 minutes)
 
# (3 minutes)
 +
 +
time mongo off --eval "var collection = 'products', limit = 100000" variety.js > off_schema_100000.txt
 +
# (75 minutes)
 +
 +
time mongo off --eval "var collection = 'products'" variety.js > off_schema_all.txt
 +
# (more than two days)
       
</pre>
 
</pre>

Navigation menu