Difference between revisions of "Data quality stats"

From Open Food Facts wiki
Jump to navigation Jump to search
(+ Data quality category)
(+ dashboard link)
 
Line 4: Line 4:
 
These data are stored daily and can be requested here: https://mirabelle.openfoodfacts.org/off-stats/data_quality_stats
 
These data are stored daily and can be requested here: https://mirabelle.openfoodfacts.org/off-stats/data_quality_stats
  
You can check a [https://mirabelle.openfoodfacts.org/off-stats?sql=select+%28year+%7C%7C+%27-%27+%7C%7C+month+%7C%7C+%27-%27+%7C%7C+day%29+as+date%2C+country%0D%0A%2C+max%28cast%28total_nb_of_products+as+numeric%29%29+as+total_nb_of_products%0D%0A%2C+max%28cast%28products_with_errors+as+numeric%29%29+as+pr_with_errors%0D%0A%2C+round%28max%28cast%28products_with_errors+as+numeric%29%2A1.0%29%2Fmax%28cast%28total_nb_of_products+as+numeric%29%2A1.0%29%2A100%2C2%29+as+percent%0D%0A%2C+max%28cast%28products_w_issues_but_no_image+as+numeric%29%29+as+pr_w_issues_but_no_image%0D%0A%2C+max%28cast%28products_wo_category+as+numeric%29%29+as+pr_wo_category%0D%0A%2C+max%28cast%28products_wo_ingredients+as+numeric%29%29+as+pr_wo_ingredients%0D%0A%2C+max%28cast%28products_wo_nutrition_facts+as+numeric%29%29+as+pr_wo_nutrition_facts%0D%0A%2C+max%28cast%28products_wo_packaging_data+as+numeric%29%29+as+pr_wo_packaging_data%0D%0Afrom%0D%0A%28%0D%0A++select+a1.year%2C+a1.month%2C+a1.day%2C+a1.country%0D%0A++%2C+case+when+a1.property+%3D%3D+%22total_nb_of_products%22+then+a1.value%0D%0A++end+as+total_nb_of_products%0D%0A++%2C+case+when+a1.property+%3D%3D+%22products_with_errors%22+then+a1.value%0D%0A++end+as+products_with_errors%0D%0A++%2C+case+when+a1.property+%3D%3D+%22products_w_issues_but_no_image%22+then+a1.value%0D%0A++end+as+products_w_issues_but_no_image%0D%0A++%2C+case+when+a1.property+%3D%3D+%22products_wo_category%22+then+a1.value%0D%0A++end+as+products_wo_category%0D%0A++%2C+case+when+a1.property+%3D%3D+%22products_wo_ingredients%22+then+a1.value%0D%0A++end+as+products_wo_ingredients%0D%0A++%2C+case+when+a1.property+%3D%3D+%22products_wo_nutrition_facts%22+then+a1.value%0D%0A++end+as+products_wo_nutrition_facts%0D%0A++%2C+case+when+a1.property+%3D%3D+%22products_wo_packaging_data%22+then+a1.value%0D%0A++end+as+products_wo_packaging_data%0D%0A++from+data_quality_stats+a1%0D%0A%29%0D%0Awhere+true%0D%0Aand+%28%2F%2Acountry+%3D+%22world%22+or%2A%2F+country+REGEXP+%22world%7Cfr%7Ces%7Cde%7Cit%7Cuk%7Cus%22%29%0D%0Aand+date+%3E+%222022-12-01%22%0D%0Agroup+by+year%2C+month%2C+day%2C+country%0D%0Alimit+900&_hide_sql=1#g.mark=line&g.x_column=date&g.x_type=temporal&g.y_column=percent&g.y_type=quantitative&g.color_column=country more readable version for main european countries here].
+
You can check a [https://mirabelle.openfoodfacts.org/off-stats?sql=select+%28year+%7C%7C+%27-%27+%7C%7C+month+%7C%7C+%27-%27+%7C%7C+day%29+as+date%2C+country%0D%0A%2C+max%28cast%28total_nb_of_products+as+numeric%29%29+as+total_nb_of_products%0D%0A%2C+max%28cast%28products_with_errors+as+numeric%29%29+as+pr_with_errors%0D%0A%2C+round%28max%28cast%28products_with_errors+as+numeric%29%2A1.0%29%2Fmax%28cast%28total_nb_of_products+as+numeric%29%2A1.0%29%2A100%2C2%29+as+percent%0D%0A%2C+max%28cast%28products_w_issues_but_no_image+as+numeric%29%29+as+pr_w_issues_but_no_image%0D%0A%2C+max%28cast%28products_wo_category+as+numeric%29%29+as+pr_wo_category%0D%0A%2C+max%28cast%28products_wo_ingredients+as+numeric%29%29+as+pr_wo_ingredients%0D%0A%2C+max%28cast%28products_wo_nutrition_facts+as+numeric%29%29+as+pr_wo_nutrition_facts%0D%0A%2C+max%28cast%28products_wo_packaging_data+as+numeric%29%29+as+pr_wo_packaging_data%0D%0Afrom%0D%0A%28%0D%0A++select+a1.year%2C+a1.month%2C+a1.day%2C+a1.country%0D%0A++%2C+case+when+a1.property+%3D%3D+%22total_nb_of_products%22+then+a1.value%0D%0A++end+as+total_nb_of_products%0D%0A++%2C+case+when+a1.property+%3D%3D+%22products_with_errors%22+then+a1.value%0D%0A++end+as+products_with_errors%0D%0A++%2C+case+when+a1.property+%3D%3D+%22products_w_issues_but_no_image%22+then+a1.value%0D%0A++end+as+products_w_issues_but_no_image%0D%0A++%2C+case+when+a1.property+%3D%3D+%22products_wo_category%22+then+a1.value%0D%0A++end+as+products_wo_category%0D%0A++%2C+case+when+a1.property+%3D%3D+%22products_wo_ingredients%22+then+a1.value%0D%0A++end+as+products_wo_ingredients%0D%0A++%2C+case+when+a1.property+%3D%3D+%22products_wo_nutrition_facts%22+then+a1.value%0D%0A++end+as+products_wo_nutrition_facts%0D%0A++%2C+case+when+a1.property+%3D%3D+%22products_wo_packaging_data%22+then+a1.value%0D%0A++end+as+products_wo_packaging_data%0D%0A++from+data_quality_stats+a1%0D%0A%29%0D%0Awhere+true%0D%0Aand+%28%2F%2Acountry+%3D+%22world%22+or%2A%2F+country+REGEXP+%22world%7Cfr%7Ces%7Cde%7Cit%7Cuk%7Cus%22%29%0D%0Aand+date+%3E+%222022-12-01%22%0D%0Agroup+by+year%2C+month%2C+day%2C+country%0D%0Alimit+900&_hide_sql=1#g.mark=line&g.x_column=date&g.x_type=temporal&g.y_column=percent&g.y_type=quantitative&g.color_column=country more readable version for main European countries here].
 +
 
 +
Based on these data, we also publish a [https://mirabelle.openfoodfacts.org/-/dashboards/data-quality-dashboard dashboard related to data quality and data completeness].
  
 
=== Ingredients analysis ===
 
=== Ingredients analysis ===
 
See: [[Ingredients Analysis Quality]]
 
See: [[Ingredients Analysis Quality]]
 
[[Category:Data quality]]
 
[[Category:Data quality]]

Latest revision as of 11:08, 29 June 2023

We're computing and historize daily some data quality stats.

Data quality errors and data completeness

These data are stored daily and can be requested here: https://mirabelle.openfoodfacts.org/off-stats/data_quality_stats

You can check a more readable version for main European countries here.

Based on these data, we also publish a dashboard related to data quality and data completeness.

Ingredients analysis

See: Ingredients Analysis Quality