Difference between revisions of "Recipe/Theory/Distribution"

From Open Food Facts wiki
Jump to navigation Jump to search
 
(21 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
In order to get an idea how a recipe changes depending on the ingredients, it is necessary to have an idea on the distribution of the nutritional values. Once we have an idea of the distribution, we can estimate the deviations for each nutrient of an ingredient.
 
In order to get an idea how a recipe changes depending on the ingredients, it is necessary to have an idea on the distribution of the nutritional values. Once we have an idea of the distribution, we can estimate the deviations for each nutrient of an ingredient.
  
From a theoretical point of view we expect that each nutrient follows a beta distribution. The reason for this, is that the values are bounded on [0,1]. For values that are far enough away from 0 and 1, we probably will have a gaussian distribution. (there are already some wiki-pages on this. which?).
+
From a theoretical point of view we expect that each nutrient follows a beta distribution. The reason for this, is that the values are bounded on [0,1]. For values that are far enough away from 0 and 1, we probably will have a gaussian distribution. ([[Nutritional_values_probability_theory|earlier thoughts]]).
  
 
This page will do some more analysis on the distributions. For this the clean-up category of [https://world.openfoodfacts.org/category/almonds Almonds] will be used.
 
This page will do some more analysis on the distributions. For this the clean-up category of [https://world.openfoodfacts.org/category/almonds Almonds] will be used.
Line 26: Line 26:
 
The QQ-plot now behaves much better:
 
The QQ-plot now behaves much better:
 
[[File:AlmondsFatsQQPlotNoOutliers.png|thumb|center|Almonds Fats QQ-plot without outliers]]
 
[[File:AlmondsFatsQQPlotNoOutliers.png|thumb|center|Almonds Fats QQ-plot without outliers]]
Note that the sample quantiles seem very similar at -1. I suspect that these are theoretical values taken from Ciqual or similar. Could this be an explanation for the second peak? The Ciqual value is 52.5
+
Note that the sample quantiles seem very similar at -1. I suspect that these are theoretical values taken from Ciqual or similar.  
 +
 
 +
Could this be an explanation for the second peak? The Ciqual value is 52.5, which corresponds to the main peak.
 +
 
 +
The second peak can also be due to US products, which are often heavily rounded dur to the small portion size. Visual inspection of the data seems to confirm this.
 +
 
 +
Assuming a Gaussian distribution: we have for the fats:
 +
mean: 52.96 ± 0.09; standard deviation: 2.98 ± 0.06
 +
 
 +
== Other nutrients ==
 +
=== Saturated fats ===
 +
<gallery>
 +
AlmondsSatFatCullenFreyNoOutliers.png | Cullen and Frey
 +
AlmondsSatFatDensityNoOutliers.png | Density
 +
AlmondsSatFatQQQNoOutliers.png | QQ
 +
</gallery>
 +
The outlier clipping did not remove the outliers sufficiently.
 +
Assuming a Gaussian distribution: we have for the proteins:
 +
mean: 3.94576837 ± 0.01586728; standard deviation: 0.47548934 ± 0.01121986
 +
 
 +
=== Carbohydrates ===
 +
<gallery>
 +
AlmondsCarbsCullenFreyNoOutliers.png |Cullen and Frey
 +
AlmondsCarbohydratesDensity.png |Density
 +
AlmondsCarbsQQNoOutliers.png |QQ
 +
</gallery>
 +
There is a double peak. What is the origin?
 +
Assuming a Gaussian distribution: we have for the carbohydrates:
 +
mean: 12.1578758 ± 0.2292885; standard deviation: 7.6425692 ± 0.1621315
 +
 
 +
=== Sugars ===
 +
<gallery>
 +
AlmondsSugarCullenFrey.png |Cullen and Frey
 +
AlmondsSugarDensity.png |Density
 +
AlmondsSugarQQ.png |QQ
 +
</gallery>
 +
There is a double peak. But is does not have a large impact on the QQ.
 +
Assuming a Gaussian distribution: we have for the carbohydrates:
 +
mean: 3.99291498 ± 0.02268404; standard deviation: 0.71301550 ± 0.01604004
 +
 
 +
=== Fiber ===
 +
<gallery>
 +
AlmondsFiberCullenFrey.png |Cullen and Frey
 +
Almonds fiber density no outliers.png |Density
 +
Almonds fiber QQ no outliers.png |QQ
 +
</gallery>
 +
There seems to be a double peak
 +
Assuming a Gaussian distribution: we have for the fiber:
 +
mean: 11.01403250 ± 0.07121081; standard deviation: 1.85284989 ± 0.05035364
 +
 
 +
=== Proteins ===
 +
<gallery>
 +
AlmondsProteinsCullenFreyNoOutliers.png |Cullen and Frey
 +
AlmondsProteinsDensity.png |Density
 +
AlmondsProteinsQQWithoutOutliers.png |QQ
 +
</gallery>
 +
There seems to be a double peak. Could this be to using the same value often?
 +
Assuming a Gaussian distribution: we have for the proteins:
 +
mean: 22.30955951 ± 0.06496679; standard deviation: 2.12213807 ± 0.04593846
 +
 
 +
=== Salt ===
 +
<gallery>
 +
AlmondsSaltCullenFrey.png | Cullen and Frey
 +
AlmondsSaltDensity.png | Density
 +
AlmondsSaltQQ.png |QQ
 +
</gallery>
 +
There is a double peak. Probably due to salted almonds. The distributions are close to 0, so a gaussian distribution is no longer valid.
 +
Assuming a Gaussian distribution: we have for the proteins:
 +
mean: 0.47843662 ± 0.02529823; standard deviation: 0.67409232 ± 0.01788855
 +
 
 +
== Conclusions ==
 +
* A gaussian distribution is good enough if the data lies far away (0.1?) from 0 (or 1);
 +
* The OFF data to use must be really consistent in terms of processing and added ingredients. In this case we should select unsalted, shelled and blanched almonds;
 +
* Removing the outliers well is required to get reliable standard deviations;
 +
* Visual check of density profiles helps a lot;
 +
* Cullen & Frey plots help to decide whether the data is clean enough for fitting;
 +
* QQ plots serves the same purpose;
 +
 
 +
 
 +
[[Category:Recipe]]

Latest revision as of 08:50, 27 February 2025

In order to get an idea how a recipe changes depending on the ingredients, it is necessary to have an idea on the distribution of the nutritional values. Once we have an idea of the distribution, we can estimate the deviations for each nutrient of an ingredient.

From a theoretical point of view we expect that each nutrient follows a beta distribution. The reason for this, is that the values are bounded on [0,1]. For values that are far enough away from 0 and 1, we probably will have a gaussian distribution. (earlier thoughts).

This page will do some more analysis on the distributions. For this the clean-up category of Almonds will be used.

First look

Using a Cullen and Frey graph we can find the most appropriate distribution that fits the data. For the fat percentage of almonds, this gives this graph:

Almonds fats Cullen and Frey

One can see that there is no distribution that fits the data (red dot) well. (do not understand the bootstrap dots). The actual distribution is much more distorted.

Another way to look at this is a Q-Q plot:

Almonds fats quantile-quantile plot

In this plot we expect the values to be around the line. There is clearly a large tail at the bottom end that distorts the distribution. There are clearly issues with these products and should be flagged as thus.

Second look

A solution could be to get rid of these strange values. Using the standards approach of removing everything below and above a value based on 1.5 times the interquartile distance, we can redo the above graphs.

The Cullen and Frey graph then transforms to:

Almonds Fat Cullen and Frey polt with outliers removed.

The distribution is now very close to what we expect for a Gaussian distribution (the star). Looking at the density plot:

Almonds Fats Density distribution without outliers

The density plot seems to suggest that there are two Gaussian distributions, which might explain the offset in the Cullen and Frey plot.

The QQ-plot now behaves much better:

Almonds Fats QQ-plot without outliers

Note that the sample quantiles seem very similar at -1. I suspect that these are theoretical values taken from Ciqual or similar.

Could this be an explanation for the second peak? The Ciqual value is 52.5, which corresponds to the main peak.

The second peak can also be due to US products, which are often heavily rounded dur to the small portion size. Visual inspection of the data seems to confirm this.

Assuming a Gaussian distribution: we have for the fats:

mean: 52.96 ± 0.09; standard deviation: 2.98 ± 0.06

Other nutrients

Saturated fats

The outlier clipping did not remove the outliers sufficiently. Assuming a Gaussian distribution: we have for the proteins:

mean: 3.94576837 ± 0.01586728; standard deviation: 0.47548934 ± 0.01121986

Carbohydrates

There is a double peak. What is the origin? Assuming a Gaussian distribution: we have for the carbohydrates:

mean: 12.1578758 ± 0.2292885; standard deviation: 7.6425692 ± 0.1621315

Sugars

There is a double peak. But is does not have a large impact on the QQ. Assuming a Gaussian distribution: we have for the carbohydrates:

mean: 3.99291498 ± 0.02268404; standard deviation: 0.71301550 ± 0.01604004

Fiber

There seems to be a double peak Assuming a Gaussian distribution: we have for the fiber:

mean: 11.01403250 ± 0.07121081; standard deviation: 1.85284989 ± 0.05035364

Proteins

There seems to be a double peak. Could this be to using the same value often? Assuming a Gaussian distribution: we have for the proteins:

mean: 22.30955951 ± 0.06496679; standard deviation: 2.12213807 ± 0.04593846

Salt

There is a double peak. Probably due to salted almonds. The distributions are close to 0, so a gaussian distribution is no longer valid. Assuming a Gaussian distribution: we have for the proteins:

mean: 0.47843662 ± 0.02529823; standard deviation: 0.67409232 ± 0.01788855

Conclusions

  • A gaussian distribution is good enough if the data lies far away (0.1?) from 0 (or 1);
  • The OFF data to use must be really consistent in terms of processing and added ingredients. In this case we should select unsalted, shelled and blanched almonds;
  • Removing the outliers well is required to get reliable standard deviations;
  • Visual check of density profiles helps a lot;
  • Cullen & Frey plots help to decide whether the data is clean enough for fitting;
  • QQ plots serves the same purpose;