Recipe/Theory/Distribution
In order to get an idea how a recipe changes depending on the ingredients, it is necessary to have an idea on the distribution of the nutritional values. Once we have an idea of the distribution, we can estimate the deviations for each nutrient of an ingredient.
From a theoretical point of view we expect that each nutrient follows a beta distribution. The reason for this, is that the values are bounded on [0,1]. For values that are far enough away from 0 and 1, we probably will have a gaussian distribution. (earlier thoughts).
This page will do some more analysis on the distributions. For this the clean-up category of Almonds will be used.
First look
Using a Cullen and Frey graph we can find the most appropriate distribution that fits the data. For the fat percentage of almonds, this gives this graph:
One can see that there is no distribution that fits the data (red dot) well. (do not understand the bootstrap dots). The actual distribution is much more distorted.
Another way to look at this is a Q-Q plot:
In this plot we expect the values to be around the line. There is clearly a large tail at the bottom end that distorts the distribution. There are clearly issues with these products and should be flagged as thus.
Second look
A solution could be to get rid of these strange values. Using the standards approach of removing everything below and above a value based on 1.5 times the interquartile distance, we can redo the above graphs.
The Cullen and Frey graph then transforms to:
The distribution is now very close to what we expect for a Gaussian distribution (the star). Looking at the density plot:
The density plot seems to suggest that there are two Gaussian distributions, which might explain the offset in the Cullen and Frey plot.
The QQ-plot now behaves much better:
Note that the sample quantiles seem very similar at -1. I suspect that these are theoretical values taken from Ciqual or similar.
Could this be an explanation for the second peak? The Ciqual value is 52.5, which corresponds to the main peak.
The second peak can also be due to US products, which are often heavily rounded dur to the small portion size. Visual inspection of the data seems to confirm this.
Assuming a Gaussian distribution: we have for the fats:
mean: 52.96 ± 0.09; standard deviation: 2.98 ± 0.06
Other nutrients
Saturated fats
The outlier clipping did not remove the outliers sufficiently. Assuming a Gaussian distribution: we have for the proteins:
mean: 3.94576837 ± 0.01586728; standard deviation: 0.47548934 ± 0.01121986
Carbohydrates
There is a double peak. What is the origin? Assuming a Gaussian distribution: we have for the carbohydrates:
mean: 12.1578758 ± 0.2292885; standard deviation: 7.6425692 ± 0.1621315
Sugars
There is a double peak. But is does not have a large impact on the QQ. Assuming a Gaussian distribution: we have for the carbohydrates:
mean: 3.99291498 ± 0.02268404; standard deviation: 0.71301550 ± 0.01604004
Fiber
There seems to be a double peak Assuming a Gaussian distribution: we have for the fiber:
mean: 11.01403250 ± 0.07121081; standard deviation: 1.85284989 ± 0.05035364
Proteins
There seems to be a double peak. Could this be to using the same value often? Assuming a Gaussian distribution: we have for the proteins:
mean: 22.30955951 ± 0.06496679; standard deviation: 2.12213807 ± 0.04593846
Salt
There is a double peak. Probably due to salted almonds. The distributions are close to 0, so a gaussian distribution is no longer valid. Assuming a Gaussian distribution: we have for the proteins:
mean: 0.47843662 ± 0.02529823; standard deviation: 0.67409232 ± 0.01788855
Conclusions
- A gaussian distribution is good enough if the data lies far away (0.1?) from 0 (or 1);
- The OFF data to use must be really consistent in terms of processing and added ingredients. In this case we should select unsalted, shelled and blanched almonds;
- Removing the outliers well is required to get reliable standard deviations;
- Visual check of density profiles helps a lot;
- Cullen & Frey plots help to decide whether the data is clean enough for fitting;
- QQ plots serves the same purpose;