Changes

Jump to navigation Jump to search
2,563 bytes added ,  11:17, 28 December 2020
Line 167: Line 167:  
Possible origins are now [https://world.openfoodfacts.org/category/olive-oils-from-france France] (30 products), [https://world.openfoodfacts.org/category/olive-oils-from-greece Greece] (43 products) and [https://world.openfoodfacts.org/category/olive-oils-from-italy Italy] (49 products). We should add at least Tunisia, Maroc, Algeria, Spain, Argentina and South Africa to these.
 
Possible origins are now [https://world.openfoodfacts.org/category/olive-oils-from-france France] (30 products), [https://world.openfoodfacts.org/category/olive-oils-from-greece Greece] (43 products) and [https://world.openfoodfacts.org/category/olive-oils-from-italy Italy] (49 products). We should add at least Tunisia, Maroc, Algeria, Spain, Argentina and South Africa to these.
   −
In addition the countries can be subdivided into regions, which correspond to official PDO's. We can slowly add the [https://www.oliveoilmarket.eu/origins/europian-pdos/ PDO's] as the appear in OFF. Only 72 olive oils have a PDO label. There are a lot if labels missing, as there are more products in the combined PDO categories.
+
In addition the countries can be subdivided into regions, which correspond to official PDO's. Only 72 olive oils have a PDO label. Many olive oils PDO's seem to be defined as categories, but not yet well integrated in the olive oils category.
   −
It is unclear whether these subcategories exhibit also differences in nutritional values. Before we can determine that we need more categorisations.
+
It is unclear whether these subcategories exhibit also differences in nutritional values. Before we can determine that we need more products in each PDO category.
    
=== New categories ===
 
=== New categories ===
By looking more closely at all the products we can identify other categories. Only if there are a lot of products in each category or if the nutritional values deviate to much, it is worthwhile to create these new products
+
By looking more closely at all the products we can identify other categories. Only if there are a lot of products in each category (or required if the nutritional values of these products deviate to much), it is worthwhile to create these new categories:
 
* pure olive oils: the current olive oils category should be renamed to pure olive oils to indicate that these products contain only one ingredient. This helps also to distinguish from the other olive oils.
 
* pure olive oils: the current olive oils category should be renamed to pure olive oils to indicate that these products contain only one ingredient. This helps also to distinguish from the other olive oils.
* [https://world.openfoodfacts.org/cgi/search.pl?action=process&tagtype_0=categories&tag_contains_0=contains&tag_0=olive%20oil%20sprays&sort_by=unique_scans_n&page_size=20 olive oil sprays] for olive oil contain other ingredients to make it sprayable. As the serving is only 0.25g, the nutritional values per serving are all zero (thanks to rounding).
+
* [https://world.openfoodfacts.org/cgi/search.pl?action=process&tagtype_0=categories&tag_contains_0=contains&tag_0=olive%20oil%20sprays&sort_by=unique_scans_n&page_size=20 olive oil sprays] contain other ingredients to make it sprayable. As the serving is only 0.25g, the nutritional values per serving are all zero (thanks to rounding).
 
* enhanced olive oils: these olive oils have added vitamins for children(?)
 
* enhanced olive oils: these olive oils have added vitamins for children(?)
 
* [https://world.openfoodfacts.org/cgi/search.pl?action=process&search_terms=flavoured%20olive%20oils&tagtype_0=categories&tag_contains_0=contains&tag_0=flavoured%20olive%20oils&sort_by=unique_scans_n&page_size=20 flavoured olive oils]: these olive oils have added flavours (garlic, etc.)
 
* [https://world.openfoodfacts.org/cgi/search.pl?action=process&search_terms=flavoured%20olive%20oils&tagtype_0=categories&tag_contains_0=contains&tag_0=flavoured%20olive%20oils&sort_by=unique_scans_n&page_size=20 flavoured olive oils]: these olive oils have added flavours (garlic, etc.)
 
* Olive oil blends: some oils are a blend of refined and virgin olive oils. Or a blend of virgin or extra virgin oils.
 
* Olive oil blends: some oils are a blend of refined and virgin olive oils. Or a blend of virgin or extra virgin oils.
* Unfiltered olive oils: some oils show that they are unfiltered. This might be another category.
+
* Unfiltered olive oils: some oils indicate that they are unfiltered. Does this imply a lower fat percentage?
    
=== NOVA ===
 
=== NOVA ===
Line 191: Line 191:     
== Normalisation ==
 
== Normalisation ==
As show in the previous section the Olive Oils category can be split into 3 subgroups, based on how the nutritional values are reported:
+
As show in the previous section the Olive Oils category can be split into 3 (or more) subgroups, based on how the nutritional values are reported on the package:
 
* per 100g
 
* per 100g
 
* per 100ml
 
* per 100ml
Line 197: Line 197:  
And maybe there is even a fourth group: per weight serving.
 
And maybe there is even a fourth group: per weight serving.
   −
If we know how the nutritional data for a product is reported, we can normalise that data. And with the normalised data we have a consistent dataset, which can be used to get the real nutritional values.
+
=== Formula's ===
=== Categorisation ===
+
If we know how the nutritional data for a product is reported, we can normalise that data. And with the normalised data we have a consistent dataset, which can be used to get the real nutritional values. We will normalise on the per 100g values with the formula:
The first step to categorise each product into each of the groups.
     −
==== Volumetric serving ====
+
    N<sub>100g</sub> = N<sub>100ml</sub> * sg
The ''per volumetric serving group'' is the easiest. We need to know the type of nutritional table that has been used to extract the data from. Unfortunately this is not registered, so we need to find a proxy for this.
     −
We could use the serving size field for this: if it contains ml, it might be taken from a per serving nutritional table. I have seen exceptions however. Unfortuately the web-interface does not allow me to search on that.
+
In which:
 +
* N<sub>100g</sub> is the nutritional value per 100g
 +
* N<sub>100ml</sub> is the nutritional value per 100ml
 +
* sg the specific gravity of olive oils, for which I use 0.911
   −
A better proxy is the existence of the transfat field. If there is data in that field, it is most likely a US or Canadian style nutritional table. Again this not a guarantee as other countries mark transfat as well. This results in 724 products. However some 60 products of this transfat sample have fat percentages around 100%, so we have false positives. We can add a fat limit of 94%, resulting in 629 products.
+
The conversion from a volumetric serving to 100ml is:
 +
  N<sub>100ml</sub> = N<sub>serving</sub> * 100 / size<sub>serving</sub>
   −
The US/Canadian style nutritional tables do not show the energy in kJoule. Thus if there is a value in that field, it is not US/Canadian. This results in 746 products. This is a low number, inducating that the field has not been filled in very well.
+
in which:
 +
* N<sub>100ml</sub> is the nutritional value per 100 ml
 +
* N<sub>serving</sub> is the nutritional value per serving
 +
* size<sub>serving</sub> the serving size in ml
 +
 
 +
From the histograms shown earlier, it is clear that the data does not correspond to these formula's. We need to add a correction factor, which accounts for rounding errors to account for the real serving size.
 +
 
 +
Thus:
 +
 
 +
  N<sub>100g</sub> = N<sub>serving</sub> * 100 * sg * C / size<sub>serving</sub>
 +
 
 +
or
 +
 
 +
  N<sub>100g</sub> = N<sub>serving</sub> * 100 * sg / size<sub>real serving</sub>
 +
 
 +
with C the added correction factor, which corrects the serving size:
 +
  size<sub>real serving </sub> = size<sub>serving</sub> / C
 +
We assume that C has the same value for all nutritional values.
 +
 
 +
=== Groups ===
 +
The first step to categorise each product into each of the groups. The group that has nutritional values per 100g will left as it is, so we need to concentrate only on the two other groups.
 +
 
 +
==== Volumetric serving group ====
 +
The ''per volumetric serving'' group is the easiest. Any product that has a US style nutritional table falls in this group. Unfortunately this is not registered, so we need to find a proxy for this table.
 +
 
 +
We could use the serving size field for this: if it contains 15 ml, it is probably taken from a per serving nutritional table. I have seen exceptions however. Unfortunately the web-interface does not allow me to search on that. So this is not an option.
 +
 
 +
A better proxy is the existence of the transfat field. If there is data in that field, it is most likely from an US or Canadian style nutritional table. Again this not a guarantee as other countries mark transfat as well. This results in 724 products.
 +
 
 +
However some 60 products of this transfat sample have fat percentages around 100%, so we have false positives. We can add a fat limit of 94%, i.e. any product that has fat percentage less than 94% belongs to this group. With this we exclude the products that probably have the nutritional values listed per 100g. This second filter results in 629 products.
    
==== Per 100 ml ====
 
==== Per 100 ml ====
Defining this group of products is more difficult. We have only the fat percentage itself as indication. As the fat percentage can not be larger than 100%, we can use the specific gravity (91.1%) as limit. This assumes all oils have the same specific gravity.
+
Defining this group of products is more difficult. We only have the fat percentage itself to go on. Assuming a fat percentage of 100% and a specific gravity of 0.911, the listed fat percentage should be lower than 91.1 gram (per 100g) for products that have a nutritional table per 100 ml.
   −
Looking at some products, the canonical rounded value is 91%, but higher values are seen as well. So we could take a limit 92%. This results in 1475 products.  
+
Looking at some products, the canonical rounded value is 91g, but higher values are seen as well. So we could take a limit 92%. This will result in 1475 products.  
   −
This will lead to some false postives. So it is better to remove first the US-products and then extract these groups.
+
There will be an overlap with the products with a US-stye nutritional table, so it is better to subtract these US-products first.
   −
=== Normalisation ===
+
==== Summary ====
We need to convert the volumetric data to weight data. For this we can use the correction factors, which we determined earlier:
+
This grouping procedure results in group sizes of:
* per 100 ml: divide by 0.911
+
* per 100g : 94 products
* per 15 ml: divide by 0.933
+
* per 100ml: 2235 products
 +
* per vol. serving: 636 products
    
=== Calculations ===
 
=== Calculations ===
In order to get correct values, the data has been exported in CSV-format. Imported in Numbers, pruned (all non-relevant data removed), scaled by the normalisation factors, merged into one table, and finally plotted and averaged.
+
In order to normalise the data, it has been exported in XLSX-format. Imported in Numbers, pruned (all non-relevant data removed), scaled by the normalisation factors, merged into one table, and finally plotted and averaged.
 
  −
Only the specific gravity needed to be adapted in order to get only a few fat percentages below 100%
      
=== Results ===
 
=== Results ===
  −
==== Grouping results ====
  −
For making the three groups
  −
  −
This resulted in group sizes of:
  −
* for 100g : 94 products
  −
* for 100ml: 235 products
  −
* for serving: 636 products
      
==== Distributions ====
 
==== Distributions ====
Line 241: Line 263:     
===== Fat =====
 
===== Fat =====
[[File:OliveOilsFatNormalisedDistribtion20201217.png|thumb|center|Distribution of normalised fat percentage (17-dec-2020)]]
+
[[File:OliveOilsFatNormalisedDistribution20201224.png|thumb|center|Normalised fat percentage distribution (24-dec-2020)]]
The normalisation has worked well. All three distributions have their maximum at 100%. The largest group of 100ml has a tail below the 100%. This might be due to filtration or different specific gravities. I need to redo the graph: the x-axis label is wrong.
+
If we use the standard formulas, we get quite some olive oils with a fat percentage larger than 100%. These olive oils belong to the ''per 100ml''-group. A correction of 1% of the specific gravity, i.e. 0.920 instead of 0.911, is enough to have a maximum at 100%.
 +
 
 +
There are quite some products below the 100% line. Lowering the specific gravity by a few promille, is sufficient to increase the fat percentage. So are these really different specific gravities?
   −
===== ° Saturated fat =====
+
There is also a group of products that have a specific gravity of 101.4%. What is up with these?
The distribution spans some 4%. Normalisation seems to have worked well.
  −
[[File:OliveOilsSaturatedFatNormalised20201217.png|thumb|center|Distribution on normalised saturated fat percentage (17-dec-2020)]]
     −
===== Mono-unsaturated fat =====
+
A closer look reveals that many of these products come from the USDA import or have 1 Tbsp serving size. So they belong to the per-serving-group, but as they lack the nutritional values that such products should have, like transfat, cholesterol, etc. So how can I recognise these? The original per serving data is not available, only the 100 ml data. This can be seen from the values that are a result of conversion, like 93.333333. This is the same fat percentage that we see in the per-serving group. Thus if a product has a fat percentage between 92.9 and 93.6, it should belong to the per serving group.
The range of mono-unsaturated fat spans some 10%. It seems there are two peaks visible in the three groups.
+
[[File:OliveOilsNormalisedFatDistributionThree20201226.png|thumb|center|Olive oils corrected normalised fat distribution (26-dec-2020)]]
[[File:OliveOilsMonosaturatedFatNormalised20201217.png|thumb|center|Distribution of normalised mono-unsaturated fat percentage (17-dec-2020)]]
+
The corrected normalised fat distribution is shown in the graph above. Now some 1400 products have a fat percentage of 100%. Every product that falls the 100% either have indeed less than 100% fat, or are the victim of rounding errors.
 +
 
 +
There still lie a few products under 98% in the per 100ml group. These products have a fat percentage of 90g per 100ml. Is this a rounding issue?
   −
===== Energy (kcal) =====
+
===== Saturated fat =====
[[File:OliveOilsEnergyKcalNormalised20201217.png|thumb|center|Distribution of normalised Energy (kcal) (17-dec-2020)]]
+
The distribution spans some 4%. The corrected normalisation seems to have worked well.
The three peaks seem to indicate that there are three groups. The peak seen around 857 kcal seems a off. We would expect it to be around 900 kcal. Products in this peak belong to 100g group and the per serving group. It looks like the normalisation (which was based on the fats), did not work out well. So what happened?
+
[[File:OliveOilsSaturatedFatNormalisedCorrected20201226.png|thumb|center|Corrected normalised saturated fat distribution (26-dec-2020)]]
   −
Th 100g-group products seem to come from the USDA import, but the data was per 100g and not per serving as you would expect for this import. So the normalisation was already done, but wrong.
+
===== Mono-unsaturated fat =====
 +
The range of mono-unsaturated fat spans some 10% with a peak at 71.5% and at 78.5%. The two peaks are present in all four groups. No explanation occurs to me.
   −
The canonical energy (kcal) value per serving is 120 kcal. And the canonical energy (kcal) value for the 100g-group is 900 kcal, i.e. 7.5 times larger. Where could this 7.5 factor come from? The conversion to 100 ml plus de conversion to gram, only provides a factor of 7.14. Hence the value of 857 kcal, which we see in the distribution.
+
[[File:OliveOilsMonoUnsaturatedFatDistributionNormalisedCorrected20201227.png|thumb|center|Corrected normalised mono-unsaturated fat distribution (27-dec-2020)]]
   −
I wonder whether this difference is due to rounding issues. The US style nutritional table likes to have a simple serving size: 1 tablespoon, which translates to 15ml. But are tablespoons in de US measured in milliliters? It says so on the packages. If we calculate the tablespoon-size from 120 and 900 kcal, we get a size of 14.63 ml, i.e. rounded 15 ml. We better use this value for normalisation.
+
===== Energy (kcal) =====
 +
[[File:OliveOilsEnergyKcalNormalisedCorrected20201227.png|thumb|center|Corrected normalised kcal-energy distribution (27-dec-2020)]]
 +
The distribution shows three peaks:
 +
* 856 kcal combines the serving, 100g and the serving in 100ml group
 +
* 870 kcal
 +
* 895 kcal
   −
Or 123 kcal per 15 ml would have been more accurate.
+
The corrected normalisation has well worked out for the serving group, but not at all for the 100g group. We expect that the peak at 856 is the correct one. So what should the correction factor be if we want the peaks to match?
 +
[[File:OliveOilsKcalEnergyDistributionFudged20201227.png|thumb|center|Fudged kcal-energy distribution (27-dec-2020)]]
 +
The peak is roughly correct with a factor 1.055 instead of the 1.01 used earlier. No idea where this comes from.
    
==== Correlations ====
 
==== Correlations ====
Line 301: Line 333:  
| Poly-unsaturated fat (g) || 7.1 || 11.2 || 14.3
 
| Poly-unsaturated fat (g) || 7.1 || 11.2 || 14.3
 
|}
 
|}
 +
==== Accuracy and Variation ====
 +
How accurate are the listed nutritional values? And has the apparent variation any meaning?
 +
 +
Many products use the canonical values as indication for their nutritional values, like 3700kJ or 900 kcal. These are probably average values. Note that the listed accuracy is one or two significant digits. This means that it is not worthwhile to add more significant digits. Normally this implies that the variation is ± 50 kcal or 50 kJ (200 kJ?). This corresponds to the spread of values seen in the graphs above.
 +
 +
One can wonder about the usefulness of products that display the nutritional values with four or five significant digits. Are those values just a reflection of statistical errors, actual differences in olive oils (year, olive, region?)
    
== OFF Conclusions and thoughts ==
 
== OFF Conclusions and thoughts ==
3,393

edits

Navigation menu