Project:Open-Prices
The goal of this project is to store prices of food products and to make them available to the public, through a REST API and web interface.
The data is licensed under OdBL 1.0, images (proof) are licensed under CC-BY-SA 4.0.
Important links & documents
- Visuals & marketing assets folder for Open Prices on Canva: https://www.canva.com/folder/FAF65Zi0H8w
Data sources
We currently consider the following data sources:
- crowdsourced price data
- via a frontend application, filling in a simple form. With ideally a proof (either a store receipt, or a picture of the product with its price tag)
- via the import of past purchases, obtained through GDPR request to stores (fidelity cards :)
- prices obtained from users on the web using a web extension
- prices provided by retailers, through the API
Crowdsourcing the data collect has the advantage of mobilizing the community around the project and of getting traction and visibility. We want to reach a critical mass of data to make the project useful, and we think that crowdsourcing is the best way to achieve this goal. Scan parties in stores can be organized to collect data.
We currently don't allow scraping as a data source for a first version, as it bears some legal concerns and technical challenges to overcome:
- the database protection law in Europe may prevent us from extracting the price from the website, if we don't have the authorization of the website owner
- the price can be different depending on the user selected store. Furthermore, some retailers (in the UK for example) start to adapt their prices depending on the user profile (if they are identified as a regular customer, or if they are a new customer). This makes it difficult to have a consistent price for a product.
Current status
The project is deployed on https://prices.openfoodfacts.org. It is built with:
- a FastAPI backend to store data (Github code)
- a Vue.js frontend to add prices: no link yet (Github code)
An integration with the existing https://openfoodfacts-explorer.vercel.app application was also developped.
We're not displaying the price on Open Food Facts website or application yet, as it has to be thoroughly considered first (the website and the application attract a lot of traffic).
Challenges
- On receipt images, there may be some personal data (name, fidelity card number, etc.). We need to find a way to remove this data before publishing the image.
- Find a way to collect efficiently prices (probably through a mobile app)
🎯 Roadmap
Usage ideas
| Description | Usage |
|---|---|
| For a given product, price trends | |
| For a given product, price comparison between stores | |
| For a given product, price comparison for the same store chain (geographic price differences) | |
| For a given product, list of stores where it is sold | |
| Better compare similar products (in addition to Nutri-score, NOVA & Eco-score) - price per kilo - Should I take the national brand, the retailer brand, the budget retailer brand ? | |
| See if ingredient composition align linearly with price (eg: more meat = pricier) | |
| Support shrinkflation investigation (compare products after a change of size/price) | |
| Support inflation investigation (evolution of prices in time) | |
| See if quality labels (eg organic) have an effect on prices | |
| Leverage the other Open Food Facts databases (products, beauty, pet food) to store prices for products other than food | |
| Help gather information about which products are available in your nearby store, as a side effect of price collection | |
| Keep a history of your purchases via your uploaded receipts | |
| Help gather OpenStreetMap POI information and add missing shops |
How can I help Open Prices ?
- Adding prices
- Requesting your loyalty card data: GDPR request
- Helping document the loyalty card data process
📚 Documentation
FAQ
What happens when a shop goes out of business and gets removed from OSM?
Discussion: https://openfoodfacts.slack.com/archives/C02Q6KM7F/p1724084895885059
Is it possible to upload a pdf receipt with the new version of the OFF app?
Discussion: https://openfoodfacts.slack.com/archives/C02Q6KM7F/p1723559332138529
i've uploaded a receipt and added a couple of the items from it, but may have to finish before i've gotten to all of them - is it possible to resume at a later point or would i have to re-upload the receipt and go through it again?
Discussion: https://openfoodfacts.slack.com/archives/C02Q6KM7F/p1722881061582649
How can I find stores in Open Prices just with OSM node?
Edge cases
Recycling in Norway
In Norway the price tags for beverages packaged in PET plastic bottles don't include the recycling deposit in the price. That's usually displayed with small text separate from the main price. Because of this it can be hard to make out from product shelf photos what the total price is. Up till now I've not added up the total price of product + deposit (except for when adding receipts), but the deposit is usually the same every time based on the drink's volume. Less than 500ml is 2 NOK and more than that is 3 NOK. Should there maybe be a separate field for the recycle deposit to not confuse potential future OCR bots trying to extract prices from the images, or should I start including the deposit in the total price going forward? https://openfoodfacts.slack.com/archives/C02Q6KM7F/p1722185150520809
Public data about prices
France
- An resource I find really interesting: https://rnm.franceagrimer.fr/prix?M3027:12MOIS
France Agri Mer (a French national agency) publishes regularly prices of vegetables and fruits, in bulk or in stores.
Israel
Papers to investigate
- https://www.semantic-web-journal.net/system/files/swj3700.pdf
- https://dl.acm.org/doi/abs/10.1145/3605098.3636200
- https://www.linkedin.com/feed/update/urn:li:activity:7198558427755765760/ A maybe interesting idea: instead of the receipt, take a photo of product list + prices on the screen of self checkout automaton
Marketing & Reads
- https://www.reddit.com/r/france/comments/1cyrov7/mon_supermarch%C3%A9_me_vole_%C3%A9pisode_2/
- https://www.ft.com/content/0ea5410f-da90-42dc-8be4-62bbdbd47762?accessToken=zwAAAY-qfgM4kc8OpUEP2pBC3NOL5GK729R3Yg.MEYCIQD3lF9nUmbAqCEp-Qw5uY3S22fYpzH0is-xkluZ9V_8SAIhAKvUGDKvM5oPHuel4ACDMVT1qWblekCH4HQF4qCaN-O5&segmentId=e95a9ae7-622c-6235-5f87-51e412b47e97&shareType=enterprise&shareId=1984a8fa-da57-40e2-a531-46926df0db05
Get in touch
|
|---|
- Slack channel: #prices
📆 Weekly meeting on Wesdnesdays
- Weekly meeting: every Wednesday at 2PM (CET), feel free to join if you're interested in the project: https://meet.google.com/oin-hiqp-tmd
🎨 Design & Mockups
- Visuals & marketing assets folder for Open Prices on Canva: https://www.canva.com/folder/FAF65Zi0H8w
- Mockups on Figma:
Timeline
| Date | Description |
|---|---|
| 08/11/2023 |