Difference between revisions of "Global stores taxonomy"

From Open Food Facts wiki
Jump to navigation Jump to search
 
(54 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
See [[Global taxonomies]] for instructions.
 
See [[Global taxonomies]] for instructions.
 +
 +
The taxonomy was moved to GitHub: https://github.com/openfoodfacts/openfoodfacts-server/blob/master/taxonomies/stores.txt
 
[[Category:Global_Taxonomies]]
 
[[Category:Global_Taxonomies]]
 +
[[Category:Stores]]
 +
== Introduction ==
 +
The stores taxonomy contains a list of locations where products (food products, beauty products, etc.) can be obtained. A related store brands taxonomy describes the brand names (store chains) of the individual stores.
  
== Current status ==
+
Most of the data related to these taxonomies is obtained through third parties (Wikidata, OpenStreetMap). The main interest of OFF is the relation between products, stores and brands.
 
 
Draft
 
== Note ==
 
  
* We need to associate stores with brands that are sold in thoses stores (Auchan for Auchan…)
+
== Store entries ==
* We should try to add Wikidata relations (and thus OSM ones) whenever possible
+
An entry in the taxonomy describes a single location. This taxonomy can combine data pulled from various sources: receipts from prices, osm, wikidata, etc or provided by the user (honour system). OFF should be careful not copy to much data already maintained by other parties. And if data is copied, it is to provide a snapshot in time.
 +
=== Receipts ===
 +
[[File:Example receipt heading.png|100 px|thumb|right|Example receipt heading]]
 +
The basis could be the receipts that users have added on prices. As all OFF data, any information should be backed up by image proofs. In the case for a store, this could be a receipt. The store information available on the receipt can be extracted to the taxonomy. What can be extracted depends on the receipt. In the example shown here, it is possible to extract the store name, the operator (useful for franchises), the address, phone number, siret, naf and tva. Analysus of more receipts might reveal other country specific information.
 +
Hopefully this information is enough to unambiguously identify a store.
  
<pre>
+
=== Prices ===
# French stores
+
The stores taxonomy can be based on [https://prices.openfoodfacts.org/ OFF Prices]. The receipts corresponding to a shop are entered on Prices. Prices has a unique identifier for each shop, which should be used as a linking pin. In addition Prices offers a link with Openstreetmap, which can be used to pull in additional information. Some of this information is already cached by Prices.
fr:Auchan
+
=== Open Street Map ===
 +
All stores are linked to [https://www.openstreetmap.org/ Open Street Map] (OSM) through the unique OSM id. The link refers to a basic object in OSM:
 +
* Node - is used to refer to a single location, like [https://www.openstreetmap.org/node/25213337 BP Service];
 +
* (Closed) Way - is used to refer to a building, like [https://www.openstreetmap.org/way/31348922 IKEA];
 +
* Relation - is used to refer multiple ways used by the shop, like [https://www.openstreetmap.org/relation/1235874 Carrefour];
  
fr:A 2 Pas
+
Using a Way to designate a shopping location might lead to inconsistencies, like [https://www.openstreetmap.org/way/4447066 this] actual location. Which shop on the street is meant? Using a relation might lead to inconsistencies as well, like [https://www.openstreetmap.org/relation/7868535 this] actual location. This can be limited to OSM objects that have the tag ''shop''.
  
fr:Carrefour
+
Several of the fields are pulled from OSM (name, postcode, city, country, lat, lon). These need to be regularly updated, so that OSM is followed. Any deleted location should be kept.
  
fr:Carrefour Market
+
=== Example Taxonomy Entry===
 +
An example for an entry / location / store / shop in the taxonomy:<br>
 +
<nowiki>store_brand:en: en:Biocoop</nowiki><br>
 +
<nowiki>price_id:en: 45</nowiki><br>
 +
<nowiki>osm_id:en: 9815975601</nowiki><br>
 +
<nowiki>osm_type:en: NODE</nowiki><br>
 +
<nowiki>osm_display_name:en: en:"Biocoop, Rue de Tolbiac, Quartier de la Maison-Blanche, Paris 13e Arrondissement, Paris, Île-de-France, France métropolitaine, 75013, France"</nowiki><br>
 +
<nowiki>osm_address_postcode:en: 75013</nowiki><br>
 +
<nowiki>osm_address_city:en: en:Paris</nowiki><br>
 +
<nowiki>osm_address_country:en: en:france</nowiki><br>
 +
<nowiki>osm_lat:en: 48.8259597</nowiki><br>
 +
<nowiki>osm_lon:en: 2.3511541</nowiki><br>
 +
== Store brand entries ==
 +
The current implementation of OFF already supports a field for Stores. It is a free text field where the user can enter whatever s/he wants. A variety of entries is the consequence. But most of all it is used to enter the value for the Store brand, i.e. the name of the store as is shown outside the store and on receipts.
 +
=== Fields ===
 +
* store-brand - the name of the store as found on receipts
 +
* wikidata - the entry on wikidata that contains more information on the supermarket chain.
 +
* countries - a list of countries where the chain currently operates (has stores). This information can be pulled from wikidata.
 +
* expiration-date-country - some kind of date when store no longer has any stores in a specific country. For instance user chain [https://www.wikidata.org/wiki/Q925132 Dia] used to be active in France and OFF still has products for this.
 +
=== Comments ===
 +
Note that this can also be imported from OSM by means of the receipts. On OSM it is indicated with the tag ''brand''.
  
fr:Carrefour City
+
OSM also indicates a link to wikidata (for instance [https://www.wikidata.org/wiki/Q151954 Lidl]).
type:en:City center store
 
  
fr:Carrefour Contact
 
  
fr:Monoprix
+
=== Example Taxonomy Entry ===
 +
<nowiki>en:store-brand: en:Biocoop</nowiki>
 +
<br>
 +
<nowiki>countries:en: en:france</nowiki>
 +
<br>
 +
<nowiki>wikidata:en: Q2904039</nowiki>
  
fr:Monop'
+
== Lifecycle ==
 +
The lifecycle for the two taxonomies is related, so they will be discussed together.
 +
=== Kickstart ===
 +
The kickstart describes the initial, very first introduction of the taxonomies.
 +
==== Store brands ====
 +
The store brands taxonomy could be started with the current entries in the Stores field. Unfortunately the quality of these current entries seems rather low. It is preferred to start with a high quality list pulled in from external sources, like all [https://www.wikidata.org/wiki/Q18043413 supermarket] chains listed on Wikidata.
  
fr:Casino
+
As a subsequent step all existing entries could be mapped to this store brands taxonomy.
  
fr:Leader Price
+
==== Stores ====
 +
The stores taxonomy can initially be pulled from OFF Prices. The data only needs to be formatted in the correct way. The stores information provided by OFF Prices will then be used as a starting point.
  
fr:Picard
+
=== Maintenance ===
 +
==== Editing ====
 +
Editing involves changing existing items in the taxonomies.
 +
==== Extending ====
 +
Extending involves adding new items to the taxonomies through all possible applications.
 +
* Stores Taxonomy
 +
** OFF Prices - new stores can be found through new location_id and can be double checked against OSM id;
 +
** Productopener - use OSM id to check whether something is new;
 +
** API - use OSM id to check whether something is new;
 +
* Store Brands Taxonomy
 +
** OFF Prices - new store brands via OSM and wikidata;
 +
** Productopener - new store brands via OSM and wikidata;
 +
** API - new store brands via OSM and wikidata;
  
fr:Thiriet
+
== Spinoff ==
 +
Information on stores, operators and owners is not the main goal of OFF. OFF prefers that this is maintained by more specialised other parties. However OFF does gather some information through receipts for instance (full address, phone number, opening hours, operator, siret, tva, naf). Users are encouraged to add this kind of information to OpenStreetMap or wikidata.
  
fr:Naturalia
+
== Use cases ==
 +
* OFF Prices - To have the prices of products for individual stores and brands.
 +
* https://github.com/danslimmon/oscar Throw out a package, add to shopping list, potentially you can do an online order/click and collect (because the store * is known, and has wikidata, and a url)
 +
* Walk into an area with a geofence - https://github.com/alltheplaces/alltheplaces / open streetmap - have home assistant push a notification to get the three items from that Wikidata:brand store that are on your list
 +
* I need (uncommon ingredient or food); from the places it is sold which is closest to me?
 +
*I scanned an item. I'm standing inside an OSM shop/supermarket at (geofence). It has a Wikidata:brand. Would I like to add this as the store that sells this food?
 +
* Provides a way via name suggestion index to get a clear logo for a given brand of it is a store brand with more data captured in open food facts, Provides some insight into supply chains
 +
=== Robotoff ===
 +
* We currently maintain a handmade list of stores that can be safely inferred from OCR and automatically applied: https://github.com/openfoodfacts/robotoff/blob/main/data/ocr/store_regex.txt
  
fr:Lidl
+
== Discussion points ==
country:en:France,en:Germany
 
wikidata:en:Q151954
 
  
fr:Leclerc, E-Leclerc
+
* What information should be stored by OFF and what can be left to OSM or Wikidata?
 +
* What to be done with shops that go out of business?
  
fr:Leclerc Drive
+
== Note ==
 
+
* We need to associate stores with brands that are sold in thoses stores (Auchan for Auchan…)
fr:Intermarché
+
* We should try to add Wikidata relations (and thus OSM ones) whenever possible
 
+
* Chains - if a store is part of a chain with multiple stores it can be added through a wikidata link
fr:Franprix
 
 
 
fr:Cora
 
 
 
fr:Bio C' Bon
 
 
 
fr:Magasins U
 
 
 
fr:Hyper U
 
 
 
fr:U Express
 
type:en:City center store
 
 
 
fr:Super U
 
 
 
fr:La vie claire
 
 
 
# Belgique
 
 
 
fr:Delhaize
 
 
 
fr:Colruyt
 
 
 
# Spanish stores
 
 
 
es:Dia
 
 
 
es:El Corte Ingles
 
 
 
# English Stores
 
 
 
en:Woolworth's
 
 
 
en:Tesco
 
 
 
en:Sainsbury's
 
 
 
en:Sainsbury's Local
 
 
 
en:Sainsbury's Online
 
 
 
en:Marks and Spencer
 
 
 
en:Waitrose
 
 
 
en:Morrisons
 
 
 
# German stores
 
 
 
de:Aldi
 
wikidata:en:Q125054
 
#geographically seperated
 
 
 
de:Aldi Nord
 
wikidata:en:Q41171373
 
 
 
de:Aldi Süd
 
wikidata:en:Q41171672
 
 
 
de:Alnatura
 
wikidata:en:Q876811
 
 
 
de:BackWerk
 
wikidata:en:Q798298
 
#self serving bakery
 
 
 
de:Café Kranzler
 
wikidata:en:Q119168
 
#listed in database as as Cafe-kranzler
 
 
 
de:dm-drogerie markt
 
wikidata:en:Q266572
 
#listed in database as Dm, Dm-drogeriemarkt
 
 
 
de:Edeka
 
wikidata:en:Q701755
 
 
 
de:Getränke Hoffmann
 
wikidata:en:Q19284021
 
#listed in database as Getranke-hofman
 
 
 
de:Hussel
 
wikidata:en:Q17123688
 
 
 
de:Ikea
 
wikidata:en:Q54078
 
 
 
de:Kaiser's Tengelmann, Kaiser's, Tengelmann
 
wikidata:en:Q897187
 
#listed in database as Kaisers, Kaiser-s
 
 
 
de:Karstadt
 
wikidata:en:Q182910
 
#listed in database as Karstadt-lebensmittel, Karstadt-lebensmittel-rewe
 
 
 
de:Kaufland
 
wikidata:en:Q685967
 
 
 
de:Lidl
 
wikidata:en:Q151954
 
#listed in database as Lidl-deutschland
 
 
 
de:Metro
 
 
 
de:Migros
 
wikidata:en:Q680727
 
#not to be confused with Migross
 
 
 
de:Müller
 
wikidata:en:Q1958759
 
#listed in database as Muller
 
 
 
de:Netto
 
wikidata:en:Q552652
 
#some products could also be wrongly added for Q879858
 
 
 
de:Netto Marken-Discount
 
wikidata:en:Q879858
 
#listed in database as Netto-marken-discount
 
 
 
de:Norma
 
wikidata:en:Q450180
 
 
 
de:Penny
 
wikidata:en:Q284688
 
 
 
de:Real
 
wikidata:en:Q698473
 
 
 
de:Reformhaus Engehardt
 
#reformhaus-engelhardt.de
 
#listed in database as Reformhaus-engelhardt
 
 
 
de:Reichelt
 
wikidata:en:Q1709874
 
 
 
de:Rewe
 
wikidata:en:Q16968817
 
 
 
de:Rossmann
 
wikidata:en:Q316004
 
#listed in database as (wrongly) Rossman
 
 
 
</pre>
 

Latest revision as of 09:52, 16 August 2024

See Global taxonomies for instructions.

The taxonomy was moved to GitHub: https://github.com/openfoodfacts/openfoodfacts-server/blob/master/taxonomies/stores.txt

Introduction

The stores taxonomy contains a list of locations where products (food products, beauty products, etc.) can be obtained. A related store brands taxonomy describes the brand names (store chains) of the individual stores.

Most of the data related to these taxonomies is obtained through third parties (Wikidata, OpenStreetMap). The main interest of OFF is the relation between products, stores and brands.

Store entries

An entry in the taxonomy describes a single location. This taxonomy can combine data pulled from various sources: receipts from prices, osm, wikidata, etc or provided by the user (honour system). OFF should be careful not copy to much data already maintained by other parties. And if data is copied, it is to provide a snapshot in time.

Receipts

Example receipt heading

The basis could be the receipts that users have added on prices. As all OFF data, any information should be backed up by image proofs. In the case for a store, this could be a receipt. The store information available on the receipt can be extracted to the taxonomy. What can be extracted depends on the receipt. In the example shown here, it is possible to extract the store name, the operator (useful for franchises), the address, phone number, siret, naf and tva. Analysus of more receipts might reveal other country specific information. Hopefully this information is enough to unambiguously identify a store.

Prices

The stores taxonomy can be based on OFF Prices. The receipts corresponding to a shop are entered on Prices. Prices has a unique identifier for each shop, which should be used as a linking pin. In addition Prices offers a link with Openstreetmap, which can be used to pull in additional information. Some of this information is already cached by Prices.

Open Street Map

All stores are linked to Open Street Map (OSM) through the unique OSM id. The link refers to a basic object in OSM:

  • Node - is used to refer to a single location, like BP Service;
  • (Closed) Way - is used to refer to a building, like IKEA;
  • Relation - is used to refer multiple ways used by the shop, like Carrefour;

Using a Way to designate a shopping location might lead to inconsistencies, like this actual location. Which shop on the street is meant? Using a relation might lead to inconsistencies as well, like this actual location. This can be limited to OSM objects that have the tag shop.

Several of the fields are pulled from OSM (name, postcode, city, country, lat, lon). These need to be regularly updated, so that OSM is followed. Any deleted location should be kept.

Example Taxonomy Entry

An example for an entry / location / store / shop in the taxonomy:
store_brand:en: en:Biocoop
price_id:en: 45
osm_id:en: 9815975601
osm_type:en: NODE
osm_display_name:en: en:"Biocoop, Rue de Tolbiac, Quartier de la Maison-Blanche, Paris 13e Arrondissement, Paris, Île-de-France, France métropolitaine, 75013, France"
osm_address_postcode:en: 75013
osm_address_city:en: en:Paris
osm_address_country:en: en:france
osm_lat:en: 48.8259597
osm_lon:en: 2.3511541

Store brand entries

The current implementation of OFF already supports a field for Stores. It is a free text field where the user can enter whatever s/he wants. A variety of entries is the consequence. But most of all it is used to enter the value for the Store brand, i.e. the name of the store as is shown outside the store and on receipts.

Fields

  • store-brand - the name of the store as found on receipts
  • wikidata - the entry on wikidata that contains more information on the supermarket chain.
  • countries - a list of countries where the chain currently operates (has stores). This information can be pulled from wikidata.
  • expiration-date-country - some kind of date when store no longer has any stores in a specific country. For instance user chain Dia used to be active in France and OFF still has products for this.

Comments

Note that this can also be imported from OSM by means of the receipts. On OSM it is indicated with the tag brand.

OSM also indicates a link to wikidata (for instance Lidl).


Example Taxonomy Entry

en:store-brand: en:Biocoop
countries:en: en:france
wikidata:en: Q2904039

Lifecycle

The lifecycle for the two taxonomies is related, so they will be discussed together.

Kickstart

The kickstart describes the initial, very first introduction of the taxonomies.

Store brands

The store brands taxonomy could be started with the current entries in the Stores field. Unfortunately the quality of these current entries seems rather low. It is preferred to start with a high quality list pulled in from external sources, like all supermarket chains listed on Wikidata.

As a subsequent step all existing entries could be mapped to this store brands taxonomy.

Stores

The stores taxonomy can initially be pulled from OFF Prices. The data only needs to be formatted in the correct way. The stores information provided by OFF Prices will then be used as a starting point.

Maintenance

Editing

Editing involves changing existing items in the taxonomies.

Extending

Extending involves adding new items to the taxonomies through all possible applications.

  • Stores Taxonomy
    • OFF Prices - new stores can be found through new location_id and can be double checked against OSM id;
    • Productopener - use OSM id to check whether something is new;
    • API - use OSM id to check whether something is new;
  • Store Brands Taxonomy
    • OFF Prices - new store brands via OSM and wikidata;
    • Productopener - new store brands via OSM and wikidata;
    • API - new store brands via OSM and wikidata;

Spinoff

Information on stores, operators and owners is not the main goal of OFF. OFF prefers that this is maintained by more specialised other parties. However OFF does gather some information through receipts for instance (full address, phone number, opening hours, operator, siret, tva, naf). Users are encouraged to add this kind of information to OpenStreetMap or wikidata.

Use cases

  • OFF Prices - To have the prices of products for individual stores and brands.
  • https://github.com/danslimmon/oscar Throw out a package, add to shopping list, potentially you can do an online order/click and collect (because the store * is known, and has wikidata, and a url)
  • Walk into an area with a geofence - https://github.com/alltheplaces/alltheplaces / open streetmap - have home assistant push a notification to get the three items from that Wikidata:brand store that are on your list
  • I need (uncommon ingredient or food); from the places it is sold which is closest to me?
  • I scanned an item. I'm standing inside an OSM shop/supermarket at (geofence). It has a Wikidata:brand. Would I like to add this as the store that sells this food?
  • Provides a way via name suggestion index to get a clear logo for a given brand of it is a store brand with more data captured in open food facts, Provides some insight into supply chains

Robotoff

Discussion points

  • What information should be stored by OFF and what can be left to OSM or Wikidata?
  • What to be done with shops that go out of business?

Note

  • We need to associate stores with brands that are sold in thoses stores (Auchan for Auchan…)
  • We should try to add Wikidata relations (and thus OSM ones) whenever possible
  • Chains - if a store is part of a chain with multiple stores it can be added through a wikidata link