Open Food Facts Days 2024/splitting things up workshop

From Open Food Facts wiki
Revision as of 12:41, 23 September 2024 by Teolemon (talk | contribs)
Jump to navigation Jump to search

Small survey with questions:

  • How did you came to OFF (what did you want to do)
  • What did you then discover, want to do

We tried to group answers.

Analysis:

some clusters:

  • support open data (publishing / capturing)
  • data analysis
    • ecology
    • nutrition
  • community
  • technical contribution
  • prices
  • Open Data is about raw data (eg. Prices).

Nutrition and ecology are about analysing data, we might have common interfaces for those.

Community is about collaboration, and communication.

We then put tools with those dimensions:

  • community: nutri-patrol / web content
  • community & open data: data quality
  • open data / raw data: facts / database / api / pictures / labels / taxonomies (but it's also interpretative, not fully raw)
  • analysis:
    • ecology: eco interpretation of taxonomy (eg. agrobalyse)
    • nutrition: taxonomy, nutriscore, nova,
  • prices: Open Prices (with in fact all the dimensions)
  • Some other dimensions:
  • data capture: producer platform:
  • data rendering / visualisation: search-a-licious, personal scoring, knowledge panels definition
  • Mobile app and web site have those two dimensions.

Open prices is on all the dimensions indeed.

There are different stages on data:

  • raw data eg. image or contribution (and the source)
  • OCR and so on to get more data
  • ML etc. to extract
  • We speak about data duplication, is it ok, or do we need only one database:
  • it's better to split sensitive information for example
  • duplication is ok if you have a single source of trust for each information
  • off-query and data-warehouse is also a good way to have cross analysis (removing sensible data for example)
  • We spoke about latency if you have many components.

There are technics to takle that like having events and going more on asynchronous rendering.

Something are quite difficult to replicate on different services.

When a product is at early stage (like open price), it's ok to mix things in a single product,

but keeping a good architecture to be able to split it later.

How should we also split raw data ? It's quite obvious that things like taxonomies should be re-usable in different part.

There is also the library vs service way of componentizing.

Shan't have libraries for taxonomies, ingredients parsing

Retrieved from "https://wiki.openfoodfacts.org/index.php?title=Open_Food_Facts_Days_2024/splitting_things_up_workshop&oldid=24298"