Difference between revisions of "Client-side libraries for personalized product filtering and ranking"

From Open Food Facts wiki
Jump to navigation Jump to search
Line 17: Line 17:
 
= Personalized product filtering and ranking =
 
= Personalized product filtering and ranking =
  
This section presents the general principles used to do personalized client-side product filtering and ranking.
+
This section presents the general principles and algorithms used to do personalized client-side product filtering and ranking.
  
Apps can implement those principles directly or use the Flutter or Javascript library.
+
Apps can implement those principles and algorithms directly using the Open Food Facts API, or simply use the Flutter or Javascript library.
  
 
== User preferences ==
 
== User preferences ==
  
 
The user preferences are stored locally on the client (mobile app or browser).
 
The user preferences are stored locally on the client (mobile app or browser).
 +
 +
=== Available preference attributes and localized descriptions ===
 +
 +
Apps can query the Open Food Facts API to retrieve all the different preference attributes that are supported by Open Food Facts. The "lc" (language code) parameter can be used to retrieve the attribute names and descriptions in a specific language.
 +
 +
e.g. https://world.openfoodfacts.org/api/v2/attribute_groups?lc=en : list of all available preference attributes in English
 +
 +
The possible preference values for each attributes can also be retrieved through the API:
 +
 +
e.g. https://world.openfoodfacts.org/api/v2/preferences?lc=en : list of all possible preferences values in English
  
 
=== User preferences format ===
 
=== User preferences format ===
Line 31: Line 41:
 
The preferences are saved in name / value pairs (hash table / associative array) of attribute ids mapping to string values.
 
The preferences are saved in name / value pairs (hash table / associative array) of attribute ids mapping to string values.
  
The list of attribute ids will be sent by the server, along with localized names, descriptions, warnings etc.
+
The list of attribute ids is sent by the server, along with localized names, descriptions, warnings etc.
  
 
String values:
 
String values:
Line 59: Line 69:
 
== Product Attributes ==
 
== Product Attributes ==
  
For each product, the server returns a match value for each attribute in a standardized format.
+
For each product in a set of search results, the server returns a match value for each attribute in a standardized format.
  
 
See [[Product Attributes]].
 
See [[Product Attributes]].
Line 65: Line 75:
 
== Scoring ==
 
== Scoring ==
  
On the client side, a library computes a score for each product returned by a search:
+
On the client side, the app (or a library) computes a score for each product returned by a search:
  
 
=== Input and output values ===
 
=== Input and output values ===

Revision as of 13:09, 20 April 2021

Summary

To allow users to get personalized search results, mobile and web apps can use our Flutter or Javascript libraries to filter and rank products according to user preferences. The filtering and ranking is done on the client side so that user personal preferences are kept on the client.

Client-side libraries for personalized product filtering and ranking is the 3rd of the 4 sub-tasks of the Project:Personalized_Search funded by the NGI0 Discovery Fund managed by NlNet.

Overview

Open-food-facts-personalized-search-project-overview.png

Diagram source: https://vecta.io/app/edit/-M2XyVv8ZoaLNrW-zQoT

  1. The Open Food Facts mobile app (and 3rd party apps) make generic search requests that do not contain user preferences
  2. The server returns a big number of generic results
  3. The app uses the user preferences stored locally to personalize the search results

Personalized product filtering and ranking

This section presents the general principles and algorithms used to do personalized client-side product filtering and ranking.

Apps can implement those principles and algorithms directly using the Open Food Facts API, or simply use the Flutter or Javascript library.

User preferences

The user preferences are stored locally on the client (mobile app or browser).

Available preference attributes and localized descriptions

Apps can query the Open Food Facts API to retrieve all the different preference attributes that are supported by Open Food Facts. The "lc" (language code) parameter can be used to retrieve the attribute names and descriptions in a specific language.

e.g. https://world.openfoodfacts.org/api/v2/attribute_groups?lc=en : list of all available preference attributes in English

The possible preference values for each attributes can also be retrieved through the API:

e.g. https://world.openfoodfacts.org/api/v2/preferences?lc=en : list of all possible preferences values in English

User preferences format

To make it easier to develop libraries to show / edit / store preferences, and to use them to rank products, we define a standard format to save user preferences.

The preferences are saved in name / value pairs (hash table / associative array) of attribute ids mapping to string values.

The list of attribute ids is sent by the server, along with localized names, descriptions, warnings etc.

String values:

  • "unset" : the user has not set a preference for the attribute (in ranking, treated like "not_important")
  • "not_important"
  • "important"
  • "very_important"
  • "mandatory"

The user preferences structure may not contain all possible attributes (e.g. if the server adds new attributes). In that case unexisting values are considered "unset".

Notes:

  • String values are used instead of 0, 1, 2, 3, 4 etc. so that the JSON is clearer to understand for humans, and so that we have the flexibility to add other values later (e.g. "somewhat_important")

Example

{
  "nutriscore": "very_important",
  "nova": "not_important",
  "labels_organic": "very_important",
  "labels_fair_trade": "important"
}

Product Attributes

For each product in a set of search results, the server returns a match value for each attribute in a standardized format.

See Product Attributes.

Scoring

On the client side, the app (or a library) computes a score for each product returned by a search:

Input and output values

  • Input values for scoring
    • Locally saved user preferences
    • Product attributes returned by the server API
  • Output value
    • Score

Algorithm

  • for each requirement in user preferences
    • if requirement is mandatory, score = score + match * 4
      • and exclude product is match is less than 20%
      • or mark product as unknown if the attribute value is unknown (e.g. the server is missing some data to compute the match for the attribute)
    • if requirement is very important, score = score + match * 2
    • if requirement is important, score = score + match

Flutter library / Dart package

Flutter is a framework to create mobile apps for Android and iOS and web apps, with a single code base. Flutter applications are written in the Dart language.

The easiest way for apps to personalize Open Food Facts search results is to use the openfoodfacts Dart package.

The openfoodfacts dart package has been extended to add support for the new Product Attributes created for the Personal Search project. It also contains functions to manage user preferences, and filter and rank products according to those preferences.

Example Flutter app with personalized search results

Smoothie is a new open source Flutter app that demonstrates how to use the openfoodfacts Dart package to personalize search results. Users can select their preferences, and then get personalized search results according to those preferences.

Flutter-smoothie-preferences.png

Javascript library

The Javascript library can be used on websites and browser extensions. It is used in particular on the Open Food Facts website to display personalized search results.