Changes

Jump to navigation Jump to search
no edit summary
Line 1: Line 1:  +
== Summary ==
 +
 +
This page contains the functional specifications for the [[Server-side product indexing and search]] task of the [[Project:Personalized_Search]] project.
 +
 +
It lists the options we are considering and the one we choose.
    
== Search API ==
 
== Search API ==
    
The search API is called by the app to retrieve a high number of generic search results (products) that match a query.
 
The search API is called by the app to retrieve a high number of generic search results (products) that match a query.
 +
 +
<div style="background-color:#ddddff;border:1px solid black;padding:1em">
 +
There are some choices we need to make:
 +
# '''How do we declare what kind of products we want to get back (matching)'''
 +
## Recommendation: by category + by keyword
 +
# '''Which of the matching products do we send back (ordering)'''
 +
## Recommendation: query-independent sort key based on product data completeness, data quality, product quality and product popularity
 +
</div>
    
=== Match (required) ===
 
=== Match (required) ===
Line 43: Line 56:  
* It could be very expensive to retrieve similar products in real-time or to pre-compute all of them
 
* It could be very expensive to retrieve similar products in real-time or to pre-compute all of them
 
* Does not work for category search of freely typed search queries
 
* Does not work for category search of freely typed search queries
 +
 +
==== Match a given set of products ====
 +
 +
Given the barcodes of multiple products, return the corresponding products.
 +
 +
This is to be used to make in-store personalized recommendations: the user scans all possible product choices he/she is contemplating (e.g. 5 different breakfast cereals), and we return the corresponding products, which can then be filtered/ranked according to the user preferences.
    
=== Optional filters ===
 
=== Optional filters ===
Line 54: Line 73:  
We can filter by country of availability, by store (or stores) of availability, and number of scans (a proxy for wide availability)
 
We can filter by country of availability, by store (or stores) of availability, and number of scans (a proxy for wide availability)
   −
== Server-side indexing ==
+
=== Non-personalized sort order ===
   −
Based on the type of queries we want to support, we may be able and may need to pre-compute and store new information in the database to better support the queries.
+
It is unrealistic to return all possible results to the calling app for all queries. E.g. if the query is "cookies", we cannot return tens of thousands of results. So we will need to use some kind of reasonable sort order that returns the most useful results for the app, without limiting the options for personnalization of the results.
   −
=== Similarity ===
+
Possible criteria to include in the sort order:
   −
If we decide to support product match, we need to pre-compute and store lists of similar products.
+
==== Completeness of data ====
   −
=== Non personalized sort order ===
+
Products for which we have enough data (e.g. ingredients, nutrition facts)
   −
It is unrealistic to return all possible results to the calling app for all queries. E.g. if the query is "cookies", we cannot return tens of thousands of results. So we will need to use some kind of reasonable sort order that returns the most useful results for the app, without limiting the options for personnalization of the results.
+
==== Data quality ====
   −
Possible criteria to include in the sort order:
+
Products for which there are no data inconsistencies (e.g. warnings on nutrition facts)
    
==== Product quality ====
 
==== Product quality ====
Line 75: Line 94:     
We could also include a measure of product popularity (e.g. derived from scans) in the sort order.
 
We could also include a measure of product popularity (e.g. derived from scans) in the sort order.
 +
 +
=== Number of results returned ===
 +
 +
We need to return at least 100 results so that the app has enough results to personalize the results set.
 +
 +
We could support more than 100 results, and possibly offer to request further results (e.g. an app could request 100 results, and then decide to request 100 more).
 +
 +
=== Returned fields ===
 +
 +
We could either return a fixed set of fields that includes all fields needed for personalization, or possibly let the app declare which fields it wants returned.
 +
 +
== Server-side indexing ==
 +
 +
Based on the type of queries we want to support, we may be able and may need to pre-compute and store new information in the database to better support the queries.
 +
 +
=== Similarity ===
 +
 +
If we decide to support product match, we need to pre-compute and store lists of similar products.
 +
 +
=== Sort order key computation ===
 +
 +
We should pre-compute the sort order key for all products.
 +
 +
== Performance ==
 +
 +
* 80% of search queries need to return results in less than 5 seconds.
 +
* 95% of search queries need to return results in less than 10 seconds.
 +
 +
Search queries should not overload the OFF servers (API server + MongoDB server)
 +
 +
[[Category:Project:Personalized_Search]]
 +
[[Category:Search]]

Navigation menu