Changes

Jump to navigation Jump to search
760 bytes removed ,  15:29, 19 June 2023
no edit summary
Line 1: Line 1:  
== Overview ==
 
== Overview ==
 
This document serves as a technical proposal for a new search API - V3 ([[Open Food Facts Search API Version 2|see V2 here]]).
 
This document serves as a technical proposal for a new search API - V3 ([[Open Food Facts Search API Version 2|see V2 here]]).
 +
 +
 +
'''IMPORTANT:''' our goal have shifted a bit on this project. See [https://docs.google.com/document/d/1mibE8nACcmen6paSrqT9JQk5VbuvlFUXI1S93yHCK2I/edit Search-a-licious roadmap architecture notes]
    
=== Goals ===
 
=== Goals ===
Line 92: Line 95:  
     field: str
 
     field: str
 
     value: str
 
     value: str
     # One of eq, ne, like, without
+
     # One of eq, ne, like
 
     operator: str = 'eq'
 
     operator: str = 'eq'
   Line 99: Line 102:  
     field: str
 
     field: str
 
     value: float
 
     value: float
     # One of eq, ne, lt, gt, without
+
     # One of eq, ne, lt, gt
 
     operator: str = 'eq'
 
     operator: str = 'eq'
   Line 106: Line 109:  
     field: str
 
     field: str
 
     value: datetime.datetime
 
     value: datetime.datetime
     # One of eq, ne, lt, gt, without
+
     # One of lt, gt
 
     operator: str = 'eq'
 
     operator: str = 'eq'
   Line 136: Line 139:  
The remaining APIs have several commonalities:
 
The remaining APIs have several commonalities:
   −
* Only fields with a value are returned in the JSON response, to keep response size down without needing to specify fields manually
   
* An optional ''response_fields'' parameter is provided, to limit the fields in the response further
 
* An optional ''response_fields'' parameter is provided, to limit the fields in the response further
 
* POST is used, to support a complex request body
 
* POST is used, to support a complex request body
Line 144: Line 146:  
* ''like'' operator, which does not need to match to exact fields, but rather will match by use of the [https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-snowball-tokenfilter.html snowball token filter] in Elasticsearch if the field supports it.
 
* ''like'' operator, which does not need to match to exact fields, but rather will match by use of the [https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-snowball-tokenfilter.html snowball token filter] in Elasticsearch if the field supports it.
 
* ''gt'', ''lt'' (greater than, less than) operators are provided.
 
* ''gt'', ''lt'' (greater than, less than) operators are provided.
* To maintain backwards compatibility, a ''without'' operator is provided, in which case the value will be ignored. Note that the value is still required, so a dummy value should be used (it was feared that making this optional would lead to client side bugs). This should be well documented.
  −
* The without operator will match empty data, as well as the empty string and 0 for numeric fields. This supports queries such as "Get foods without carbohydrates" for backwards comptability, even though that would be better served by making a query where carbohydrates = 0 (ensuring that missing data is not returned in the results).
   
* There is no support for an OR operator (which is supported in the [[Open Food Facts Search API Version 2|V2 API]]), however clients can perform this logic themselves if they wish through multiple API calls.
 
* There is no support for an OR operator (which is supported in the [[Open Food Facts Search API Version 2|V2 API]]), however clients can perform this logic themselves if they wish through multiple API calls.
   Line 157: Line 157:  
== Work Plan ==
 
== Work Plan ==
   −
* Completed (locally):
+
* Completed
 
** Product document definition
 
** Product document definition
 
** Bulk import script
 
** Bulk import script
 
** API definition
 
** API definition
** API implementation (partly implemented)
+
** API implementation
 
** Docker, Elasticvue configuration
 
** Docker, Elasticvue configuration
 +
** API tests
 +
** Redis Integration
 
* TODO:
 
* TODO:
** Proposal alignment
+
** Deploy (without any traffic)
** OFF-search repo creation, initial commit
  −
** Finish API implementation, unit tests
  −
** Redis reader
  −
** Deploy (without any traffic) - will need assistance for this, as well as Nginx configuration
   
** Redis writer on the Perl side
 
** Redis writer on the Perl side
 
** Final testing
 
** Final testing
Line 181: Line 179:  
* Robotoff could use the same cluster
 
* Robotoff could use the same cluster
 
* Current search APIs could be switched over (with a translation layer)
 
* Current search APIs could be switched over (with a translation layer)
 +
 +
[[Category:Search]]
199

edits

Navigation menu