Changes

Jump to navigation Jump to search
→‎Backend: more details about backend
Line 23: Line 23:  
** Python is widely adopted (one of the main languages taught at school).
 
** Python is widely adopted (one of the main languages taught at school).
 
** It scales well.
 
** It scales well.
** Part of the team knows Python.
+
** Part of the team is already using Python.
 
* Building an API from scratch would have been too long, so choose the FastAPI framework which integrates interesting benefits:
 
* Building an API from scratch would have been too long, so choose the FastAPI framework which integrates interesting benefits:
** [https://www.tutlinks.com/fastapi-with-postgresql-crud-async/ Building an API with FastAPI is very easy]: it's possible to build a simple API with its generated dynamic documentation in a few hours.
+
**[https://www.tutlinks.com/fastapi-with-postgresql-crud-async/ Building an API with FastAPI is very easy]: it's possible to build a simple API with its self generated documentation (openapi standard).
 
** Said to be fast: https://www.techempower.com/ benchmarks are showing it is ~3 times faster than very popular Python framework such as Django or Flask.
 
** Said to be fast: https://www.techempower.com/ benchmarks are showing it is ~3 times faster than very popular Python framework such as Django or Flask.
 
** FastAPI can be run with or without a web server.
 
** FastAPI can be run with or without a web server.
 +
** RESTful compliance is largely handled by FastAPI itself.
    
==== Database ====
 
==== Database ====
Line 35: Line 36:  
* It could lead to 10 times more records as the total number of products gathered by Open Food Facts.
 
* It could lead to 10 times more records as the total number of products gathered by Open Food Facts.
   −
The data model simplicity lead us to choose a classical RDBMS known for its scalability: PostGRESQL. Postgre has also interesting advantages and features which can be useful for Folksonomy Engine (fully open source, transactions, stored procedures, triggers, JSON...).
+
The data model simplicity lead us to choose a classical RDBMS known for its scalability: PostgreSQL. PostgreSQL has also interesting advantages and features which can be useful for Folksonomy Engine (fully open source, transactions, stored procedures, triggers, JSON...).
    
==== Data model ====
 
==== Data model ====
For performance concerns, we separated the data model implementation into two tables:
+
For performance concerns, we separated the data model implementation into two parts :
   −
* one small table deals with the live data
+
* one small table deals with the live data (current record versions)
 
* the other one stores all the past versions of the records.
 
* the other one stores all the past versions of the records.
 +
Triggers at PostgreSQL level are used to maintain integrity between the current and archived versions of the records.
    
=== Frontend ===
 
=== Frontend ===
Line 53: Line 55:  
=== Tests ===
 
=== Tests ===
 
If not specified, all our tests are made on a Raspberry Pi 4: because of the very low price of it, hardware -- and tests -- are easily reproducible.
 
If not specified, all our tests are made on a Raspberry Pi 4: because of the very low price of it, hardware -- and tests -- are easily reproducible.
 +
 +
All tests have been run with:
 +
* a standard installation, as [https://github.com/openfoodfacts/folksonomy_api/blob/main/INSTALL.md described here]
 +
* no optimization at all
 +
* <code>uvicorn</code> launched with 2 workers.
 +
$ uvicorn folksonomy.api:app --reload --host 192.168.0.42 --workers 2
 +
 +
We used the classical <code>ab</code> test tool from the Apache Foundation, and a custom script to test new key/value injection (see bellow).
    
Testing 10,000 queries to <code>/ping</code> (includes a database request) give the following results:
 
Testing 10,000 queries to <code>/ping</code> (includes a database request) give the following results:
Line 59: Line 69:  
  Requests per second:    307.11 [#/sec] (mean)
 
  Requests per second:    307.11 [#/sec] (mean)
 
  Time per request:      32.562 [ms] (mean)
 
  Time per request:      32.562 [ms] (mean)
 +
Testing 10,000 queries to a particular key/value gives the following results:
 +
$ ab -n 10000 -c 10 http://192.168.0.42:8000/product/6389599748279
 +
 +
Requests per second:    227.45 [#/sec] (mean)
 +
Time per request:      43.966 [ms] (mean)
 
Publishing 2,000 new key/value pairs on 2,000 different products give the following result:
 
Publishing 2,000 new key/value pairs on 2,000 different products give the following result:
 
  $ time ./fe_test.sh
 
  $ time ./fe_test.sh
29

edits

Navigation menu