Line 1: |
Line 1: |
| [[Category:ProductOpener]] | | [[Category:ProductOpener]] |
| [[Category:Developer]] | | [[Category:Developer]] |
− | = Infrastructure =
| + | [[Category:Roadmap]] |
| | | |
| This page describes the hardware/software infrastructure for the Open Food Facts + Open Beauty Facts projects. | | This page describes the hardware/software infrastructure for the Open Food Facts + Open Beauty Facts projects. |
| + | |
| + | == Infrastructure planning == |
| + | |
| + | * Hardware needs: [[Infrastructure Planning]] |
| + | |
| + | == 2021 roadmap == |
| + | * Please expand desirable objectives (not hardware needs) |
| + | * |
| + | * … |
| | | |
| == Current infrastructure == | | == Current infrastructure == |
| | | |
− | '''off1''', dedicated free.org server:
| + | === off1 === |
− | * Hardware configuration:
| + | Dedicated free.org server: |
− | ** Dell 1U server
| + | |
− | ** Disks: | + | '''Hardware configuration:''' |
− | *** 2x4 TB SATA HDD (hardware RAID1) : /srv
| + | * Dell 1U R440 server (1 Xeon Silver 4110: 8c/16t) |
− | *** 2x14 TB SATA HDD (hardware RAID1) : /srv2 (images, imports)
| + | * RAM: 64GB (DDR4@2666) |
− | * Running services:
| + | * Disks: |
− | ** OS: debian 9.5
| + | ** 2x4 TB SATA HDD (hardware RAID1) : /srv |
− | ** 1 nginx reverse proxy to serve static files
| + | ** 2x14 TB SATA HDD (hardware RAID1) : /srv2 (images, imports) |
− | ** 1 apache + mod_perl to generate dynamic pages + API
| + | '''Running services:''' |
− | *** running 4 separate instances on different ports: OFF, OBF, OPFF, OPF
| + | * OS: debian 9.5 |
| + | * 1 nginx reverse proxy to serve static files |
| + | * 1 apache + mod_perl to generate dynamic pages + API |
| + | ** running 4 separate instances on different ports: OFF, OBF, OPFF, OPF |
| + | |
| + | === off2 === |
| + | Dedicated free.org server: |
| + | |
| + | '''Hardware configuration:''' |
| + | * Dell 1U R440 server (1 Xeon Silver 4110: 8c/16t) |
| + | * RAM: 64GB (DDR4@2666) |
| + | * Disks: |
| + | ** 2x4 TB SATA HDD (hardware RAID1) : /srv |
| + | ** 2x14 TB SATA HDD (hardware RAID1) : /srv2 (images, imports) |
| + | '''Running services:''' |
| + | * OS: debian 9.5 |
| + | * 1 apache + mod_perl (not used) |
| + | * 1 MongoDB (production) |
| + | * Robotoff (dockerized) |
| + | |
| + | === robotoff === |
| + | Hetzner - cpx51 |
| + | |
| + | '''Hardware configuration:''' |
| + | * 16 vCPU |
| + | * Storage: 50GB |
| + | * RAM: 32GB |
| + | '''Running services:''' |
| + | * Tensorflow Serving (Tensorflow ML models) |
| + | * Robotoff ANN API: logo similarity service, memory-hungry |
| + | |
| + | === no name yet === |
| + | Dedicated OVH - soyoustart server (rented): |
| + | |
| + | '''Hardware configuration:''' |
| + | * E3-SAT-1-32 |
| + | * Storage: SoftRaid 3x2To SATA |
| + | * RAM: 32GB DDR3 1333 MHz |
| + | * CPU: Xeon E3-1245v2 (4c/8th) |
| + | '''Running services:''' |
| + | * wiki |
| + | * zammad (soon) |
| + | |
| + | === dev === |
| + | Dedicated OVH - soyoustart server (rented): |
| + | |
| + | '''Hardware configuration:''' |
| + | * Super Micro server (1 Xeon D-1520) |
| + | * Storage: 4 x 2TB HDD, software RAID1 / /boot + LVM /srv /srv2 |
| + | * RAM: 32GB (DDR4@2666) |
| + | * CPU: Xeon D-1520 |
| + | '''Running services:''' |
| + | * OS: debian 9.8 |
| + | * 1 nginx reverse proxy |
| + | * 1 apache + mod_perl (https://openfoodfacts.dev - auth: off / off) |
| + | * 1 mongodb |
| + | |
| + | === dev2 === |
| + | Dedicated server (self-hosted by [[User:Cquest|cquest]]) |
| + | |
| + | '''Hardware configuration:''' |
| + | * Dell R710 server (2U) |
| + | * Storage (ZFS pool) with: |
| + | ** 6 x 2TB HITACHI HUS72402 (SAS) |
| + | ** 500GB Samsung 860 SSD: OS and 256GB read cache (L2ARC) |
| + | ** 16GB Intel Optane SSD: write journal (ZIL) |
| + | * RAM: 96GB (DDR3@1333) |
| + | * CPU: 2 Xeon X5675 = 12 cores @ 3.07GHz |
| + | '''Running services:''' |
| + | * OS: Proxmox 6 (debian 10 based) |
| + | |
| + | '''Network''': |
| + | * uplink: optic fiber from OVH (1G down/500M up) with dedicated IPv4 and IPv6/72 |
| + | * ethernet 1Gbps (bonded for backup) |
| + | * infiniband 40Gbps (proxmox cluster backbone + access to storage server) |
| | | |
− | '''off2''', dedicated free.org server:
| + | === backup === |
− | * Hardware configuration:
| + | Shared storage server (self-hosted by [[User:Cquest|cquest]]) |
− | ** Dell 1U server
| |
− | ** Disks:
| |
− | *** 2x4 TB SATA HDD (hardware RAID1) : /srv
| |
− | *** 2x14 TB SATA HDD (hardware RAID1) : /srv2 (images, imports)
| |
− | * Running services:
| |
− | ** OS: debian 9.5
| |
− | ** 1 apache + mod_perl (not used)
| |
− | ** 1 MongoDB (production)
| |
| | | |
− | '''dev''', dedicated OVH server (rented): | + | '''Hardware configuration:''' |
− | * Hardware configuration: | + | * Supermicro storage server (4U) |
− | ** Storage: 4 x 2TB HDD, software RAID1 / /boot + LVM /srv /srv2 | + | * Storage (ZFS pool) with: |
− | ** RAM: 32GB
| + | ** 36 SAS bays (populated disks evolving with time are needs) |
− | ** CPU: Xeon D-1520
| + | ** 1TB Samsung 860 SSD: OS and 256GB read cache (L2ARC) |
− | * Running services:
| + | ** 16GB Intel Optane SSD: write journal (ZIL) |
− | ** OS: debian 9.8
| + | * RAM: 128GB |
− | ** 1 robotoff (dockerized)
| + | * CPU: |
− | ** 1 nginx reverse proxy | + | '''Running services:''' |
− | ** 1 apache + mod_perl (https://world.dev.openfoodfacts.org - auth: off / off)
| + | * OS: Proxmox 6 (debian 10 based) |
− | ** 1 mongodb | + | * Local NFS access from dev2 to /srv /srv2 backups (nghtly rsync + ZFS snapshots) |
| + | * Proxmox CT/VM replication |
| | | |
− | === History === | + | === ovh1 === |
| + | VM/CT server for computing usages. |
| + | |
| + | See [[Infrastructure/VM cluster]]. |
| + | |
| + | '''Hardware configuration:''' |
| + | * CPU: AMD Epyc 7451 (24c/48t) |
| + | * RAM: 256GB |
| + | * 1 GB/s public bandwith; 3 GB/s private bandwith (10.0.0.1) on vRack |
| + | * 2 x 1TB SSD NVMe |
| + | |
| + | Partition / storage: |
| + | * Ext4 root partition: 32Gb on RAID1 /dev/md2 |
| + | * 512MB swap |
| + | * 920GB ZFS pool on remaining space (mirror) |
| + | |
| + | '''Running services:''' |
| + | * OS: Proxmox 6 (based on debian 10 buster) |
| + | |
| + | === ovh2 === |
| + | VM/CT server for computing usages (same as ovh1). |
| + | |
| + | See [[Infrastructure/VM cluster]]. |
| + | |
| + | '''Hardware configuration:''' |
| + | * CPU: AMD Epyc 7451 (24c/48t) |
| + | * RAM: 256GB |
| + | * 1 GB/s public bandwith; 3 GB/s private bandwith (10.0.0.2) on vRack |
| + | * 2 x 1TB SSD NVMe |
| + | |
| + | Partition / storage: |
| + | * Ext4 root partition: 32Gb on RAID1 /dev/md2 |
| + | * 512MB swap |
| + | * 920GB ZFS pool on remaining space (mirror) |
| + | |
| + | '''Running services:''' |
| + | * OS: Proxmox 6 (based on debian 10 buster) |
| + | |
| + | === ovh3 === |
| + | Mainly storage server, VM/CT host is needed. |
| + | |
| + | See [[Infrastructure/VM cluster]]. |
| + | |
| + | '''Hardware configuration:''' |
| + | * CPU: Intel Xeon-D 1541 (8c/16t) |
| + | * RAM: 32GB |
| + | * 1 GB/s public bandwith; 3 GB/s private bandwith (10.0.0.3) |
| + | |
| + | Partition / storage: |
| + | * Ext4 root partition: 32Gb on SSD |
| + | * 512MB swap |
| + | * 5GB ZFS ZIL (write journal) |
| + | * 472GB ZFS cache (ARC2 read cache) |
| + | |
| + | * ZFS storage (6 x 12TB HDD) in RAIDZ2 (2 disks for redundancy, around 42TB available) |
| + | |
| + | '''Running services:''' |
| + | * OS: Proxmox 6 (based on debian 10 buster) |
| + | |
| + | === ovh4 === |
| + | Mainly storage server, VM/CT host is needed. |
| + | |
| + | See [[Infrastructure/VM cluster]]. |
| + | |
| + | '''Hardware configuration:''' |
| + | * CPU: Intel Xeon-D 1541 (8c/16t) |
| + | * RAM: 32GB |
| + | * 1 GB/s public bandwith; 3 GB/s private bandwith (10.0.0.4) |
| + | |
| + | Partition / storage: |
| + | * Ext4 root partition: 32Gb on SSD |
| + | * 512MB swap |
| + | * 5GB ZFS ZIL (write journal) |
| + | * 472GB ZFS cache (ARC2 read cache) |
| + | |
| + | * ZFS storage (6 x 12TB HDD) in RAIDZ2 (2 disks for redundancy, around 42TB available) |
| + | |
| + | '''Running services:''' |
| + | * OS: Proxmox 6 (based on debian 10 buster) |
| + | |
| + | == History == |
| | | |
| From 2012 to 2016, OFF and OBF have been running on part of an OVH server (also used for other purposes): | | From 2012 to 2016, OFF and OBF have been running on part of an OVH server (also used for other purposes): |
Line 680: |
Line 836: |
| | | |
| | | |
| + | ===== Generate CSS and download dependencies ===== |
| + | |
| + | <pre> |
| + | /srv/off/yarn install |
| + | /srv/off/yarn run build |
| + | </pre> |
| | | |
| + | The commands above might fail if the correct versions of yarn and nodejs are not installed. To fix: |
| + | |
| + | <pre> |
| + | apt remove cmdtest |
| + | apt remove yarn |
| + | curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - |
| + | echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list |
| + | apt-get update |
| + | apt-get install yarn |
| + | curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash - |
| + | apt-get install -y nodejs |
| + | yarn install |
| + | yarn run build |
| + | </pre> |
| | | |
| ==== Cron jobs ==== | | ==== Cron jobs ==== |