Changes

Jump to navigation Jump to search
Some general explanations about the cluster setup and features.
Line 1: Line 1: −
Open Food Facts uses a cluster to host different virtual machines (VM).
+
Open Food Facts uses a Proxmox based cluster to host different virtual machines (VM) on OVH provided servers.
 +
 
 +
The cluster is made of [[Infrastructure|4 physical machines]] ("nodes" or "hosts" in Proxmox jargon):
 +
* ovh1 and ovh2 are '''computation''' oriented nodes: '''24 cores, 256 GB RAM''', 1TB nvme SSD
 +
* ovh3 and ovh4 are '''storage''' oriented nodes: 32GB RAM, '''6x12 TB HDD''' + 512GB NVMe cache
 +
 
 +
ovh1 and ovh3 are in Roubaix datacenter, ovh2 and ovh4 in Strasbourg.
 +
 
 +
At initial setup (january 2021), v6.3 of proxmox has been installed (based on Debian 10 "buster").
 +
 
 +
Proxmox '''GUI''' is available on any of the cluster nodes on port 8006.
 +
 
 +
At the networking level, a vRack links the cluster nodes with a 3Gbps private network used to access data on storage servers and replicate data between nodes.
 +
 
 +
All storage is managed using '''ZFS''' which provides:
 +
* volume management (like lvm)
 +
* redundancy (like mdadm)
 +
* encryption (like luks)
 +
* compression
 +
* snapshots
 +
* quota
 +
 
 +
Snapshots allow efficient synchronization between remote storage, and is used extensively by Proxmox to replicate data across the nodes. Snapshots simplify backups and allow rollbacks.
 +
 
 +
Proxmox allows full virtualization (VM, using QEMU) are containerization (CT using LXC). For performance reason, LXC CTs are prefered over QEMU VMs.
 +
 
 +
All resources are shared and dynamically allocated, thus can be reallocated at any time with reboot.
   −
The cluster is made of [[Infrastructure|4 physical machines]]: ovh1, ovh2, ovh3, ovh4.
      
== Usage guidelines (to be completed) ==
 
== Usage guidelines (to be completed) ==
 
Here is a few guides to follow for all new virtual machines:
 
Here is a few guides to follow for all new virtual machines:
# MUST: no direct root access without SSH key
+
# MUST: no direct root access on the nodes, even with SSH key.
# MUST: indirect root access with sudo, only with SSH connexion
+
# MUST: sudoers (root access using sudo) limited to SSH key based authentication
# SHOULD: use SSH keys published on Github: giving access to a server is then simple and secure: <pre>curl https://github.com/CharlesNepote.keys | tee -a /home/charles/.ssh/authorized_keys</pre>
+
# SHOULD: use SSH keys published on Github: giving access to a server is then simple and secure: <pre>curl https://github.com/CharlesNepote.keys | tee -a ~/.ssh/authorized_keys</pre>
# SHOULD: take care of production resources: use "nice" for scripts manually launched. Stéphane's tip: just use <pre>nice ./mycommand -arg1 foo -arg2 bar</pre> (nice default to lower the priority).
+
# SHOULD: take care of production resources: use "nice" / "ionice" for scripts manually launched. Stéphane's tip: just use <pre>nice ./mycommand whatever arguments</pre> (nice default to lower the priority). CPU and I/O priorities can be set if needed at the virtualization level.
29

edits

Navigation menu