Changes

Jump to navigation Jump to search
Line 1: Line 1: −
=== How it works ===
+
Also see [[Product Opener Installation]]
   −
This page contains the definition of the hierarchy of states of product pages in Open Food Facts.
+
This page lists all the steps necessary to install Product Opener on Debian, including a development environment.
   −
For more details, see [[Global taxonomies]]
+
All instructions below are for user "stephane", please use your own name. :-)
   −
== Taxonomy definition ==
+
== Open Food Facts test / development server ==
    +
OFF test / development server is a small dedicated server ("kimsufi 2G", the cheap sub-brand of OVH).
 +
 +
* Linux ks3095298.kimsufi.com 3.10.23-xxxx-std-ipv6-64 #1 SMP Tue Mar 18 14:48:24 CET 2014 x86_64 GNU/Linux
 +
* CPU: Intel(R) Atom(TM) CPU 230 @ 1.60GHz - Cores: 2 - Cache: 512KB
 +
* RAM: 2 Gb
 +
* Disk: 1 Tb
 +
 +
== Debian install ==
 +
 +
Fresh Debian install with OVH / kimsufi install process:
 +
 +
* Step 1:
 +
** OS - Type: Basic
 +
** Debian 7.5 stable (Wheezy) - debian 7 - 64 bits
 +
* Step 3: Partitions
 +
** Default:
 +
*** / ext4 20 Gb
 +
*** /home/ ext4 979.5 Gb
 +
*** swap 1 x 512 Mb
 +
 +
== DNS ==
 +
 +
ks3095298.kimsufi.com IP address: 94.23.195.82
 +
 +
Create 2 A records in the DNS zone of the domain you want to use:
 +
* openfoodfacts.ovh 94.23.195.82
 +
* *.openfoodfacts.ovh 94.23.195.82
 +
 +
== Install ==
 +
 +
=== Update Debian ===
 +
 +
Usually done by OVH Debian install:
 +
 +
<pre>
 +
apt-get update
 +
apt-get upgrade
 +
</pre>
 +
 +
=== Security ===
 +
 +
Install fail2ban and sudo:
 +
 +
<pre>
 +
apt-get install fail2ban
 +
apt-get install sudo
 +
</pre>
 +
 +
Create an user and give it sudo access:
 +
 +
<pre>
 +
adduser stephane
 +
adduser stephane sudo
 +
</pre>
 +
 +
 +
=== Install software used by Product Opener ===
 +
 +
<pre>
 +
apt-get install mongodb
 +
 +
apt-get install memcached
 +
 +
apt-get install imagemagick
 +
 +
apt-get install tesseract-ocr
 +
apt-get install tesseract-ocr-fra
 +
 +
apt-get install exim4
 +
dpkg-reconfigure exim4-config
 +
- Internet Site mail is sent by smtp 
 +
- 127.0.0.1
 +
 +
apt-get install geoip-bin geoip-database
 +
</pre>
 +
 +
=== Install libraries used by Product Opener ===
 +
 +
<pre>
 +
apt-get install zlib1g-dev
 +
</pre>
 +
 +
 +
=== Install development tools ===
 +
 +
Note: the Product Opener code is currently on a private bitbucket.org repository and is in the process of being made publicly available (open source). If you want to help us to clean / document Product Opener to open it, request access to the bitbucket repository: stephane@openfoodfacts.org
 +
 +
<pre>
 +
apt-get install build-essential
 +
apt-get install git
 +
 +
git clone https://stephane@bitbucket.org/openfoodfacts/product-opener.git
 +
</pre>
 +
 +
(get the link for your user on bitbucket.org)
 +
 +
 +
=== Apache 2.2 + mod_perl: Product Opener backend ===
 +
 +
==== Build ====
 +
 +
<pre>
 +
apt-get install libperl-dev
 +
 +
cd /home/stephane
 +
mkdir src
 +
cd src
 +
wget http://apache.crihan.fr/dist//httpd/httpd-2.2.29.tar.gz
 +
tar xvfz httpd-2.2.29.tar.gz
 +
cd httpd-2.2.29
 +
./configure --with-mpm=prefork --prefix=/home/stephane/apache  --enable-rewrite --enable-proxy --enable-proxy_http --enable-deflate  --disable-userdir --enable-headers
 +
make
 +
make install
 +
cd ~
 +
 +
cd ~/src
 +
wget http://mir2.ovh.net/ftp.apache.org/dist/perl/mod_perl-2.0.8.tar.gz
 +
tar xvfz mod_perl-2.0.8.tar.gz
 +
cd mod_perl-2.0.8
 +
perl Makefile.PL MP_APXS=/home/stephane/apache/bin/apxs
 +
make
 +
make install
 +
cd ~
 +
</pre>
 +
 +
==== Configuration ====
 +
 +
In ~/apache/conf/httpd.conf:
 +
 +
Change the port to something else than 80 (the reverse proxy will listen on 80 and forward to the backend on another port).
 +
 +
<pre>
 +
Listen 8001
 +
</pre>
 +
 +
If you are installing several Product Openers on the same machine, use different ports.
 +
 +
Change the Apache user to your user:
 +
 +
<pre>
 +
User stephane
 +
Group stephane
 +
</pre>
 +
 +
Add to apache/conf/httpd.conf:
    
<pre>
 
<pre>
 +
LoadModule perl_module modules/mod_perl.so
   −
en:Empty
+
PerlWarn Off
fr:Vide
+
PerlRequire /home/stephane/product-opener/cgi/startup.pl
es:Vacío
  −
pt:Vazio
  −
ro:Gol
  −
he:ריק
     −
en:Checked
  −
fr:Vérifié
  −
es:Comprobado
  −
pt:Verificado
  −
ro:Verificat
  −
he:נבדק
     −
en:To be checked
+
<Location /cgi>
fr:A vérifier
+
SetHandler perl-script
es:Pendiente de comprobación
+
PerlResponseHandler ModPerl::Registry
pt:Pendente de verificação
+
PerlOptions +ParseHeaders
ro:De verificat
+
Options +ExecCGI
he:לבדיקה
+
Order allow,deny
 +
Allow from all
 +
</Location>
   −
en:Complete
  −
fr:Complet
  −
es:Completado
  −
pt:Completo
  −
ro:Complet
  −
he:הושלם
     −
en:To be completed
+
<VirtualHost *>
fr:A compléter
+
DocumentRoot /home/stephane/product-opener/html
es:A completar
+
ServerName openfoodfacts.ovh
pt:A completar
+
ErrorLog /home/stephane/logs/error_log
ro:De completat
+
CustomLog /home/stephane/logs/access_log combined
he:להשלמה
+
ScriptAlias /cgi/ "/home/stephane/product-opener/cgi/"
 +
</VirtualHost>
   −
en:Expiration date to be completed
+
PerlPostReadRequestHandler My::ProxyRemoteAddr
fr:Date limite à compléter
+
</pre>
es:Fecha límite de consumo a completar
  −
pt:Data de validade a completar
  −
ro:De completat data de expirare
  −
he:תאריך תפוגה להשלמה
     −
en:Expiration date completed
+
=== Apache 2.4 - light weight reverse proxy ===
fr:Date limite complétée
  −
es:Fecha límite de consumo agregada
  −
pt:Data de validade completada
  −
ro:Data de expirare completată
  −
he:תאריך התפוגה הושלם
     −
en:Product name to be completed
+
==== Build ====
fr:Nom du produit à compléter
  −
es:Nombre de producto a completar
  −
pt:Nome do produto a completar
  −
ro:De completat numele produsului
  −
he:שם המוצר להשלמה
     −
en:Quantity to be completed
+
<pre>
fr:Quantité à compléter
+
apt-get install libpcre3 libpcre3-dev
es:Cantidad a completar
  −
pt:Quantidade a completar
  −
ro:Cantitatea de completat
  −
he:כמות להשלמה
     −
en:Packaging to be completed
+
cd /home/stephane/src
fr:Emballage à compléter
+
wget http://apache.websitebeheerjd.nl//httpd/httpd-2.4.12.tar.gz
es:Envase a completar
+
tar xvfz httpd-2.4.12.tar.gz
pt:Embalagem a completar
+
cd httpd-2.4.12
ro:De completat ambalarea
  −
he:אריזה להשלמה
     −
en:Brands to be completed
+
cd srclib
fr:Marques à compléter
+
wget http://mir2.ovh.net/ftp.apache.org/dist//apr/apr-1.5.1.tar.gz
es:Marcas a completar
+
wget http://mir2.ovh.net/ftp.apache.org/dist//apr/apr-util-1.5.4.tar.gz 
pt:Marcas a completar
+
tar xvfz apr-1.5.1.tar.gz
ro:Mărci de completat
+
tar xvfz apr-util-1.5.4.tar.gz
he:מותגים להשלמה
+
ln -s apr-1.5.1 apr
 +
ln -s apr-util-1.5.4 apr-util
 +
cd ..
   −
en:Categories to be completed
+
./configure --prefix=/home/stephane/proxy --enable-rewrite --enable-proxy --enable-proxy_http --disable-userdir --enable-headers --enable-expires --enable-mime --enable-so --with-included-apr
fr:Catégories à compléter
  −
es:Categorías a completar
  −
pt:Categorias a completar
  −
ro:Categorii de completat
  −
he:קטגוריות להשלמה
     −
en:Characteristics completed
+
make
fr:Caractéristiques complétées
+
make install
es:Características completadas
+
cd ~
pt:Características completadas
+
</pre>
ro:Caracteristici completate
  −
he:מאפיינים הושלמו
     −
en:Characteristics to be completed
+
==== Configuration ====
fr:Caractéristiques à compléter
  −
es:Características a completar
  −
pt:Características a completar
  −
ro:Caracteristici de completat
  −
he:מאפיינים להשלמה
     −
en:Ingredients completed
+
in proxy/conf/httpd.conf :
fr:Ingrédients complétés
+
* uncomment #LoadModule rewrite_module modules/mod_rewrite.so
es:Ingredientes completados
+
* uncomment #LoadModule slotmem_shm_module modules/mod_slotmem_shm.so
pt:Ingredientes completados
  −
ro:Ingrediente completate
  −
he:הרכיבים הושלמו
     −
en:Ingredients to be completed
+
Add in httpd.conf :
fr:Ingrédients à compléter
  −
es:Ingredientes a completar
  −
ro:Ingrediente de completat
  −
he:רכיבים להשלמה
     −
en:Nutrition facts completed
+
<pre>
fr:Informations nutritionnelles complétées
+
<VirtualHost *>
es:Valores nutricionales completados
+
DocumentRoot /home/stephane/product-opener/html
pt:Valores nutricionais completados
+
ServerName world.openfoodfacts.ovh
ro:Valori nutriționale completate
+
ServerAlias *.openfoodfacts.ovh
he:מפרט תזונתי הושלם
     −
en:Nutrition facts to be completed
+
ErrorLog /home/stephane/logs/proxy_error_log
fr:Informations nutritionnelles à compléter
+
CustomLog /home/stephane/logs/proxy_access_log combined
es:Valores nutricionales a completar
+
ServerAdmin stephane@openfoodfacts.org
pt:Valores nutricionais a completar
  −
ro:Valori nutriționale de completat
  −
he:מפרט תזונתי להשלמה
     −
en:Photos validated
+
<Directory "/home/stephane/product-opener/html">
fr:Photos validées
+
    Options Indexes FollowSymLinks
es:Imágenes validadas
+
    Require all granted
pt:Imagens validadas
+
</Directory>
ro:Fotografii validate
+
ProxyPreserveHost On
he:התמונות אומתו
+
RewriteEngine on
 +
RewriteCond  %{REQUEST_URI}  !/./
 +
RewriteRule ^(/cgi/.*)$ http://localhost:8001$1 [P,L]
 +
RewriteMap escape int:escape
 +
RewriteRule ^/favicon.ico$ /favicon.ico [L]
 +
RewriteCond  %{REQUEST_URI}  !^/images/
 +
RewriteCond  %{REQUEST_URI}  !^/js/
 +
RewriteCond  %{REQUEST_URI}  !^/rss/
 +
RewriteCond  %{REQUEST_URI}  !^/robots
 +
RewriteCond  %{REQUEST_URI}  !^/clicks/
 +
RewriteCond  %{REQUEST_URI}  !^/data/
 +
RewriteCond  %{REQUEST_URI}  !^/files/
 +
RewriteRule  ^(.*)$ http://localhost:8001/cgi/display.pl?${escape:$1} [P,L,QSA]
 +
</VirtualHost>
   −
en:Photos to be validated
  −
fr:Photos à valider
  −
es:Imágenes a validar
  −
pt:Imagens a validar
  −
ro:Fotografii de validat
  −
he:תמונות לאימות
     −
en:Photos uploaded
+
<VirtualHost *>
fr:Photos envoyées
+
DocumentRoot /home/stephane/product-opener/html
es:Imágenes agregadas
+
ServerName openfoodfacts.ovh
pt:Imagens enviadas
+
ErrorLog /home/stephane/logs/proxy_error_log
ro:Fotografii încărcate
+
CustomLog /home/stephane/logs/proxy_access_log combined
he:התמונות הועלו
+
DirectoryIndex index.html index.shtml
 +
<Directory "/home/stephane/product-opener/html">
 +
    Options -Indexes +FollowSymLinks +Includes
 +
    Require all granted
 +
</Directory>
 +
RewriteEngine on
 +
RewriteCond %{HTTP_HOST} ^stephane\.openfoodfacts\.ovh
 +
RewriteRule ^/products$ /products.shtml [L]
   −
en:Photos to be uploaded
+
</VirtualHost>
fr:Photos à envoyer
  −
es:Imágenes a agregar
  −
pt:Imagens a enviar
  −
ro:Fotografii de încărcat
  −
he:תמונות להעלאה
      
</pre>
 
</pre>
 +
 +
 +
 +
=== Install Perl modules ===
 +
 +
If possible, use apt-get to install packages directly. If they are not available, you can use cpan to install the modules.
 +
 +
<pre>
 +
apt-get install libwww-perl libimage-magick-perl
 +
apt-get install  libxml-encoding-perl libtext-unaccent-perl libmime-lite-perl
 +
apt-get install libcache-memcached-fast-perl libjson-perl libclone-perl  libgraphviz-perl
 +
apt-get install libmime-lite-perl  # needs to be done after installing exim4
 +
apt-get install libcrypt-passwdmd5-perl libencode-detect-perl libgraphics-color-perl libbarcode-zbar-perl
 +
apt-get install libxml-feedpp-perl libmongodb-perl liburi-find-perl
 +
</pre>
 +
 +
Modules that could not be installed using apt-get:
 +
 +
<pre>
 +
cpan
 +
install URI::Escape::XS
 +
install Image::OCR::Tesseract # needs imagemagick installed first
 +
</pre>
 +
 +
Modules that HAVE to be installed using cpan:
 +
 +
As of Feb 3rd 2015, the MongoDB perl module installed through "apt-get install libmongodb-perl" is too old (0.45):
 +
 +
<pre>
 +
perl -MMongoDB -e 'print $MongoDB::VERSION . "\n"'
 +
0.45
 +
</pre>
 +
 +
This causes an error when trying to access a list of "tags" such as categories:
 +
 +
<pre>
 +
Software error:
 +
Not an ARRAY reference at /home/stephane/product-opener/cgi/Blogs/Display.pm line 797.
 +
</pre>
 +
 +
A newer MongoDB perl module needs to be installed with cpan.
 +
 +
<pre>
 +
cpan
 +
install MongoDB
 +
</pre>
 +
 +
<pre>
 +
perl -MMongoDB -e 'print $MongoDB::VERSION . "\n"'
 +
v0.708.0.0
 +
</pre>
 +
 +
cgi/startup.pl preloads a lot of modules, you can check that those modules are installed by running:
 +
 +
<pre>
 +
cd /home/stephane/product-opener/cgi
 +
perl startup.pl
 +
</pre>
 +
 +
=== Create some paths and links ===
 +
 +
<pre>
 +
cd /home/stephane/product-opener/cgi
 +
ln -s ./ Blogs
 +
ln -s S
22

edits

Navigation menu