Line 1: |
Line 1: |
− | === How it works ===
| + | Also see [[Product Opener Installation]] |
| | | |
− | This page contains the definition of the hierarchy of states of product pages in Open Food Facts. | + | This page lists all the steps necessary to install Product Opener on Debian, including a development environment. |
| | | |
− | For more details, see [[Global taxonomies]]
| + | All instructions below are for user "stephane", please use your own name. :-) |
| | | |
− | == Taxonomy definition == | + | == Open Food Facts test / development server == |
| | | |
| + | OFF test / development server is a small dedicated server ("kimsufi 2G", the cheap sub-brand of OVH). |
| + | |
| + | * Linux ks3095298.kimsufi.com 3.10.23-xxxx-std-ipv6-64 #1 SMP Tue Mar 18 14:48:24 CET 2014 x86_64 GNU/Linux |
| + | * CPU: Intel(R) Atom(TM) CPU 230 @ 1.60GHz - Cores: 2 - Cache: 512KB |
| + | * RAM: 2 Gb |
| + | * Disk: 1 Tb |
| + | |
| + | == Debian install == |
| + | |
| + | Fresh Debian install with OVH / kimsufi install process: |
| + | |
| + | * Step 1: |
| + | ** OS - Type: Basic |
| + | ** Debian 7.5 stable (Wheezy) - debian 7 - 64 bits |
| + | * Step 3: Partitions |
| + | ** Default: |
| + | *** / ext4 20 Gb |
| + | *** /home/ ext4 979.5 Gb |
| + | *** swap 1 x 512 Mb |
| + | |
| + | == DNS == |
| + | |
| + | ks3095298.kimsufi.com IP address: 94.23.195.82 |
| + | |
| + | Create 2 A records in the DNS zone of the domain you want to use: |
| + | * openfoodfacts.ovh 94.23.195.82 |
| + | * *.openfoodfacts.ovh 94.23.195.82 |
| + | |
| + | == Install == |
| + | |
| + | === Update Debian === |
| + | |
| + | Usually done by OVH Debian install: |
| + | |
| + | <pre> |
| + | apt-get update |
| + | apt-get upgrade |
| + | </pre> |
| + | |
| + | === Security === |
| + | |
| + | Install fail2ban and sudo: |
| + | |
| + | <pre> |
| + | apt-get install fail2ban |
| + | apt-get install sudo |
| + | </pre> |
| + | |
| + | Create an user and give it sudo access: |
| + | |
| + | <pre> |
| + | adduser stephane |
| + | adduser stephane sudo |
| + | </pre> |
| + | |
| + | |
| + | === Install software used by Product Opener === |
| + | |
| + | <pre> |
| + | apt-get install mongodb |
| + | |
| + | apt-get install memcached |
| + | |
| + | apt-get install imagemagick |
| + | |
| + | apt-get install tesseract-ocr |
| + | apt-get install tesseract-ocr-fra |
| + | |
| + | apt-get install exim4 |
| + | dpkg-reconfigure exim4-config |
| + | - Internet Site mail is sent by smtp |
| + | - 127.0.0.1 |
| + | |
| + | apt-get install geoip-bin geoip-database |
| + | </pre> |
| + | |
| + | === Install libraries used by Product Opener === |
| + | |
| + | <pre> |
| + | apt-get install zlib1g-dev |
| + | </pre> |
| + | |
| + | |
| + | === Install development tools === |
| + | |
| + | Note: the Product Opener code is currently on a private bitbucket.org repository and is in the process of being made publicly available (open source). If you want to help us to clean / document Product Opener to open it, request access to the bitbucket repository: stephane@openfoodfacts.org |
| + | |
| + | <pre> |
| + | apt-get install build-essential |
| + | apt-get install git |
| + | |
| + | git clone https://stephane@bitbucket.org/openfoodfacts/product-opener.git |
| + | </pre> |
| + | |
| + | (get the link for your user on bitbucket.org) |
| + | |
| + | |
| + | === Apache 2.2 + mod_perl: Product Opener backend === |
| + | |
| + | ==== Build ==== |
| + | |
| + | <pre> |
| + | apt-get install libperl-dev |
| + | |
| + | cd /home/stephane |
| + | mkdir src |
| + | cd src |
| + | wget http://apache.crihan.fr/dist//httpd/httpd-2.2.29.tar.gz |
| + | tar xvfz httpd-2.2.29.tar.gz |
| + | cd httpd-2.2.29 |
| + | ./configure --with-mpm=prefork --prefix=/home/stephane/apache --enable-rewrite --enable-proxy --enable-proxy_http --enable-deflate --disable-userdir --enable-headers |
| + | make |
| + | make install |
| + | cd ~ |
| + | |
| + | cd ~/src |
| + | wget http://mir2.ovh.net/ftp.apache.org/dist/perl/mod_perl-2.0.8.tar.gz |
| + | tar xvfz mod_perl-2.0.8.tar.gz |
| + | cd mod_perl-2.0.8 |
| + | perl Makefile.PL MP_APXS=/home/stephane/apache/bin/apxs |
| + | make |
| + | make install |
| + | cd ~ |
| + | </pre> |
| + | |
| + | ==== Configuration ==== |
| + | |
| + | In ~/apache/conf/httpd.conf: |
| + | |
| + | Change the port to something else than 80 (the reverse proxy will listen on 80 and forward to the backend on another port). |
| + | |
| + | <pre> |
| + | Listen 8001 |
| + | </pre> |
| + | |
| + | If you are installing several Product Openers on the same machine, use different ports. |
| + | |
| + | Change the Apache user to your user: |
| + | |
| + | <pre> |
| + | User stephane |
| + | Group stephane |
| + | </pre> |
| + | |
| + | Add to apache/conf/httpd.conf: |
| | | |
| <pre> | | <pre> |
| + | LoadModule perl_module modules/mod_perl.so |
| | | |
− | en:Empty
| + | PerlWarn Off |
− | fr:Vide
| + | PerlRequire /home/stephane/product-opener/cgi/startup.pl |
− | es:Vacío
| |
− | pt:Vazio
| |
− | ro:Gol
| |
− | he:ריק
| |
| | | |
− | en:Checked
| |
− | fr:Vérifié
| |
− | es:Comprobado
| |
− | pt:Verificado
| |
− | ro:Verificat
| |
− | he:נבדק
| |
| | | |
− | en:To be checked
| + | <Location /cgi> |
− | fr:A vérifier
| + | SetHandler perl-script |
− | es:Pendiente de comprobación
| + | PerlResponseHandler ModPerl::Registry |
− | pt:Pendente de verificação
| + | PerlOptions +ParseHeaders |
− | ro:De verificat
| + | Options +ExecCGI |
− | he:לבדיקה
| + | Order allow,deny |
| + | Allow from all |
| + | </Location> |
| | | |
− | en:Complete
| |
− | fr:Complet
| |
− | es:Completado
| |
− | pt:Completo
| |
− | ro:Complet
| |
− | he:הושלם
| |
| | | |
− | en:To be completed
| + | <VirtualHost *> |
− | fr:A compléter
| + | DocumentRoot /home/stephane/product-opener/html |
− | es:A completar
| + | ServerName openfoodfacts.ovh |
− | pt:A completar
| + | ErrorLog /home/stephane/logs/error_log |
− | ro:De completat
| + | CustomLog /home/stephane/logs/access_log combined |
− | he:להשלמה
| + | ScriptAlias /cgi/ "/home/stephane/product-opener/cgi/" |
| + | </VirtualHost> |
| | | |
− | en:Expiration date to be completed
| + | PerlPostReadRequestHandler My::ProxyRemoteAddr |
− | fr:Date limite à compléter
| + | </pre> |
− | es:Fecha límite de consumo a completar
| |
− | pt:Data de validade a completar
| |
− | ro:De completat data de expirare
| |
− | he:תאריך תפוגה להשלמה
| |
| | | |
− | en:Expiration date completed
| + | === Apache 2.4 - light weight reverse proxy === |
− | fr:Date limite complétée
| |
− | es:Fecha límite de consumo agregada
| |
− | pt:Data de validade completada
| |
− | ro:Data de expirare completată
| |
− | he:תאריך התפוגה הושלם
| |
| | | |
− | en:Product name to be completed
| + | ==== Build ==== |
− | fr:Nom du produit à compléter
| |
− | es:Nombre de producto a completar
| |
− | pt:Nome do produto a completar
| |
− | ro:De completat numele produsului
| |
− | he:שם המוצר להשלמה
| |
| | | |
− | en:Quantity to be completed
| + | <pre> |
− | fr:Quantité à compléter
| + | apt-get install libpcre3 libpcre3-dev |
− | es:Cantidad a completar
| |
− | pt:Quantidade a completar
| |
− | ro:Cantitatea de completat
| |
− | he:כמות להשלמה
| |
| | | |
− | en:Packaging to be completed
| + | cd /home/stephane/src |
− | fr:Emballage à compléter
| + | wget http://apache.websitebeheerjd.nl//httpd/httpd-2.4.12.tar.gz |
− | es:Envase a completar
| + | tar xvfz httpd-2.4.12.tar.gz |
− | pt:Embalagem a completar
| + | cd httpd-2.4.12 |
− | ro:De completat ambalarea
| |
− | he:אריזה להשלמה
| |
| | | |
− | en:Brands to be completed
| + | cd srclib |
− | fr:Marques à compléter
| + | wget http://mir2.ovh.net/ftp.apache.org/dist//apr/apr-1.5.1.tar.gz |
− | es:Marcas a completar
| + | wget http://mir2.ovh.net/ftp.apache.org/dist//apr/apr-util-1.5.4.tar.gz |
− | pt:Marcas a completar
| + | tar xvfz apr-1.5.1.tar.gz |
− | ro:Mărci de completat
| + | tar xvfz apr-util-1.5.4.tar.gz |
− | he:מותגים להשלמה
| + | ln -s apr-1.5.1 apr |
| + | ln -s apr-util-1.5.4 apr-util |
| + | cd .. |
| | | |
− | en:Categories to be completed
| + | ./configure --prefix=/home/stephane/proxy --enable-rewrite --enable-proxy --enable-proxy_http --disable-userdir --enable-headers --enable-expires --enable-mime --enable-so --with-included-apr |
− | fr:Catégories à compléter
| |
− | es:Categorías a completar
| |
− | pt:Categorias a completar
| |
− | ro:Categorii de completat
| |
− | he:קטגוריות להשלמה
| |
| | | |
− | en:Characteristics completed
| + | make |
− | fr:Caractéristiques complétées
| + | make install |
− | es:Características completadas
| + | cd ~ |
− | pt:Características completadas
| + | </pre> |
− | ro:Caracteristici completate
| |
− | he:מאפיינים הושלמו
| |
| | | |
− | en:Characteristics to be completed
| + | ==== Configuration ==== |
− | fr:Caractéristiques à compléter
| |
− | es:Características a completar
| |
− | pt:Características a completar
| |
− | ro:Caracteristici de completat
| |
− | he:מאפיינים להשלמה
| |
| | | |
− | en:Ingredients completed
| + | in proxy/conf/httpd.conf : |
− | fr:Ingrédients complétés
| + | * uncomment #LoadModule rewrite_module modules/mod_rewrite.so |
− | es:Ingredientes completados
| + | * uncomment #LoadModule slotmem_shm_module modules/mod_slotmem_shm.so |
− | pt:Ingredientes completados
| |
− | ro:Ingrediente completate
| |
− | he:הרכיבים הושלמו
| |
| | | |
− | en:Ingredients to be completed
| + | Add in httpd.conf : |
− | fr:Ingrédients à compléter
| |
− | es:Ingredientes a completar
| |
− | ro:Ingrediente de completat
| |
− | he:רכיבים להשלמה
| |
| | | |
− | en:Nutrition facts completed
| + | <pre> |
− | fr:Informations nutritionnelles complétées
| + | <VirtualHost *> |
− | es:Valores nutricionales completados
| + | DocumentRoot /home/stephane/product-opener/html |
− | pt:Valores nutricionais completados
| + | ServerName world.openfoodfacts.ovh |
− | ro:Valori nutriționale completate
| + | ServerAlias *.openfoodfacts.ovh |
− | he:מפרט תזונתי הושלם
| |
| | | |
− | en:Nutrition facts to be completed
| + | ErrorLog /home/stephane/logs/proxy_error_log |
− | fr:Informations nutritionnelles à compléter
| + | CustomLog /home/stephane/logs/proxy_access_log combined |
− | es:Valores nutricionales a completar
| + | ServerAdmin stephane@openfoodfacts.org |
− | pt:Valores nutricionais a completar
| |
− | ro:Valori nutriționale de completat
| |
− | he:מפרט תזונתי להשלמה
| |
| | | |
− | en:Photos validated
| + | <Directory "/home/stephane/product-opener/html"> |
− | fr:Photos validées
| + | Options Indexes FollowSymLinks |
− | es:Imágenes validadas
| + | Require all granted |
− | pt:Imagens validadas
| + | </Directory> |
− | ro:Fotografii validate
| + | ProxyPreserveHost On |
− | he:התמונות אומתו
| + | RewriteEngine on |
| + | RewriteCond %{REQUEST_URI} !/./ |
| + | RewriteRule ^(/cgi/.*)$ http://localhost:8001$1 [P,L] |
| + | RewriteMap escape int:escape |
| + | RewriteRule ^/favicon.ico$ /favicon.ico [L] |
| + | RewriteCond %{REQUEST_URI} !^/images/ |
| + | RewriteCond %{REQUEST_URI} !^/js/ |
| + | RewriteCond %{REQUEST_URI} !^/rss/ |
| + | RewriteCond %{REQUEST_URI} !^/robots |
| + | RewriteCond %{REQUEST_URI} !^/clicks/ |
| + | RewriteCond %{REQUEST_URI} !^/data/ |
| + | RewriteCond %{REQUEST_URI} !^/files/ |
| + | RewriteRule ^(.*)$ http://localhost:8001/cgi/display.pl?${escape:$1} [P,L,QSA] |
| + | </VirtualHost> |
| | | |
− | en:Photos to be validated
| |
− | fr:Photos à valider
| |
− | es:Imágenes a validar
| |
− | pt:Imagens a validar
| |
− | ro:Fotografii de validat
| |
− | he:תמונות לאימות
| |
| | | |
− | en:Photos uploaded
| + | <VirtualHost *> |
− | fr:Photos envoyées
| + | DocumentRoot /home/stephane/product-opener/html |
− | es:Imágenes agregadas
| + | ServerName openfoodfacts.ovh |
− | pt:Imagens enviadas
| + | ErrorLog /home/stephane/logs/proxy_error_log |
− | ro:Fotografii încărcate
| + | CustomLog /home/stephane/logs/proxy_access_log combined |
− | he:התמונות הועלו
| + | DirectoryIndex index.html index.shtml |
| + | <Directory "/home/stephane/product-opener/html"> |
| + | Options -Indexes +FollowSymLinks +Includes |
| + | Require all granted |
| + | </Directory> |
| + | RewriteEngine on |
| + | RewriteCond %{HTTP_HOST} ^stephane\.openfoodfacts\.ovh |
| + | RewriteRule ^/products$ /products.shtml [L] |
| | | |
− | en:Photos to be uploaded
| + | </VirtualHost> |
− | fr:Photos à envoyer
| |
− | es:Imágenes a agregar
| |
− | pt:Imagens a enviar
| |
− | ro:Fotografii de încărcat
| |
− | he:תמונות להעלאה
| |
| | | |
| </pre> | | </pre> |
| + | |
| + | |
| + | |
| + | === Install Perl modules === |
| + | |
| + | If possible, use apt-get to install packages directly. If they are not available, you can use cpan to install the modules. |
| + | |
| + | <pre> |
| + | apt-get install libwww-perl libimage-magick-perl |
| + | apt-get install libxml-encoding-perl libtext-unaccent-perl libmime-lite-perl |
| + | apt-get install libcache-memcached-fast-perl libjson-perl libclone-perl libgraphviz-perl |
| + | apt-get install libmime-lite-perl # needs to be done after installing exim4 |
| + | apt-get install libcrypt-passwdmd5-perl libencode-detect-perl libgraphics-color-perl libbarcode-zbar-perl |
| + | apt-get install libxml-feedpp-perl libmongodb-perl liburi-find-perl |
| + | </pre> |
| + | |
| + | Modules that could not be installed using apt-get: |
| + | |
| + | <pre> |
| + | cpan |
| + | install URI::Escape::XS |
| + | install Image::OCR::Tesseract # needs imagemagick installed first |
| + | </pre> |
| + | |
| + | Modules that HAVE to be installed using cpan: |
| + | |
| + | As of Feb 3rd 2015, the MongoDB perl module installed through "apt-get install libmongodb-perl" is too old (0.45): |
| + | |
| + | <pre> |
| + | perl -MMongoDB -e 'print $MongoDB::VERSION . "\n"' |
| + | 0.45 |
| + | </pre> |
| + | |
| + | This causes an error when trying to access a list of "tags" such as categories: |
| + | |
| + | <pre> |
| + | Software error: |
| + | Not an ARRAY reference at /home/stephane/product-opener/cgi/Blogs/Display.pm line 797. |
| + | </pre> |
| + | |
| + | A newer MongoDB perl module needs to be installed with cpan. |
| + | |
| + | <pre> |
| + | cpan |
| + | install MongoDB |
| + | </pre> |
| + | |
| + | <pre> |
| + | perl -MMongoDB -e 'print $MongoDB::VERSION . "\n"' |
| + | v0.708.0.0 |
| + | </pre> |
| + | |
| + | cgi/startup.pl preloads a lot of modules, you can check that those modules are installed by running: |
| + | |
| + | <pre> |
| + | cd /home/stephane/product-opener/cgi |
| + | perl startup.pl |
| + | </pre> |
| + | |
| + | === Create some paths and links === |
| + | |
| + | <pre> |
| + | cd /home/stephane/product-opener/cgi |
| + | ln -s ./ Blogs |
| + | ln -s S |