Changes

Jump to navigation Jump to search
m
Line 1: Line 1:  
Also see [[Product Opener Installation]]
 
Also see [[Product Opener Installation]]
 +
[[Category:ProductOpener]]
 +
This page lists all the steps necessary to install Product Opener on Debian, including a development environment.
 +
 +
= Warning! Those instructions are outdated and kept for reference only =
 +
 +
Please see http://en.wiki.openfoodfacts.org/Infrastructure for up-to-date install instructions.
 +
 +
= Installation on Jessie =
 +
 +
We start from a clean Jessie install.
 +
 +
== Get the code and configure it ==
 +
 +
<pre>
 +
apt-get install git
 +
 +
git clone <repo> /srv/openfoodfacts
 +
mkdir /srv/openfoodfacts/users
 +
mkdir /srv/openfoodfacts/products
 +
chown -R www-data:www-data /srv/openfoodfacts
 +
 +
cd /srv/openfoodfacts/cgi
 +
rm Blogs
 +
ln -s . Blogs
 +
ln -s SiteLang_off.pm SiteLang.pm
 +
</pre>
 +
 +
Edit a few variables in Config2.pm:
 +
<pre>
 +
$server_domain = "yourdomain.com";
 +
$www_root="/srv/openfoodfacts/html";
 +
$data_root="/srv/openfoodfacts";
 +
</pre>
 +
 +
Also in Config_off.pm:
 +
<pre>
 +
$domain = 'yourdomain.com';
 +
$contact_email = 'webmaster@yourdomain.com';
 +
$admin_email = 'webmaster@yourdomain.com';
 +
</pre>
 +
 +
And in startup.pl:
 +
<pre>
 +
use lib "/srv/openfoodfacts/cgi/";
 +
</pre>
 +
 +
== Install packaged prerequisites ==
 +
 +
<pre>
 +
apt-get install apache2 build-essential geoip-bin geoip-database imagemagick libbarcode-zbar-perl libcache-memcached-fast-perl libclone-perl libcrypt-passwdmd5-perl libdatetime-format-mail-perl \
 +
libencode-detect-perl libgeo-ip-perl libgraphics-color-perl libgraphviz-perl libhtml-defang-perl libimage-magick-perl libjson-perl libmime-lite-perl libmime-lite-perl libmongodb-perl libperl-dev \
 +
libtest-nowarnings-perl libtext-unaccent-perl liburi-escape-xs-perl liburi-find-perl libwww-perl libxml-encoding-perl libxml-feedpp-perl memcached mongodb tesseract-ocr tesseract-ocr-fra zlib1g-dev
 +
</pre>
 +
 +
== Prepare CPAN ==
 +
If you never used CPAN before, you can automatically configure it:
 +
<pre>
 +
(echo y;echo o conf prerequisites_policy follow;echo o conf commit)|cpan
 +
</pre>
 +
 +
== Install a few Perl modules from CPAN ==
 +
 +
<pre>
 +
cpan Net::IDN::Punycode
 +
cpan Encode::Punycode
 +
cpan GraphViz2
 +
cpan Image::OCR::Tesseract
 +
cpan Algorithm::CheckDigits
 +
cpan XML::Simple
 +
</pre>
 +
 +
Note: on Debian Jessie, you might need to install the follwoing to be able to get cpan to install GraphViz2
 +
<pre>
 +
apt-get install libtest-version-perl
 +
</pre>
 +
 +
== Rebuild Apache 2.2 and mod_perl ==
 +
 +
<pre>
 +
cd /usr/local/src/
 +
wget http://apache.crihan.fr/dist/httpd/httpd-2.2.31.tar.gz
 +
tar xvfz httpd-2.2.31.tar.gz
 +
cd httpd-2.2.31
 +
./configure --with-mpm=prefork --prefix=/usr/local  --enable-rewrite --enable-proxy --enable-proxy_http --enable-deflate --disable-userdir --enable-headers
 +
make
 +
make install
 +
 +
cd /usr/local/src/
 +
wget http://mir2.ovh.net/ftp.apache.org/dist/perl/mod_perl-2.0.9.tar.gz
 +
tar xvfz mod_perl-2.0.9.tar.gz
 +
cd mod_perl-2.0.9
 +
perl Makefile.PL MP_APXS=/usr/local/bin/apxs
 +
make
 +
make install
 +
 +
cat >> /usr/local/conf/httpd.conf <<EOF
 +
Listen 8001
 +
 +
LoadModule perl_module modules/mod_perl.so
 +
 +
PerlWarn Off
 +
PerlRequire /srv/openfoodfacts/cgi/startup.pl
 +
 +
<VirtualHost *:8001>
 +
  DocumentRoot /srv/openfoodfacts/html
 +
  ServerName yourdomain.com
 +
  ErrorLog /var/log/apache2/error.log
 +
  CustomLog /var/log/apache2/access.log combined
 +
  ScriptAlias /cgi/ /srv/openfoodfacts/cgi/
 +
 +
  <Location /cgi>
 +
    SetHandler perl-script
 +
    PerlResponseHandler ModPerl::Registry
 +
    PerlOptions +ParseHeaders
 +
    Options +ExecCGI
 +
    Order allow,deny
 +
    Allow from all
 +
  </Location>
 +
</VirtualHost>
 +
 +
PerlPostReadRequestHandler My::ProxyRemoteAddr
 +
EOF
 +
 +
sed -i -e 's/^Listen 80$/#Listen 80/' /usr/local/conf/httpd.conf
 +
sed -i -e 's/^User.*/User www-data/' /usr/local/conf/httpd.conf
 +
sed -i -e 's/^Group.*/Group www-data/' /usr/local/conf/httpd.conf
 +
 +
/usr/local/bin/apachectl -k start
 +
</pre>
 +
 +
== Configure standard Apache 2 ==
 +
 +
<pre>
 +
a2enmod proxy_http
 +
a2enmod rewrite
 +
 +
cat >> /etc/apache2/sites-available/product-opener-proxy.conf <<EOF
 +
<VirtualHost *>
 +
  DocumentRoot /srv/openfoodfacts/html
 +
  ServerName yourdomain.com
 +
  ServerAlias *.yourdomain.com
 +
 +
  <Directory "/srv/openfoodfacts/html">
 +
    Options Indexes FollowSymLinks
 +
    Require all granted
 +
  </Directory>
 +
  ProxyPreserveHost On
 +
  RewriteEngine on
 +
  RewriteCond  %{REQUEST_URI}  !/./
 +
  RewriteRule ^(/cgi/.*)$ http://localhost:8001$1 [P,L]
 +
  RewriteMap escape int:escape
 +
  RewriteRule ^/favicon.ico$ /favicon.ico [L]
 +
  RewriteCond  %{REQUEST_URI}  !^/images/
 +
  RewriteCond  %{REQUEST_URI}  !^/js/
 +
  RewriteCond  %{REQUEST_URI}  !^/foundation/
 +
  RewriteCond  %{REQUEST_URI}  !^/rss/
 +
  RewriteCond  %{REQUEST_URI}  !^/robots
 +
  RewriteCond  %{REQUEST_URI}  !^/clicks/
 +
  RewriteCond  %{REQUEST_URI}  !^/data/
 +
  RewriteCond  %{REQUEST_URI}  !^/files/
 +
  RewriteRule  ^(.*)$ http://localhost:8001/cgi/display.pl?${escape:$1} [P,L,QSA]
 +
</VirtualHost>
 +
EOF
 +
 +
a2ensite product-opener-proxy.conf
 +
a2dissite 000-default.conf
 +
service apache2 restart
 +
</pre>
 +
 +
You should now have a working Productopener instance running at http://yourdomain.com/. Enjoy!
 +
 +
= Installation on Debian Jessie with stock Apache and nginx =
 +
 +
<pre>
 +
Vous êtes sur le point d'installer un OS sur le serveur ks3095298.kimsufi.com. Choisissez le type d'installation que vous souhaitez utiliser :
 +
 +
-> Installer à partir d'un template OVH
 +
 +
Debian 8.1 stable (Jessie) (stable) - debian8
 +
 +
root@ks3095298:~# uname -a
 +
Linux ks3095298.kimsufi.com 3.14.32-xxxx-grs-ipv6-64 #5 SMP Wed Sep 9 17:24:34 CEST 2015 x86_64 GNU/Linux
 +
 +
perl -v
 +
 +
This is perl 5, version 20, subversion 2 (v5.20.2) built for x86_64-linux-gnu-thread-multi
 +
 +
 +
apt-get update
 +
apt-get upgrade
 +
 +
apt-get install fail2ban
 +
apt-get install sudo
 +
 +
apt-get install mongodb
   −
This page lists all the steps necessary to install Product Opener on Debian, including a development environment.
+
MongoDB shell version: 2.4.10
 +
 
 +
vi /etc/mongodb.conf
 +
- change dbpath
 +
- uncomment :
 +
#port = 27017
 +
#nohttpinterface = true
 +
 
 +
 
 +
apt-get install git
 +
 
 +
apt-get install apache2
 +
apt-get install libapache2-mod-perl2
 +
apt-get install modperl
 +
 
 +
adduser off
 +
 
 +
apt-get install imagemagick
 +
apt-get install memcached
 +
apt-get install tesseract-ocr tesseract-ocr-fra
 +
apt-get install exim4
 +
dpkg-reconfigure exim4-config
 +
apt-get install geoip-bin geoip-database
 +
apt-get install zlib1g-dev
 +
 
 +
apt-get install libwww-perl libimage-magick-perl libxml-encoding-perl libtext-unaccent-perl libmime-lite-perl libcache-memcached-fast-perl libjson-perl libclone-perl  libgraphviz-perl libmime-lite-perl libcrypt-passwdmd5-perl libencode-detect-perl libgraphics-color-perl libbarcode-zbar-perl libxml-feedpp-perl libmongodb-perl liburi-find-perl libxml-simple-perl
 +
 
 +
 
 +
cp -a /home/openfoodfacts-bitbucket/product-opener/cgi /home/off/cgi/
 +
 
 +
/home/off/cgi# rm Blogs
 +
root@ks3095298:/home/off/cgi# ln -s /home/off/cgi Blogs
 +
 
 +
ln -s SiteLang_off.pm SiteLang.pm
 +
 
 +
vi /home/off/startup.pl
 +
# Needs to be configured
 +
use lib "/home/off/cgi/";
 +
 
 +
 
 +
ln -s ../sites-available/001-off.conf  001-off.conf
 +
service apache2 restart
 +
 
 +
To see errors, missing perl modules etc.
 +
 
 +
/var/log/apache2# tail -f error.log
 +
 
 +
cpan
 +
install URI::Escape::XS
 +
install Encode::Punycode
 +
install GraphViz2
 +
install HTML::Defang
 +
install Algorithm::CheckDigits
 +
install Geo::IP
 +
install Image::OCR::Tesseract
 +
DateTime::Format::Mail
 +
DateTime::Format::CLDR
 +
DateTime::Locale
 +
 
 +
 
 +
 
 +
 
 +
Issues:
 +
   
 +
MongoDB does not start:
 +
   
 +
    service mongodb start
 +
root@ks3095298:/home/off/cgi# ps -ef | grep mongo
 +
root    21814 23471  0 23:56 pts/0    00:00:00 grep mongo
 +
 
 +
 
 +
in /var/log/syslog:
 +
   
 +
    Jan 11 00:05:05 ks3095298 systemd[1]: Starting An object/document-oriented database...
 +
Jan 11 00:05:05 ks3095298 systemd[1]: Started An object/document-oriented database.
 +
Jan 11 00:05:05 ks3095298 mongod[22404]: Mon Jan 11 00:05:05.434 terminate() called, printing stack (if implemented for platform):
 +
Jan 11 00:05:05 ks3095298 mongod[22404]: 0xa6d3c6 0x57e204 0x7212186b7bb6 0x7212186b7c01 0x7212186b7e19 0x7212192b666d 0xa6070d 0xa61a67 0xa5f8ca 0xa5f979 0x5a626c 0x5a65d2 0x5a6a93 0x5a6ab9 0x5a6e5c 0x56647b 0x721217dbab45 0x57e05c
 +
Jan 11 00:05:05 ks3095298 mongod[22404]: /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x26) [0xa6d3c6]
 +
 
 +
 
 +
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com --recv EA312927
 +
 
 +
 
 +
Text::Unaccent was causing Apache to segfault, replaced the module with pure perl
 +
 
 +
remote_ip has been replaced in Apache 2.4, need to check if we do get the X-Forwarder-For ip directly now
 +
 
 +
 
 +
Loading products from /products in the mongodb database:
 +
   
 +
update_all_products_from_dir_in_mongodb.pl
 +
   
 +
   
 +
 
 +
Trying to use nginx for the reverse proxy:
 +
   
 +
apt-get install nginx  (need port 80 to be available, stop apache or change its port first)
 +
 
 +
 
 +
 
 +
root@ks3095298:/etc/nginx/sites-enabled# more /etc/nginx/sites-available/off
 +
##
 +
# You should look at the following URL's in order to grasp a solid understanding
 +
# of Nginx configuration files in order to fully unleash the power of Nginx.
 +
# http://wiki.nginx.org/Pitfalls
 +
# http://wiki.nginx.org/QuickStart
 +
# http://wiki.nginx.org/Configuration
 +
#
 +
# Generally, you will want to move this file somewhere, and start with a clean
 +
# file but keep this around for reference. Or just disable in sites-enabled.
 +
#
 +
# Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples.
 +
##
 +
 
 +
# Default server configuration
 +
#
 +
server {
 +
        listen 80 default_server;
 +
        listen [::]:80 default_server;
 +
 
 +
        server_name openfoodfacts.ovh *.openfoodfacts.ovh;
 +
 
 +
        # SSL configuration
 +
        #
 +
        # listen 443 ssl default_server;
 +
        # listen [::]:443 ssl default_server;
 +
        #
 +
        # Self signed certs generated by the ssl-cert package
 +
        # Don't use them in a production server!
 +
        #
 +
        # include snippets/snakeoil.conf;
 +
 
 +
        root /home/off/html;
 +
 
 +
        # Add index.php to the list if you are using PHP
 +
        index index.html index.htm index.nginx-debian.html;
 +
 
 +
        location ~ ^/(images|js|rss|data|files|resources|foundation)/ {
 +
                # First attempt to serve request as file, then
 +
                # as directory, then fall back to displaying a 404.
 +
                try_files $uri $uri/ =404;
 +
        }
 +
 
 +
        location = /robots.txt {
 +
                try_files $uri $uri/ =404;
 +
        }
 +
 
 +
        location / {
 +
                proxy_set_header Host $host;
 +
                proxy_set_header X-Real-IP $remote_addr;
 +
                proxy_pass http://127.0.0.1:8001/cgi/display.pl?;
 +
        }
 +
 
 +
        location /cgi/ {
 +
                proxy_set_header Host $host;
 +
                proxy_set_header X-Real-IP $remote_addr;
 +
                proxy_pass http://127.0.0.1:8001;
 +
        }
 +
 
 +
        # deny access to .htaccess files, if Apache's document root
 +
        # concurs with nginx's one
 +
        #
 +
        #location ~ /\.ht {
 +
        #      deny all;
 +
        #}
 +
}
 +
 
 +
 
 +
</pre>
 +
 
 +
= Old notes =
    
All instructions below are for user "stephane", please use your own name. :-)
 
All instructions below are for user "stephane", please use your own name. :-)
Line 120: Line 485:  
make
 
make
 
make install
 
make install
 +
cd ~
    +
cd ~/src
 
wget http://mir2.ovh.net/ftp.apache.org/dist/perl/mod_perl-2.0.8.tar.gz  
 
wget http://mir2.ovh.net/ftp.apache.org/dist/perl/mod_perl-2.0.8.tar.gz  
 
tar xvfz mod_perl-2.0.8.tar.gz
 
tar xvfz mod_perl-2.0.8.tar.gz
Line 127: Line 494:  
make
 
make
 
make install
 
make install
 +
cd ~
 
</pre>
 
</pre>
    
==== Configuration ====
 
==== Configuration ====
   −
In apache/conf/httpd.conf:
+
In ~/apache/conf/httpd.conf:
    
Change the port to something else than 80 (the reverse proxy will listen on 80 and forward to the backend on another port).
 
Change the port to something else than 80 (the reverse proxy will listen on 80 and forward to the backend on another port).
Line 177: Line 545:  
PerlPostReadRequestHandler My::ProxyRemoteAddr
 
PerlPostReadRequestHandler My::ProxyRemoteAddr
 
</pre>
 
</pre>
      
=== Apache 2.4 - light weight reverse proxy ===
 
=== Apache 2.4 - light weight reverse proxy ===
Line 196: Line 563:  
tar xvfz apr-1.5.1.tar.gz
 
tar xvfz apr-1.5.1.tar.gz
 
tar xvfz apr-util-1.5.4.tar.gz  
 
tar xvfz apr-util-1.5.4.tar.gz  
ln -s apr-1.5.1 ap
+
ln -s apr-1.5.1 apr
 
ln -s apr-util-1.5.4 apr-util
 
ln -s apr-util-1.5.4 apr-util
 
cd ..  
 
cd ..  
Line 203: Line 570:     
make
 
make
make install
+
sudo make install
 +
cd ~
 
</pre>
 
</pre>
    
==== Configuration ====
 
==== Configuration ====
   −
in proxy/conf/httpd.conf :
+
in ~/proxy/conf/httpd.conf :
 
* uncomment #LoadModule rewrite_module modules/mod_rewrite.so
 
* uncomment #LoadModule rewrite_module modules/mod_rewrite.so
 
* uncomment #LoadModule slotmem_shm_module modules/mod_slotmem_shm.so
 
* uncomment #LoadModule slotmem_shm_module modules/mod_slotmem_shm.so
Line 223: Line 591:  
CustomLog /home/stephane/logs/proxy_access_log combined
 
CustomLog /home/stephane/logs/proxy_access_log combined
 
ServerAdmin stephane@openfoodfacts.org
 
ServerAdmin stephane@openfoodfacts.org
 +
 +
<Location "/foundation/foundation-icons" >
 +
Header set Access-Control-Allow-Origin "*"
 +
</Location>
 +
 +
<Location "/images/products" >
 +
Header set Link "<http://creativecommons.org/licenses/by-sa/3.0/>; rel='license'; title='CC-BY-SA 3.0'"
 +
</Location>
 +
    
<Directory "/home/stephane/product-opener/html">
 
<Directory "/home/stephane/product-opener/html">
Line 241: Line 618:  
RewriteCond  %{REQUEST_URI}  !^/data/
 
RewriteCond  %{REQUEST_URI}  !^/data/
 
RewriteCond  %{REQUEST_URI}  !^/files/
 
RewriteCond  %{REQUEST_URI}  !^/files/
 +
RewriteCond  %{REQUEST_URI}  !^/resources/
 +
RewriteCond  %{REQUEST_URI}  !^/foundation/
 
RewriteRule  ^(.*)$ http://localhost:8001/cgi/display.pl?${escape:$1} [P,L,QSA]
 
RewriteRule  ^(.*)$ http://localhost:8001/cgi/display.pl?${escape:$1} [P,L,QSA]
 
</VirtualHost>
 
</VirtualHost>
Line 262: Line 641:     
</pre>
 
</pre>
  −
      
=== Install Perl modules ===
 
=== Install Perl modules ===
Line 327: Line 704:  
ln -s ./ Blogs
 
ln -s ./ Blogs
 
ln -s S
 
ln -s S
 +
</pre>
 +
 +
=== Update IP to country GeoIP database ===
 +
 +
Product Opener uses the free legacy GeoIP Lite database from Maxmind:
 +
* http://search.cpan.org/~maxmind/Geo-IP-1.45/lib/Geo/IP.pm
 +
* http://dev.maxmind.com/geoip/legacy/geolite/
 +
 +
The database installed with apt-get install geoip-database can be very old.
 +
To update manually, download the new databases to:
 +
 +
<pre>
 +
/usr/share/GeoIP# ls -lrt
 +
total 7256
 +
-rw-r--r-- 1 root root 1103675  5 déc.  2013 GeoIPv6.dat
 +
-rw-r--r-- 1 root root  600352  5 déc.  2013 GeoIP.dat
 +
 +
 +
wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
 +
gzip -d GeoIP.dat.gz
 +
</pre>

Navigation menu