Changes

Jump to navigation Jump to search
m
Line 1: Line 1:  
Also see [[Product Opener Installation]]
 
Also see [[Product Opener Installation]]
 +
[[Category:ProductOpener]]
 +
This page lists all the steps necessary to install Product Opener on Debian, including a development environment.
 +
 +
= Warning! Those instructions are outdated and kept for reference only =
 +
 +
Please see http://en.wiki.openfoodfacts.org/Infrastructure for up-to-date install instructions.
 +
 +
= Installation on Jessie =
 +
 +
We start from a clean Jessie install.
 +
 +
== Get the code and configure it ==
 +
 +
<pre>
 +
apt-get install git
 +
 +
git clone <repo> /srv/openfoodfacts
 +
mkdir /srv/openfoodfacts/users
 +
mkdir /srv/openfoodfacts/products
 +
chown -R www-data:www-data /srv/openfoodfacts
 +
 +
cd /srv/openfoodfacts/cgi
 +
rm Blogs
 +
ln -s . Blogs
 +
ln -s SiteLang_off.pm SiteLang.pm
 +
</pre>
 +
 +
Edit a few variables in Config2.pm:
 +
<pre>
 +
$server_domain = "yourdomain.com";
 +
$www_root="/srv/openfoodfacts/html";
 +
$data_root="/srv/openfoodfacts";
 +
</pre>
 +
 +
Also in Config_off.pm:
 +
<pre>
 +
$domain = 'yourdomain.com';
 +
$contact_email = 'webmaster@yourdomain.com';
 +
$admin_email = 'webmaster@yourdomain.com';
 +
</pre>
 +
 +
And in startup.pl:
 +
<pre>
 +
use lib "/srv/openfoodfacts/cgi/";
 +
</pre>
 +
 +
== Install packaged prerequisites ==
 +
 +
<pre>
 +
apt-get install apache2 build-essential geoip-bin geoip-database imagemagick libbarcode-zbar-perl libcache-memcached-fast-perl libclone-perl libcrypt-passwdmd5-perl libdatetime-format-mail-perl \
 +
libencode-detect-perl libgeo-ip-perl libgraphics-color-perl libgraphviz-perl libhtml-defang-perl libimage-magick-perl libjson-perl libmime-lite-perl libmime-lite-perl libmongodb-perl libperl-dev \
 +
libtest-nowarnings-perl libtext-unaccent-perl liburi-escape-xs-perl liburi-find-perl libwww-perl libxml-encoding-perl libxml-feedpp-perl memcached mongodb tesseract-ocr tesseract-ocr-fra zlib1g-dev
 +
</pre>
 +
 +
== Prepare CPAN ==
 +
If you never used CPAN before, you can automatically configure it:
 +
<pre>
 +
(echo y;echo o conf prerequisites_policy follow;echo o conf commit)|cpan
 +
</pre>
 +
 +
== Install a few Perl modules from CPAN ==
 +
 +
<pre>
 +
cpan Net::IDN::Punycode
 +
cpan Encode::Punycode
 +
cpan GraphViz2
 +
cpan Image::OCR::Tesseract
 +
cpan Algorithm::CheckDigits
 +
cpan XML::Simple
 +
</pre>
 +
 +
Note: on Debian Jessie, you might need to install the follwoing to be able to get cpan to install GraphViz2
 +
<pre>
 +
apt-get install libtest-version-perl
 +
</pre>
 +
 +
== Rebuild Apache 2.2 and mod_perl ==
 +
 +
<pre>
 +
cd /usr/local/src/
 +
wget http://apache.crihan.fr/dist/httpd/httpd-2.2.31.tar.gz
 +
tar xvfz httpd-2.2.31.tar.gz
 +
cd httpd-2.2.31
 +
./configure --with-mpm=prefork --prefix=/usr/local  --enable-rewrite --enable-proxy --enable-proxy_http --enable-deflate --disable-userdir --enable-headers
 +
make
 +
make install
 +
 +
cd /usr/local/src/
 +
wget http://mir2.ovh.net/ftp.apache.org/dist/perl/mod_perl-2.0.9.tar.gz
 +
tar xvfz mod_perl-2.0.9.tar.gz
 +
cd mod_perl-2.0.9
 +
perl Makefile.PL MP_APXS=/usr/local/bin/apxs
 +
make
 +
make install
 +
 +
cat >> /usr/local/conf/httpd.conf <<EOF
 +
Listen 8001
 +
 +
LoadModule perl_module modules/mod_perl.so
 +
 +
PerlWarn Off
 +
PerlRequire /srv/openfoodfacts/cgi/startup.pl
 +
 +
<VirtualHost *:8001>
 +
  DocumentRoot /srv/openfoodfacts/html
 +
  ServerName yourdomain.com
 +
  ErrorLog /var/log/apache2/error.log
 +
  CustomLog /var/log/apache2/access.log combined
 +
  ScriptAlias /cgi/ /srv/openfoodfacts/cgi/
 +
 +
  <Location /cgi>
 +
    SetHandler perl-script
 +
    PerlResponseHandler ModPerl::Registry
 +
    PerlOptions +ParseHeaders
 +
    Options +ExecCGI
 +
    Order allow,deny
 +
    Allow from all
 +
  </Location>
 +
</VirtualHost>
 +
 +
PerlPostReadRequestHandler My::ProxyRemoteAddr
 +
EOF
 +
 +
sed -i -e 's/^Listen 80$/#Listen 80/' /usr/local/conf/httpd.conf
 +
sed -i -e 's/^User.*/User www-data/' /usr/local/conf/httpd.conf
 +
sed -i -e 's/^Group.*/Group www-data/' /usr/local/conf/httpd.conf
 +
 +
/usr/local/bin/apachectl -k start
 +
</pre>
 +
 +
== Configure standard Apache 2 ==
 +
 +
<pre>
 +
a2enmod proxy_http
 +
a2enmod rewrite
 +
 +
cat >> /etc/apache2/sites-available/product-opener-proxy.conf <<EOF
 +
<VirtualHost *>
 +
  DocumentRoot /srv/openfoodfacts/html
 +
  ServerName yourdomain.com
 +
  ServerAlias *.yourdomain.com
 +
 +
  <Directory "/srv/openfoodfacts/html">
 +
    Options Indexes FollowSymLinks
 +
    Require all granted
 +
  </Directory>
 +
  ProxyPreserveHost On
 +
  RewriteEngine on
 +
  RewriteCond  %{REQUEST_URI}  !/./
 +
  RewriteRule ^(/cgi/.*)$ http://localhost:8001$1 [P,L]
 +
  RewriteMap escape int:escape
 +
  RewriteRule ^/favicon.ico$ /favicon.ico [L]
 +
  RewriteCond  %{REQUEST_URI}  !^/images/
 +
  RewriteCond  %{REQUEST_URI}  !^/js/
 +
  RewriteCond  %{REQUEST_URI}  !^/foundation/
 +
  RewriteCond  %{REQUEST_URI}  !^/rss/
 +
  RewriteCond  %{REQUEST_URI}  !^/robots
 +
  RewriteCond  %{REQUEST_URI}  !^/clicks/
 +
  RewriteCond  %{REQUEST_URI}  !^/data/
 +
  RewriteCond  %{REQUEST_URI}  !^/files/
 +
  RewriteRule  ^(.*)$ http://localhost:8001/cgi/display.pl?${escape:$1} [P,L,QSA]
 +
</VirtualHost>
 +
EOF
 +
 +
a2ensite product-opener-proxy.conf
 +
a2dissite 000-default.conf
 +
service apache2 restart
 +
</pre>
 +
 +
You should now have a working Productopener instance running at http://yourdomain.com/. Enjoy!
 +
 +
= Installation on Debian Jessie with stock Apache and nginx =
 +
 +
<pre>
 +
Vous êtes sur le point d'installer un OS sur le serveur ks3095298.kimsufi.com. Choisissez le type d'installation que vous souhaitez utiliser :
 +
 +
-> Installer à partir d'un template OVH
 +
 +
Debian 8.1 stable (Jessie) (stable) - debian8
 +
 +
root@ks3095298:~# uname -a
 +
Linux ks3095298.kimsufi.com 3.14.32-xxxx-grs-ipv6-64 #5 SMP Wed Sep 9 17:24:34 CEST 2015 x86_64 GNU/Linux
 +
 +
perl -v
 +
 +
This is perl 5, version 20, subversion 2 (v5.20.2) built for x86_64-linux-gnu-thread-multi
 +
 +
 +
apt-get update
 +
apt-get upgrade
 +
 +
apt-get install fail2ban
 +
apt-get install sudo
 +
 +
apt-get install mongodb
   −
This page lists all the steps necessary to install Product Opener on Debian, including a development environment.
+
MongoDB shell version: 2.4.10
 +
 
 +
vi /etc/mongodb.conf
 +
- change dbpath
 +
- uncomment :
 +
#port = 27017
 +
#nohttpinterface = true
 +
 
 +
 
 +
apt-get install git
 +
 
 +
apt-get install apache2
 +
apt-get install libapache2-mod-perl2
 +
apt-get install modperl
 +
 
 +
adduser off
 +
 
 +
apt-get install imagemagick
 +
apt-get install memcached
 +
apt-get install tesseract-ocr tesseract-ocr-fra
 +
apt-get install exim4
 +
dpkg-reconfigure exim4-config
 +
apt-get install geoip-bin geoip-database
 +
apt-get install zlib1g-dev
 +
 
 +
apt-get install libwww-perl libimage-magick-perl libxml-encoding-perl libtext-unaccent-perl libmime-lite-perl libcache-memcached-fast-perl libjson-perl libclone-perl  libgraphviz-perl libmime-lite-perl libcrypt-passwdmd5-perl libencode-detect-perl libgraphics-color-perl libbarcode-zbar-perl libxml-feedpp-perl libmongodb-perl liburi-find-perl libxml-simple-perl
 +
 
 +
 
 +
cp -a /home/openfoodfacts-bitbucket/product-opener/cgi /home/off/cgi/
 +
 
 +
/home/off/cgi# rm Blogs
 +
root@ks3095298:/home/off/cgi# ln -s /home/off/cgi Blogs
 +
 
 +
ln -s SiteLang_off.pm SiteLang.pm
 +
 
 +
vi /home/off/startup.pl
 +
# Needs to be configured
 +
use lib "/home/off/cgi/";
 +
 
 +
 
 +
ln -s ../sites-available/001-off.conf  001-off.conf
 +
service apache2 restart
 +
 
 +
To see errors, missing perl modules etc.
 +
 
 +
/var/log/apache2# tail -f error.log
 +
 
 +
cpan
 +
install URI::Escape::XS
 +
install Encode::Punycode
 +
install GraphViz2
 +
install HTML::Defang
 +
install Algorithm::CheckDigits
 +
install Geo::IP
 +
install Image::OCR::Tesseract
 +
DateTime::Format::Mail
 +
DateTime::Format::CLDR
 +
DateTime::Locale
 +
 
 +
 
 +
 
 +
 
 +
Issues:
 +
   
 +
MongoDB does not start:
 +
   
 +
    service mongodb start
 +
root@ks3095298:/home/off/cgi# ps -ef | grep mongo
 +
root    21814 23471  0 23:56 pts/0    00:00:00 grep mongo
 +
 
 +
 
 +
in /var/log/syslog:
 +
   
 +
    Jan 11 00:05:05 ks3095298 systemd[1]: Starting An object/document-oriented database...
 +
Jan 11 00:05:05 ks3095298 systemd[1]: Started An object/document-oriented database.
 +
Jan 11 00:05:05 ks3095298 mongod[22404]: Mon Jan 11 00:05:05.434 terminate() called, printing stack (if implemented for platform):
 +
Jan 11 00:05:05 ks3095298 mongod[22404]: 0xa6d3c6 0x57e204 0x7212186b7bb6 0x7212186b7c01 0x7212186b7e19 0x7212192b666d 0xa6070d 0xa61a67 0xa5f8ca 0xa5f979 0x5a626c 0x5a65d2 0x5a6a93 0x5a6ab9 0x5a6e5c 0x56647b 0x721217dbab45 0x57e05c
 +
Jan 11 00:05:05 ks3095298 mongod[22404]: /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x26) [0xa6d3c6]
 +
 
 +
 
 +
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com --recv EA312927
 +
 
 +
 
 +
Text::Unaccent was causing Apache to segfault, replaced the module with pure perl
 +
 
 +
remote_ip has been replaced in Apache 2.4, need to check if we do get the X-Forwarder-For ip directly now
 +
 
 +
 
 +
Loading products from /products in the mongodb database:
 +
   
 +
update_all_products_from_dir_in_mongodb.pl
 +
   
 +
   
 +
 
 +
Trying to use nginx for the reverse proxy:
 +
   
 +
apt-get install nginx  (need port 80 to be available, stop apache or change its port first)
 +
 
 +
 
 +
 
 +
root@ks3095298:/etc/nginx/sites-enabled# more /etc/nginx/sites-available/off
 +
##
 +
# You should look at the following URL's in order to grasp a solid understanding
 +
# of Nginx configuration files in order to fully unleash the power of Nginx.
 +
# http://wiki.nginx.org/Pitfalls
 +
# http://wiki.nginx.org/QuickStart
 +
# http://wiki.nginx.org/Configuration
 +
#
 +
# Generally, you will want to move this file somewhere, and start with a clean
 +
# file but keep this around for reference. Or just disable in sites-enabled.
 +
#
 +
# Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples.
 +
##
 +
 
 +
# Default server configuration
 +
#
 +
server {
 +
        listen 80 default_server;
 +
        listen [::]:80 default_server;
 +
 
 +
        server_name openfoodfacts.ovh *.openfoodfacts.ovh;
 +
 
 +
        # SSL configuration
 +
        #
 +
        # listen 443 ssl default_server;
 +
        # listen [::]:443 ssl default_server;
 +
        #
 +
        # Self signed certs generated by the ssl-cert package
 +
        # Don't use them in a production server!
 +
        #
 +
        # include snippets/snakeoil.conf;
 +
 
 +
        root /home/off/html;
 +
 
 +
        # Add index.php to the list if you are using PHP
 +
        index index.html index.htm index.nginx-debian.html;
 +
 
 +
        location ~ ^/(images|js|rss|data|files|resources|foundation)/ {
 +
                # First attempt to serve request as file, then
 +
                # as directory, then fall back to displaying a 404.
 +
                try_files $uri $uri/ =404;
 +
        }
 +
 
 +
        location = /robots.txt {
 +
                try_files $uri $uri/ =404;
 +
        }
 +
 
 +
        location / {
 +
                proxy_set_header Host $host;
 +
                proxy_set_header X-Real-IP $remote_addr;
 +
                proxy_pass http://127.0.0.1:8001/cgi/display.pl?;
 +
        }
 +
 
 +
        location /cgi/ {
 +
                proxy_set_header Host $host;
 +
                proxy_set_header X-Real-IP $remote_addr;
 +
                proxy_pass http://127.0.0.1:8001;
 +
        }
 +
 
 +
        # deny access to .htaccess files, if Apache's document root
 +
        # concurs with nginx's one
 +
        #
 +
        #location ~ /\.ht {
 +
        #      deny all;
 +
        #}
 +
}
 +
 
 +
 
 +
</pre>
 +
 
 +
= Old notes =
    
All instructions below are for user "stephane", please use your own name. :-)
 
All instructions below are for user "stephane", please use your own name. :-)
Line 198: Line 563:  
tar xvfz apr-1.5.1.tar.gz
 
tar xvfz apr-1.5.1.tar.gz
 
tar xvfz apr-util-1.5.4.tar.gz  
 
tar xvfz apr-util-1.5.4.tar.gz  
ln -s apr-1.5.1 ap
+
ln -s apr-1.5.1 apr
 
ln -s apr-util-1.5.4 apr-util
 
ln -s apr-util-1.5.4 apr-util
 
cd ..  
 
cd ..  
Line 205: Line 570:     
make
 
make
make install
+
sudo make install
 +
cd ~
 
</pre>
 
</pre>
    
==== Configuration ====
 
==== Configuration ====
   −
in proxy/conf/httpd.conf :
+
in ~/proxy/conf/httpd.conf :
 
* uncomment #LoadModule rewrite_module modules/mod_rewrite.so
 
* uncomment #LoadModule rewrite_module modules/mod_rewrite.so
 
* uncomment #LoadModule slotmem_shm_module modules/mod_slotmem_shm.so
 
* uncomment #LoadModule slotmem_shm_module modules/mod_slotmem_shm.so
Line 225: Line 591:  
CustomLog /home/stephane/logs/proxy_access_log combined
 
CustomLog /home/stephane/logs/proxy_access_log combined
 
ServerAdmin stephane@openfoodfacts.org
 
ServerAdmin stephane@openfoodfacts.org
 +
 +
<Location "/foundation/foundation-icons" >
 +
Header set Access-Control-Allow-Origin "*"
 +
</Location>
 +
 +
<Location "/images/products" >
 +
Header set Link "<http://creativecommons.org/licenses/by-sa/3.0/>; rel='license'; title='CC-BY-SA 3.0'"
 +
</Location>
 +
    
<Directory "/home/stephane/product-opener/html">
 
<Directory "/home/stephane/product-opener/html">
Line 243: Line 618:  
RewriteCond  %{REQUEST_URI}  !^/data/
 
RewriteCond  %{REQUEST_URI}  !^/data/
 
RewriteCond  %{REQUEST_URI}  !^/files/
 
RewriteCond  %{REQUEST_URI}  !^/files/
 +
RewriteCond  %{REQUEST_URI}  !^/resources/
 +
RewriteCond  %{REQUEST_URI}  !^/foundation/
 
RewriteRule  ^(.*)$ http://localhost:8001/cgi/display.pl?${escape:$1} [P,L,QSA]
 
RewriteRule  ^(.*)$ http://localhost:8001/cgi/display.pl?${escape:$1} [P,L,QSA]
 
</VirtualHost>
 
</VirtualHost>
Line 264: Line 641:     
</pre>
 
</pre>
  −
      
=== Install Perl modules ===
 
=== Install Perl modules ===
Line 329: Line 704:  
ln -s ./ Blogs
 
ln -s ./ Blogs
 
ln -s S
 
ln -s S
 +
</pre>
 +
 +
=== Update IP to country GeoIP database ===
 +
 +
Product Opener uses the free legacy GeoIP Lite database from Maxmind:
 +
* http://search.cpan.org/~maxmind/Geo-IP-1.45/lib/Geo/IP.pm
 +
* http://dev.maxmind.com/geoip/legacy/geolite/
 +
 +
The database installed with apt-get install geoip-database can be very old.
 +
To update manually, download the new databases to:
 +
 +
<pre>
 +
/usr/share/GeoIP# ls -lrt
 +
total 7256
 +
-rw-r--r-- 1 root root 1103675  5 déc.  2013 GeoIPv6.dat
 +
-rw-r--r-- 1 root root  600352  5 déc.  2013 GeoIP.dat
 +
 +
 +
wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
 +
gzip -d GeoIP.dat.gz
 +
</pre>

Navigation menu