Line 1:
Line 1:
Also see [[Product Opener Installation]]
Also see [[Product Opener Installation]]
+
[[Category:ProductOpener]]
+
[[Category:Infrastructure]]
+
This page lists all the steps necessary to install Product Opener on Debian, including a development environment.
+
+
= Warning! Those instructions are outdated and kept for reference only =
−
This page lists all the steps necessary to install Product Opener on Debian, including a development environment.
+
Please see http://en.wiki.openfoodfacts.org/Infrastructure for up-to-date install instructions.
= Installation on Jessie =
= Installation on Jessie =
Line 12:
Line 17:
apt-get install git
apt-get install git
−
cd /srv
+
git clone <repo> /srv/openfoodfacts
−
git clone .../openfoodfacts/
mkdir /srv/openfoodfacts/users
mkdir /srv/openfoodfacts/users
mkdir /srv/openfoodfacts/products
mkdir /srv/openfoodfacts/products
Line 43:
Line 47:
</pre>
</pre>
−
In /srv/openfoodfacts/cgi/Display.pm, swap lines 3750-3752 and
+
== Install packaged prerequisites ==
−
3769-3771 (location of Javascript things)
−
== Install packaged prerequisites ==
+
<pre>
+
apt-get install apache2 build-essential geoip-bin geoip-database imagemagick libbarcode-zbar-perl libcache-memcached-fast-perl libclone-perl libcrypt-passwdmd5-perl libdatetime-format-mail-perl \
+
libencode-detect-perl libgeo-ip-perl libgraphics-color-perl libgraphviz-perl libhtml-defang-perl libimage-magick-perl libjson-perl libmime-lite-perl libmime-lite-perl libmongodb-perl libperl-dev \
+
libtest-nowarnings-perl libtext-unaccent-perl liburi-escape-xs-perl liburi-find-perl libwww-perl libxml-encoding-perl libxml-feedpp-perl memcached mongodb tesseract-ocr tesseract-ocr-fra zlib1g-dev
+
</pre>
+
== Prepare CPAN ==
+
If you never used CPAN before, you can automatically configure it:
<pre>
<pre>
−
apt-get install mongodb memcached imagemagick tesseract-ocr tesseract-ocr-fra geoip-bin geoip-database libwww-perl libimage-magick-perl libxml-encoding-perl libtext-unaccent-perl libmime-lite-perl libcache-memcached-fast-perl libjson-perl libclone-perl libgraphviz-perl libmime-lite-perl libcrypt-passwdmd5-perl libencode-detect-perl libgraphics-color-perl libbarcode-zbar-perl libxml-feedpp-perl libmongodb-perl liburi-find-perl liburi-escape-xs-perl libtest-nowarnings-perl libhtml-defang-perl libgeo-ip-perl libdatetime-format-mail-perl libperl-dev zlib1g-dev apache2 build-essential
+
(echo y;echo o conf prerequisites_policy follow;echo o conf commit)|cpan
</pre>
</pre>
Line 59:
Line 68:
cpan GraphViz2
cpan GraphViz2
cpan Image::OCR::Tesseract
cpan Image::OCR::Tesseract
+
cpan Algorithm::CheckDigits
+
cpan XML::Simple
+
</pre>
+
+
Note: on Debian Jessie, you might need to install the follwoing to be able to get cpan to install GraphViz2
+
<pre>
+
apt-get install libtest-version-perl
</pre>
</pre>
Line 65:
Line 81:
<pre>
<pre>
cd /usr/local/src/
cd /usr/local/src/
−
wget http://apache.crihan.fr/dist//httpd/httpd-2.2.29.tar.gz
+
wget http://apache.crihan.fr/dist/httpd/httpd-2.2.31.tar.gz
−
tar xvfz httpd-2.2.29.tar.gz
+
tar xvfz httpd-2.2.31.tar.gz
−
cd httpd-2.2.29
+
cd httpd-2.2.31
./configure --with-mpm=prefork --prefix=/usr/local --enable-rewrite --enable-proxy --enable-proxy_http --enable-deflate --disable-userdir --enable-headers
./configure --with-mpm=prefork --prefix=/usr/local --enable-rewrite --enable-proxy --enable-proxy_http --enable-deflate --disable-userdir --enable-headers
make
make
Line 73:
Line 89:
cd /usr/local/src/
cd /usr/local/src/
−
wget http://mir2.ovh.net/ftp.apache.org/dist/perl/mod_perl-2.0.8.tar.gz
+
wget http://mir2.ovh.net/ftp.apache.org/dist/perl/mod_perl-2.0.9.tar.gz
−
tar xvfz mod_perl-2.0.8.tar.gz
+
tar xvfz mod_perl-2.0.9.tar.gz
−
cd mod_perl-2.0.8
+
cd mod_perl-2.0.9
perl Makefile.PL MP_APXS=/usr/local/bin/apxs
perl Makefile.PL MP_APXS=/usr/local/bin/apxs
make
make
Line 139:
Line 155:
RewriteCond %{REQUEST_URI} !^/images/
RewriteCond %{REQUEST_URI} !^/images/
RewriteCond %{REQUEST_URI} !^/js/
RewriteCond %{REQUEST_URI} !^/js/
+
RewriteCond %{REQUEST_URI} !^/foundation/
RewriteCond %{REQUEST_URI} !^/rss/
RewriteCond %{REQUEST_URI} !^/rss/
RewriteCond %{REQUEST_URI} !^/robots
RewriteCond %{REQUEST_URI} !^/robots
Line 154:
Line 171:
You should now have a working Productopener instance running at http://yourdomain.com/. Enjoy!
You should now have a working Productopener instance running at http://yourdomain.com/. Enjoy!
+
+
= Installation on Debian Jessie with stock Apache and nginx =
+
+
<pre>
+
Vous êtes sur le point d'installer un OS sur le serveur ks3095298.kimsufi.com. Choisissez le type d'installation que vous souhaitez utiliser :
+
+
-> Installer à partir d'un template OVH
+
+
Debian 8.1 stable (Jessie) (stable) - debian8
+
+
root@ks3095298:~# uname -a
+
Linux ks3095298.kimsufi.com 3.14.32-xxxx-grs-ipv6-64 #5 SMP Wed Sep 9 17:24:34 CEST 2015 x86_64 GNU/Linux
+
+
perl -v
+
+
This is perl 5, version 20, subversion 2 (v5.20.2) built for x86_64-linux-gnu-thread-multi
+
+
+
apt-get update
+
apt-get upgrade
+
+
apt-get install fail2ban
+
apt-get install sudo
+
+
apt-get install mongodb
+
+
MongoDB shell version: 2.4.10
+
+
vi /etc/mongodb.conf
+
- change dbpath
+
- uncomment :
+
#port = 27017
+
#nohttpinterface = true
+
+
+
apt-get install git
+
+
apt-get install apache2
+
apt-get install libapache2-mod-perl2
+
apt-get install modperl
+
+
adduser off
+
+
apt-get install imagemagick
+
apt-get install memcached
+
apt-get install tesseract-ocr tesseract-ocr-fra
+
apt-get install exim4
+
dpkg-reconfigure exim4-config
+
apt-get install geoip-bin geoip-database
+
apt-get install zlib1g-dev
+
+
apt-get install libwww-perl libimage-magick-perl libxml-encoding-perl libtext-unaccent-perl libmime-lite-perl libcache-memcached-fast-perl libjson-perl libclone-perl libgraphviz-perl libmime-lite-perl libcrypt-passwdmd5-perl libencode-detect-perl libgraphics-color-perl libbarcode-zbar-perl libxml-feedpp-perl libmongodb-perl liburi-find-perl libxml-simple-perl
+
+
+
cp -a /home/openfoodfacts-bitbucket/product-opener/cgi /home/off/cgi/
+
+
/home/off/cgi# rm Blogs
+
root@ks3095298:/home/off/cgi# ln -s /home/off/cgi Blogs
+
+
ln -s SiteLang_off.pm SiteLang.pm
+
+
vi /home/off/startup.pl
+
# Needs to be configured
+
use lib "/home/off/cgi/";
+
+
+
ln -s ../sites-available/001-off.conf 001-off.conf
+
service apache2 restart
+
+
To see errors, missing perl modules etc.
+
+
/var/log/apache2# tail -f error.log
+
+
cpan
+
install URI::Escape::XS
+
install Encode::Punycode
+
install GraphViz2
+
install HTML::Defang
+
install Algorithm::CheckDigits
+
install Geo::IP
+
install Image::OCR::Tesseract
+
DateTime::Format::Mail
+
DateTime::Format::CLDR
+
DateTime::Locale
+
+
+
+
+
Issues:
+
+
MongoDB does not start:
+
+
service mongodb start
+
root@ks3095298:/home/off/cgi# ps -ef | grep mongo
+
root 21814 23471 0 23:56 pts/0 00:00:00 grep mongo
+
+
+
in /var/log/syslog:
+
+
Jan 11 00:05:05 ks3095298 systemd[1]: Starting An object/document-oriented database...
+
Jan 11 00:05:05 ks3095298 systemd[1]: Started An object/document-oriented database.
+
Jan 11 00:05:05 ks3095298 mongod[22404]: Mon Jan 11 00:05:05.434 terminate() called, printing stack (if implemented for platform):
+
Jan 11 00:05:05 ks3095298 mongod[22404]: 0xa6d3c6 0x57e204 0x7212186b7bb6 0x7212186b7c01 0x7212186b7e19 0x7212192b666d 0xa6070d 0xa61a67 0xa5f8ca 0xa5f979 0x5a626c 0x5a65d2 0x5a6a93 0x5a6ab9 0x5a6e5c 0x56647b 0x721217dbab45 0x57e05c
+
Jan 11 00:05:05 ks3095298 mongod[22404]: /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x26) [0xa6d3c6]
+
+
+
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com --recv EA312927
+
+
+
Text::Unaccent was causing Apache to segfault, replaced the module with pure perl
+
+
remote_ip has been replaced in Apache 2.4, need to check if we do get the X-Forwarder-For ip directly now
+
+
+
Loading products from /products in the mongodb database:
+
+
update_all_products_from_dir_in_mongodb.pl
+
+
+
+
Trying to use nginx for the reverse proxy:
+
+
apt-get install nginx (need port 80 to be available, stop apache or change its port first)
+
+
+
+
root@ks3095298:/etc/nginx/sites-enabled# more /etc/nginx/sites-available/off
+
##
+
# You should look at the following URL's in order to grasp a solid understanding
+
# of Nginx configuration files in order to fully unleash the power of Nginx.
+
# http://wiki.nginx.org/Pitfalls
+
# http://wiki.nginx.org/QuickStart
+
# http://wiki.nginx.org/Configuration
+
#
+
# Generally, you will want to move this file somewhere, and start with a clean
+
# file but keep this around for reference. Or just disable in sites-enabled.
+
#
+
# Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples.
+
##
+
+
# Default server configuration
+
#
+
server {
+
listen 80 default_server;
+
listen [::]:80 default_server;
+
+
server_name openfoodfacts.ovh *.openfoodfacts.ovh;
+
+
# SSL configuration
+
#
+
# listen 443 ssl default_server;
+
# listen [::]:443 ssl default_server;
+
#
+
# Self signed certs generated by the ssl-cert package
+
# Don't use them in a production server!
+
#
+
# include snippets/snakeoil.conf;
+
+
root /home/off/html;
+
+
# Add index.php to the list if you are using PHP
+
index index.html index.htm index.nginx-debian.html;
+
+
location ~ ^/(images|js|rss|data|files|resources|foundation)/ {
+
# First attempt to serve request as file, then
+
# as directory, then fall back to displaying a 404.
+
try_files $uri $uri/ =404;
+
}
+
+
location = /robots.txt {
+
try_files $uri $uri/ =404;
+
}
+
+
location / {
+
proxy_set_header Host $host;
+
proxy_set_header X-Real-IP $remote_addr;
+
proxy_pass http://127.0.0.1:8001/cgi/display.pl?;
+
}
+
+
location /cgi/ {
+
proxy_set_header Host $host;
+
proxy_set_header X-Real-IP $remote_addr;
+
proxy_pass http://127.0.0.1:8001;
+
}
+
+
# deny access to .htaccess files, if Apache's document root
+
# concurs with nginx's one
+
#
+
#location ~ /\.ht {
+
# deny all;
+
#}
+
}
+
+
+
</pre>
= Old notes =
= Old notes =
Line 380:
Line 592:
CustomLog /home/stephane/logs/proxy_access_log combined
CustomLog /home/stephane/logs/proxy_access_log combined
ServerAdmin stephane@openfoodfacts.org
ServerAdmin stephane@openfoodfacts.org
+
+
<Location "/foundation/foundation-icons" >
+
Header set Access-Control-Allow-Origin "*"
+
</Location>
+
+
<Location "/images/products" >
+
Header set Link "<http://creativecommons.org/licenses/by-sa/3.0/>; rel='license'; title='CC-BY-SA 3.0'"
+
</Location>
+
<Directory "/home/stephane/product-opener/html">
<Directory "/home/stephane/product-opener/html">
Line 398:
Line 619:
RewriteCond %{REQUEST_URI} !^/data/
RewriteCond %{REQUEST_URI} !^/data/
RewriteCond %{REQUEST_URI} !^/files/
RewriteCond %{REQUEST_URI} !^/files/
+
RewriteCond %{REQUEST_URI} !^/resources/
+
RewriteCond %{REQUEST_URI} !^/foundation/
RewriteRule ^(.*)$ http://localhost:8001/cgi/display.pl?${escape:$1} [P,L,QSA]
RewriteRule ^(.*)$ http://localhost:8001/cgi/display.pl?${escape:$1} [P,L,QSA]
</VirtualHost>
</VirtualHost>
Line 482:
Line 705:
ln -s ./ Blogs
ln -s ./ Blogs
ln -s S
ln -s S
+
</pre>
+
+
=== Update IP to country GeoIP database ===
+
+
Product Opener uses the free legacy GeoIP Lite database from Maxmind:
+
* http://search.cpan.org/~maxmind/Geo-IP-1.45/lib/Geo/IP.pm
+
* http://dev.maxmind.com/geoip/legacy/geolite/
+
+
The database installed with apt-get install geoip-database can be very old.
+
To update manually, download the new databases to:
+
+
<pre>
+
/usr/share/GeoIP# ls -lrt
+
total 7256
+
-rw-r--r-- 1 root root 1103675 5 déc. 2013 GeoIPv6.dat
+
-rw-r--r-- 1 root root 600352 5 déc. 2013 GeoIP.dat
+
+
+
wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
+
gzip -d GeoIP.dat.gz
+
</pre>