Line 1: |
Line 1: |
| Also see [[Product Opener Installation]] | | Also see [[Product Opener Installation]] |
| + | [[Category:ProductOpener]] |
| + | This page lists all the steps necessary to install Product Opener on Debian, including a development environment. |
| + | |
| + | = Warning! Those instructions are outdated and kept for reference only = |
| + | |
| + | Please see http://en.wiki.openfoodfacts.org/Infrastructure for up-to-date install instructions. |
| + | |
| + | = Installation on Jessie = |
| + | |
| + | We start from a clean Jessie install. |
| + | |
| + | == Get the code and configure it == |
| + | |
| + | <pre> |
| + | apt-get install git |
| + | |
| + | git clone <repo> /srv/openfoodfacts |
| + | mkdir /srv/openfoodfacts/users |
| + | mkdir /srv/openfoodfacts/products |
| + | chown -R www-data:www-data /srv/openfoodfacts |
| + | |
| + | cd /srv/openfoodfacts/cgi |
| + | rm Blogs |
| + | ln -s . Blogs |
| + | ln -s SiteLang_off.pm SiteLang.pm |
| + | </pre> |
| + | |
| + | Edit a few variables in Config2.pm: |
| + | <pre> |
| + | $server_domain = "yourdomain.com"; |
| + | $www_root="/srv/openfoodfacts/html"; |
| + | $data_root="/srv/openfoodfacts"; |
| + | </pre> |
| + | |
| + | Also in Config_off.pm: |
| + | <pre> |
| + | $domain = 'yourdomain.com'; |
| + | $contact_email = 'webmaster@yourdomain.com'; |
| + | $admin_email = 'webmaster@yourdomain.com'; |
| + | </pre> |
| + | |
| + | And in startup.pl: |
| + | <pre> |
| + | use lib "/srv/openfoodfacts/cgi/"; |
| + | </pre> |
| + | |
| + | == Install packaged prerequisites == |
| + | |
| + | <pre> |
| + | apt-get install apache2 build-essential geoip-bin geoip-database imagemagick libbarcode-zbar-perl libcache-memcached-fast-perl libclone-perl libcrypt-passwdmd5-perl libdatetime-format-mail-perl \ |
| + | libencode-detect-perl libgeo-ip-perl libgraphics-color-perl libgraphviz-perl libhtml-defang-perl libimage-magick-perl libjson-perl libmime-lite-perl libmime-lite-perl libmongodb-perl libperl-dev \ |
| + | libtest-nowarnings-perl libtext-unaccent-perl liburi-escape-xs-perl liburi-find-perl libwww-perl libxml-encoding-perl libxml-feedpp-perl memcached mongodb tesseract-ocr tesseract-ocr-fra zlib1g-dev |
| + | </pre> |
| + | |
| + | == Prepare CPAN == |
| + | If you never used CPAN before, you can automatically configure it: |
| + | <pre> |
| + | (echo y;echo o conf prerequisites_policy follow;echo o conf commit)|cpan |
| + | </pre> |
| + | |
| + | == Install a few Perl modules from CPAN == |
| + | |
| + | <pre> |
| + | cpan Net::IDN::Punycode |
| + | cpan Encode::Punycode |
| + | cpan GraphViz2 |
| + | cpan Image::OCR::Tesseract |
| + | cpan Algorithm::CheckDigits |
| + | cpan XML::Simple |
| + | </pre> |
| + | |
| + | Note: on Debian Jessie, you might need to install the follwoing to be able to get cpan to install GraphViz2 |
| + | <pre> |
| + | apt-get install libtest-version-perl |
| + | </pre> |
| + | |
| + | == Rebuild Apache 2.2 and mod_perl == |
| + | |
| + | <pre> |
| + | cd /usr/local/src/ |
| + | wget http://apache.crihan.fr/dist/httpd/httpd-2.2.31.tar.gz |
| + | tar xvfz httpd-2.2.31.tar.gz |
| + | cd httpd-2.2.31 |
| + | ./configure --with-mpm=prefork --prefix=/usr/local --enable-rewrite --enable-proxy --enable-proxy_http --enable-deflate --disable-userdir --enable-headers |
| + | make |
| + | make install |
| + | |
| + | cd /usr/local/src/ |
| + | wget http://mir2.ovh.net/ftp.apache.org/dist/perl/mod_perl-2.0.9.tar.gz |
| + | tar xvfz mod_perl-2.0.9.tar.gz |
| + | cd mod_perl-2.0.9 |
| + | perl Makefile.PL MP_APXS=/usr/local/bin/apxs |
| + | make |
| + | make install |
| + | |
| + | cat >> /usr/local/conf/httpd.conf <<EOF |
| + | Listen 8001 |
| + | |
| + | LoadModule perl_module modules/mod_perl.so |
| + | |
| + | PerlWarn Off |
| + | PerlRequire /srv/openfoodfacts/cgi/startup.pl |
| + | |
| + | <VirtualHost *:8001> |
| + | DocumentRoot /srv/openfoodfacts/html |
| + | ServerName yourdomain.com |
| + | ErrorLog /var/log/apache2/error.log |
| + | CustomLog /var/log/apache2/access.log combined |
| + | ScriptAlias /cgi/ /srv/openfoodfacts/cgi/ |
| + | |
| + | <Location /cgi> |
| + | SetHandler perl-script |
| + | PerlResponseHandler ModPerl::Registry |
| + | PerlOptions +ParseHeaders |
| + | Options +ExecCGI |
| + | Order allow,deny |
| + | Allow from all |
| + | </Location> |
| + | </VirtualHost> |
| + | |
| + | PerlPostReadRequestHandler My::ProxyRemoteAddr |
| + | EOF |
| + | |
| + | sed -i -e 's/^Listen 80$/#Listen 80/' /usr/local/conf/httpd.conf |
| + | sed -i -e 's/^User.*/User www-data/' /usr/local/conf/httpd.conf |
| + | sed -i -e 's/^Group.*/Group www-data/' /usr/local/conf/httpd.conf |
| + | |
| + | /usr/local/bin/apachectl -k start |
| + | </pre> |
| + | |
| + | == Configure standard Apache 2 == |
| + | |
| + | <pre> |
| + | a2enmod proxy_http |
| + | a2enmod rewrite |
| + | |
| + | cat >> /etc/apache2/sites-available/product-opener-proxy.conf <<EOF |
| + | <VirtualHost *> |
| + | DocumentRoot /srv/openfoodfacts/html |
| + | ServerName yourdomain.com |
| + | ServerAlias *.yourdomain.com |
| + | |
| + | <Directory "/srv/openfoodfacts/html"> |
| + | Options Indexes FollowSymLinks |
| + | Require all granted |
| + | </Directory> |
| + | ProxyPreserveHost On |
| + | RewriteEngine on |
| + | RewriteCond %{REQUEST_URI} !/./ |
| + | RewriteRule ^(/cgi/.*)$ http://localhost:8001$1 [P,L] |
| + | RewriteMap escape int:escape |
| + | RewriteRule ^/favicon.ico$ /favicon.ico [L] |
| + | RewriteCond %{REQUEST_URI} !^/images/ |
| + | RewriteCond %{REQUEST_URI} !^/js/ |
| + | RewriteCond %{REQUEST_URI} !^/foundation/ |
| + | RewriteCond %{REQUEST_URI} !^/rss/ |
| + | RewriteCond %{REQUEST_URI} !^/robots |
| + | RewriteCond %{REQUEST_URI} !^/clicks/ |
| + | RewriteCond %{REQUEST_URI} !^/data/ |
| + | RewriteCond %{REQUEST_URI} !^/files/ |
| + | RewriteRule ^(.*)$ http://localhost:8001/cgi/display.pl?${escape:$1} [P,L,QSA] |
| + | </VirtualHost> |
| + | EOF |
| + | |
| + | a2ensite product-opener-proxy.conf |
| + | a2dissite 000-default.conf |
| + | service apache2 restart |
| + | </pre> |
| + | |
| + | You should now have a working Productopener instance running at http://yourdomain.com/. Enjoy! |
| + | |
| + | = Installation on Debian Jessie with stock Apache and nginx = |
| + | |
| + | <pre> |
| + | Vous êtes sur le point d'installer un OS sur le serveur ks3095298.kimsufi.com. Choisissez le type d'installation que vous souhaitez utiliser : |
| + | |
| + | -> Installer à partir d'un template OVH |
| + | |
| + | Debian 8.1 stable (Jessie) (stable) - debian8 |
| + | |
| + | root@ks3095298:~# uname -a |
| + | Linux ks3095298.kimsufi.com 3.14.32-xxxx-grs-ipv6-64 #5 SMP Wed Sep 9 17:24:34 CEST 2015 x86_64 GNU/Linux |
| + | |
| + | perl -v |
| + | |
| + | This is perl 5, version 20, subversion 2 (v5.20.2) built for x86_64-linux-gnu-thread-multi |
| + | |
| + | |
| + | apt-get update |
| + | apt-get upgrade |
| + | |
| + | apt-get install fail2ban |
| + | apt-get install sudo |
| + | |
| + | apt-get install mongodb |
| | | |
− | This page lists all the steps necessary to install Product Opener on Debian, including a development environment.
| + | MongoDB shell version: 2.4.10 |
| + | |
| + | vi /etc/mongodb.conf |
| + | - change dbpath |
| + | - uncomment : |
| + | #port = 27017 |
| + | #nohttpinterface = true |
| + | |
| + | |
| + | apt-get install git |
| + | |
| + | apt-get install apache2 |
| + | apt-get install libapache2-mod-perl2 |
| + | apt-get install modperl |
| + | |
| + | adduser off |
| + | |
| + | apt-get install imagemagick |
| + | apt-get install memcached |
| + | apt-get install tesseract-ocr tesseract-ocr-fra |
| + | apt-get install exim4 |
| + | dpkg-reconfigure exim4-config |
| + | apt-get install geoip-bin geoip-database |
| + | apt-get install zlib1g-dev |
| + | |
| + | apt-get install libwww-perl libimage-magick-perl libxml-encoding-perl libtext-unaccent-perl libmime-lite-perl libcache-memcached-fast-perl libjson-perl libclone-perl libgraphviz-perl libmime-lite-perl libcrypt-passwdmd5-perl libencode-detect-perl libgraphics-color-perl libbarcode-zbar-perl libxml-feedpp-perl libmongodb-perl liburi-find-perl libxml-simple-perl |
| + | |
| + | |
| + | cp -a /home/openfoodfacts-bitbucket/product-opener/cgi /home/off/cgi/ |
| + | |
| + | /home/off/cgi# rm Blogs |
| + | root@ks3095298:/home/off/cgi# ln -s /home/off/cgi Blogs |
| + | |
| + | ln -s SiteLang_off.pm SiteLang.pm |
| + | |
| + | vi /home/off/startup.pl |
| + | # Needs to be configured |
| + | use lib "/home/off/cgi/"; |
| + | |
| + | |
| + | ln -s ../sites-available/001-off.conf 001-off.conf |
| + | service apache2 restart |
| + | |
| + | To see errors, missing perl modules etc. |
| + | |
| + | /var/log/apache2# tail -f error.log |
| + | |
| + | cpan |
| + | install URI::Escape::XS |
| + | install Encode::Punycode |
| + | install GraphViz2 |
| + | install HTML::Defang |
| + | install Algorithm::CheckDigits |
| + | install Geo::IP |
| + | install Image::OCR::Tesseract |
| + | DateTime::Format::Mail |
| + | DateTime::Format::CLDR |
| + | DateTime::Locale |
| + | |
| + | |
| + | |
| + | |
| + | Issues: |
| + | |
| + | MongoDB does not start: |
| + | |
| + | service mongodb start |
| + | root@ks3095298:/home/off/cgi# ps -ef | grep mongo |
| + | root 21814 23471 0 23:56 pts/0 00:00:00 grep mongo |
| + | |
| + | |
| + | in /var/log/syslog: |
| + | |
| + | Jan 11 00:05:05 ks3095298 systemd[1]: Starting An object/document-oriented database... |
| + | Jan 11 00:05:05 ks3095298 systemd[1]: Started An object/document-oriented database. |
| + | Jan 11 00:05:05 ks3095298 mongod[22404]: Mon Jan 11 00:05:05.434 terminate() called, printing stack (if implemented for platform): |
| + | Jan 11 00:05:05 ks3095298 mongod[22404]: 0xa6d3c6 0x57e204 0x7212186b7bb6 0x7212186b7c01 0x7212186b7e19 0x7212192b666d 0xa6070d 0xa61a67 0xa5f8ca 0xa5f979 0x5a626c 0x5a65d2 0x5a6a93 0x5a6ab9 0x5a6e5c 0x56647b 0x721217dbab45 0x57e05c |
| + | Jan 11 00:05:05 ks3095298 mongod[22404]: /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x26) [0xa6d3c6] |
| + | |
| + | |
| + | sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com --recv EA312927 |
| + | |
| + | |
| + | Text::Unaccent was causing Apache to segfault, replaced the module with pure perl |
| + | |
| + | remote_ip has been replaced in Apache 2.4, need to check if we do get the X-Forwarder-For ip directly now |
| + | |
| + | |
| + | Loading products from /products in the mongodb database: |
| + | |
| + | update_all_products_from_dir_in_mongodb.pl |
| + | |
| + | |
| + | |
| + | Trying to use nginx for the reverse proxy: |
| + | |
| + | apt-get install nginx (need port 80 to be available, stop apache or change its port first) |
| + | |
| + | |
| + | |
| + | root@ks3095298:/etc/nginx/sites-enabled# more /etc/nginx/sites-available/off |
| + | ## |
| + | # You should look at the following URL's in order to grasp a solid understanding |
| + | # of Nginx configuration files in order to fully unleash the power of Nginx. |
| + | # http://wiki.nginx.org/Pitfalls |
| + | # http://wiki.nginx.org/QuickStart |
| + | # http://wiki.nginx.org/Configuration |
| + | # |
| + | # Generally, you will want to move this file somewhere, and start with a clean |
| + | # file but keep this around for reference. Or just disable in sites-enabled. |
| + | # |
| + | # Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples. |
| + | ## |
| + | |
| + | # Default server configuration |
| + | # |
| + | server { |
| + | listen 80 default_server; |
| + | listen [::]:80 default_server; |
| + | |
| + | server_name openfoodfacts.ovh *.openfoodfacts.ovh; |
| + | |
| + | # SSL configuration |
| + | # |
| + | # listen 443 ssl default_server; |
| + | # listen [::]:443 ssl default_server; |
| + | # |
| + | # Self signed certs generated by the ssl-cert package |
| + | # Don't use them in a production server! |
| + | # |
| + | # include snippets/snakeoil.conf; |
| + | |
| + | root /home/off/html; |
| + | |
| + | # Add index.php to the list if you are using PHP |
| + | index index.html index.htm index.nginx-debian.html; |
| + | |
| + | location ~ ^/(images|js|rss|data|files|resources|foundation)/ { |
| + | # First attempt to serve request as file, then |
| + | # as directory, then fall back to displaying a 404. |
| + | try_files $uri $uri/ =404; |
| + | } |
| + | |
| + | location = /robots.txt { |
| + | try_files $uri $uri/ =404; |
| + | } |
| + | |
| + | location / { |
| + | proxy_set_header Host $host; |
| + | proxy_set_header X-Real-IP $remote_addr; |
| + | proxy_pass http://127.0.0.1:8001/cgi/display.pl?; |
| + | } |
| + | |
| + | location /cgi/ { |
| + | proxy_set_header Host $host; |
| + | proxy_set_header X-Real-IP $remote_addr; |
| + | proxy_pass http://127.0.0.1:8001; |
| + | } |
| + | |
| + | # deny access to .htaccess files, if Apache's document root |
| + | # concurs with nginx's one |
| + | # |
| + | #location ~ /\.ht { |
| + | # deny all; |
| + | #} |
| + | } |
| + | |
| + | |
| + | </pre> |
| + | |
| + | = Old notes = |
| | | |
| All instructions below are for user "stephane", please use your own name. :-) | | All instructions below are for user "stephane", please use your own name. :-) |
Line 120: |
Line 485: |
| make | | make |
| make install | | make install |
| + | cd ~ |
| | | |
| + | cd ~/src |
| wget http://mir2.ovh.net/ftp.apache.org/dist/perl/mod_perl-2.0.8.tar.gz | | wget http://mir2.ovh.net/ftp.apache.org/dist/perl/mod_perl-2.0.8.tar.gz |
| tar xvfz mod_perl-2.0.8.tar.gz | | tar xvfz mod_perl-2.0.8.tar.gz |
Line 127: |
Line 494: |
| make | | make |
| make install | | make install |
| + | cd ~ |
| </pre> | | </pre> |
| | | |
| ==== Configuration ==== | | ==== Configuration ==== |
| | | |
− | In apache/conf/httpd.conf: | + | In ~/apache/conf/httpd.conf: |
| | | |
| Change the port to something else than 80 (the reverse proxy will listen on 80 and forward to the backend on another port). | | Change the port to something else than 80 (the reverse proxy will listen on 80 and forward to the backend on another port). |
Line 177: |
Line 545: |
| PerlPostReadRequestHandler My::ProxyRemoteAddr | | PerlPostReadRequestHandler My::ProxyRemoteAddr |
| </pre> | | </pre> |
− |
| |
| | | |
| === Apache 2.4 - light weight reverse proxy === | | === Apache 2.4 - light weight reverse proxy === |
Line 196: |
Line 563: |
| tar xvfz apr-1.5.1.tar.gz | | tar xvfz apr-1.5.1.tar.gz |
| tar xvfz apr-util-1.5.4.tar.gz | | tar xvfz apr-util-1.5.4.tar.gz |
− | ln -s apr-1.5.1 ap | + | ln -s apr-1.5.1 apr |
| ln -s apr-util-1.5.4 apr-util | | ln -s apr-util-1.5.4 apr-util |
| cd .. | | cd .. |
Line 203: |
Line 570: |
| | | |
| make | | make |
− | make install | + | sudo make install |
| + | cd ~ |
| </pre> | | </pre> |
| | | |
| ==== Configuration ==== | | ==== Configuration ==== |
| | | |
− | in proxy/conf/httpd.conf : | + | in ~/proxy/conf/httpd.conf : |
| * uncomment #LoadModule rewrite_module modules/mod_rewrite.so | | * uncomment #LoadModule rewrite_module modules/mod_rewrite.so |
| * uncomment #LoadModule slotmem_shm_module modules/mod_slotmem_shm.so | | * uncomment #LoadModule slotmem_shm_module modules/mod_slotmem_shm.so |
Line 223: |
Line 591: |
| CustomLog /home/stephane/logs/proxy_access_log combined | | CustomLog /home/stephane/logs/proxy_access_log combined |
| ServerAdmin stephane@openfoodfacts.org | | ServerAdmin stephane@openfoodfacts.org |
| + | |
| + | <Location "/foundation/foundation-icons" > |
| + | Header set Access-Control-Allow-Origin "*" |
| + | </Location> |
| + | |
| + | <Location "/images/products" > |
| + | Header set Link "<http://creativecommons.org/licenses/by-sa/3.0/>; rel='license'; title='CC-BY-SA 3.0'" |
| + | </Location> |
| + | |
| | | |
| <Directory "/home/stephane/product-opener/html"> | | <Directory "/home/stephane/product-opener/html"> |
Line 241: |
Line 618: |
| RewriteCond %{REQUEST_URI} !^/data/ | | RewriteCond %{REQUEST_URI} !^/data/ |
| RewriteCond %{REQUEST_URI} !^/files/ | | RewriteCond %{REQUEST_URI} !^/files/ |
| + | RewriteCond %{REQUEST_URI} !^/resources/ |
| + | RewriteCond %{REQUEST_URI} !^/foundation/ |
| RewriteRule ^(.*)$ http://localhost:8001/cgi/display.pl?${escape:$1} [P,L,QSA] | | RewriteRule ^(.*)$ http://localhost:8001/cgi/display.pl?${escape:$1} [P,L,QSA] |
| </VirtualHost> | | </VirtualHost> |
Line 262: |
Line 641: |
| | | |
| </pre> | | </pre> |
− |
| |
− |
| |
| | | |
| === Install Perl modules === | | === Install Perl modules === |
Line 327: |
Line 704: |
| ln -s ./ Blogs | | ln -s ./ Blogs |
| ln -s S | | ln -s S |
| + | </pre> |
| + | |
| + | === Update IP to country GeoIP database === |
| + | |
| + | Product Opener uses the free legacy GeoIP Lite database from Maxmind: |
| + | * http://search.cpan.org/~maxmind/Geo-IP-1.45/lib/Geo/IP.pm |
| + | * http://dev.maxmind.com/geoip/legacy/geolite/ |
| + | |
| + | The database installed with apt-get install geoip-database can be very old. |
| + | To update manually, download the new databases to: |
| + | |
| + | <pre> |
| + | /usr/share/GeoIP# ls -lrt |
| + | total 7256 |
| + | -rw-r--r-- 1 root root 1103675 5 déc. 2013 GeoIPv6.dat |
| + | -rw-r--r-- 1 root root 600352 5 déc. 2013 GeoIP.dat |
| + | |
| + | |
| + | wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz |
| + | gzip -d GeoIP.dat.gz |
| + | </pre> |