Platform for producers - Cheatsheet

Some details about the internals of the Producers Platform that can be of interest to admins and developers.

Introduction to the Producers Platform

On the Producers Platform, each food manufacturer can have its own mini Open Food Facts with only its products.

Producers can:

• import product data and photos in a variety of ways
• get reports on data quality and product quality (and even suggestions on how they could improve it)
• publish their product data and photos in the public Open Food Facts database

How it works

Organizations

Users on Open Food Facts can declare themselves as professional users representing a company or brand. They can associate their account with an organization (only one) by editing their profile (or when they create their account).

If the organization does not exist yet, it is created right away. Otherwise, an e-mail is sent to the producers platform admins, and admins can edit the user profile to add him/her to the organization.

There isn't currently a user management system to allow the creators of organization to manage users, review and accept requests by other users to join their organization, or invite other users.

Importing data and photos to the producers platform

Product edit form

Producers can manually add or edit products using the regular web edit form.

Bulk images upload form

Producers can upload images from multiple products using a single form. Images need to have the barcode in their filename so that we can associate each image file with the right product. We also try to recognize "ingredients", "nutrition", "packaging" etc. in the file names to automatically select the images.

Spreadsheet file import form

Producers can upload product data with a spreadsheet file (LibreOffice, Excel or CSV). We provide a template file, but producers can use any format for the columns names and order. When the spreadsheet file is uploaded, we try to auto-guess what each column is (based on its title and content), and producers can adjust the matching if needed. The matching is saved for subsequent uploads.

GS1 GSDN imports

Mapping of GS1 organization identifiers (GLN, Party Name) to OFF organizations

In GS1 data, organizations are identified with a GLN, that is associated to a "party name".

GS1 product data often has organizations identified as the "brand owner" and/or as the "information provider".

e.g.

      "brandOwner" : {
         "gln" : "3010836800139",
         "partyName" : "BONDUELLE SAS"
      },
      "informationProviderOfTradeItem" : {
         "gln" : "3010836800139",
         "partyName" : "BONDUELLE EUROPE LONG LIFE"
      },

We currently use the informationProviderOfTradeItem if it is present and not empty.

By default, we map the party name to create the OFF organization id. e.g. "BONDUELLE EUROPE LONG LIFE" becomes org-bonduelle-europe-long-life.

In practice, some GS1 data may have different party names for the same GLN (e.g. the CodeOnline data above), and some big organizations have different GLNs for different lines of products.

Admins of the producers platform can edit organization profiles to list the different GLNs that the organization should be associated with.

Merging of data

Data from Equadis or CodeOnline will be merged with existing data (if the organization name from the GS1 GLN or party name matches, otherwise the GS1 data will be imported to a different organization).

Producers can also send extra data and/or pictures on top of imported Equadis or CodeOnline data, or edit it manually.

Equadis

Equadis customers can send realtime product data updates to Open Food Facts. On the OFF side, product updates are processed nightly.

CodeOnline (under development)

CodeOnline publishes an export file in open data. The export is updated monthly.

Admins of the producers platform can edit organization profiles to indicate that the CodeOnline data should not be loaded (for instance if we already have real time data for products of that organization).

Handling of conflicts

For data that comes from GS1 / GSDN (e.g. through Equadis or CodeOnline Food), we skip the data if it less recent than the last publication date we have in sources_fields:org-gs1:publicationDateTime. We use lastChangeDateTime if publicationDateTime is not available (e.g. for CodeOnline).

This is to avoid importing monthly exports of CodeOnline over more recent realtime Equadis data.

Exporting data and photos from the producers platform to the public database

Data sources

Data sources are available in the product JSON and can be listed through the /data-sources facet.

Product owners

Owners are available in the product JSON and can be listed through the /owners facet.

In the public database, the owner field is imported from the producers platform only for organizations that correspond to producers. It is not kept when the owner is an user, or an organization that corresponds to a label or an external database (e.g. org-label-biopartenaire or org-database-usda).

Handling of conflicts

If a product already has an owner specified in the public database, then exports from a different owner will be skipped. Admins can request an export with "overwrite owner" if they want the export to proceed, in which case the products will be assigned to the new owner.

Technical details

The producers platform is a separate instance of Product Opener. It differs from the public instance in that products are always associated with 1 owner, which can be an organization or an user.

The owners thare users are only used in very rare instances, e.g. for users that need to import a lot of product photos that they took.

The owner id is of the form org-organization-id or user-user-id (e.g. org-nestle-france or user-stephane).

For each product, the owner id is stored in the "owner" field, and in the "owners_tags" array (with only one entry). The "owners_tags" field is used to make the /owners facet work without extra code (this may change in the future).

Each product is stored in /srv/off-pro/products/[owner-id]/ (instead of /srv/off/products/ on the public platform). Similarly, images are in /srv/off-pro/html/images/products/[owner-id]/.