Experiments/GenAI

Generative AI is the field of Artificial intelligence that generates contents (text, images) from prompts or contents. The resulting technologies might have interesting usage for Open Food Facts.

This page list useful resources.

From Open Food Facts

• Describe me here: https://github.com/openfoodfacts/openfoodfacts-ai/issues/289
• Describe me here: https://openfoodfacts.slack.com/archives/CF0286BRQ/p1705611584634089
• Describe me here: https://openfoodfacts.slack.com/archives/CF0286BRQ/p1702632877529589
• Describe me here: https://openfoodfacts.slack.com/archives/CF0286BRQ/p1702322101115109
• Describe me here: https://openfoodfacts.slack.com/archives/C1UNFJVQV/p1712323105541929 - Henadzi Rabkin I have started fine tuning with LLaVA back then in February, then postponed the project. Here's a link on my colab https://colab.research.google.com/drive/1REw1PdZLg1Wy3MAGORIGVe1qIhxn_V2x?usp=sharing - TIMG: https://openfoodfacts.slack.com/archives/C1UNFJVQV/p1709093097394419

From elsewhere

• Google Experiment - Mix 2 cuisines together and create wacky recipes
• interesting concept, quite disruptive of OFF: https://foodanalyze.com/ - no permanent storage, no barcode orientation and the associated HackerNews thread: https://news.ycombinator.com/item?id=41259425
• https://openfoodfacts.slack.com/archives/C02VDSWHT/p1703301573620989
• https://engineering.hellofresh.com/recipes-and-generative-ai-6d74a107860c
• FYI , OpenAI has deployed image input, and interesting use cases are emerging, https://twitter.com/jackloganjohns/status/1712600735366393912?t=R8PYuJuEPSPi9-s6JYj2mQ&s=19
• Accidentally found today this dataset, might be helpful, see "Intended use " section https://agdatacommons.nal.usda.gov/articles/dataset/SNAPMe_A_Benchmark_Dataset_of_Food_Photos_with_Food_Records_for_Evaluation_of_Computer_Vision_Algorithms_in_the_Context_of_Dietary_Assessment/24856449

• Carrefour playing with virtual avatars and ChatGPT for the Nutri-Score. https://lehub.laposte.fr/repere/chatgpt-carrefour-cree-son-premier-avatar-grace-intelligence-artificielle-generative
• (Open Products Facts) Interesting Demo from Google about troubleshooting a record player: https://youtu.be/XEzRZ35urlk?t=3031
• Interesting demo from Google about meal planning: https://youtu.be/XEzRZ35urlk?t=2852
• Circle to search on food: https://youtu.be/XEzRZ35urlk?t=4814
• a recap video with a Microwave meal planning on a budget (cc @Raphael Odini) https://youtu.be/XEzRZ35urlk?t=3219 and a Camera troubleshooting (Open Products Facts)
• Climate Q&A chatGPT https://huggingface.co/spaces/Ekimetrics/climate-question-answering
  
  Assistants are going to try and remember you are allergic to peanuts.

Client side LLMs

• ​​https://medium.com/@kavinask007/pitfalls-in-trying-to-run-mistral-7b-model-locally-in-browser-using-rust-wasm-4668c129abd0
• upcoming feature of the next version of Chrome: embedded local LLM: https://huggingface.co/blog/Xenova/run-gemini-nano-in-your-browser
• https://developer.chrome.com/docs/ai
• https://news.ycombinator.com/item?id=40834600
• 65% of the userbase worldwide will get it, so it might be worth investigating and see if it’s useful enough for spellcheck

Opensource multimodal

• paligemma (opensource multimodal)
• See https://llava.hliu.cc/ 2023

Prompt database

• Open Food Facts prompt database: https://docs.google.com/spreadsheets/d/1Iq4_vJcPEuCNQa9wa5aX3sES_67q5MJZT9-3E3SX8g4/edit?gid=808139179#gid=808139179