USE OPEN-SOURCE LLMS IN POSTGRESQL with Ollama and new open-source extension pgai
🦙 𝗪𝗵𝗮𝘁 𝗶𝘀 𝗢𝗹𝗹𝗮𝗺𝗮?
In contrast to proprietary language models, open-source models are private, free to use (hardware costs aside), can run locally, and can be easily customized for your use case. But getting started with open-source LLMs can be tricky. Enter Ollama.
Ollama is the easiest and most popular way to get up and running with open-source language models. It enables easy access and usage of a variety of open-source embedding and generation models, like Llama 3, Mistral, Phi 3, Gemma and more.
🐘 𝗪𝗵𝗮𝘁 𝗶𝘀 𝗽𝗴𝗮𝗶?
Pgai is an open-source PostgreSQL extension that brings AI models closer to your PostgreSQL data. We built pgai to make AI engineering more accessible to developers familiar with PostgreSQL, making it easier for them to build RAG and search.
🧰 𝗪𝗵𝗮𝘁 𝗰𝗮𝗻 𝗜 𝗱𝗼 𝘄𝗶𝘁𝗵 𝗽𝗴𝗮𝗶 𝗮𝗻𝗱 𝗢𝗹𝗹𝗮𝗺𝗮?
𝗘𝗺𝗯𝗲𝗱𝗱𝗶𝗻𝗴 𝗰𝗿𝗲𝗮𝘁𝗶𝗼𝗻: You can create embeddings on data in PostgreSQL tables using popular open-source embedding models like BERT, Meta’s llama 3, and Nomic Embed. Pgai stores embeddings in the pgvector data type, making it easy to perform search and RAG with pgvector and pgvectorscale after embeddings are created.
𝗥𝗔𝗚 𝗮𝗻𝗱 𝗟𝗟𝗠 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴: You can now perform RAG and LLM reasoning tasks on data in PostgreSQL tables, leveraging state of the art open-source models like Meta’s Llama 3, Mistral, Gemma, Qwen, Phi 3 and more. This unlocks common reasoning tasks like summarization, categorization, and data enrichment, all with a SQL query rather than an entire data pipeline.
See the blog at the end of this post for three examples with code.
👏 Big shoutout to John Pruitt, AI engineer at Timescale, who added Ollama support to pgai in less than a week.
🧑💻𝗩𝗲𝗿𝘆 𝗰𝗼𝗼𝗹! 𝗛𝗼𝘄 𝗰𝗮𝗻 𝗜 𝗴𝗲𝘁 𝘀𝘁𝗮𝗿𝘁𝗲𝗱?
Pgai is open-source under the PostgreSQL license, and free to use on any PostgreSQL database. You can find installation instructions on the pgai GitHub repository (see end of post). It’s also available on any database service in Timescale’s PostgreSQL cloud platform.
See comments for resources to get started!
#ollama #pgvector #opensourcellms #opensourceai #aiengineer
