Both AI research and engineering use Pytorch, but there is not a single shared library between the RAG and agent research and engineering communities. I have read hundreds of papers on RAG and agents, almost none of which use frameworks like Langchain and LlamaIndex. I have talked to many ML engineers building RAG and agent products; most people only use the utility functions of these tools like text splitters or output parsers. The majority of people who use these two popular frameworks are software engineers who don't have much background in AI. Building RAG and agent applications is no different from training a model; you just currently manually tune your prompts. You need evaluations, you need iterations, you need to get the in-context learning working well and gather more data to automate your prompting process and to move into model finetuning. The problem with researchers’ code is that it has no abstraction at all. It only works on the benchmarking dataset. They lack powerful string processing that can generalize the tool calls. Their code is hard to read as it is not organized. Thus, it is really hard to adapt to production. The problems with the frameworks are that they have abstracted so many layers but are not supported with really strong engineering to provide robust adaptation to different use cases. They are more like no-code solutions, which are perfect for people to use out-of-the-box, but terrible for products and research. There can be a solution great for both the ML research and the product community: a library that provides strong string processing, an easy-to-understand tool interface to support tool calls, various output formats, and various access to different models with great monitoring capabilities like TensorBoard for Pytorch. That is the mission and purpose of LightRAG. Let researchers and engineers work on what matters: prompts, datasets, evaluations, and model finetuning without building everything from scratch alone with subpar solutions or using an overly engineered framework which requires them to spend more time figuring out how to customize instead of on what matters. I know I'm not a popular voice, but this is super important, and I think the majority of the community is moving in the wrong direction, even some of those people I looked up to and considered the best in the field. If you are passionate about a library that really unites RAG and Agent research and engineering, please work with me. I know I'm not able to do this alone.
Chris Blodgett
2mo
The problem I have with the RAG frameworks is most of the stuff really isn't really that useful and the layers of abstraction just over complicates and makes them not very flexible. Like the prompt template in langchang - it is basically just a glorified python f"" string with {} passed in as its parameters. The 'chains' basically come down to you prompting the models in very specific ways and restricting the output so much that you can funnel it to do what you want it to , that's pretty much it. I don't need a framework when the gist of the matter involves creating a string to pass to a LLM :) Most of the work is just coming up with the prompts you want, in the order you want, with enough code around the responses to get what you need out of them for what you're trying to do. Most of what I've seen in the frameworks can be done with a fairly minimal amount of python code and then you'll know exactly what is going on in the most efficient way possible.
Laurentiu Petrea
2mo
100% my experience. We started with LangChain and now we are removing it and going “native” for reasons you already mentioned. Whenever I was looking for scripts to fine-tune, it was a mess because everything was so disorganised and poorly written. (we fixed that) - same for RAG. Bridging the gap is needed, software engineers need to learn AI without the abstraction layers of popular frameworks and ML engineers need to write code that others can work with. Probably that’s why MLOps is perceived as so hard: someone needs to make sense of the code. Wrapping FastAPI on top and dockerising is piece of cake. What do you need help with?
Ion Moșnoi
2mo
i mostly use langchain to save retrieve information using Faiss, i dont use character splitter, because it does not make any sense to split the text at random when it passed 1000 characters. I use NER to detect titles, paragrams, and other important concept features, in order to split and link the chunks based on their properties, different application, requires different NER classes. It will be hard to make engineers and researchers to use same library.
If you are doing production grade RAG systems you should not use Langchain or similar abstraction libraries. There is no holy grail library for RAG, and probably never will be, since RAG does not generalize to all problems and use cases. The libs get you 80 % there and then you need to throw them away to deliver the final hardest 20 %. Instead of these one-for-all libraries, assemble custom RAG for your specific use case from smaller building blocks. Maybe that helps you to build your own library. The current abstraction level in the libraries is too high. I think the Unix philosophy of doing one thing and doing it well is the correct approach.
Building based on other people’s framework takes more time than using all the existing basic programming tools to build our own for such simple problems like RAG and is less flexible. Why would people learn to use such tools like langchain at all? Just a waste of time.
Leo Walker
2mo
I always enjoy it when I see the LangChain or LlamaIndex team replicating the research papers with their tools the ones that come to mind are the RAPTOR approach to RAG. https://youtu.be/jbGchdTL7d0?si=bQt4OhpAllikImlR
Kevin Tran
2mo
Practically, this is very much agreeable. I think the best framework is one that is high level enough that you can iterate your ideas quickly without having to start from scratch and that you still have a lot of flexibility to adapt to your specific use case. For example, pandas, scikit-learn, huggingface etc seem like great frameworks for their intended purposes. LightRAG sounds promising based on your information. I just joined your Slack channel.
David Xue
2mo
This might be the best or the boldest perspective I’ve seen on the challenges between research and practical application in RAG and fine-tuned. I’ve experienced similar frustrations over the last six months, initially excited by the potential of frameworks like Langchain and LlamaIndex, only to be disappointed by their inflexibility due to excessive abstraction. I completely agree that bridging the gap between practical application and research is daunting yet crucial. I greatly appreciate the efforts being made with LightRAG to address these issues. I might contribute not on the research side but possibly on the fund， marketing or community aspects. Appreciate your work and perservance.
Siddhesh Gunjal
2mo
This is something I have realised too. We are currently using haystack by deepset to build our production level products. It has a good balance of abstraction and supports building good RAG systems with 3rd party embedding stores. But sometimes it's a hassle to bring everything to one place for different use cases. So yes, I really loved your idea for LightRAG and I would love to contribute.
LightRAG | AI researcher | x MetaAI
2moThank to everyone for your support. I never planned a library. I am solving a few hard problems with RAG and agents in our production, and only then found that we have learned so much from both the research and the engineering, and we have our own library already. I did think hard on open-sourcing it or not. It is core tech foundation to our startup. But I think we can do something great here if we can open-source it and to unite the community to build something amazing. To join the discussion on such a library, please join the Discord (light-rag) channel. https://discord.gg/ezzszrRZvT Looking forward to taking more input for the final direction and structure of the library. Here is my previous post on LightRAG: https://www.linkedin.com/posts/li-yin-ai_join-the-people-search-copilot-discord-server-activity-7188204575042990080-A_rQ?utm_source=share&utm_medium=member_desktop