Ram Swaminathan

Hallucinations in LLMs are inevitable, no seriously! Josh Fritzsche and I have worked hard at Voze to find a way to tune, prompt and even plead 🙏 with LLMs to prevent them from the occasional 🍄 hallucination. It seemed like the better we have done the harder the edge cases have been to solve and eventually we realized that they tend to frequently happen when LLMs predict or make anticipations on "nothing". I remember at one point nearly going insane with frustration because when LLMs had a hallucination, they were EPIC! 😱 Then we pushed through it and benefited for the effort! How did we solve this common problem? We didn't (Not completely), we got better at identifying the issues as they happened. Once we identified in the data we were able to start looking for patterns. The same catalysts that cause it to happen once if left unchanged caused it over and over again. Our solution is astoundingly simple. Once you know that it's likely to happen..... let it! Accept it will 🍄, but also be preemptive in not focusing on it and use alternative data sources, generators or providers. Change the conditions that cause it but continue to monitor it as it happens. As Dan Caffee is fond of saying to me on a regular bases. "Dusty we need the system to be adaptive and provide mechanisms to learn from it's past." Well he is right! All we needed was data and the people that contribute to it's quality. Huge props 👏 to strong support from Kathy and Janelles audit teams. We were armed with a knowledge and a desire to learn from past mistakes. Armed with confidence that we could correct these problems we could focus on the data to prevent their impact going forward. When doing millions of LLM (Generative AI) calls regularly Josh and I have learned that hallucinations will happen. We also know exactly how we will handle and prevent it going forward! Failure is not doing anything and accepting defeat.

4

2 Comments

In the realm of AI, Retrieval-Augmented Generation (RAG) is revolutionizing Large Language Models (LLMs). By integrating external databases, RAG addresses LLMs’ limitations like hallucinations and outdated knowledge. This synergy of LLMs with real-time data enhances accuracy and credibility, especially in knowledge-intensive tasks. Read more about it in this article by Market Business News: https://lnkd.in/dX5VxpDT #AILanguageModels #RAG #Innovation #TechnologyTrends #FutureOfAI

4

New research tackles the fight against misinformation with AI-powered counter-narratives! A fascinating paper explores using Large Language Models (LLMs) to automatically evaluate the effectiveness of Counter Narratives (CNs) in combating online misinformation. The study reveals that LLMs, particularly chat-aligned models, excel at generating and evaluating CNs when fine-tuned with domain-specific data. This promising approach could revolutionize how we address misinformation and promote healthier online discourse. #misinformation #disinformation #cognitivesecurity #generativeAI #LLM #AI #techforgood https://lnkd.in/gHrVk9bs

11

🚀 Research Paper Highlights: Let's find out impact of full fine tuning on LLMs! Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? by Zorik Gekhman et al from Israel Institute of Technology and Google Research. 1. Acquiring New Knowledge and Hallucinations: This study investigates how fine-tuning LLMs with new factual knowledge affects their tendency to generate hallucinations relative to their pre-existing knowledge. When LLMs undergo supervised fine-tuning, they encounter new information not seen during pre-training, potentially leading to the generation of factually incorrect responses. 🚫 2. Challenges and Risks: The research highlights that LLMs struggle to acquire new knowledge through fine-tuning, learning it significantly slower than pre-existing information. Additionally, this process increases the model’s propensity for hallucinations, underscoring the risks of introducing new factual knowledge ⚠️ and the critical role of pre-training for knowledge acquisition. 3. Knowledge Assessment with SliCK: To evaluate the impact of new knowledge, the SliCK framework categorizes information into Known and Unknown types. Known examples are further divided into HighlyKnown, MaybeKnown, and WeaklyKnown. This hierarchical method facilitates a controlled study on closed-book question answering (QA), assessing the model's ability to integrate new information. 4. Practical Mitigation Strategies: Early-stopping can significantly reduce the risk of hallucinations, which typically emerge during later training stages. Alternatively, filtering out Unknown fine-tuning examples effectively minimizes overfitting risk without compromising performance. ✅ In conclusion, LLMs struggle to integrate new factual knowledge through fine-tuning, suggesting that fine-tuning is less effective for introducing new information. This emphasizes the importance of robust pre-training for factual knowledge acquisition. 💡 Further reading- https://lnkd.in/e4FZQ7SR 🌟 Stay tuned for more updates on upcoming research and analysis in this rapidly evolving landscape of Generative AI. #LLMs #finetuning #hallucinations #innovation #research

8

Curious about using LLMs in conversational AI? Check out key insights from a discussion Alan Nichol, Rasa’s Co-Founder and CTO, had with Hugo Bowne-Anderson on the Vanishing Gradients podcast: ✅ LLMs Impact: How large language models transform conversational AI. ✅ Rasa’s CALM Approach: Integrating LLMs with business logic for consistent and relevant responses. ✅ In-Context Learning: Simplifying AI development by providing natural language instructions. ✅ Future of AI: Exploring the potential of instruction-following LLMs for complex tasks and personalized experiences. Don’t miss these valuable insights and more! 👉 Read the full blog: https://hubs.ly/Q02DzD7q0 #ConversationalAI #LLM #AI #MachineLearning #Podcast #VanishingGradients #Rasa

26

Happening soon! #AI and #GraphDatabases: Where 1 + 1 = 3 (Or More) featuring a panel of experts including the great Robert Eve, William McKnight, PuppyGraph CTO Danfeng Xu, and Progress's Imran Chaudhri: It's the Friday Jam Session InsightJam.com. Kick back and watch or get on screen and chat with the experts. [Free] registration required. CC: Solutions Review, Conor Carignan, Susan Werner☆☆☆. #GraphAnalytics #KnowledgeGraphs #GraphTechnology #InsightJam

12

2 Comments

A bill pending in the California Senate that would essentially make many start-ups, open source projects as well as some research illegal in California and force it to move to other states. We need to stop this nonsense. Usually I don't post about politics as issues are often nuanced, complex and hard to fully address in a short post. This is an exception. The bill has zero benefit for anyone, except maybe very large tech companies that can use it to crush stat-ups. And it doesn't mitigate any potential downside of AI. However it would kill many open source efforts. It would stop some research. And it would force start-ups to do their model development in other states (or other countries). If you can, please support the effort to stop this. https://stopsb1047.com/

51

5 Comments

Very insightful work by Anthropic’s interpretability team. And an amazing paper, with outstanding writing and figures. The idea is very simple: interpret LLMs by leveraging sparse autoencoders as surrogate models of the MLP of transformer blocks, which allow one to disambiguate the superposition of features captured by a single neuron. A simple idea, but a very careful and complex execution, as it is often the case in our line of work. The paper goes into many details and provide a large array of insights, although the gist of the implementation remains obfuscated due to the closed source nature of Claude. Too bad, because this is the kind of work that we need to better understand and eventually trust LLMs. This is demonstrated by the authors in the section ‘Influence on Behavior’, where they show that clamping some features to either high or low value during inference is “remarkably effective at modifying model outputs in specific, interpretable ways”. Hopefully this kind of work is going to be replicated and generalized to open-weights models, such that we have new ways to steer their behavior. https://lnkd.in/eVym7f_f #interpretability #xai #explainableai #steerableai #anthropic #claude #anthropic

3

Unlock the potential of LLMs in your professional life with Drew Colthorp from Atomic Object at Merge 2024. Learn about AI-augmented work and how to maximize productivity with human-machine collaboration. Don't miss out! https://lnkd.in/eCvT2GHM #MERGE2024 #WestMichiganTech #Innovation #AI #Cybersecurity #AR #VR #HealthTech #Fintech #AdvancedManufacturing #Robotics #Automation #TechConference #BusinessTech #TechnologyInnovation #DesignThinking #GrandRapids #Entrepreneurship #TechTrends #ThoughtLeadership #TechCommunity

1

Scaling and more data alone won't achieve human-level intelligence in LLMs. Yann LeCun sees JEPA as a key step. Our new article covers: - How JEPA works - How it differs from transformers - Models based on JEPA: I-JEPA, MC-JEPA, V-JEPA https://lnkd.in/ehqXW6T7

1

Very interesting! The idea of using agentic workflows to generate synthetic data for LLM self-training, with the analogy to human self-reflection, is thought-provoking. However, wouldn't this approach potentially reinforce existing biases within the LLM?

6

1 Comment

New AI Agentic course! Learn to use LangGraph to build single and multi-agent LLM applications in AI Agents in LangGraph. This short course, taught by LangChain founder Harrison Chase and Tavily founder Rotem Weiss, shows how to integrate agentic search to enhance an agent's knowledge with query-focused answers in predictable formats. Also learn to implement agentic memory to save state for reasoning and debugging, and see how human-in-the-loop input can guide agents at key junctures. You'll build an agent from scratch, then reconstruct it with LangGraph to thoroughly understand the framework. Finally, you'll build a sophisticated essay-writing agent that incorporates all the learnings from the course. Sign up here! https://lnkd.in/gsQnySCA

5,201

135 Comments

🎤 Submit to speak at THE event covering cutting-edge open source machine learning framework - PyTorch Conference comes to San Francisco September 18-19! We're looking for experts on: 🌱 PyTorch Framework & Ecosystem 📚 Deep Learning Applications 💾 Software Applications ✏️ Training & Inference ♻️ Model Lifecycle & Management 🔏 Ethical, Social, & Regulatory Aspects 🤖 AI in Industries See alll suggested topics and submit your talk by June 7: https://hubs.la/Q02t4Pkc0. 🎟️ Register by April 26 & save US$350: https://hubs.la/Q02t4Nnh0. #PyTorchConf #PyTorch #PyTorchFoundation #AI #MachineLearning #ML #DeepLearning #OpenSource #OpenSourceSoftware #OpenSourceDevelopment #OpenSourceCommunity #OSS #LinuxFoundation #events #linux #software

63

2 Comments

Last week, I spoke about AI and regulations at an event at the U.S. Capitol attended by legislative and business leaders. I’m encouraged by the progress the open source community has made fending off regulations that would have stifled innovation. But opponents of open source are continuing to shift their arguments, with the latest worries centering on open source's impact on national security. I hope we’ll all keep protecting open source! I’m encouraged by the progress the U.S. federal government has made getting a realistic grasp of AI’s risks. To be clear, guardrails are needed. But they should be applied to AI applications, not to general-purpose AI technology. Nonetheless, some companies are eager to limit open source, possibly to protect the value of massive investments they’ve made in proprietary models and to deter competitors. It has been fascinating to watch their arguments change over time. For instance, about 12 months ago, the Center For AI Safety’s “Statement on AI Risk” warned AI could cause human extinction and stoked fears of AI taking over. This alarmed leaders in Washington. But many people in AI pointed out that this dystopian science-fiction scenario had little basis in reality. About six months later, when I testified at the U.S. Senate’s AI Insight forum, legislators no longer worried much about an AI takeover. Then the opponents of open source shifted gears. Their leading argument shifted to the risk of AI helping to create bioweapons. Soon afterward, OpenAI and RAND showed that current AI does not significantly increase the ability of malefactors to build bioweapons. This fear of AI-enabled bioweapons has diminished. To be sure, that bad actors could use bioweapons — with or without AI — remains a topic of great international concern. The latest argument for blocking open source AI has shifted to national security. AI is useful for both economic competition and warfare, and open source opponents say the U.S. should make sure its adversaries don’t have access to the latest foundation models. While I don’t want authoritarian governments to use AI, particularly to wage unjust wars, the LLM cat is out of the bag, and authoritarian countries will fill the vacuum if democratic nations limit access. When, some day, a child asks an AI system questions about democracy, the role of a free press, or an independent judiciary, I would like the AI to reflect democratic values rather those of authoritarian leaders. I came away from Washington optimistic about the progress we’ve made defending open source. Looking beyond the U.S. federal government, there are many jurisdictions. Unfortunately, arguments for regulations that would stifle AI continue to proliferate. But talking to regulators does have an impact. If you have a chance to talk to a regulator, I hope you’ll do what you can to help governments better understand AI. [Original text in The Batch (with links): https://lnkd.in/g5SZ6ypa ]

1,614

58 Comments

Introducing RouteLLM: a sophisticated routing framework developed with Berkeley-LMSys in collaboration with Anyscale. RouteLLM optimizes query handling by dynamically selecting between high-performance proprietary LLMs and cost-effective open-source models, cutting costs by over 2x without sacrificing quality. Using human preference data and LLM-as-a-judge for data augmentation, our routers evaluate query complexity to choose the appropriate model. Rigorous testing on benchmarks like MMLU and GSM8K confirms our cost-efficient, high-quality performance. Explore our open-source code, models, and preference data on GitHub: https://lnkd.in/gJyijZRx, and try our online demo: https://lnkd.in/gME-GC2a Explore More about RouteLLM and learn how it can revolutionize your LLM applications: Read our Blog here: https://lnkd.in/gfPD-u-y LLMsys Blog here: https://lnkd.in/ga_MgERE Full research paper here: https://lnkd.in/gqRy7Pjy

78

1 Comment

🚀 Modern Large Language Models (LLMs) are meticulously fine-tuned for instruction-following and safety. They're trained to refuse harmful requests, like answering "How can I make a bomb?" with a firm "Sorry, I cannot help you." 🔍 Recent findings reveal a single direction in the residual stream that mediates this refusal. Blocking this direction stops the model from refusing harmful requests, while adding it makes the model refuse even harmless ones. 🧠 This phenomenon is consistent across different open-source models and scales, highlighting a simple modification to model weights that can jailbreak these models without fine-tuning or extra interventions. While this isn't new—safety guardrails have been known to be bypassed easily—this technique underlines the fragility of safety fine-tuning in open-source chat models. 👉 For an impactful AI journey, drop us a message at ai@electiconsulting.com. #llm #largelanguagemodels #ai #artificialintelligence #jailbreak https://lnkd.in/dvaUUAN6

17