Dr. Thomas Steiner

Very insightful work by Anthropic’s interpretability team. And an amazing paper, with outstanding writing and figures. The idea is very simple: interpret LLMs by leveraging sparse autoencoders as surrogate models of the MLP of transformer blocks, which allow one to disambiguate the superposition of features captured by a single neuron. A simple idea, but a very careful and complex execution, as it is often the case in our line of work. The paper goes into many details and provide a large array of insights, although the gist of the implementation remains obfuscated due to the closed source nature of Claude. Too bad, because this is the kind of work that we need to better understand and eventually trust LLMs. This is demonstrated by the authors in the section ‘Influence on Behavior’, where they show that clamping some features to either high or low value during inference is “remarkably effective at modifying model outputs in specific, interpretable ways”. Hopefully this kind of work is going to be replicated and generalized to open-weights models, such that we have new ways to steer their behavior. https://lnkd.in/eVym7f_f #interpretability #xai #explainableai #steerableai #anthropic #claude #anthropic

3

Our larger goal at Kùzu Inc. is to make graph databases and knowledge graphs accessible to a much larger community, through our focus on ease-of-use, scalability and performance. Toward this end, Semih Salihoğlu is preparing a series of videos introducing the core concepts of #graph databases and their feature set. ✅ If you are new to graph databases, this series is a great way to become familiar with the key concepts to get started with graphs. ✅ If you're familiar with graph data models, this series is a great way to understand the principles behind Kùzu and to begin using it in your projects. The first video in the series is now out! Give it a watch, and share around 🤝🏽. Subscribe to our YouTube channel for more such videos to be released in the coming weeks! https://lnkd.in/gqdGTqYf Video: https://lnkd.in/gsejgP8w Kùzu GitHub: https://lnkd.in/dUXZH6sT

14

That pesky ECDSA signature refuses to go away, and we are struggling with it in a distributed world of trust. Luckily EdDSA is much more scalable, and now used in many distributed signature applications (and where we can split the private key up into secret shares and distribute them over several nodes). But we have a problem ... elliptic curve methods are going to be cracked by quantum computers. So can we take our PQC signature methods and scale them to distributed signatures? Well, no, as Dilithium doesn't support it. But Raccoon can, and also has better side channel resistance (and with a similar performance and key size to Dilithium. So, meet Raccoon: https://lnkd.in/d2QMHSM9

30

2 Kommentare

🚀 Modern Large Language Models (LLMs) are meticulously fine-tuned for instruction-following and safety. They're trained to refuse harmful requests, like answering "How can I make a bomb?" with a firm "Sorry, I cannot help you." 🔍 Recent findings reveal a single direction in the residual stream that mediates this refusal. Blocking this direction stops the model from refusing harmful requests, while adding it makes the model refuse even harmless ones. 🧠 This phenomenon is consistent across different open-source models and scales, highlighting a simple modification to model weights that can jailbreak these models without fine-tuning or extra interventions. While this isn't new—safety guardrails have been known to be bypassed easily—this technique underlines the fragility of safety fine-tuning in open-source chat models. 👉 For an impactful AI journey, drop us a message at ai@electiconsulting.com. #llm #largelanguagemodels #ai #artificialintelligence #jailbreak https://lnkd.in/dvaUUAN6

17

25 authors come together to explore the societal impact of open source, with a focus on open #AI in this excellent article: https://hubs.la/Q02rbS450 #OpenSource #Stanford #OpenAI

1

#Performance of #LLMs. The adoption of #LLMs is rapidly growing, but the difficulty arises when trying to apply them beyond experimental settings. Deploying LLMs in an actual environment is complex, and one of the significant challenges is effectively evaluating their performance. Evaluation of LLMs can be based on different factors like toxicity, helpfulness, correctness and more. This article by Surya Prakash Pathak from Red Hat explores various evaluation techniques and metrics employed in assessing the performance of LLMs, with a particular emphasis on retrieval-augmented generation (#RAG) use cases. RAG, a pivotal use case for different enterprises, focuses on combining retrieval methods with the generation capabilities of LLMs. This unique approach involves not only generating content based on learned patterns but also leveraging pre-existing information through retrieval mechanisms. Scott Creeley :: Stephen Flaherty https://lnkd.in/gJTBagjv

13

2 Kommentare

Ex-OpenAI Superalignment employee Leopold Aschenbrenner published a lengthy paper this week about how he thinks we could get to AGI (Artificial General Intelligence) by 2027. THAT IS SOON. It’s a long and complicated document but basically posits that between increases in compute, algorithmic improvements and a few ‘mysterious’ other things, we could actually speed this up. The implications are pretty significant overall and it’s worth diving into what he has to say even if you don’t agree with it. #openAI #technology #chatGPT #research #machinelearning #ai

2

New estimate of when we will run out of human-generated data for training LLMs: 2028. The recent trend towards on-device LLMs is consistent with wanting higher-quality, smaller datasets (like edited books and peer-reviewed journals) instead of all the internet. #AI

23

1 Kommentar

Key Qualities of Interpretability of Large Language Models for Decision Making Apps 🔻 If LLMs were more integrated into critical decision-making processes, concerns regarding their interpretability will come to the forefront. Interpretability refers to the ability to understand and explain the reasoning behind a model’s outputs. In the context of decision-making applications, interpretability is of utmost importance for several reasons. First, it enables users to trust the model’s predictions by providing insights into the underlying decision-making process. This transparency helps in identifying potential biases and ensuring fairness in the model’s outputs. Second, interpretable models facilitate accountability, as they allow stakeholders to understand why certain decisions were made and trace back the reasoning to specific inputs or factors. Finally, interpretability promotes user acceptance and adoption, as people are more likely to rely on systems that they can comprehend and validate. However, achieving interpretability in LLMs is a challenging task due to their complex architectures and the vast amount of knowledge they encode. These models often operate as black boxes, making it difficult to unravel the intricate patterns and relationships learned during training. Traditional explanation methods, such as feature importance or attention mechanisms, are impossible or too complex for production and even if they were possible, they may not fully capture the nuances of LLMs’ decision-making processes. Therefore, there is a growing need for novel approaches that can provide meaningful and faithful explanations of LLM outputs. We explore two promising post-hoc approaches for enhancing the interpretability of LLMs in decision-making applications. The first approach focuses on reasoning explanations through prompting techniques, such as Chain-of-Thought (CoT) prompting and its extension, Self-Entailment-Alignment Chain-of-Thought (SEA-CoT). These methods encourage LLMs to generate step-by-step explanations of their reasoning process, providing a transparent view of how the model arrives at its decisions. The second approach leverages counterfactual generation using LLMs to create alternative scenarios that highlight the impact of specific factors on the model’s outputs. By examining how the model’s predictions change under different counterfactual conditions, we can gain valuable insights into the causal relationships learned by the model.

48

8 Kommentare

LoRA: Low-Rank Adaptation Of Large Language Models: A Brief Summary 💡 🖥 Discover LoRA, a powerful tool revolutionizing the fine-tuning of pre-trained language models. By leveraging two lower-dimensional matrices to exploit the lower intrinsic rank of a weight matrix, LoRA offers a remarkable enhancement in computational and parametric efficiencies. #LoRA #LanguageModels #Efficiency

2

In computational linguistics, researchers are focused on enhancing language models' ability to extract precise information from extensive textual data. Recent studies introduce "retrieval heads," specialized attention mechanisms that selectively focus on critical segments of large texts, improving targeted data retrieval. These heads, integrated into transformer-based models like LLaMA and Yi, significantly enhance accuracy and efficiency, particularly in handling long-context scenarios. The methodology involves thorough experiments across various models, including the Needle-in-a-Haystack test, which embeds specific information to measure precision. Results show models with active retrieval heads outperform those without, with accuracy dropping notably when these heads are masked. The empirical evidence underscores the effectiveness of retrieval heads in improving information retrieval precision and reliability. This research deepens our understanding of attention mechanisms in large-scale text processing, paving the way for more efficient and accurate language models. https://lnkd.in/dNNPvMXw https://lnkd.in/ddbHX6EA Innovation Hacks AI #aiinnovation #generativeai

1

I decided to see if Claude 3.5 Sonnet is as good as Anthropic claims. I came up with 5 tests to pit it against OpenAI's GPT-4o. 1. Handwriting: My handwriting is terrible so I wrote my prompt in a notebook. https://lnkd.in/d-q4Rprj 2. Making a complete game: Using Python create a tower defense game from scratch. https://lnkd.in/d7ZaiyP8 3. Vector art: Make me a cool spaceship logo. https://lnkd.in/dbxQN3Jn 4. A funny story: Write a tale about a humorous cat. https://lnkd.in/dAyrmT8x 5. The big debate: Should AI have rights? https://lnkd.in/dK4EG7GE

8

2 Kommentare

AnchorGT: A Novel Attention Architecture for Graph Transformers as a Flexible Building Block to Improve the Scalability of a Wide Range of Graph Transformer Models Transformers have taken the machine learning world by storm with their powerful self-attention mechanism, achieving state-of-the-art results in areas like natural language processing and computer vision. However, when it came to graph data, which is... https://lnkd.in/eedeAnTa #AI #ML #Automation

2

Our next video is out! This is the first of a 2-part video in which Prof. Semih Salihoğlu explains about Resource Description Framework (RDF), a popular data model in graph DBMSs. If you're curious about graph modeling and are intimidated by the terminology used (especially when it comes to RDF), this video has you covered! ✅ Fundamentals of RDF: What are resources, URIs, literals, and (subject, predicate, object) triples, and how do these form a graph? ✅ Advantages of using RDF as a data model for complex and irregular domains and its flexibility in handling irregularities and connections between objects ✅ How RDF allows you to query both data and schema information in a uniform way. In the 2nd video (coming soon!), we will also go over RDF's powerful knowledge representation and reasoning capabilities, and their potential in the age of AI. Stay tuned! https://lnkd.in/gZM79U44

30

Now published in OJ-CSYS: "Novel Bounds for Incremental Hessian Estimation With Application to Zeroth-Order Federated Learning," by Alessio Maritan, Luca Schenato and Subhrakanti Dey. Link: https://lnkd.in/gN27SuPH The Hessian matrix conveys important information about the curvature, spectrum and partial derivatives of a function, and is required in a variety of tasks. However, computing the exact Hessian is prohibitively expensive for high-dimensional input spaces, and is just impossible in zeroth-order optimization, where the objective function is a black-box of which only input-output pairs are known. In this work we address this relevant problem by providing a rigorous analysis of an Hessian estimator available in the literature, allowing it to be used as a provably accurate replacement of the true Hessian matrix. The Hessian estimator is randomized and incremental, and its computation requires only point function evaluations. We provide non-asymptotic convergence bounds on the estimation error and derive the minimum number of function queries needed to achieve a desired accuracy with arbitrarily high probability. In the second part of the paper we show a practical application of our results, introducing a novel optimization algorithm suitable for non-convex and black-box federated learning. #optimization #dataprivacy #federatedlearning

3

Cleanlab Introduces the Trustworthy Language Model (TLM) that Addresses the Primary Challenge to Enterprise Adoption of LLMs: Unreliable Outputs and Hallucinations https://zurl.co/yqQO #ai #enterpriseai #genai #llm

1