Jessica Yuan

LLMs can output tokens 200% faster using multi-token prediction. And it’s now been open sourced by Meta. Here’s how it works — Multi-token prediction extends standard language modeling -- it predicts several future tokens at once instead of just the next one. - First, the model uses a shared core to make a context representation from its past tokens. - Then, it predicts multiple future tokens in parallel using separate prediction layers. This approach is much more memory-efficient, so during training, it can handle large vocabulary sizes without overloading the GPU. And at inference, it can decode much faster. Why does it work so well? According to the paper, it’s because multi-token prediction is able to reduce the mismatch between training and inference. This makes the model better at handling long-term dependencies. Paper: https://lnkd.in/ge_quFaS HuggingFace models: https://lnkd.in/gUgzsbtZ #AI #LLMs

So I just set up an astronomer.io account and I'm familiarizing myself with Airflow. The concept of DAGs (Directed Acyclic Graphs) seemed confusing at first but they really are just a set a functions that make up the steps of your workflow. You can see examples of different DAGs in the astronomer registry https://lnkd.in/esjueWXr

It sounds like Milliman data assets would be perfect for Health Economics & Outcomes Research! HEOR (pronounced like Eeyore but with an audible “h” at the start), uses methodologies to evaluate the impact of healthcare decisions on both financial and medical interests. The goal of a HEOR analysis is to determine the best treatment option (i.e., pharmaceuticals, medical devices, and other healthcare services) to optimize both clinical outcomes and affordability to promote an efficient, equitable, and high-quality healthcare system. HEOR analysts can evaluate not just cost- and clinical-effectiveness, but also the cost of illness to include evaluating the health burden, economic burden, and intangible costs of diseases on the patient and society. These methods can be applied in a variety of business contexts where understanding the likely financial impact of possible future outcomes is beneficial, especially to think holistically on mitigating risk and costs over time. #millimanlifesciences #milliman #healthactuaries #datascienceheor #healtheconomicsoutcomesresearch #actuariesanddatascientistssittinginatree -- Enjoy this post? Follow @Caitlin (me 😊) for more! 

🚀 Exciting News! I’ve launched a Substack newsletter where I share thoughts on how data leaders can improve their teams' efficacy by building a strong, data-driven culture. The motivation for this came after reflecting on my career transition from UX Research to Data Science. While there's quite a bit that's different between the two fields, the need to see your team have an impact is a constant. But I've noticed interesting variability in how each discipline approaches this challenge -- and more often than not, these teams aren't collaborating to their mutual benefit. There's room to improve! If you'd like to follow along, click the link below to subscribe, or peruse some of my early posts. Looking forward to hearing what you think!

Impact Analytics was featured in The Wall Street Journal for our data surrounding the impact of GLP-1 weight loss drugs on retail sizing. This article highlights the crucial role we're playing in helping retailers adapt to shifting consumer needs. Check out the full story: #retailtrends #ai #allocation #inventorymanagement #glp #ozempic #glpdrugs #weightlossdrugs #supplychain #ImpactAnalytics

Your users have preferences! Sometimes their preferences are really concentrated on one category/format/product, and other times their preferences are distributed across a lot of those things. This is a deceptively hard characterization to make at scale. But what if you could do just that with a tool you had in your data science toolkit already, just waiting to be repurposed for something else? I wrote about it here (see comment for the link to the blog post!).

My last couple posts have come down hard on the anti-AI side, so in the interest of even-handedness, I wrote about some AI/AI-adjacent products I've seen that actually work pretty well: https://lnkd.in/gzafncc9 I do maintain that humility and auditability are important in any of these products, but there are some genuinely good tools out there. Just look at them for what they are and without the hype!

Last year, at Data Day Texas, I had the pleasure of attending Joe Reis' conference closing fireside chat. I walked away not only with new insights but also his book, "Fundamentals of Data Engineering". It's a solid read, offering straightforward, tool-agnostic strategies for architecture and data pipelines. Better yet, Joe is aware that organizations have different needs as they mature from nascent data cultures to dynamic data driven enterprises, with recommendations for each phase of that journey. Fast forward to earlier this year, and I was blown away by Sol Rashidi during her opening keynote at Data Day Texas 2024. As a former college rugby star turned data practitioner and C-suite leader, she opened her speech by sharing how rugby taught her the resilience, courage, and grit needed to tackle tough challenges head-on, then proceeded to wow the audience with gem after gem on getting things done in the world of data. Her one hour talk was so packed with honesty and wisdom that I immediately pre-ordered her book, "Your AI Survival Guide." That book arrives at my home today. So, you can imagine my excitement learning that Joe and Sol are teaming up to teach a Maven course! This is a dream pairing, each complementing each other's style and experience. Successful data initiatives require awareness of the cultural and interpersonal dynamics of an organization and the skill to navigate them. I intend to learn and improve this with Joe and Sol. Check out the course in the link I've shared. Join and we'll end up as cohort-mates! #Data #AI #Leadership #ProfessionalDevelopment #CareerAdvancement

Enhancing Multi-Agent Systems with Latent Communication: Insights from Nest and RAG Models Reading about Nearest Neighbor Speculative Decoding (Nest) and Retrieval-Augmented Generation (RAG) has sparked new ideas for my multi-agent system (MAS) research. While RAG uses explicit text retrieval to augment model outputs, Nest leverages latent states for efficient inter-agent communication. This approach can enhance MAS interactions by dynamically adjusting strategies and using speculative decoding to achieve high-quality, fast generation. In a smart city traffic system, for example, agents like traffic lights and autonomous vehicles can share latent states for real-time adjustments, improving efficiency and response times. Nest's dynamic and flexible communication method, contrasted with RAG's explicit text-based approach, offers a robust framework for advancing MAS in complex, collaborative tasks. https://lnkd.in/etEX3CGW

Been thinking about how LLMs/Transformers and teaching kids to read are related lately. I've been listening to "Sold a Story" Podcast: https://lnkd.in/e_9fbZ5t about the Reading Wars in schools. To oversimplify, it contrasts phonetic learning vs whole word/queuing methods to teach kids how to read. Highly recommend it if you have kids. As I've listened to the podcast it makes me think about NLP/GenAI concepts, like tokenization, embeddings, attention, transformers. Transformers seem like queuing methodology and tokenization seems related to phonemes. Definitely some nuance missed here. I've always thought the english language and probably others are extremely cumbersome for trying to get thoughts/emotions out of one's head and to the others (especially as an introvert!) Not sure there is a point to this post other than to wonder what is the most efficient mechanism for communicating with other people as both a knowledge transfer vehicle and an emotional one. I also need to do some research around how cognitive scientists in the reading field have related their work to the Transformer/Gen AI world.

🎉 Meet David K., the wizard behind the curtain at Numbers Station, leading the Data Janitor crew! 🧹✨ We all know LLMs and AI Models are all about the data it learns from! 🚀 Dive into David's latest blog where he breaks down the mysteries of data cleaning and why it's super crucial for training smart, unbiased AI. Think of it as the AI's diet—better ingredients, better outcomes! 🍏➡️🤖 🛡️ Get the inside scoop on how Numbers Station keeps their data top-notch with cool tricks like consistency checks and question re-phrasing. It’s not just about having loads of data; it's about having the right data. 👏 Big shoutout to David K. for making data cleaning a thing of beauty at Numbers Station. 🌟 Check out how our efforts are making AI smarter and more reliable every day. 🔗 https://lnkd.in/eTkYrz_p #DataScience #MachineLearning #Innovation 💬 Got questions or wanna chat about data? Drop a comment below or hit us up to see how clean data can rev up your AI engines! 🚀👀

🚀 I'm thrilled to share groundbreaking insights from our latest research at #Databricks, highlighting the significant productivity benefits of the Databricks Assistant. Our recent study reveals how the Assistant is revolutionizing how data teams operate, making data analysis more efficient and intuitive than ever before. In the blog post, I cover three key takeaways from our investigation and discuss some of the areas we'll be investing in for the remainder of the year to continue enhancing our AI-powered authoring assistant. Read the full article here: https://lnkd.in/eGU96ZQN #DataScience #Productivity #DatabricksAssistant #AI #DataDriven #LLM

Is Generative AI just a bubble that's about to pop? Aiden Gomez from Cohere doesn't seem to think so and it's hard to disagree with much of what is said in this interview. https://lnkd.in/gXeJSncZ 1. LLMs are well beyond stochastic parrots - they can apply learned patterns to new inputs. 2. In the vast majority of applications it is about augmenting humans by allowing them to work faster and more effectively. 3. Emphasis is not on making the largest model possible, but on models that enterprises find useful and can reasonably consume. 4. Despite the expensive compute, the path to revenue is not far down the road - but it won't happen by wishful thinking... 5. The real risks are not existential, but rather misapplication/abuse of the technology. 6. The bar set by LLMs is being raised all the time and whilst there may exist an upper limit to their capabilities, it's likely we can get the rate of hallucinations to be at least on par with humans!

For our R&D at Scenario, we shifted away from our custom planning algorithm to doing more data exploration across economics, social, necessities, medical, and lifestyle. Data for what?... Insights of course. What kind?... Life scenarios. Huh?... Everyday I hear 7 voices in my head and the hardest ones to hear are... - 2 in lower income and - 3 in middle income. Jen... - low income, works at a fast food restaurant in California. - takes classes at community college, - wants to study business, - lives in a meh apartment with a roommate, - wants to get a cat for her sanity but roommate allergic, - parks her car on the street, weirdos around, - in a meh neighborhood, - wears baggy clothes to detract attention, - carries a mace, - gets dropped off home by a friend if her class is at night. There are millions like her. Besides her inflation-adjusted earnings not pacing with the customers she serves at work... Her situation is difficult to improve economically = financially, professionally, educationally. If given more specifics about Jen, think like an algorithm... What is the best path for Jen to have a better life [scenario]? It could be a mix of... - Change major, take different classes, what? - Drop out, take a bootcamp, which? - Pay it with credit card? - Save up, how much? why? - Move to reduce cost, but where? - Work for free, gain skills, for how long? - Then get better job, which? better life, how? Jen is not sure what's possible, but if she knew, she wants to know how to make it happen. The solution is not a specific vertical: finance, education, jobs, housing. ...cuz life is not a specific vertical. But for what matters to most people... The solution is economic. But even deeper, Jen's why: she wants a good life. Related... I'm glad this exists to hopefully help people like Jen: Techstars Economic Mobility Powered by Samvid Ventures #startups #economics #goodlife #techstars

Today, the Snowflake AI Research Team is thrilled to introduce Snowflake Arctic, a top-tier enterprise-focused LLM that pushes the frontiers of cost-effective training and openness. Arctic is efficiently intelligent and truly open. "Snowflake and AWS are aligned in the belief that generative AI will transform virtually every customer experience we know," said David Brown, Vice President Compute and Networking, AWS. "With AWS, Snowflake was able to customize its infrastructure to accelerate time-to-market for training Snowflake Arctic. Using Amazon EC2 P5 instances with Snowflake’s efficient training system and model architecture co-design, Snowflake was able to quickly develop and deliver a new, enterprise-grade model to customers. And with plans to make Snowflake Arctic available on AWS, customers will have greater choice to leverage powerful AI technology to accelerate their transformation." https://lnkd.in/eUnAAAKg

Sitting in on an insightful talk from Noah Arliss and Hossein Ahmadi on core platform performance improvements at Snowflake. If you want to learn about the impact of these improvements (as measured by the Snowflake Performance Index) and also catch up on all of our latest platform announcements, follow this link: https://lnkd.in/grNNtsQe

I think the problem with generative ai right now is that people still think of it as generative ai instead of just another data engineering problem. It needs to get more mundane. The real challenge, IMO, is not the prompt engineering everyone obsesses about, or even LLMs themselves. I've read a paper that essentially showed LLM results converge regardless of technique when trained on the same dataset. You can make an LLM faster, but at a certain point you can't make it better without different data. So this isn't really about LLM tech. It's about unstructured data management. And as far as I know, NOBODY has brought a compelling solution for that to the market that allows for integrating well within existing structured data management solutions. If we're supposed to build the future on generative ai, eventually the wild experimentation will settle down into the mundanity of repeatable processes managed by data engineering teams. We need thoughtfully engineered solutions for ingest, cleaning, transformation, automated metadata generation, governance, cataloging, observability, etc. They especially need to play well within existing structured data solutions. Governanace is great, but I already have a governance platform and I really don't want a second one. Until the industry arrives at that level of sophistication, Gen AI will continue to be a toy instead of a reliable tool. p.s. - zero generative ai was used for this post.

Big day, Snowflake has just invested in Omni! “The combination of Omni and Snowflake helps us build exceptional data products and scale our operations, and I'm excited to see what they help us do next." - Boris Jabes, Founder and CEO of Census Learn more in this post from Jamie Davidson 👇

2024 is the year of AI value, and time for companies to get out of PoC purgatory. “I really want to start driving use of AI at scale,” Luke G., head of analytics and AI at TD told The Wall Street Journal. “I think now is the time.” https://gag.gl/w2TUIR In his recent RevX keynote, Domino Data Lab's Nick Elprin detailed how companies do this today. Watch and learn how to accelerate AI impact: https://gag.gl/oN6hJo #AI #datascience #GenerativeAI