Speaking with Machine Learning Street Talk (MLST), Cohere Co-Founder Nick Frosst offers a fresh and practical take on evaluating Large Language Models. Nick emphasizes that while benchmarks provide a standardized way to measure LLM performance, they may not capture the specific needs and requirements of enterprises. Cohere focuses on creating models that excel in real-world business tasks, ensuring practical and tangible value for businesses.
This is easily my favorite talk on AI I've heard. I listened to it twice back-to-back. I highly recommend people listen to the entire interview.
That's probably the best talk of AI I've heard recently! I see many people and companies fighting over 2% increase on "almost random" benchmarks, and although we definitely need a standardized way of evaluating these models, these results very hardly show the actual performance of the models. Feedbacks from Reddit users are a much more trustworthy benchmark 😜
Machine Learning Street Talk (MLST) creates video, audio, and written content about machine learning, cognitive science, and cognitive philosophy. It's recognized as the highest-rated technical AI podcast on Spotify.
I saw the whole podcast on youtube and the amazing part is when Nick says “we’re here to solve real world problems not to built artifical gods”.
Really enjoyed this conversation. One key point was the affirmation of Cohere's genuine mission on solving real world problems using LLMs.
Certainly one of the best ML Street Talks
Very informative.
Watch the full video: https://www.youtube.com/watch?v=4JF1V2hzGKE