Paul King

LinkedIn team (Juan Pablo Bottaro Karthik Ramgopal) shared a helpful deep dive into their deployment of an LLM-powered job search assistant. Highly recommend this read! Few highlights:  💡 Relatively easy to setup RAG-based pipeline. Achieved 80% of the basic experience in 1 month, then 4 more months to reach 95% mark. 💡 Evaluating the quality of application outputs is hard. I hear this from many enterprises developing LLM applications. (check out Okareo by Matthew Wyman Boris Selitser) 💡 Balancing latency with quality is important. While certain techniques like Chain of Thought could improve accuracy and reduce hallucinations, it will increase response times, which can affect user experience. https://lnkd.in/gtGGwf5P #largelanguagemodels #RAG #EBR #LLM #ml #ai #hallucinations #evaluations #

4

Here’s one way to use ChatGPT in Patient Services. Well technically, not ChatGPT, but Large Language Models (LLMs). LLMs are a type of AI and they are the underlying technology that power ChatGPT. What makes LLMs so powerful? They’re really good at analyzing and processing text. You know who deals with a lot of text? Field Reimbursement & Hub teams. We’ve been exploring the boring applications for LLMs that don’t make the headlines but could be very valuable to some teams. Here’s one example: ✅ Automatically processing EOBs for HCP reimbursement If you deal with a buy & bill drug, you’re probably processing lots of EOBs to reimburse providers for copay assistance. Today, many hub teams manually receive their EOBs as faxes. But these teams get lots of faxes. So usually this fax gets added to a backlog of all faxes. Someone then reviews every fax. Whenever it’s an EOB, they have to review it and figure out if it meets the criteria for processing payment. This process is obviously very manual. And if you don’t do this in a timely fashion, HCPs get frustrated, which negatively affects their experience on your program. They also will call you more frequently to get a status update on their reimbursement. This makes your phone lines busier & can hurt your KPIs like first-call-resolution. We can use AI to automate this process. We can use LLMs to automatically detect the EOBs within your faxes. And we can review the text in the EOBs to streamline this process. We can do this in a secure, HIPAA compliant way. This lets you process your claims faster, free up your staff, and reduce the number of calls that you have to deal with. It also improves the HCP experience on your program. 📣 If you face this problem, and you’re interesting in learning more, feel free to DM me. I’ll also be at Asembia if you want to connect there. ❓If you’re in patient services, field reimbursement, or hubs, here's my question for you: Which day-to-day processes in your life do you want to see streamlined?

7

We're thrilled to announce M12's participation in DatologyAI's Series A! Datology is a leading data curation platform that empowers businesses to unlock the true potential of their data for AI and machine learning initiatives. As the field of AI continues to evolve, data curation will play an increasingly critical role. We believe Datology is at the forefront of this revolution, providing the tools and expertise necessary to build robust, reliable, and impactful AI models. We're already seeing SLM (small language models), including Microsoft phi-2, tout "data curation" as a major driver of improved model performance and accuracy. It's no secret that clean, curated data is the fuel for high-performing AI models. However, the status quo is to throw (big) data at the problem. Datology re-imagines how models are pre-trained by shifting the focus from parameters to data quality and data accuracy at the data curation phase. While we think big data will remain the foundation, we also believe curated, contextualized data will drive the next frontier of AI advancement. Congrats Ari Morcos Bogdan Gaza Matthew Leavitt! M12, Microsoft's Venture Fund Michelle Gonzalez Michael Stewart Todd Graham Peter B. Tina McNulty Valentina Escudero Jose Clautier Noah Bradford #data #microsoft

143

8 Comments

TRUST TAKES YEARS TO BUILD SECONDS TO BREAK, & FOREVER TO REPAIR One of the biggest risks posed by Gen AI is a further & irreversible erosion of trust. How can technology help counter this risk?

22

2 Comments

Can’t agree more on this as we are seeing more early stage startups building more useful AI-native agents instead of copilots on existing workflow.

4

Tiny is mighty, it seems. This week, tech giants Microsoft and Apple both released open source language models with a small footprint. What’s powerful here — if these language models can run on-device, you truly have AI in your pocket that runs anytime, anywhere, even without the internet. This could potentially open up a whole horizon of use-cases of AI in the physical world — like running constantly in the background of a device like an iphone or accessed where there’s no internet connectivity (like mid-flight or in a war-torn area). Taking the musings one step further: because it can run locally on a device like a phone, these small LMs could potentially be finetuned or inferenced across a peer-to-peer network. Making it possible to be really collaborative, and a lot cheaper to train and run. This feels like the cusp of something big. All eyes on WWDC on June 10. For more weekly downloads on AI and creativity: I write a letter on Fridays at thereview.strangevc.com

24

Highlighting: "Nature has created this virtuous cycle of seeing and doing, powered by spatial intelligence," she said." (Fei Fei Li) ----- Spatial Intelligence sounds 1000x more useful than 'generative' AI. Our world is 3D, not 2D. Understanding 3D requires simulations that are spatially rendered in 3D. 3D is the measure of our world. We use 3D for games, robotics, drones, autonomous vehicles, digital twins and more. Not 'gen' AI word salad bullshitting. Gen AI is either a dead end branch or the tiniest part of a more complex 3D future. Bring on spatial intelligence. No bubble, just work. ----- Reuters: "Exclusive: Stanford AI leader Fei-Fei Li building 'spatial intelligence' startup." (Katie Paul, Anna Tong, & Krystal Hu) (May 3, 2024) "Prominent computer scientist Fei-Fei Li is building a startup that uses human-like processing of visual data to make artificial intelligence (AI) capable of advanced reasoning, sources told Reuters, in what would be a leap forward for the technology. Li is widely known as the "godmother of AI," a title derived from the "godfathers" moniker often used to refer to a trio of researchers who won the computing world's top prize, the Turing Award, in 2018 for their breakthroughs in AI technology. One source pointed to a talk Li gave at the TED conference in Vancouver last month, in which she said the cutting edge of research involved algorithms that could plausibly extrapolate what images and text would look like in three-dimensional environments and act upon those predictions, using a concept called "spatial intelligence." To illustrate the idea, she showed a picture of a cat with its paw outstretched, pushing a glass toward the edge of a table. In a split second, she said, the human brain could assess "the geometry of this glass, its place in 3D space, its relationship with the table, the cat and everything else," then predict what would happen and take action to prevent it. "Nature has created this virtuous cycle of seeing and doing, powered by spatial intelligence," she said. Her own lab at Stanford University was trying to teach computers "how to act in the 3D world," she added, for example by using a large language model to get a robotic arm to perform tasks like opening a door and making a sandwich in response to verbal instructions. Li made her name in the AI field by developing a large-scale image dataset called ImageNet that helped usher in a generation of computer vision technologies that could identify objects reliably for the first time. By jumping into the startup world, Li is joining a race among the hottest AI companies to teach their algorithms common sense in order to overcome the limitations of current technologies like large language models, which have a tendency to spit out nonsensical falsehoods in the midst of otherwise dazzling human-like responses." (keep reading) Reuters: https://lnkd.in/gmU8Ga2B #ai #notgenai #3d #spatialintelligence #spatialcomputing #ourworldis3d

12

4 Comments

"So getting back to NVIDIA, I don’t see any force stopping the juggernaut in the next few years, other than a huge decline in the need for AI compute, and that seems unlikely. Any startup competitor could easily be acquired by NVIDIA for tens of billions of dollars and it would barely be material given NVIDIAs market cap. And while the other major chip companies are mounting a challenge, particularly AMD, the demand for GPUs is so great, and will be for the next few years, there is room for massive growth for multiple players. When NVIDIA eventually slows down, I suspect it will come from one of two areas. The first is a surprising shift in market use cases. Given the long development cycles of computer chips, it’s hard to be as responsive to the market demands as software companies can be. And product designers have to choose some level of predictability and reliability in their components and their partners, which is why they favor proven tech over innovative new designs most of the time. But the way NVIDIA benefited because their chips for graphics cards happened to be a good match for new AI workloads - I think something similar could happen to slow them down. Some new popular AI workload will be a better fit for something other than a GPU, but a chip that is already established in the market in other ways. Or secondly, it comes from a company that is building their own chips and decides to get into the broader semiconductor business. This means Apple, Amazon, Facebook, or most likely - Google. But all those companies, even if they can compete technically with NVIDIA, lack the infrastructure to sell and support chips and those who develop them. Building those capabilities will be slow." https://lnkd.in/dFpj5hqv

1

1 Comment

Sam Altman in the MIT Technology Review yesterday: the killer app for AI is a "super-competent colleague that knows absolutely everything about my whole life, every email, every conversation I’ve ever had, but doesn’t feel like an extension.” It could tackle some tasks instantly, he said, and for more complex ones it could go off and make an attempt, but come back with questions for you if it needs to. /quote These agents can be created now, and there's important work to do making them trustworthy, reliable, accurate, data-driven, and compliant. https://lnkd.in/gCsKmhZ6

9

3 Comments

Researchers have credibly documented that AI is now capable of lying to humans "Recent research showed that as LLMs become more complex, they express emergent properties and abilities that were neither predicted nor intended by their designers. Next to abilities such as learning from examples, self-reflecting, doing chain-of-thought reasoning, utilizing human-like heuristics, and many others, researchers recently discovered that state-of-the-art LLMs are able to solve a range of basic theory of mind tasks. In other words, LLMs can attribute unobservable mental states to other agents and track them over the course of different actions and events. Most notably, LLMs excel at solving false belief tasks, which are widely used to measure theory of mind in humans. However, this brings a rather fundamental question to the table: If LLMs understand that agents can hold false beliefs, can they also induce these beliefs? If so, this would mean that deception abilities emerged in LLMs." https://lnkd.in/eHymv-7S

13

5 Comments

The honeymoon phase of generative AI (gen AI) is over. As most organizations are learning, it is relatively easy to build gee-whiz gen AI pilots but turning them into at-scale capabilities is another story. The difficulty in making that leap goes a long way to explaining why just 11 percent of companies have adopted gen AI at scale, according to our latest tech trends research. #ai #generativeai #genai #scaling #thefundraisingangel

12

Why do companies roll back Generative AI features more frequently than "traditional" ones? Unlike non-AI logic, Generative AI is unpredictable and no amount of pre-release testing can ensure its behavior in real-world usage. Even companies like Google are not immune; https://lnkd.in/eTsszVzG. #AI #LLM #TrustworthyAI #Google

15

2 Comments

Sparse autoenconders have shown promising early results in interpreting LLMs and could potentially be the next frontier for programming them too. Recent results from Anthropic and OpenAI show we can derive 'features' using autoencoders, which represent activations of neurons that correspond to concepts that we can understand. Amplifying certain features results in clear downstream model behaviour, which is why Linus Lee described feature steering as an important area of interface research. Shawn swyx W described it as '"representation engineering writ large". It'll be interesting to follow the progress of interpretability research and the surface area of model tuning that this unlocks. https://lnkd.in/eCkuchm6

26

2 Comments

The LLM will see you now: Google and DeepMind released a paper on Monday describing Med-Gemini, a group of advanced AI models targeting healthcare applications. The authors claim that Med-Gemini is outperforming competing models such as OpenAI’s GPT-4. During the pandemic we saw the rise of Telehealth which has largely come back to earth but could doctors be consulting more often with AI? (We know they already Google stuff) #Healthcare #AI

12

9 Comments

I think it's import to know that large language models generally suck when it comes to numbers and why. I recently read an article on how poorly LLMs reason, and the problem they used was of the "Bob has X sisters and Bob's sister Mary has Y brothers, how many brothers does Bob have" variety. This requires the LLM to work with numbers by calculating how many siblings there are in order to produce the right answer, and it's something that LLMs have fundamental problems with. It's not an example of poor reasoning, it's an example of LLMs being bad at working with numbers. And they're bad at working with numbers because of "tokenization". Computers can't work with text. They can only process numbers. If you want a computer to do anything with text, there needs to be a numerical representation of the text for the computer to work with. And the way LLMs like GPT-4 turn text into numbers coincidentally makes it very hard for them to work with numbers. For computers, text needs to be split up into smaller parts, and each part is assigned a unique number. If we give each letter in the English alphabet a number like a=1, b=2, etc. then the computer can say 20, 8, 5, 19, 9, 19: 20=t 8=h 5=e 19=s 9=i 19=s And we can see: "thesis" That's what tokenization is. The letters are tokens. 's' is one token with the number 19, and this token appeared twice in "thesis". But separate letters make it difficult to understand the higher meaning behind words. Think how hard it is to understand what someone is saying if they spell every letter individually. It's even worse for computers. It might seem reasonable to just use words as the different parts. "thesis" could just get it's own number. Let's say '1'. And we'll give "the" the number '2'. Then '2,1' would mean "the", "thesis". That approach didn't work though. There's too many words! Especially when you want to include other languages like Mandarin. What modern language models use instead is tokens that are 1-6 characters long. Not just letters, but not full words either: they use tokens as building blocks for sentences. For example: "the" = 1 ' ' (space) = 2 "sis" = 3 "ter" = 4 "the thesis" is represented as 1,2,1,3: 1: "the" 2: " " 1: "the" 3: "sis" "the sister" is represented as 1,2,3,4: 1: "the" 2: " " 3: "sis" 4: "ter" We re-used token 3 ("sis") to represent both "thesis" and "sister". That's how LLMs build sentences. There's a token visualizer for GPT-4 here: https://lnkd.in/gpuQrGP3 So why do LLMs perform badly when it comes to numbers? Because those numbers are also represented as tokens! "5 + 12 + 3 = x" could be represented with the tokens "5 + 12 + " "3 =" " x" It's a mess! LLMs can't really do math, because the tokenization makes it impossible for the LLM to operate on a number basis. It can't separate out "+" or "=". For math, we'd have to go back to the separate-letters method. A model can either produce poems, answer questions and hold conversations, or do math, but not both.

7

This is (has been for a decade plus) my issue with use of even weak-AI for things like medical imaging, forget design or coding or writing. https://lnkd.in/gbukQBkP If you missed it before: correct. This has been shared before, but is not well discussed, or even swept under the rug in the hype cycle. But if you didn't know, for weak-AI/LLM/ML: We Do Not Know why it says what it says. Unlike evaluating any human's work where they can say "I saw this, that, and over here, consistent with X" the AI goes "Positive" and a reviewer would have to do their work from scratch. So, not a time saver and the way evaluations work, not likely to go well. When things start going wrong with medical systems, environmental or traffic monitoring, finance, etc. we won't know till it has made mistakes and ruined economies, outright killed people. And then we won't know how to fix it, because it is a black box. THIS is why I am not just against Gen-AI for ethical reasons (theft) but think we are headed down the wrong CS-level path entirely, and evolutionary improvements won't fix the underlying Big Problem with it.

1

2 Comments

The next big thing in AI after LLMs are VLMs. Large Language Models allow computers to chat with humans. However using images or videos in those chats is still resulting in bad outcomes. Often when ChatGPT is asked to generate text inside images, there are spelling mistakes. This is where VLMs or Visual Language Models come in. They should be able to describe what is in an image or a video in words which afterwards can be used in an LLM. So imagine a VLM security camera which explains in text that there is a long queue in front of the two open shopping checkouts, employee X just served three lattes while employee y has been talking to employee z, an unknown person has just broken the window, a fox got into the chicken stables, a house fire started,... If VLMs can translate what happens in the world around us then robots will be able to interact a lot easier with it. Tesla self-driving software is already using VLMs to describe what is happening around it, e.g. stop sign at 50m. What do you think the future of VLMs will be? https://lnkd.in/eGsj9qb2

10

2 Comments