Raghu Hiremagalur

We are excited to announce the launch of Ursa engine, our Kafka API-compatible data streaming engine that is built on top of Lakehouse. Ursa is not something entirely new; it is built on our many years of experience building and operating #ApachePulsar and #ApacheKafka. To learn more about the #Ursa engine, I recommend our #PulsarSummit keynote presentation and announcement blog post. Here, I’d like to delve deeper into some of its features: 1. Making Kafka API a first-class citizen: In #ApachePulsar, we developed a unified messaging model combining queuing and streaming semantics into one protocol, aimed at enhancing data sharing between business applications. Although powerful and increasingly adopted by enterprises and unicorns, the requirement for application rewrites is impractical in today’s economy. Instead of building data gravity around a protocol, we open up access to the robust stream storage layer by extending support to the Kafka protocol, transitioning from a single protocol to a multi-protocol platform, facilitating greater flexibility and integration. 2. Achieve Stream-Table duality via Lakehouse Storage: Data streams commonly end up in data lakehouses, where integrating and maintaining them can be complex and resource-intensive. Ursa simplifies this by storing data streams as lakehouse tables in object storage. This integration not only streamlines operations but also allows seamless access to operational data for analytical processes. Thanks to its multi-modal storage engine which fosters data sharing across teams and departments. 3. Object storage as the primary storage solution: To better align with cloud economics, particularly for large workloads, Ursa makes ZooKeeper and BookKeeper optional and utilizes a cost-optimized write-ahead log based solely on object storage with leaderless and stateless brokers. This architecture leverages object storage for both data replication and storage, minimizing inter-AZ data transfers and reducing costs for latency-relaxed, high-volume workloads. By integrating these features at the multi-tenant level rather than at isolated cluster levels, Ursa eliminates costly data duplication and movement across clusters, allowing teams to map directly to tenants and be able to share data across teams and departments more efficiently. We believe that the future of data streaming platforms lies in being open and multi-faceted, fundamentally different from traditional data platforms. They are designed not just to store and process data, but to enhance organizational capabilities by fostering data sharing across the organization and integration with other systems. More and more data streaming platforms will look more like Ursa, with multi-protocol, multi-tenant, and multi-modal capabilities. Check out our announcement blog post about Ursa: https://lnkd.in/ghEKTYE6 #ApacheKafka #Lakehouse #DataStreaming

100

1 Comment

I like this idea of "technical taste". It gets into where software engineering is as much creative art as it is science. Four great takeaways (IMHO) for software devs in this era in particular: 1. Aspiring engineers should cultivate a sense of curiosity, experiment with different tools and technologies, and embrace a mindset of continuous learning. 2. AI has the potential to streamline processes and enhance productivity for engineers, but it may also lead to disruptions in traditional software development workflows. 3. Developing technical taste and judgment is essential for making informed decisions about which technologies and approaches to pursue. 4. Collaboration and open-mindedness are key to leveraging the full potential of AI and staying ahead of technological advancements. I also really liked these two quotes from Sam Schillace: 1. "The right time to do something is when you have that feeling in the pit of your stomach that's like, 'oh, this is a great idea and it's going to suck to build because nothing's ready yet.'" 2. "Technical taste is like, 'how well have you consolidated that set of experiences and heuristics into judgment that you can apply accurately when you see new things?'" 3. "It may be the case that very small teams can do very large projects, or like we were talking about before, it may be the case we're just going to get really ambitious about what we try to do with the same size of teams, which is kind of where I would put my money." Great job on continuing to put out great episodes, Brett Berson and team First Round Capital! #ai #engineering #softwaredevelopment

5

Symflower Launches DevQualityEval: A New Benchmark for Enhancing Code Quality in Large Language Models Symflower has recently introduced DevQualityEval, an innovative evaluation benchmark and framework designed to elevate the code quality generated by large language models (LLMs). This release will allow developers to assess and improve LLMs’ capabilities in real-world software development scenarios. DevQualityEval offers a standardized benchmark and framework that allows developers to measure & compare the performance of various LLMs in generating high-quality code. This tool is useful for evaluating the effectiveness of LLMs in handling complex programming tasks and generating reliable test cases. By providing detailed metrics and comparisons, DevQualityEval aims to guide developers and users of LLMs in selecting suitable models for their needs. The framework addresses the challenge of assessing code quality comprehensively, considering factors such as code compilation success, test coverage, and the efficiency of generated code. This multi-faceted approach ensures that the benchmark is robust and provides meaningful insights into the performance of different LLMs. Read our full take on 'DevQualityEval': https://lnkd.in/guRuBjaB GitHub: https://lnkd.in/grssvCVR Symflower #artificialintelligence #ai #datascience #llms

44

3 Comments

What’s Next: RunWhen's Kyle Forster talks open source and microservices SDxCentral CEO Matt Palmer speaks with RunWhen's Kyle Forster about a better way for companies to support gigantic code bases.

2

AI Reading for Sunday May 12! Speculating about OpenAI's Monday announcement. https://lnkd.in/e4nA2xKd More rumors of what ChatGPT might show. - THE DECODER Top guesses according to Perplexity, others, and yours truly - Project Q* which possibly learns to solve complex multi-step reasoning problems with something like Q-learning and tree search. - Something with agents and improvements to Assistants API which Sam Altman has been hyping - Better voice assistant - Not GPT-5 per Sam Altman, but maybe 4.5 with a long context and multimodality to catch up to Google, and price reductions for GPT-4 matching cheaper startups - Improved AI studio dev tools. - Maybe OpenAI should release a small local open source model and runtime environment for finetuning it. That space is buzzing but the technical bar to get into it is high for most people. Or maybe, a sarcasm detector. Wow, a true leap for mankind. Great job, Einsteins. - PCMAG OpenAI removed 'our most advanced' from the description of GPT-4 ... a more advanced 4.5 or something coming? Either they have poor opsec or are good at getting attention with headfakery. - Reddit ARM to launch AI chips to compete with Nvidia. But their business model is they don't sell chips, they license designs to vendors like Apple and Amazon and Nvidia, so this is new for them. ARM might have pricing power and a roadmap to capture more of the value chain, also might push customers to e.g. RISC-V. - Nikkei Asia WSJ covers NYC AI startups in NYC like HuggingFace, Runway, Pinecone, which are raising money at 1/2 the pace of Bay Area - WSJ Pro Alibaba leverages cloud business to become leading AI startup investor in China. - FT Wikipedia adds an LLM-powered Chrome extension for fact-checking. - Chrome Store How did that robot commencement speaker work out? Looks kinda weird and creepy. - YouTube Researchers test AI systems' ability to solve the New York Times' connections puzzle Tesla exec leaves, suggests it's a toxic work environment and "The recent layoffs that are rocking the company and its morale have thrown this harmony out of balance and it's hard to see the long-game." - Futurism Videos of cheerleaders engaging in off-limits activities were circulated. The cheerleaders and cops said they were deepfakes. A woman was accused of deepfaking them. Not deepfakes, although possibly underhanded shenanigans of unclear motivation. - the Guardian

2

This is a very interesting article on sparse vectors, which can help overcome limitations of vector search related to domain specific keywords. And of course, the code in the example is clear and clean! Sparse vectors have a few values and many zeros. Dense vectors have limited or no zeroes. But really, what we're talking about semantic search on terms where the context is not present or harder to infer. The author presents this as specialized keyword search such as medical terms or other domain specific terminology. "These scores reflect the significance of each term in the context of the document or query, guiding the model to allocate more weight to terms that are likely to be more meaningful for retrieval purposes." Highly recommend reading. Nice work Paul Iusztin. And a nod to Qdrant and their vector search technology that supports sparse vectors. https://lnkd.in/gcBWfTg8

3

pgvectorscale: A Game-Changer for Vector Similarity Search 🚀 Timescale's #pgvectorscale extension for PostgreSQL has achieved a remarkable milestone, matching the performance of Pinecone, a dedicated vector database, while being 75% more cost-effective. 💰 In their latest blog post, Timescale showcases the impressive speed and efficiency of pgvectorscale, which leverages the power of PostgreSQL to deliver lightning-fast vector similarity search. This breakthrough opens up new possibilities for applications like semantic search, recommendation systems, and more. 🔍 Key takeaways: 🌟 pgvectorscale now matches Pinecone's performance in vector similarity search 💸 pgvectorscale is 75% more cost-effective compared to Pinecone 🔧 Easy integration with existing PostgreSQL databases 🌐 Scalable solution for handling billions of vectors ⚡ Optimized for fast and efficient search results pgvectorscale's seamless integration with PostgreSQL allows developers to incorporate vector similarity search into their applications with ease, while benefiting from the robustness and scalability of a proven database system. 🗄️ If you're looking for a high-performance, cost-effective solution for vector similarity search, pgvectorscale is definitely worth exploring. 🔍💡 #pgvectorscale #postgresql #vectorsimilaritysearch #semanticsearch #recommendationsystems #timescale https://lnkd.in/gsfgCK32

5

Seeking a full time onsite hybrid AI Architect Engineer Manager for leading RAG and LLM Generative AI role to relocate and lead RLHF, prompt engineering for zero shot learning, one shot learning, few short learning. Engineer Architect or Manage and lead teams on Fine Tuning LLM models on PEFT, RLFL, Prompt Tuning context window, and to work on Reward model using LoRA, and to work on agent to instruct LLM on Context window and lead AI Generative LLM RAG performance for scalability. Engineer Architect or Manage and lead Generative AI LLM Lang Chain Use cases, and work on Hugging faces for encoded/weights. Engineer Architect or Manage and contribute to building RAG Retrieval Augmented Generation to retrieve external library data for tailored to specific models domain in prompt analysis. Additional talents that I have which include People first partnerships, deliver on AI architect accountability and ensure focus on diversity inclusion for delivery business value. Manage stakeholders data projects to improve business revenue. People talents that I offer are people loving manager who is humble and offers critical creative innovative AI thinking with the team. Engineer Architect or Manager the architecture for AI Lang Chain Generative AI Use Cases that need LLM text words. Contribute as Engineer Lead on improved production pipeline supply chain signal processing and be the Engineer Architect for large language text analysis (LLM). Research for teams can be more productive in using Lang Smith for LLM large language models. Assist on curating sandbox to develop bidirectional decoder/encoder transformer (BERT) masking language next sentence (MLM). Lead again on OpenAI GPT for optimized performance to build chat what-if analysis for the business. Manage Engineer to lead scaling the team on development with performing LLM Chain and predictions models and build predict messages that pull from the Hugging Face Hub objects and prompt engineering with prompt templates and human message responses.

3

Tips a hat to SourseAI - “A narrow ICP definition and deep customer empathy” Tanya Hyams-Young & Paul Jefferies are seeing the fly wheel effect of vertical SaaS. Note to Aussie startups that take this route - think hard about the TAM, here and abroad. You’ll probably conclude you’re going to need to move to “the rest of the world” pretty quickly. See you in a week EMEA! #verticalsaas #ai #decisionintelligence #telecoms #mvno

11

5 Comments

Thanks for sharing these insights. What I am seeing is that orgs test an idea using a proprietary #LLM like #OpenAI, then put it into production with an open source LLM using a development platform like #Llamaindex. That way, private data remains private.

1

To everyone finding it challenging to grasp how to utilize Large Language Models (LLMs), think of LLMs as an operating system. Instead of trying to develop your own LLM modules, which most companies lack the resources or budget to do, focus on building applications on top of existing LLM platforms. It’s similar to how apps are created for iOS or Android. #genai #llm #llmapps

15

1 Comment

New post on what happens after you add a control plane and local storage for observability: you get 1000x the telemetry when it matters. https://lnkd.in/gAvkk-qP

58

1 Comment

A story as old as time The Innovators’ Dilemma. Darlene succinctly summarizes the antidote: - Prioritize research and experimentation FOR disruption. Focu on how to cannibalize the own business, not just improve it… - Be willing to pivot and adjust plans when faced with issues or changing market conditions. Don’t see the pivot as a failure. All learning is valuable… - Don't pass on investments or partnerships that could drive growth and innovation even if the ROI isn’t immediately known… Innovation isn’t a guarantee. But without it, there is. #innovatorsdilemma #corporateinnovation

12

2 Comments

Nice to see Agent Architecture making a comeback to enhance the performance and reliability of LLMs. Andrew NG followers will find this article unsurprising; working on small, specialized agents boosts speed and reduces hallucinations. #AgentArchitecture #LLMs #AI https://lnkd.in/dGmhu6d6

9

6 Comments

LinkedIn team (Juan Pablo Bottaro Karthik Ramgopal) shared a helpful deep dive into their deployment of an LLM-powered job search assistant. Highly recommend this read! Few highlights:  💡 Relatively easy to setup RAG-based pipeline. Achieved 80% of the basic experience in 1 month, then 4 more months to reach 95% mark. 💡 Evaluating the quality of application outputs is hard. I hear this from many enterprises developing LLM applications. (check out Okareo by Matthew Wyman Boris Selitser) 💡 Balancing latency with quality is important. While certain techniques like Chain of Thought could improve accuracy and reduce hallucinations, it will increase response times, which can affect user experience. https://lnkd.in/gtGGwf5P #largelanguagemodels #RAG #EBR #LLM #ml #ai #hallucinations #evaluations #

4

AWS continues to add capabilities to Amazon Bedrock that make it the easiest way to build high quality, low latency, and cost-effective GenAI applications. Today the team announced:   - Custom Model Import, which lets customers easily bring their own models (from SageMaker and S3) to Bedrock to leverage its leading GenAI features, such as RAG, agents, Knowledge Bases (guardrails), inference deployment, and advanced customization to keep teaching and refining models.   - Model Evaluation, which lets customers quickly select predefined evaluation criteria (like accuracy and robustness) and run reports that help them understand which is the best model for them.   - Meta’s new Llama 3 models, now available on Bedrock and can support a wide range of use cases including text summarization and classification, sentiment analysis, language translation, and code generation.   - New Cohere Models, Command R and R+, are state-of-the-art FMs customers can use to build enterprise-grade generative AI applications with advanced RAG capabilities, in 10 languages, to support their global business operations. These will arrive on Bedrock in the coming weeks.   - New Amazon Titan models with Amazon Titan Image Generator (GA today) helping customers produce studio-quality images, at low cost, using natural language prompts (and applies an invisible watermark to help reduce the spread of disinformation)… along with Amazon Titan Text Embeddings V2 model, which reduces storage and compute costs, while out-performing other leading models on text embeddings.   Tens of thousands of customers are already building real-world production-ready GenAI apps and we're excited for even more customers to get started. https://lnkd.in/g4rkak47

2,849

53 Comments