Bryan Ngo

Introducing a little side-project that I've been building! 🔥 genson-rs: a blazing-fast JSON Schema inference engine that can derive the schema from gigabytes of JSON data in mere seconds 🚀 written in Rust genson-rs runs 25-75x faster than its python counterpart, and generally performs much better than most (if not all) other open-source schema inference tools out there! The speed it offers makes it a viable option for large-scale schema inference both online and locally. Check it out at: https://lnkd.in/gnpbUPg5 if you are curious about how it benchmarks against other tools and the optimizations applied! Any issues or feedback are appreciated! 🙌  #rust #jsonschema

45

Xian Ke, this is incredible! The speed at which you and Vishal Punjabi leveraged crewAI, Streamlit, and various LLM models to create Finbot, a powerful company analysis tool for investors, was mind-blowing. ICYMI, Finbot is here: https://lnkd.in/eSQD69qA I couldn't help but chuckle when I saw the Finbot name - it reminded me of the Wall Street Bets crowd. Maybe you can get them to yolo their life savings based on Finbot's analysis? (Kidding, of course. Or am I? 💎🙌) But in all seriousness, your enthusiasm and passion for AI and LLMs is contagious. It's engineers like you who are pushing the boundaries of what's possible with AI and paving the way for a brighter future.🏆 I'd love to extend an invitation to any other software engineers from the bootcamp to join our Devs Do AI Discord server. It's a great place for engineers without an ML / AI background to connect with like-minded devs, share knowledge, and collaborate on LLM projects. DM me for the invite link. 🤝

13

I can't believe the key takeaway I had for middle and high school girl students this weekend in a programming camp was something even expert software engineers forget sometimes: "software engineering is much more than coding". I had the chance to talk to these young bright minds at Santa Clara university. I explained, hopefully in an intuitive way, how humans coordinate at large scale to build products like YouTube. And mainly I wanted to bust the myth that software engineering is a job where you code heads down all day and don't need to talk to people. Three main qualities I highlighted in a good engineer: - knows their users - team player - problem solver That made me think though. The meetings and all those office hours, and 1:1s spent "socializing", while on the surface seem like unproductive hours, they're indeed my job now. And that's true of anybody who's growing up the ladder in organizations. Human coordination is a beautiful problem :)

140

3 Comments

I built a 2 bit “Gen AI”* from scratch. Transformer are the core architecture of all gen AI systems, but the attention mechanism in Transformers has allways been a bit difficult for me to reason about. Last night, I trained a transformer to work as a logical “and” gate on a two bit binary sequence. I basically treated the bits as tokens and then just used a single self attention head to try and update the encoding so that all bits would turn to 1 in the “and” case and all bits would turn to 0 in all other cases. It was super fun and I’m learning a ton, I have a bunch of ideas to build on this and keep growing my practical understanding… but my initial algorithm also took 12 parameters and 10k training iterations in order to get a good output. Where I could get a perfect output immediately with 2 transistors. In addition to helping me understand transformers, it was also a nice reminder that AI shouldn’t do everything. Sometimes it’s just impractical.

24

3 Comments

Principal Engineer Staff Engineer Distinguished Engineer (my favourite) I’m going to say it and it’s likely a little rhetorical: I dislike all these terms immensely. Please can someone give me the canonical description for what they are supposed to mean? (I know I can use a search engine or AI to do it… I want to know what people think they mean). The industry I work in is in danger of disappearing up its own backside. What’s wrong with: Software engineer Why do we need to differentiate? The only reason I can see is signposting to people with differing levels of experience or different sets of responsibilities. If we need to do that then, I’m not sure our teams are set up correctly in the first place. #software #softwareengineering #softwaredevelopment #productdevelopment

100

105 Comments

Anyone in my network interested in LLM correctness (or knows someone who might be interested)? Please reach out and I'd love to bounce some ideas off. Correctness verification for LLMs is very difficult and may require new definitions of correctness and new ways of verification: 1. Classic testing theory says that if we partition the input domain well, then we only need to test finite boundary conditions. This may not apply to LLMs. 2. Part of the challenge also comes from the non-determinism of the LLMs. Even if we set the temperature to zero, there is still non-determinism if the model is retrained with up-to-date data or fine-tuned with domain-specific data. For more context, please take a look at the discussion thread with Martin Casado on Twitter. #artificialintelligence #machinelearning #deeplearning https://lnkd.in/egKqJ2xB

10

1 Comment

Now we have two first class data catalogs integrated with MS Fabric now available in Azure. Of course DataBricks is the heavy heavy favorite in the enterprise lakehouses as well as AI (as of the conference that just finished 🤓)

6

I have a new blog post that discusses our journey in speeding up Docker builds for a large rails monolith - https://lnkd.in/gT9HVnWv

18

The Power of AI-Driven Pair Programming As a Staff Engineer at Artium, I've witnessed firsthand the transformative power of integrating AI tools, particularly Large Language Models (LLMs), into pair programming. This innovative approach is revolutionizing the way we code and collaborate, and I'm excited to share my insights with you. One of the most significant benefits of utilizing LLMs like GPT in pair programming is the dramatic acceleration of coding processes. By providing real-time feedback and code suggestions, these AI models reduce the time spent on routine tasks and debugging, allowing teams to push updates and features at an unprecedented pace. This increased speed and efficiency is a game-changer in the fast-paced world of software development. Moreover, LLMs leverage vast datasets to suggest coding solutions that adhere to best practices, significantly improving the quality and reliability of software products. This AI-driven guidance helps minimize errors and enhance overall code performance, ensuring that we deliver top-notch solutions to our clients. Another crucial aspect of AI-driven pair programming is the optimal utilization of resources. By handling repetitive tasks, AI allows developers to focus on complex problem-solving and creative innovation. This shift not only optimizes human staff allocation but also amplifies the creative potential of development teams, leading to more groundbreaking solutions. AI models in pair programming also serve as an on-demand knowledge base, teaching new frameworks and languages, and keeping the team updated with the latest industry standards. This fosters a culture of continuous learning and professional growth, which is essential in the ever-evolving tech landscape. Furthermore, with AI, the benefits of pair programming are no longer confined to office hours. Developers can interact with AI partners anytime, ensuring continuous productivity and making software development more adaptable to varying work schedules. This flexibility is crucial in today's diverse and dynamic work environments. In my experience, the AI-human partnership in software development not only increases operational efficiency but also paves the way for more innovative solutions. As we continue to explore and understand AI's potential, it's clear that the fusion of AI with human ingenuity in pair programming is just the beginning of a larger transformative journey in tech. I'm curious to hear your thoughts and experiences. What AI tools have you integrated into your development process, and how have they transformed your workflow? Let's discuss how we can further harness AI to enhance our productivity and creativity in tech. #SoftwareDevelopment #PairProgramming #AIinTech #Innovation #TechCommunity

5

Training large complex models can pose lot of performance and infrastructure challenges. With Anyscale we can easily overcome these challenges. Some examples: - Ray Data allows you scale computationally intensive operations like preprocessing in a separate CPU cluster and leverage GPUs for model training. This ensures efficient use of GPUs. - Ray Data provides mechanism to stream data from cloud storage to your processing layers to Training workers. This ensures there is no idle time in GPU workers waiting for the data. - Ray Train provides out of box solutions for fault tolerance, so the training can resume seamlessly on an application crash or system crash. If you are interested in more such techniques, please join our webinar here: https://lnkd.in/g3am2nth

10

Who is Benjamin Ott? I'm a dad to 3 amazing kids, a musician (bass and guitar), a software engineer, a novice golfer who enjoys the time out with friends, and an avid machine learning enthusiast (with a PhD in the field). For work, I build infrastructure for generative AI models. This has been more than just a job for me - it's a front-row seat to history in the making. With a background in math, computer science, and machine learning, I've always been drawn to challenges that blend complex theory with practical applications. This work isn’t just about writing code; it's about paving the digital highways for AI that can create, imagine, and interact in human-like ways. It’s a great responsibility and an adventure, helping to push the limits of technology in a field that's capturing the world's imagination and reshaping our future. From building defense systems to automating the reduction of energy waste to helping to build one of the world's leading smart home assistants, my path has prepared me to contribute to technologies that are making a profound impact across the globe. I'm excited to see where generative AI can take us in the next few years and I'm buckled in for what promises to be a wild ride. I think it's going to be an exciting one, bringing along inspiring transformation as it helps to push the frontiers of various industries by amplifying the abilities of creators.

78

18 Comments

A data-driven approach to driving wound healing outcomes Marcella Gomez, University of California, Santa Cruz https://lnkd.in/gAkWZYJT Meet: https://lnkd.in/g4AAWkcK YouTube Stream: https://lnkd.in/gwtqaTmf Talk details: https://lnkd.in/gWn3B3cU Subscribe for talk announcements: https://lnkd.in/g5ciuNuX Abstract: Precision medicine requires an ability to predict the response of an individual to a prescribed treatment regimen a priori. Thus, advancement in the field is challenged by a lack of predictive models and, arguably, a lack of time-series information for a highly dynamic system. Here, we discuss work in wound healing, for accelerating wound closure. We argue that timing of treatments is as critical to consider as the choice of drug or therapy. Due to system size and complexity, data-driven methods need to be explored to develop multi-dimensional quantifiable indicators tracking systemic changes. In this work I discuss how bioelectronic devices enhanced with deep learning can help facilitate real-time sensing and actuation for automated decisions in treatment for wound healing and preliminary work in transcriptomic based classification of wound states. Bio: Marcella M. Gomez is an associate professor at UC Santa Cruz in the department of Applied Mathematics and Associate Dean for Diversity, Equity, and Inclusion for Baskin Engineering. She received her PhD from Caltech in 2015 and a B.S. from UC Berkeley in 2009; both degrees in Mechanical Engineering. Her research interests are in the broad field of bio-control leveraging methods in machine learning and control theory. Applications range from controlling single-cell response to driving complex systems such as wound healing. #modeling #simulation #medicine #healing #trauma #predictivemodeling #datascience #data #woundhealing #biology #bioengineering #systemanalysis #sensing #omics

5

"The programming language is now human. Everybody in the world is now a programmer." - Jensen Huang Prompts are the new programs. GPT agents the new apps. Those of us trained in classical deterministic programming need to adjust our mental models to get better at leveraging GenAI tools. Only way to get better is to practice. #GenAI

8

When I first got to Amazon, one of the things that stood out as different from my past job experience is that Amazon has a strong written document culture. Meetings typically start with documents <= 6 pages and everyone reads the document for the first part of the meeting before starting discussion -- because who has time to read documents before the meeting? Starting the meetings with a read means that everyone gets the time baked into their schedule to actually read the doc, and then you can have a productive discussion with everyone starting at the same place. This simple tweak has a few powerful impacts, but the one that has had the most impact on me personally is the page length constraint. When you require everyone to read the doc at the start of the meeting, that imposes a time constraint on how much content you can expect them to read before discussion. This constraint puts the burden on *you*, the author of the doc, to make sure that you are: 1. Concise, highlighting only the important info. The details that don't make the cut can go in an appendix for reference. 2. Prioritizing the actual decisions that you want to get out of the meeting. You asked these people to meet for a reason! To help them help you, you have to organize the information in a way that makes it as simple as possible for them to grok and contribute/decide. 3. Actually understand your own problem! Size constraints force us to distill complexity and find the right conceptual abstractions to communicate at the right level. I started using the size constraint trick in my design documents as well for these reasons as I found it made it simpler for others to understand my docs - regardless if we're speed reading in a meeting or reading on their own. Happy Writing! ✒

234

7 Comments

Its about solving the problem, not 'having AI' in the system. This is from last March, but the Wave 1 vs. Wave 2 distinction does seem to be playing out. Less 'generate me content', more 'synthesize vast amounts of relevant data and aid in decisonmaking or signal extraction'. https://lnkd.in/ehFBFC32 They don't talk about multi-agent systems here, but looking forward to more startups with a focus on leveraging agentic design patterns (https://lnkd.in/ed5rMmFe) to streamline real world workflows.

4

1 Comment

There are a lot of hyped up claims about AI being tossed around lately; especially when it comes to software engineering. After being deeply immersed in this misunderstood industry for over 20 years, I can say whole heartedly, don’t believe the hype. This article does an excellent job at describing what the craft of developing software is really like and why the reports of its death have been greatly exaggerated. https://lnkd.in/eY-59vxH

14

I took some time during my day off to build a CHIP8 Emulator in Rust! https://lnkd.in/gtFkrFAu It is a silly little emulator capable of rendering only the most basic test ROMs. It has no monetary, business, or extrinsic value. All of the code (per the GitHub link) is open-source, and it is licensed under the public domain. I wrote this because I love to code. I didn't become a software engineer to solve any problems I had, or start a billion-dollar business, or land a prestigious job. I did it because nothing, not even music (my major at the time), felt as good to me as writing a program and watching it run. I wanted to make that my career. As my high school drama club teacher said: "Do what you love and you will never work a day in your life". That philosophy has guided my career decisions throughout most of my life. Unfortunately, I've learned over the past decade that this advice does not always apply. While I may have become a software engineer because I am passionate about coding, that is not the case for many people who do. Most, I imagine, see coding as a means to an end, whether as the raw material for a world-changing product, a skill that can be leveraged for income, or the initial stepping stone in a technical leadership role. It took me too long to realize that this is not only okay but the norm. My job is to create economic value. My employers do not pay me to indulge my passion, especially when it gets in the way of doing the job well. Often one has to ship suboptimal implementations to hit critical deadlines or settle for a "good enough" architecture and move on to higher priorities. Combine this with the expanding breadth one acquires as they become more senior, passion may start inversely correlating with performance. In retrospect, my dad got it more right than my drama club teacher when he said: "If it was fun, they wouldn't call it 'work'; they'd call it 'fun'". I've spent the last few years struggling to reconcile the passion I have as a coder with the pragmatism I face as a software engineer. I've come to realize that this struggle is born out of an inability to separate the two from themselves. Coding as a hobby is different from working as a software engineer. While the tools might be the same, the attitude and approach one must take in each situation is fundamentally different. I wrote this emulator today to remind myself of the joy of crafting code, separate from the work of engineering software. If you are a passionate coder, perhaps just starting in your career, perhaps more tenured, and you've been searching for that magical flow state you feel when you hit that enter button and watch the terminal light up, I'm here to tell you that it's been there all along, lying in the implementation of a useless emulator.

23

6 Comments