Mat Velloso’s Post

How big is the bet from #Microsoft for AI? On november 2023, the #Top500 list spoted in the 3rd place an Azure Cloud supercomputer, a Microsoft supercomputer called Eagle that delivered 561 petaflops of performance, with 1.12 million computing cores, including Nvidia's H100 GPU and Intel's Xeon Platinum 8480C CPU cores. The supercomputer provides access to the #GPT-4 and other generative models hosted on Azure cloud. Runs on top of Ubuntu 22.04. https://www.top500.org/ #GENAI #GPT4 #AzureCloud #AzureSupercomputer #Supercomputer #HPC #AI

Azure and NVIDIA Partner to Drive Next-Gen GPU Acceleration in AI! 🤖 Azure and NVIDIA have joined forces 🤝 to propel the AI landscape forward. By combining Azure's scalable cloud infrastructure with NVIDIA's cutting-edge GPU technology, this collaboration introduces next-gen GPU instances. 💫 🔺 These instances are poised to revolutionize AI workloads across sectors, delivering faster ⏩ training times, heightened accuracy, and accelerated insights. 📈 From healthcare 👨⚕️ to finance 💰 and beyond, this partnership paves the way for groundbreaking AI-powered solutions. #AI #CloudComputing #Innovation #NVIDIA #Azure #GPUAcceleration

🚀 NVIDIA's latest powerhouse, the GH200 GPU, is set to revolutionize AI capabilities for select companies! Bindu Reddy of Abacus AI is thrilled to be among the first to harness this tech for open-source AI projects. With industry giants like AWS, Google Cloud, and Oracle Cloud Infrastructure gearing up to integrate GH200 into their services, the AI landscape is poised for a major leap forward. Meanwhile, competitors AMD and Intel are not far behind, with their own AI accelerators challenging NVIDIA's dominance. Get ready for a surge in AI innovation as these tech titans race to power the future! 🌐💡 #NVIDIA #GH200 #AI #MachineLearning #DeepLearning #CloudComputing #Innovation #TechNews #ArtificialIntelligence #DataScience #BigData #Analytics #OpenSource #FutureOfAI #TechTrends

At the end, is very much a matter of hardware and GPUs. A cluster of 3,584 H100s, manufactured by NVIDIA, at cloud service provider CoreWeave completed a massive GPT-3-based benchmark in just 11 minutes. And that's just the beginning. Nvidia and CoreWeave are now in the process of helping Inflection AI install a new cluster that will run 22,000 H100s. That's likeky the the largest GPU cluster for AI applications in the world, ahead of Meta’s 16,000 GPU cluster announced in May. #ai #cloud #hardware #gpu

📣There's a hot, new chip on the block! 📣 1 week after OpenAI Dev Day, comes Nvidia's special address at Supercomputing 23 where they announced their latest and greatest AI chip yet. The H200s have 1.4x more memory bandwidth and 1.8x more memory capacity than the coveted H100s. Releasing in 2Q24, AWS, GCP, Azure, and Oracle will be among the first to offer these new chips. ...Not that this announcement does anything to ease the unparalleled supply crunch for these chips in Silicon Valley right now. Nvidia's GPUs are more precious than gold and harder to find than a snowflake in the Sahara. The lead time for new orders right now is 6 months, which might as well be a lifetime in AI. It's no wonder that the stock is up +247% this year, continuing its journey up and to the right. It's the closest thing we have to a monopoly play on the computing systems used to power GenAI.

https://lnkd.in/gCJYWNc8 MosaicML, just acquired by DataBricks for $1.3B, published some interesting benchmarks for training LLMs on the AMD MI250 GPU, and said it is ~80% as fast as an NVIDIA A100. Did the world just change?. #nvidia #amd #ai #cloud #llms #mosaicml #databricks #aitraining #gpucomputing #deeplearning #neuralnetworks #technology #innovation #business credit: forbes

If AI providers are in the spotlight, the current winners are companies supplying processing power both client-side and server-side. Notably, Microsoft saw a $1B increase in market cap, and while the OpenAI deal contributes, the primary driver is the surge in earnings from its cloud business. Similarly, Nvidia's market cap rose by $2B. This suggests that the main beneficiaries of the new AI ecosystem might not be the most visible players. https://lnkd.in/eCNcyqCT

AWS re:Invent 2023 saw a game-changing announcement from AWS and NVIDIA. AWS will be the first cloud provider to offer NVIDIA GH200 Grace Hopper Superchips, interconnected with NVIDIA NVLink technology through NVIDIA DGX Cloud and running on Amazon Elastic Compute Cloud (Amazon EC2). The NVIDIA GH200 Grace Hopper Superchip-powered instances in AWS will feature 4.5 TB of HBM3e memory, a 7.2x increase compared to current-generation NVIDIA H100-powered EC2 P5 instances. This allows developers to run larger models, while improving training performance. This is serious levels of compute for training large models! 1. The NVIDIA GH200 NVL32 is a rack-scale reference design for NVIDIA GH200 Grace Hopper Superchips connected through NVLink targeted for hyperscale data centres. 2. The CPU to GPU memory interconnect is 900 GB/s, which is 7x faster than PCIe Gen 5. GPUs access CPU memory in a cache-coherent way, extending the total memory available for applications. 3. NVIDIA GH200 NVL32 is ideal for LLM training and inference, recommender systems, graph neural networks (GNNs), vector databases, and retrieval-augmented generation (RAG) models. #AWS #NVIDIA #CloudComputing #AI #EmergingTech

In the latest round of MLPerf inference benchmark scores, chip and server makers and cloud providers showed off performance of top-of-the-range hardware for the newly introduced 6B-parameter, GPT-J, large language model (LLM) inference benchmark, which is designed to indicate system performance on LLMs like Chat-GPT. #chatgpt #llms #nvidia #mlperf #eetimes #ai #chatgpt

The GenAI projects are off to the races ! But the underlying GPU resources still remain expensive and in short supply. How can you better manage your GPU cluster to optimize these precious resources and get maximum utilization of GPU, CPU, and memory resources? How can I re-prioritize AI/ML jobs and transparently migrate jobs between on-prem and cloud resources via checkpointing GPU’s? Message me if you would like an overview of our award winning platform, Memory Machine for AI ! #GenAI #AI/ML #AI #GPU #MemVerge