Dwight Diercks

The world's most advanced processors, including Intel(Intel 7), Apple(M1, M2, M3, M4 etc chips) and Nvidia(Blackwell) were researched and developed in Israel. The list goes on. Apple's 2nd largest R&D, Nvidia's 2nd largest R&D, Intel's 2nd largest R&D all in Israel. If there was no Israel today and the people inside, the advancement of software and hardware as it is today(GPUs, AI, CPUs, humanoid robots etc) would cease to exist. Not sure what it's going to take for the world to recognize Israel as a force for good, an ally, and get behind backing Israel and Israelis. Let these people get back to work, their government defend themselves and there will be a brighter future for all of us. Enough is enough.

13

thinking about the r10k - mipspro was amazing. just look at the verbose output from the software pipelining pass. i think there's pldi paper on a showdown between this algorithm and some ILP-based scheme. i should read again. .BB16.naive_matmuli: # 0xb6c  #<loop> Loop body line 263, nesting depth: 3, estimated iterations: 256  #<loop> Not unrolled: estimated schedule improvement < 10%  #<loop> Not unrolled: disable analysis w/-CG:unroll_analysis=off  #<swps>   #<swps> Pipelined loop line 263 steady state  #<swps>   #<swps>  512 iterations before pipelining  #<swps>    Not unrolled before pipelining  #<swps>  18 cycles per iteration  #<swps>   3 mem refs   ( 16% of peak)  #<swps>   7 integer ops ( 19% of peak)  #<swps>  10 instructions ( 13% of peak)  #<swps>   1 short trip threshold  #<swps>   8 integer registers used.  #<swps>   #<swps>   6 min cycles required for resources  #<swps>  17 cycles required for recurrence(s)  #<swps>   6 operations in largest recurrence  #<swps>  18 cycles in minimum schedule found  #<swps>  #<swp>   #<swp> Pipelined loop line 263 replication 0  #<swp>   #<freq>  #<freq> BB:16 frequency = 56481544.00000 (heuristic)  #<freq> BB:16 => BB:15 probability = 0.99610  #<freq> BB:16 => BB:18 probability = 0.00390

3

1 Comment

Are you wondering if you can get more performance out of our cuBLAS library for matrix multiplies applications like LLMs? Check out this great new article by Babak Hejazi for understanding matrix multiply performance on our most advanced GPUs and new features and APIs like grouped GEMMs that are important for Mixture-of-Experts (MoE) models. #cuBLAS #GPUs #MatrixMultiplication #PerformanceOptimization

144

2 Comments

AI workflows are driving demand for memory and storage to prepare data, train the model and create useful inference engines.  This is driving announcements of storage and memory products that support the various elements of these AI workflows.

51

4 Comments

Can you really speed up a CPU by 100x with some added "smarts"? The concept is intriguing - perfectly scheduling instructions and hiding memory latency. VLIW aimed to achieve this through efficient instruction packaging and multiple threads, like @Cray XMT's massively multi-threaded architectures. As someone who has worked on processor design, I understand the challenge of making this work practically with existing programs. That's why I'm somewhat skeptical of claims made by Finnish startup Flow Computing. However, I'm open to being convinced otherwise. Read more about Flow Computing's parallel processing accelerator and share your thoughts in the comments: https://lnkd.in/gRBYbBvS #computerarchitecture

4

2 Comments

Unlocking the full potential of trillion parameter #LLMs needs an integrated hardware and software platform. Learn how NVIDIA Grace Blackwell, TensorRT-LLM, and Triton accelerate trillion-parameter LLMs by up to 30x compared to previous generations. #DataCenter

72

4 Comments

Thomas Coughlin has outlined the impact of AI advancements on storage needs in his current article published by Forbes. He notes that as AI applications grow, there's an increasing demand for memory and storage solutions to manage both the extensive data used in AI training and the results utilized in inference engines. This demand benefits a variety of storage technologies including DRAM, NAND flash, HDDs, even magnetic tape (#lto), and likely, emerging non-volatile memories, particularly in endpoint AI inference applications. Quantum Corporation's #ActiveScaleColdStore provides a comprehensive storage solution tailored for AI workloads. This system integrates LTO technology with SSD, HDD into a S3 Object storage system using an innovative #tape optimized Two-Dimensional-Erasure-Code technology #2DEC , ensuring efficient handling and scalability of AI-driven data demands for low cost, high performance green computing #quantum #lto #activescalecoldstore

8

1 Comment

NIM enable enterprises to generate text 3x faster, delivers superior throughput and achieves near-100% utilization with multiple concurrent requests. NVIDIA has partnered with Hugging Face to make deploying generative AI models easier and faster. Here’s how: Optimized Models: NVIDIA NIM boosts performance, speeds up token generation, reduces costs, and enhances user experience. NVIDIA NIM: - Easy deployment across cloud, data centers, and workstations. - Utilizes TensorRT-LLM for low-latency, high-throughput AI inference. - Supports a wide range of LLMs and applications. Superior Throughput: Generate tokens up to 5x faster, translating to higher revenue for enterprises. Easy Deployment on Hugging Face: - Deploy models in minutes with NVIDIA NIM. - Start with Llama 3 8B and 70B on your preferred cloud provider. Simple Steps: - Go to the Llama 3 model page on Hugging Face. - Select ‘NVIDIA NIM Endpoints’. - Choose your cloud instance type. - Select ‘NVIDIA NIM’ and create the endpoint. - Begin making API calls in minutes! Get Started: Accelerate time to market, increase revenue with high token throughput, and lower inference costs with Llama 3 NIMs from Hugging Face.

45

1 Comment

Better look at Llama 3 on #IntelArc A770, this time without my typing slowing down the video. Other than cutting-out my slow typing⌨, it's unedited. 😊 Followed this tutorial and swapped out the model ID with "meta-llama/Meta-Llama-3-8B-Instruct". https://lnkd.in/gAT3khEn

39

OK... Intel Corporation's Pat Gelsinger was on stage today at Computex to reveal more on Xeon 6 and Gaudi. Quick takeaways: Xeon 6: - x86 CPU market is more competitive than ever - The days of one company having a massive advantage in terms of "billboard specs" are over - now it's about workload performance -Given the parity between Intel and AMD, I think Intel's acceleration engines and advanced instructions (and support in the ISV ecosystem) is going to help Xeon 6 - Happy to see server and datacenter consolidation ratios discussed - it reminds us there is a world outside of AI. - However, AMD isn't sitting still - Both companies better keep an eye on Arm Gaudi: - I like the more orthogonal approach Intel is taking toward NVIDIA with its focus on RAG and enterprise AI - it's the right approach - In line with the, Gaudi 2 and Gaudi 3 pricing makes this a much stronger (enterprise) play - All lined up behind a part that has a very competitive performance position - Intel Developer Cloud (IDC) is going to be invaluable for enterprise IT - and a gold mine for Intel Look for coverage soon!

50

1 Comment

Large Language Models (LLMs) are absolute powerhouses when it comes to generating top-notch #GenAI responses. Yet, #LLMs lack that in-depth, domain-specific knowledge that can make all the difference in certain use cases. Additionally, let's not forget the hefty computing resources and expertise required to develop and deploy them. In my latest article, I delve into why Small Language Models (SLMs) are becoming the go-to choice for specific tasks. With their more focused approach, #SLMs are more efficient than LLMs, and have a smaller compute footprint. Thus, they are more suitable to run locally on workstations or on-prem servers. Check out my article 👇 and ‘d love to hear your thoughts about it https://lnkd.in/diRYaNXF #DellTechEMEA #iWork4Dell #AMD, #Intel, #NVIDIA 

143

7 Comments

At Computex 2024, NVIDIA announced a new graphic processor architecture Vera Rubin. The architecture will use a new HBM4 memory. The Rubin graphic processor will have 8 stacks of HBM4 memory, with later versions announced featuring 12 HBM4 stacks. In addition to that, NVIDIA announced a new Vera Rubin board with Vera processor (VR200) that will unite a GPU and a CPU into a superchip. Although most information is yet to be announced, the platform will support new NVLink 6 3600GBps connections and CX9 SuperNIC up to 1600GBps connections. Besides that, NVIDIA announced Blackwell Ultra GB200 graphic processor with 12 HBM3E memory support, where memory supported will be increased from 192GB to 244GB. The release of Rubin is planned for early 2026. #NVIDIA #HBM4 #VPS #server #Blackwell

1

The technology world seems to be more engaged and prepared for new AI model releases than just a few months ago. With Meta releasing the Llama 3 model just today, I've seen performance results and readiness for platforms from Intel Corporation and Qualcomm. AMD highlighted its partnership, but hasn't showed specifics as yet. Intel has the most robust testing so far, showing results on Gaudi 2, Xeon, Core Ultra integrated graphics and Arc A770 discrete GPU: https://lnkd.in/gjB85SgQ Snapdragon says that tools "will be available" to run Llama 3 on its platforms, just not posted on their AI Hub yet: https://lnkd.in/gEqCfbqn

46

There's often confusion about NVIDIA Triton's capabilities with real-time streaming. The answer is, it fully supports them. Just published a new guide on Inferless blog which explains how by integrating SSE with NVIDIA Triton Inference Service you can build real-time AI streaming apps! 🚀 Full guide, demo, and code below. Guide: https://lnkd.in/d5XvdDZk Demo: https://lnkd.in/dD5NAbsC Code:https://lnkd.in/dAEZg63g

26

https://lnkd.in/gSekNjEX One request I always get from academic researchers is the opportunity to understand and research on real-world product traces. The blog post from Phitchaya Mangpo Phothilimthana and Saurabh Kadekodi has some nice information on synthesized storage IO traces that we released today at Google. (The methodology is also quite cool, if you haven't seen it before, I would look at the Google ASPLOS'24 paper on Thesios.)

221

1 Comment

Eight years ago, during an internal event at NVIDIA's headquarters in Santa Clara, I gathered the courage to ask our CEO a bold question: "Where do you see NVIDIA in 10 years?" The room went silent for a chilling moment before Jensen's determined response: "We will become the number 1 IT company in the world." Fast forward to today, with two years left to fulfill that vision, look at what we've achieved in eight years! Our journey thus far is a testament to our commitment, innovation, and relentless pursuit of excellence. And now, the news that NVIDIA has become the most valuable company in the world, as reported by The New York Times, stands as a resounding confirmation of Jensen's vision. This remarkable milestone showcases our unwavering dedication and the extraordinary impact of our work in the technology sector. Together, we've proven that bold aspirations, coupled with relentless effort, can indeed transform into groundbreaking achievements. Let's continue to push the boundaries of what's possible and shape the future of technology. https://lnkd.in/d947mewQ

21

Intel's CEO, Pat Gelsinger, just announced some exciting updates during his keynote at Intel Vision 2024. The latest Gaudi 3 GPU claims to be 1.3X faster than NVIDIA's H100 and comes in multiple form factors. In addition, Intel unveiled new CPU chip families including Lunar Lake NPU, Granite Rapids i.e. Gen 6 Xeon, Saphire Rapids is Gen 4 Xeon, and Emerald Rapids is Gen 5. Intel is also looking to commoditize AI with its AI PC based on Intel Core Ultra, shipped in over 5 million AI PCs. The company is focusing on Edge with its 3 laws of Edge: Economics, Physics, and Land (cheaper, faster, smaller). By 2026, 50% of AI workloads are predicted to be at the Edge. Intel's commitment to open standards was also highlighted, along with microscaling formats that reduce the memory footprint of AI/ML models without sacrificing accuracy. Lastly, Intel unveiled UltraEthernet, a networking standard to combat proprietary protocols. Exciting developments from Intel! What do you think about their latest innovations? #IntelVision2024 #AI #EdgeComputing #OpenStandards

1

Growth of Arm in the datacenter starts with availability, then adoption. With some hyperscalers rolling their own designs, everyone else has to look to the open market to see what's for sale. The only real option here is from Ampere, which recently announced a custom 3nm Arm 256-core CPU as part of the AmpereOne family. More cores, more memory channels, and a custom microarchitecture they might disclose later this year. It's a chiplet design too, showcasing where this market is going. https://lnkd.in/e6JMXCum

95

7 Comments