https://worksonmygpu.com Notes on GPU computing, AI inference, and ML research — from low-level performance optimization to new developments in AI. en https://worksonmygpu.com/posts/fp8-wgmma-register-shuffle https://worksonmygpu.com/posts/fp8-wgmma-register-shuffle Sun, 09 Aug 2026 00:00:00 GMT Beyzanur Zeybek FP16 lets you feed one WGMMA's output straight into the next — the layouts already match. FP8 doesn't get that for free: the accumulator and the next operand disagree on which thread owns which value, and fixing that costs real shuffle instructions. https://worksonmygpu.com/posts/wgmma-register-fragments https://worksonmygpu.com/posts/wgmma-register-fragments Sat, 08 Aug 2026 00:00:00 GMT Beyzanur Zeybek WGMMA showed up unexplained in two earlier posts — as a row in a diagram, then as a building block of FlashAttention-3. Here's what it actually is: an async, 128-thread-wide matrix multiply, and the register layout that makes it usable. https://worksonmygpu.com/posts/flashattention-versions https://worksonmygpu.com/posts/flashattention-versions Fri, 07 Aug 2026 00:00:00 GMT Beyzanur Zeybek Three papers, three years, three different bottlenecks. FlashAttention wasn't 'made faster' three times in a row — each version targeted something the previous one left on the table. https://worksonmygpu.com/posts/cute-layouts https://worksonmygpu.com/posts/cute-layouts Thu, 06 Aug 2026 00:00:00 GMT Beyzanur Zeybek Every CuTe layout is just a pair — a Shape and a Stride. That one idea, applied recursively, is what lets CUTLASS 3.x describe a whole GEMM's tiling — from the full matrix down to a single thread's registers — with one abstraction instead of a class per level. https://worksonmygpu.com/posts/what-is-cutlass https://worksonmygpu.com/posts/what-is-cutlass Wed, 05 Aug 2026 00:00:00 GMT Beyzanur Zeybek cuBLAS gives you a fast, opaque matrix multiply. CUTLASS gives you the same speed as composable C++ building blocks, so you can fuse, customize, and specialize — matched to the GPU's own execution and memory hierarchy. https://worksonmygpu.com/posts/gpu-architecture-history https://worksonmygpu.com/posts/gpu-architecture-history Mon, 03 Aug 2026 00:00:00 GMT Beyzanur Zeybek Tensor Cores in Volta, sparsity in Ampere, a Transformer Engine in Hopper, FP4 in Blackwell — the architectural changes that actually mattered, one generation at a time.