"🏗️ Hardware Memory bandwidth issue

Attachments

Rohan Paul on X: "🏗️ Hardware Memory bandwidth is becoming the choke point slowing down GenAI. During 2018–2022, transformer model size grew ~410× every 2 years, while memory per accelerator grew only about 2× every 2 years. And that mismatch shoves us into a “Memory-Wall” The "memory wall" is https://t.co/OUbGI5THg4"