Ashish Sabharwal, a computer scientist at the Allen Institute for AI, recently helped show that “chain-of-thought” reasoning in neural networks comes at a hefty computational cost. https://lnkd.in/edQVKiQE
Congratulations Ashish!! Whiteboard with complex math is on brand! The Quanta Magazine article is vague on details 1) some Lin Al problems are “thought” to be in complexity class outside the league of Transformers…. Which ones? 2) Chain of thought helped solve problems beyond regular Transformers. which problems? 3) Dr Chiang says we must be aware that Transformers are incapable of doing some tasks. Again example please? 4) What kind of computational resource is the bottle neck for chain of thought… memory, IO, gates, etc? I am sure these questions are answered in the academic paper. I hope they can be summarized and be understood by general engineering community. Thank you
Pawan Mishra
3mo
Well done, Ashish! I still remember you had helped me solve an almost unsolvable linear algebra problem at IIT Kanpur :)
Saikat Sen
3mo
Good to see you in the news, Ashish Sabharwal!
Vasilije Simic
3mo
Incredible!
So good to see linear algebra on the whiteboard. #mathlove