Realising that we typically give a misleadingly confident (& wrong) "this is how it happens" about not only quantum speed-up but quantum computing generally was one of my motivations for writing Beyond Weird. Scott's article here from 2021 is one of the best explanations of why.