You also get dramatic advances when you spot that you can leave out part of the problem. Algebra, for instance (and hence the whole of computer programming), derives from the realisation that you can leave out all the messy, intractable numbers.
The Salmon of Doubt, The Universe (p. 116), Harmony Books. 2002