Whenever you see a sweeping statement that a tremendous amount can come from a very small number of assumptions, you always find that it is false. There are usually a large number of implied assumptions that are far from obvious if you think about them sufficiently carefully.
volume II; lecture 26, "Lorentz Transformations of the Fields"; section 26-1, "The four-potential of a moving charge"; p. 26-2 - The Feynman Lectures on Physics (1964)