I think it absolutely necessary that the President should have the power of removing his subordinates from office; it will make him, in a peculiar manner, responsible for their conduct, and subject him to impeachment himself, if he suffers them to perpetrate with impunity high crimes or misdemeanors against the United States, or neglects to superintend their conduct, so as to check their excesses.

The Writings of James Madison: 1787-1790 (ed. 1904)