But, to be plain, you are dissatisfied with me about the negro. Quite likely there is a difference of opinion between you and myself upon that subject. I certainly wish that all men could be free, while I suppose you do not. Yet I have neither adopted, nor proposed any measure, which is not consistent with even your view, provided you are for the Union. I suggested compensated emancipation; to which you replied you wished not to be taxed to buy negroes. But I had not asked you to be taxed to buy negroes, except in such way, as to save you from greater taxation to save the Union exclusively by other means.

Letter to James C. Conkling (1863)