Problems with visual design can turn users off so quickly that they never discover all the smart choices you made with navigation or interaction design.
The Elements of User Experience: User-centered Design for the Web (ed. Peachpit Press, 2003) - ISBN: 9780735712027