On the contrary, it's because somebody knows something about it that we can't talk about physics. It's the things that nobody knows anything about that we can discuss. We can talk about the weather; we can talk about social problems; we can talk about psychology; we can talk about international finance — gold transfers we can't talk about, because those are understood — so it's the subject that nobody knows anything about that we can all talk about!
Rejoinder when told that he couldn't talk about physics, because "nobody [at this table] knows anything about it." - Surely You're Joking, Mr. Feynman! (1985)