Our self illusion is so interwoven with personal memories that when we recall an event, we believe we are retrieving a reliable episode from our history like opening a photograph album and examining a snapshot in time. If we then discover the episode never really happened, then our whole self is called into question. But that's only because we are so committed to the illusion that our self is a reliable story in the first place.

The Self Illusion: How the Social Brain Creates Identity (ed. Oxford University Press, 2012) - ISBN: 9780199897599