If you want to build a recursively self-improving AI, have it go through a billion sequential self-modifications, become vastly smarter than you, and not die, you've got to work to a pretty precise standard.

Question 12 in Less Wrong Q&A with Eliezer Yudkowsky (January 2010)