The idealistic tinge in my conception of the physical world arose out of mathematical researches on the relativity theory. In so far as I had any earlier philosophical views, they were of an entirely different complexion.
From the beginning I have been doubtful whether it was desirable for a scientist to venture so far into extra-scientific territory. The primary justification for such an expedition is that it may afford a better view of his own scientific domain.

Preface - The Nature of the Physical World (1928)