Not to get into an epistemological debate about it, but that he's been pretty consistently himself back through Eraserhead and his earlier short works, that (as Hulk above mentioned) he maintains a sort of consistent through-line that shows even in the more 'conventional' projects, and that the people who've worked with him in any capacity corroborate his essential weirdness, provides a pretty good basis.
That you're couching it in terms of "it seems" doesn't change that this is exactly what you've been doing.However, when you start getting into a Director's actual mind and actual intent, you begin to go off the reservation.