Not a complete answer (could there be one?), but too long for a comment.
why is it [Set Theory] not being taught at the very outset of math education?
It has been tried, most likely still is in certain places at different degrees. For some history and background, lookup the New Math of the '60s, possible keywords Belgium, Willy Servais, Georges Papy. Unfortunately, many of the relevant papers are behind paywalls now, but quoting from one which is not (Geert Vanpaemel - Belgian contributions to the New Math movement in Europe ...):
Servais emphasized the concept of a pédagogie ouverte, an open approach to the learning
process. [...] He wrote with enthusiasm about the choice of set theory as the foundation of mathematical
education, and in particular lauded Papy’s use of Venn diagrams and arrow-graphs, which were, in
his opinion, ideally suited to the mind of young children. He also endorsed their attempts, criticized
by some, to bring the New Math to primary schools. [...] Servais was convinced that the basic unity, acquired by the use of set theory,
provided the solution to the pedagogy that he had in mind: as the goal of mathematics teaching was
to activate the mind of the child towards grasping the mathematical structures in the world around
him, it was necessary to define these structures and to make them the backbone of the whole