Framework for Concept Introduction: A general framework can be adapted from the Zone of Proximal Development or ZPD, which is an instructional concept (due initially to Lev Vygotsky) that continues to evolve, especially in its treatment within the world of education and relation to scaffolding.
The Wikipedia definition provided (quoting Vygotsky) is:
the distance between the actual developmental level as determined by independent problem solving and the level of potential development as determined through problem solving under adult guidance, or in collaboration with more capable peers
So the two constraints of (1) whether or not the concept falls within a student's ZPD, and (2) how much time can be spent on working with the student, are probably enough to give a very basic framework for introducing a concept. In particular, if either of these constraints is not satisfied, then the concept should be avoided for the time being.
This approach is admittedly simplistic, since there are lots of things that could be covered in a course, i.e., that are potentially within students' ken given enough time, but at the expense of other material. To develop such a framework further, then, one must at least have some idea of the course goals.
Frogs, Birds, and Chunking: The "frogs and birds" metaphor refers to a notion introduced by Freeman Dyson in a talk where he begins:
Some mathematicians are birds, others
are frogs. Birds fly high in the air and
survey broad vistas of mathematics out
to the far horizon. They delight in concepts that unify our thinking and bring
together diverse problems from different parts of
the landscape. Frogs live in the mud below and see
only the flowers that grow nearby. They delight
in the details of particular objects, and they solve
problems one at a time. I happen to be a frog, but
many of my best friends are birds. The main theme
of my talk tonight is this. Mathematics needs both
birds and frogs. Mathematics is rich and beautiful
because birds give it broad visions and frogs give it
intricate details. Mathematics is both great art and
important science, because it combines generality
of concepts with depth of structures. It is stupid
to claim that birds are better than frogs because
they see farther, or that frogs are better than birds
because they see deeper. The world of mathematics is both broad and deep, and we need birds and
frogs working together to explore it (p. 212).
Dyson is discussing mathematicians, but you may wish to consider whether or not the discussion can be applied to students of mathematics, as well. Something like: Can students in mathematics classes be meaningfully divided into tadpoles and hatchlings?
My sense is that the answer, generally, is no: There might exist certain preferences, but early on students are not to be classified in this way. Instead, you must elect to provide a mix of "concepts that unify our thinking" and "details of particular objects." To share my own perception (and perhaps out myself as somewhat ranine), I think one must alternate between minor details, as objects are introduced, and broader comments as students' arsenal of examples grows.
With regard to "chunking," I refer to studies on chessboard perceptions; an early one, comparing the ability to reconstruct positions among players at different levels, can be found here. In this study and others that follow, it seems that experts are able to "chunk" concepts in a different way than novices. This can be helpful in situations that rely on one's memory capacity, but is also helpful more generally when clumping together broad ideas that can then be deconstructed.
To clarify: I bring up these three items (frogs, birds, chunking) to suggest that students be taught about new objects and some of the related details (frogs), and that more general conceptual explanations (birds) be introduced to unify the objects in a way that allows for better chunking. This allows for mastery to be attained in a domain (here, mathematics).
Loose Ends: Your question is actually four different ones.
Is there any good framework for deciding when it is a good idea to "skip the concept" and "just teach the method," knowing that students will appreciate the concept only much later in life, or possibly never?
Do you really show children why long division works?
And if you do, do you intend them to retain that concept?
Why is this skipping-of-the-concept sometimes good and sometimes bad?
I hope that I have provided helpful thoughts about 1, though I make no remarks here about utility; this is a needed consideration, but difficult to pin down when you do not know students' future paths. The answers to 2 and 3 are yes, assuming that the students have sufficient procedural fluency for this concept to be within their ZPD.
As for 4, skipping a concept can be bad because (as indicated above) it robs students of the opportunity to gain the bird's-eye-view that leads to chunking and, eventually, mastery/expertise. Skipping a concept can be good because, in most cases, an individual item covered can be generalized far beyond what could reasonably be expected in any single course. (Consider an example like Bloch's principle.)
As a final note, I think a good example of a concept that should be initially skipped is why you can write any rational number in lowest terms. It is true that, for a given rational number, you will be able to do this. However, to assert this for all rational numbers, one will need approximately the strength of the Principle of Mathematical Induction (PMI) or the Well-Ordering Principle (WOP). When teaching students (whether in elementary school or a community college developmental mathematics course) about rational numbers and discussing the process of simplification, I would not start talking about PMI or WOP.