Students have the tendency to just try to learn the examples and skip the concepts. Is there a name for this phenomenon? Has it been studied? Does it have natural origins, or is it taught in prior math classes?
I'm not familiar with the literature, but I do know that "math people" tend to severely overestimate the degree to which the rest of the population reasons from first principles. Most people exclusively reason by analogy:
Is $f(x)$ continuous? Well, based on the examples I've seen, continuous functions look "normal" and discontinuous functions look "weird." In this case $f(x)$ looks more normal than weird, so $f(x)$ is probably continuous.
Side Note: Students who exclusively reason by analogy generally don't see it as problematic that they don't know how to proceed in the absence of similar examples. When that happens, they view it as a "data" problem, not a "model" problem, so they "fix" the problem by obtaining a similar example from someone/something that seems trustworthy and adding it to their dataset of examples for future decisions.
This isn't just in math. It's across life in general. Consider stereotypes:
Is X true about thing Z? Well, based on the examples I've seen, thing Z seems similar to the type of things for which X is true. So X is probably true about thing Z.
To me, this would suggest that overreliance on analogical reasoning has natural origins (even if the specific decision rule for a particular case of analogical reasoning might be learned from other people).
Response to OP's Addendum:
. . . they [students] create "rules" for each appearance of functions in each solved example, not realizing that regardless of the appearance of the function in all these examples, we are just checking the equality of the one-sided limits . . . is this the nature of human learning, or is it a learning technique that is encouraged by the way we test them (e.g., solve an example involving a piecewise-defined function, then test them by giving another piecewise-defined function)?
I've had countless similar experiences working with students who were flat-out resistant to first-principles reasoning. You go through a zoo of different-looking examples, emphasizing that while there are different "rules" for each example, the rules all stem from the same "first principle" -- but some students just ignore the first principle and instead prefer to memorize a special rule for each special case.
Why does this happen? Isn't it easier to just remember the first principle and apply it as needed? It's less to remember, and it's guaranteed to cover the complete space of problems. The resistance to first-principles reasoning is especially weird because memorizing special cases takes more effort, and at the same time, the students who do this are typically the ones who are trying to minimize the amount of effort that they put forth to get a desired grade in their class.
I think what's happening is that these students are reaching an "abstraction ceiling" beyond which they find it exceedingly difficult to engage in first-principles reasoning -- so difficult, that it's actually less effort to memorize a bunch of special cases.
On the surface, the idea of an "abstraction ceiling" might seem weird. Doesn't a ceiling imply that there's a single topic marking the dividing line between the abstractions that a student is or isn't able to grasp? Is there really a level at which one is suddenly incapable of learning math? Yeah, I don't think that makes sense either. But that's not what I mean by the abstraction ceiling. What I mean is a "soft" threshold at which the amount of time and effort required to learn math begins to skyrocket due to friction in the learning experience.
There are numerous sources of friction that come into play as you move up the levels of math:
Maybe the new information isn't being broken down and scaffolded enough, and it's overwhelming your working memory capacity.
Maybe you're missing prerequisite knowledge.
Maybe you've covered the prerequisites, but you didn't get enough practice to hammer in sufficient mastery.
Maybe you mastered the prerequisites at one point, but you didn't get enough review to retain that information, and consequently, you forgot it.
Maybe you've lost motivation / interest due to feeling like the math is no longer relevant to your future.
Maybe you've lost motivation due to being in a class where everyone else is just as good at math as you, and math is no longer the thing that makes you "special."
And not only do these sources of friction increase as you climb higher in math, but they also compound. The more prerequisite knowledge you're missing, the harder it is to learn new material, and bigger your knowledge gaps grow. The more motivation you lose due to no longer being the best among your classmates, the worse you get relative to your peers, and the more motivation you lose.
As you climb the levels of math, these sources of friction conspire against you and eventually throw you off the train. And one of the first warning signs is when you stop understanding things at the core, and instead try to memorize special cases cookbook-style.
The unfortunate part is that, while some sources of friction can be mitigated through effective pedagogy, individual differences can stack the deck against you. For instance, it's known that working memory capacity varies between individuals. Relatedly, it's also known that people differ in their ability to generalize patterns from data, and even in their rate of forgetting. We all have cognitive differences, just like we have physical differences. It's just that the physical differences are easier to see.
We're getting into a really touchy subject, and this may be an uncomfortable (maybe even unpopular) take. But surely there's no issue accepting that, even with years of practice, not everyone can really understand Topological Quantum Field Theory at its core, right?
Say there's some math genius who takes some number of hours to master Topological Quantum Field Theory starting from high school calculus. Say a best-in-the-university math student takes twice as long, an exceptionally talented math student takes twice as long as that, a very talented (but not exceptionally so) math student takes twice as long as that, and so on. Where does it end? Keep on going and you eventually hit an infeasibly large time commitment, and keep on going after that and you eventually hit a time commitment so large that it exceeds the sum total waking hours in the average human lifespan.
And if you accept this for Topological Quantum Field Theory, then how is any different for, say, high school calculus?
In fact, it would be weird if there were a single level of math that marked the dividing line between "everyone who's learned the prerequisites can do it" versus "reserved for geniuses only." It's likely a continuous thing that starts in fairly elementary math -- the higher the level of math, the fewer the number of people are capable of really understanding it at its core, given a reasonable time commitment. Each student eventually hits their "abstraction ceiling," the amount of time and effort it takes for them to succeed in math class begins to skyrocket, they find other subjects that they enjoy more (which may or may not use the math that they've learned so far), and they take an "off ramp" into those subjects.
Of course, this is in no way an argument for giving up on students who are struggling. You support them as much as is feasible given the context of the class (see Daniel R. Collins's answer for a great example of this). But at the end of the day, when you're reflecting on how well you've done as a teacher, you need to have realistic expectations.
If you run a summer basketball camp, you might have a couple really impressive kids who make you think "wow, maybe I'll see them play professionally someday," but you're not going to turn all the attendees into future pro ball players. Most of the kids probably aren't even going to get a basketball scholarship to college. You're definitely not going to get the 5'2" kid to dunk. And that's okay. Hopefully, you'll get each kid to have fun and become the best basketball player that they have the potential to be.
Addendums to This Answer
Addendum 1: Commenters (ryang & Passer By) have pointed out that analogical reasoning is also known as inductive reasoning and first-principles reasoning is also known as deductive reasoning. You may have better luck using those terms when searching the literature.
Addendum 2: This is not to claim that analogical reasoning should be avoided entirely. Analogical reasoning generally does a good job of supplying you with a quick decision that is probably correct (or at least more likely correct than incorrect). In most situations in life that's all you need.
As so elegantly stated in Yakk's insightful answer, even professional mathematicians "have heuristics and patterns . . . which they are confident they can reduce recursively to first principles."
A typical professional mathematician doesn't determine if a function is continuous by doing an epsilon-delta test. Given $\sin(x)+\cos(x),$ they will know it is continuous without thinking; and if asked to verify, it is the sum of two continuous functions (again, not epsilon-delta).
They can probably generate an epsilon-delta proof . . . But very few mathematicians will look at $\sin(x)+\cos(x)$ and generate a first principles proof that it is continuous and use that to answer the question.
There's a difference between using analogical reasoning and over-relying on analogical reasoning. When you use analogical reasoning, you need to have a sense of its confidence (i.e. probability of correctness) and how that measures up against the stakes of the situation you're in. If the confidence does not measure up against the stakes, then you need to fall back to first-principles reasoning (provided that you have enough time to do so). People who over-rely on analogical reasoning do not do this.
Addendum 3. Some hardline pure mathematicians might argue that the entire point of math classes is training in deductive reasoning, so inductive reasoning has no place. While that may be at least partially true from the perspective of some pure math classes, especially those really niche classes focused on the axiomatic foundations of math, this answer interprets math in its most general sense, and in most areas under the gigantic umbrella of math, inductive reasoning has some place. For instance, throughout applied math there's a saying "all models are wrong but some are useful." And in machine learning specifically (an extreme case for illustrative purposes), the whole point is to use math to build automated systems that use inductive reasoning to make decisions that are correct enough to be useful.