I've been tempted to try to use spaced repetition in the past, as there is certainly some kind of linguistic component to learning mathematics. This always meets the paradox that many mathematicians work nonverbally, as pointed out in Hadamard's study of the mental habits of mathematicians. Mathematics is accumulated less by mnemotechnics than by a slow and difficult struggle to use all of one's mental capacity to produce reasonably faithful mental models, as observed many places by William Thurston, e.g. as expressed in this quote:
"Mathematics is amazingly compressible: you may struggle a long time, step by
step, to work through some process or idea from several approaches. But once you
really understand it and have the mental perspective to see it as a whole, there
is often a tremendous mental compression. You can file it away, recall it quickly
and completely when you need it, and use it as just one step in some other mental
process. The insight that goes with this compression is one of the real joys of
mathematics"
What worries me about trying spaced repetition/mnemotechnics is that these try to get the compression/memory without the long struggle to form the mental structure. Such a process is precisely the opposite of trying to store things cleanly in order to remember them quickly and accurately. Thurston's quote suggests that memory of a topic is won by engaging very painstakingly for a long time with an extraordinary mess.
This sort of thing is even reported in interviews with more algebraic thinkers like Alain Connes:
" I always proceed in
the following way : whatever the complexity of the problem, instead of trying it first on a
piece of paper, with a pencil, I just go out for a walk, and try to have all the ingredients
present in my mind, in order to start manipulating them mentally. Only after this exercise,
am I able to see clearly, think about the various steps and begin to get a mental picture.
This is a painful process which consists in gathering in your mind, in your memory, all
the elements of the problem, in order to begin manipulating them. It is an exercise that I
recommend - well, of course, different people function differently- if one wants to be able not
to depend upon paper and pencil. Because with paper and pencil, you get tempted to start
writing immediately, and if you haven’t thought long enough before, you will get nowhere.
You will get discouraged before having had enough time to create in the linguistic part of
the brain specific mental pictures that you can then manipulate, as usual, by zipping them,
transforming them into something smaller, and then moving them around."
This compression is again won by struggle. I'd be wary of "learning" mathematics via ANY sort of mnemotechnics. It's probably best to struggle with things and work for their compression without looking for a shortcut! Of course, this is just my opinion.
Perhaps there is one honest mnemotechnic mathematicians use: the simplest nontrivial example of a given phenomenon. Such an example is analogous to a model of a finite geometry, in that any such example encodes the whole theory and the example can be generated from a relatively small seed that is more compatible with our mental modules, perhaps computationally or visually. One gets the impression that such an example is more concrete since its interrelations are less disembodied, or could be generated from the simpler rules used to define the example.
Edit: The following may be helpful regarding the use of spaced repetition. It might be helpful to internalize "atomic facts" with which to work (think arithmetic facts) but is not likely to be helpful for working with these facts in creative ways, (think fancy mental arithmetic a la Art Benjamin). This said, the following excellent quote of Paul Halmos about the mathematician's work points to a potentially useful application of spaced repetition:
For the professional pure mathematician, mathematics is the logical dovetailing of a carefully selected sparse set of assumptions with their surprising conclusions via a conceptually elegant proof.
From this standpoint, it may be useful to employ spaced repetition to quickly internalize the "sparse set of assumptions" in order to begin thinking about them. This is interesting in light of Alain Connes's quote above, as well.