In which areas of mathematics is it traditional to use script letters, such as $$\mathcal{ABCDEFG}$$, and is there a pedagogical advantage to doing so?

• The Laplace transform traditionally uses a script $\mathscr{L}$. – Joel Reyes Noche Sep 20 '14 at 13:33
• Also, the letters $\cal{ABCDEFG}$ are calligraphic (the LaTeX command is \cal). The letters $\scr{ABCDEDG}$ are script (the LaTeX command is \scr). – Joel Reyes Noche Sep 20 '14 at 13:35
• I'm using the term script informally, as in Using script fonts in LaTeX, but thanks for pointing it out. – J W Sep 20 '14 at 13:47

When using categories it is common (though not universal) to denote the categories by script letters; see http://www.staff.science.uu.nl/~ooste110/syllabi/catsmoeder.pdf and http://wwwhome.ewi.utwente.nl/~fokkinga/mmf92b.pdf (page 8) for two examples. [The examples are not chosen very specifically, they just came up early on in a search for examples.]

Another example where a script letter is common is for denoting the powerset of a set $S$, as $\mathcal{P}(S)$.

Regarding the pedagogical advantage, I would say that it can be useful if one has several 'layers' or 'types of things' to distinguish them by different fonts. For example, it is common to denote sets by upper-case and their elements by lower-case letters.

If there is a third layer it can make sense to ressort to a third font, and thus use script letters. Like, categories $\mathcal{C}, \mathcal{D}$ have objects $C_1, C_2$ and $D_1, D_2$, respectively; and let $f$ and $g$ denote a morphism of $C_1, C_2$ and $D_1, D_2$, respectively.

The question now splits a bit as there are at least two separate questions:

1. Does it make sense to use several fonts in this way?

2. Is script a good choice for a font?

For 1. I would say that it can help to structure things and to have a permanent reminder what is what. It is (with different fonts) also very wide spread, for example, Greek letters for angles. Yet, one should also not overdo this. There is a risk that students become so fixated on some specific notation that they are completely off track when then confronted with a different set of notational conventions.

For 2. I would also say, yes, in some cases. What is good about the font is that it is not too different but still distinguishable. By contrast, Fraktur (as somewhat frequently used for ideals or the symmetric group and related things) can be difficult to read and write. For instance, I am under the impression that for not few students the notations $\mathfrak{S}_n$ and especially $\mathfrak{A}_n$ for the symmetric and alternating groups are not at all intuitive, since they do not really "see" an "S" and an "A" in a different font but just some unknown symbols.

What is not so good is that it can be a bit 'heavy' or 'voluminous' (for lack of better words). So, it seems better to use it for things that do not occur that frequently and/or are rather 'high' in the hierarchy, like, element $a$, in set $A$, in a collection of sets $\mathcal{A}$.

I want to expand a little on quid's answer to call attention to a couple of specific fields of math where script fonts definitely helped me pick things up.

• One big one is topology. The go-to notation for that is $$(X,\mathscr T)$$, where $$X$$ is the base set and $$\mathscr T$$ is the collection of sets that are open in $$X$$. Here, script letters indicate collections of sets, captal letters indicate sets, and lowercase letters indicate points.

• Another is axiomatic linear algebra. Here, $$v$$ might indicate a vector, $$\lambda$$ a scalar, $$L$$ a linear mapping (or a matrix, which is the same thing), and $$\mathcal V$$ a linear space.

• And, of course, high school geometry uses lowercase letters for distances, capital letters for points, Greek letters for angles, and script lowercase letters like $$\ell$$ for lines.

The big win of using different typefaces for different types from both a practical and pedagogical perspective is that you can see a variable and know off the bat what kind of "thing" it is. So if I'm in linear algebra and someone asks me if $$\mathcal U\cap\mathcal V$$ is finite-dimensional, I can leap to "Ah, those are linear spaces, so finite-dimensionality is a natural thing to wonder about." Mathematical fields can often get by with just partitioning the alphabet (like in calculus where a, b, and c are fixed parameters, f, g, and h are functions, i, j, and k are unit vectors, x, y, z are real numbers and so on), but there is an elegance to being able to say "Choose $$V\in\mathcal V$$ such that $$v\in V$$" that makes the typefaces worth mastering mathematical handwriting.

Now that schoolchildren in the US no longer learn script handwriting, I conjecture that the usage of script letters in mathematics will decline. Eventually, the usage may converge to a level similar to the usage of Fraktur letters in mathematics.

SO, to answer the question: If you are teaching young people (who have rarely or never seen script letters before), then DO NOT use script letters in your teaching.

Anecdote. I was working with some 5th graders. I was careful to print everything I wrote (no cursive). But my small letter d turned out to have a loop in its stem. So they were laughting about that, trying to write d d d in my style, and not paying attention to what I was saying.