4
$\begingroup$

There is a Mathematics Stack Exchange question on a good book on inverse problems for engineers. Here, I would like to ask for suggestions on how to approach teaching undergraduate upper-division science and engineering students about inverse problems, without requiring relatively heavy mathematical prerequisites such as functional analysis. What can be assumed is some exposure to linear algebra (including least squares, but not necessarily the singular value decomposition), calculus and introductory ordinary differential equations.

(The students I teach have also had introductory courses on statistics, control systems, signal processing and image processing, but I would like to keep the question broadly applicable and not assume a very specific background.)

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ I don't think functional analysis is needed to grasp the topic itself and solve problems. Much more critical is exposure to basic statistics to include basic least squares regressions. An undergrad DOE class is good enough here. $\endgroup$ – guest Jun 18 '18 at 4:50
  • 1
    $\begingroup$ @guest I don't agree. There are a lot of flavours of inverse problems and a large chunk of them does work with basically no statistics knowledge. However, to really understand the issue of instability, the SVD of a compact operator is really helpful. Of course, there are statistical inverse problems, but note that there are others. $\endgroup$ – Dirk Jun 19 '18 at 4:45
  • $\begingroup$ stats is the useful stuff for the target audience (undergrad science/engineering) $\endgroup$ – guest Jun 19 '18 at 10:23
3
$\begingroup$

You should start by figuring out what you want students to understand and be able to do with the material that you will teach them. This includes issues of course content but you should also consider the kinds of exercises and projects that you will assign to students.

Some questions that you should consider include:

What should the balance between theory and computational practice be?

To what extent do you want to follow a statistical approach? Classical or Bayesian or both?

How much programming do you want students to do? With what languages and libraries?

What kinds of inverse problems do you want to deal with? Inverse scattering? Inverse heat conduction? Travel time and attenuation tomography?

What mathematical prerequisites can you assume?

For the course that I teach to first year graduate students in science, engineering, and math, the answers to these questions were very clear and determined the course syllabus and the content of the book that’s wrote.

In particular, we assumed that students had background in calculus, ordinary differential equations, and probability and statistics at the undergraduate level. The students needed a fairly applied course with a strong computational focus because they would mostly be doing those kinds of computations in their own research projects. We expected them to treat inverse problems as statistical parameter estimation problems and wanted them to see both classical and Bayesian approaches. Because students might not have background in partial differential equations and almost certainly had no background in functional analysis, we did not deal with inverse scattering and limited our attention to discretized inverse problems.

Improve this answer
$\endgroup$
2
$\begingroup$

First, let me qualify this answer:

  • I am not familiar with the book mentioned in the original post.
  • I have not taught any courses.
  • My engineering courses had weekly problem sets.
  • I am not familiar with very complicated inverse problems, such as X-ray crystallography, CAT scanning, seismography, or echo-based petroleum exploration.
  • The course developer will need to identify categories of inverse problem that are likely to be relevant to the students in the future, tools that are available to the students for solving these problems, and examples of typical problems and results.

I am a mechanical engineer who works as a computer programmer, so I have the students' background described in the original post. As a student, I would have found the proposed course interesting. In my professional career, I would have applied what I learned in such a course. Indeed, I have found myself independently learning parts of such a course over the course of my career.

Here are some of the things I would hope to see in such a course. This list can be adapted to a unit that is anywhere from a week long to most of a course. A week-long unit might only have some of the introductory problem categories; a course might cover several complicated problem categories.

A one- or two- lecture overview of the topic, with:

  • The overall concept of inverse problems.

    • Given "what happened", be able to figure out "how" and "why".

  • A summary of the course's learning objectives:

    • Be able to recognize categories of inverse problems that can be solved.
    • Know how to use some typical tools to solve such problems.
    • Be able to recognize categories of inverse problems that cannot be (practically) solved (if such categories exist).
    • Be able to estimate how precise an answer is reasonable, given the nature of the inputs to a problem.
    • Be able to identify what extra information would be needed to get a more precise answer.
    • Be able to check your answers.
    • Be able to explain your answers.

  • A slideshow (with simple graphics) listing inverse problems that the students are already thoroughly familiar with, and a few that they might not be familiar with:

    • Addition/Subtraction
    • Multiplication/Division
    • Exponents/Logarithms
    • Trig functions/Inverse trig functions
    • Differentiation/Integration
    • Inverting matrices
    • Solving differential equations
    • Conversion between time domain and frequency domain
    • Inferring statistical distributions from data sets
    • Photoshop's blurring / sharpening filters

  • A slideshow (also with simple graphics) showing a few kinds of inverse problems that will be covered in the course. For each kind of problem, show:

    • A field where the problem occurs.
    • What a typical problem looks like.
    • What a typical solution looks like.

  • What tools are available to the students to solve these problems during the course, and what fancier tools they might be able to obtain in their future work.

Then teach how to solve relevant categories of problems. Start with easy ones, and move on to harder ones. The first problem set or two could cover these topics:

  • Find the formula of a linear equation.
  • Estimate the formula of a linear equation given noisy data. In the process, mention that the statistical formulas are based on an objective function, and the objective function minimizes the squared error of the data from the estimated equation.
  • Use linear interpolation to find options along a Production Possibility Frontier. (This is an economics concept, but very useful in Manufacturing Engineering.) The PPF does not have to be linear. The linear approximation can be piece-wise, and just has to be "good enough". Explain that the results from inverse problems are often unstable, so they can provide ranges of answers.
  • Use the quadratic formula to find the roots of a parabola.
  • Use Newton's method (or a similar method) to find the roots of a parabola. Explicitly discuss how the objective function is set up and used.
  • Use Newton's method (or a similar method) to find the roots of an arbitrarily complicated mathematical function of one variable.
  • Simulate convolving two functions. (Many real-world situations can be described by such a process.) For example, apply a blurring algorithm to a function. Teach how to estimate parameters of two (hidden) functions whose convolution can be seen.
  • Show more-or-less inverse convolutions. For example, blurring and sharpening filters.

Later problem sets could cover multi-variable inverse problems. Some fields that use such problems include:

  • Finance: estimating sets of parameters that would allow a system to have observed behavior. For example, a rapidly growing field of business. What are plausible combinations of fixed costs, variable costs, profit margins, productivity improvement rates that would allow a business to grow at a certain rate sustainably using internally generated cash flows? If the business is growing too fast for this, or the business does not yet have that cost structure, how much external investment is required to achieve the observed growth rate? How quickly must the external investment grow? How much external investment is required before the cost structure can reach the necessary parameters? How big will the business need to be at that point? Side lessons that can be learned: The importance of break-even analysis, and some of the risks of relying on outside financing.
  • Use of the Laplace Transform and/or Fourier Transform (and their inverses) to convert between the time domain and the frequency domain.
  • X-ray crystallography
  • CAT scanning
  • seismography
  • echo-based petroleum exploration

Try to design each problem set so that the student has to:

  • Identify why an inverse-problem needs to be solved.
  • Identify available parameters in the observed data / images.
  • Identify the category of inverse problem.
  • Identify a tool that can estimate a solution.
  • Determine the parameters of a solution.
  • Verify that the solution can generate the observed data / images.
  • Identify the limitations of the solution (such as instability / range of possible solutions, imprecision of parameters, et cetera)
  • Notice whether the solution is significantly more accurate than a much simpler solution.
  • Make a picture of the problem and solution.
  • Be able to explain the solution to someone who has never taken calculus.

Early on in the course, some of the "identify" steps may be provided for the students. But by the end of the course, the students should be able to answer these questions themselves.

Improve this answer
$\endgroup$
0
$\begingroup$

Some more recent books on inverse problems that also stress applications are:

A light introduction (short, concise, a little more focused on math, but still with good applications) is

In a similar spirit, but older and a little longer, but also very well written is

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Thank you for the suggested books. They may also be worth posting in an answer to math.stackexchange.com/questions/143501/…. Do you also have any thoughts on how to teach inverse problems to undergraduate engineering students? $\endgroup$ – J W Jun 19 '18 at 10:01
  • $\begingroup$ Why did people minus one Dirk? He had links to books and all. Seems rough. $\endgroup$ – guest Jun 23 '18 at 0:14
  • $\begingroup$ @guest: While I did not downvote Dirk's answer, I can guess one reason why someone did. My question emphasizes teaching over recommended books, there already being a Mathematics Stack Exchange question covering those. As it currently stands, Dirk's answer does not address teaching inverse problems beyond the book recommendations he gives. $\endgroup$ – J W Jun 23 '18 at 5:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.