How to generate sample problems

Question

How do professors and authors generate problems for their exams and textbooks? To what extent are problems inspired by problems from existing texts? And how does someone who understands the material thoroughly gauge the difficulty of the questions he or she is creating?

Answer 1

As an occasional textbook author and frequent reviewer of texts for several publishers, I have thought quite a bit about the issues you raise.

First, it is obviously true that really good questions migrate freely among textbooks, usually with minor changes or embellishments. In some cases it is pretty clear that a question is original in a particular book, and in that case I think authors who take a question or an idea for a question for their own book ought to give credit to the original source more often than seems to happen in practice. But many interesting questions have become so 'standard' that there is probably no telling where they originated.

Also, I have always tried to invent new questions or to think of new uses for classic ones. But this gets to your second good question about judging difficulty. I know of no way to be sure about the difficulty of a question for students than to test it out in several classes. If the question is really original, there seems no end to the surprises how students will interpret the wording or the approach they will take to solving it. Often a question can go from frustrating and time wasting to really useful by introducing it in a new way, putting it in a slightly different context, or wording it more carefully.

Sometimes, a horribly confusing question persists unchanged through multiple editions of essentially the same textbook. I know of one quirky question in a much-used, multi-author probability book that has alternated correct and incorrect answers in the back of the book through five editions. (First thing I do when a new edition comes out is to check what answer they're giving $this$ time.) It's not a really a deep or difficult question, just badly worded. One wonders what flaws in lines of communication can allow such things to happen.

Recently, one of the difficulties with some commercial Internet courses and hastily published on-line books is that the quality and variety of questions seems to suffer during whatever process of authoring and reviewing is being used. My personal view is that computer-generated questions in online support services for some textbooks can be especially problematic. In randomly changing numbers in some types of problems, the problems can become nonsensical in unexpected ways. Also, the answer-recognition software is sometimes not clever enough to recognize an essentially correct answer in an unanticipated format.

Overall, many questions assigned are engaging and useful. Even more so because many instructors work hard to assign the best questions available. However, students who encounter difficulty with a question should always keep in mind the possibility that the deficiency may be in the question (or the answer provided). Don't spend an inordinate amount of time on any one 'impossible' question without checking somehow where the real difficulty lies.