Writing quality questions is often an important part of financial education. There are exam questions and there are questions to practice for the exam. I’ve been writing questions for 17 years. There’s a rhythm and an art to it.
Maybe 10 years ago I ran into a guy—I’ll call him Owen—who did test prep for a variety of financial exams. He had a “stable” of writers who wrote thousands of questions for him. He said they could write 10 questions an hour and he kept them going night and day until a project was done. They were fast.
I tend to be a fairly fast writer, but I could never get anywhere close to 10 questions an hour. If I’m really cooking, I can creep up to six questions an hour. A more usual sustained speed is maybe five.
Now, I titled this article: “2 B or NOT 2 B. That Isn’t the Question.” That’s because most multiple choices have more than two answer choices (2 B or NOT 2 B). I could probably write way more than 10 questions in an hour if two-answer questions were the norm.
The hard part about writing a good question is writing the wrong answer choices (called “distractors”). Okay, the first wrong answer choice is usually easy (NOT 2 B), but the rest of them take finesse to make them plausible choices but still wrong. Owen’s stable of writers could not have written 10 questions in an hour if their distractors were any good.
Another hard part about writing good questions is that, if you are required to write too many questions based on a limited amount of material, you start exhausting the material. Your questions start sounding all alike.
The Texas Department of Insurance pointed a way out of this difficulty a number of years ago when it began to require providers of insurance education to use “application-based” questions in their exams.
The term “application-based” comes from Bloom’s Taxonomy of Learning Domains (which I touch on in my post “Ethics Education is Very Stocky. But Does It Stick?”). Texas regulators felt that it wasn’t enough to learn the rules governing insurance. They felt it was important that insurance professionals be able to apply the rules to factual situations: hence the requirement that 70 percent of the questions in an insurance exam for Texas must require you to do this.
Honestly, the requirement does elevate the quality of the exams and the courses they are attached to. And, since a single insurance rule can apply to many fact situations, the new type of question opens up the possibilities for question writers.
If you know what you are writing about.
Owen’s stable of writers might have a hard time writing application-based questions. You could almost write a computer algorithm to write knowledge-based questions as you don’t do much more than just manipulate the statement of a rule to create a question about the rule. To write application-based questions you need to know something about how the rule works in the real world.
Now, before I end this post, I have to confess that I’ve been talking about writing questions for instructional material that already exists. Historically, this is the most common way that financial education is put together. In the old days, you started with a book and then wrote questions about the book.
There is a better way. You start with the application of the field of knowledge in the real world. Next you figure out what you need to teach in order to bring a learner up to speed in that real world. Then you write the assessment (questions or other types of assessments). Finally, you create the instructional pieces necessary to get the learner to be successful on the assessment and, by extension, in the real world.
It’s backwards from the way it’s always been done.