2 B or NOT 2 B. That Isn’t the Question

Writing quality questions is often an important part of financial education. There are exam questions and there are questions to practice for the exam. I’ve been writing questions for 17 years. There’s a rhythm and an art to it.

Maybe 10 years ago I ran into a guy—I’ll call him Owen—who did test prep for a variety of financial exams. He had a “stable” of writers who wrote thousands of questions for him. He said they could write 10 questions an hour and he kept them going night and day until a project was done. They were fast.

I tend to be a fairly fast writer, but I could never get anywhere close to 10 questions an hour. If I’m really cooking, I can creep up to six questions an hour. A more usual sustained speed is maybe five.

Now, I titled this article: “2 B or NOT 2 B. That Isn’t the Question.” That’s because most multiple choices have more than two answer choices (2 B or NOT 2 B). I could probably write way more than 10 questions in an hour if two-answer questions were the norm.

The hard part about writing a good question is writing the wrong answer choices (called “distractors”). Okay, the first wrong answer choice is usually easy (NOT 2 B), but the rest of them take finesse to make them plausible choices but still wrong. Owen’s stable of writers could not have written 10 questions in an hour if their distractors were any good.

Another hard part about writing good questions is that, if you are required to write too many questions based on a limited amount of material, you start exhausting the material. Your questions start sounding all alike.

The Texas Department of Insurance pointed a way out of this difficulty a number of years ago when it began to require providers of insurance education to use “application-based” questions in their exams.

The term “application-based” comes from Bloom’s Taxonomy of Learning Domains (which I touch on in my post “Ethics Education is Very Stocky. But Does It Stick?”). Texas regulators felt that it wasn’t enough to learn the rules governing insurance. They felt it was important that insurance professionals be able to apply the rules to factual situations: hence the requirement that 70 percent of the questions in an insurance exam for Texas must require you to do this.

Honestly, the requirement does elevate the quality of the exams and the courses they are attached to. And, since a single insurance rule can apply to many fact situations, the new type of question opens up the possibilities for question writers.

If you know what you are writing about.

Owen’s stable of writers might have a hard time writing application-based questions. You could almost write a computer algorithm to write knowledge-based questions as you don’t do much more than just manipulate the statement of a rule to create a question about the rule. To write application-based questions you need to know something about how the rule works in the real world.

Now, before I end this post, I have to confess that I’ve been talking about writing questions for instructional material that already exists. Historically, this is the most common way that financial education is put together. In the old days, you started with a book and then wrote questions about the book.

There is a better way. You start with the application of the field of knowledge in the real world. Next you figure out what you need to teach in order to bring a learner up to speed in that real world. Then you write the assessment (questions or other types of assessments). Finally, you create the instructional pieces necessary to get the learner to be successful on the assessment and, by extension, in the real world.

It’s backwards from the way it’s always been done.

Advertisements

Serving Two Masters: The World of Continuing Education

Okay. Your license renewal period is almost over and you haven’t met your CE requirement yet. You need to find a course on something you already know so you can breeze through the course and pass the exam. You have to get through it fast. You’re out of time. Next time you’ll plan better. Maybe there’s a course that could really give you something to grow your book of business. But that’s next time.

And next time never comes.

This is the challenge for people writing continuing education courses. You see CE as a blank canvas for your creativity. You could teach great things. Training directors are in total sympathy. If they are the ones footing the bill, they may actually prefer courses that teach deep things. But they know the score. CE time is not your most reflective time. You need the hours and you need them fast.

How do you design courses for learners who are, well, resistant?

CE hours are, of course, somewhat related to the amount of time a student is expected to spend in a course. Different states try different strategies to try to force students through the material. Other states are more laissez faire. But students who are resistant are not the eager faces you’d like to see.

In my mind, you need to be pretty modest about what you can achieve. You plan your course with solid instruction, but you lace it with a few compelling take-aways. The idea is: if they get nothing else from the course, this is it. They get the hours, but they get a little tidbit of knowledge they can use.

You got your hours, just in the nick of time, but you got something extra. Maybe you’ll set aside more time next time around.

And I served two masters well. I delivered the hours as painlessly as possible. And I served up a little bit of knowledge to boot.

Cognitive Task Analysis of the Mind

All the instructional design textbooks start the process with analysis. The most famous development model is the ADDIE model: Analyze, Design, Develop, Implement, and Evaluate. In the heat of real-world course development, there is a tendency to want to jump right to the design stage or even the develop stage—because analysis is seen as expensive or because we “know” what the analysis is going to show.

Now we can’t always go through a process like a full cognitive task analysis. CTA is used to capture detailed information about jobs, such as what cognitive activities separate high performers from beginners, how experts make decisions, what knowledge subject-matter experts will tell you versus tacit knowledge that they can’t even begin to articulate, and so on. This is a very detailed process and may actually yield too much “useful information.” After all, going from beginner to expert can take a long time. Sometime the education has to go in steps.

So, if you can’t do a full CTA, do you just give up? Some do.

But in my experience, it never pays to start a project without having some kind of picture of two things: (1) a picture of the learner and (2) a picture of what you want the learner to be able to do as a result of your course. We’re not just spewing information. We want to achieve a result.

Now, it may seem silly, but I learned these two points in a play writing class. We playwrights use different terminology. We don’t talk about learners, we talk about audiences. We don’t talk about learning objectives, we talk about the major dramatic question (MDQ). To write an effective play, you have to transport your audience from here to there.

Believe it or not, playwrights often do research before writing a play. But in the field of dramatic writing, it is also proper to talk about the playwright’s imagination.

We don’t talk about imagination as much in the field of instruction. But why not? Our purpose, like the playwright’s is to capture people’s minds and change them.

So, if you can’t do a CTA—for whatever reason—you do what you can do. Hard data is still important. But you need imagination as well. Imagination isn’t day dreaming. It is hard work. Don’t skip that step. (It’s even important if you can do a CTA.)

Ethics Education Is Very Sticky. But Does It Stick?

This evening, I washed the dishes after a tasty meal of my wife’s lasagna. Naturally, my 20-something sons took the opportunity to take showers at the same time, so we all rapidly ran out of hot water.

This turned out to be a blessing in disguise for me. Hot-water cleanup of sticky cheese tends to do little but spread and smear. Turn on the cold water and the cheese becomes more brittle and washes away without contaminating anything else. Hot water is good for many cleanup chores, but cold water has it’s place.

I don’t suppose that cold water was good for my son’s showers.

A sticky subset of financial education is ethics education. Most, if not all, licensing bodies in the financial professions require ethics education. The idea is that the public—consumers of financial services—are protected by teaching financial professionals about ethical rules. But does it? And does the run-of-the-mill ethics education stick?

Being able to recite rules is different from applying them in day-to-day practice. This is true in many areas of knowledge. And this difference is captured in a framework called Bloom’s Taxonomy of Learning Domains. Different learning strategies are required to be effective in teaching, say, application of ethical rules versus the ability to merely recite them. Just like hot water is good for some cleanups but cold water is good for others.

Ethics is situational. Now, I’m not talking about situational ethics, where the rules change depending on the situation. I’m talking about rules that are meant to be followed.

The situational part is where applying ethical rules gets difficult. When you are in the heat of a situation, you need to recognize an ethical problem and you need to decide how to act on the fly. The heat of the situation makes us forget the rules we learned to recite so well.

Unless we rehearse our responses. And, of course, we can’t rehearse our responses on real clients.

What we need is what many organizations use to train people with dangerous jobs: similators. Can you imagine a pilot being trained to respond to a flight emergency without a flight simulator?

The job of a financial professional may not be so dramatic, but people’s fortunes are at stake. Ethical lapses can wipe out a couple’s retirement savings or leave parents unable to send their kids to college.

So some sort of ethics situational simulator is required. It doesn’t have to be fancy, but the purpose is to put the learner into a situation, get them to respond, and give them feedback on whether their response was appropriate or not.

By giving the learner experience in solving ethical dilemmas in simulation, maybe we can keep them out of hot water when they’re out on the job.

Learning Objectives Can Mean More for the Designers Than for the Learners

Have you ever opened a course, online or a textbook, and skipped right over the statement of learning objectives? I have. Even if I read them, they don’t really make sense to me. Not yet.

I need a little orientation first.

Back in 1985, Robert Gagné said that there are nine steps to effective instruction. Providing learning objectives is Step 2, not Step 1. The first step is getting the learner’s attention. This involves laying out the problem that will be solved by the learning experience that is about to unfold and making clear why the learner should care. (Check out Don Clark’s Big Dog website for the entire nine steps.)

Only then do you provide the roadmap. It’s: Here’s the need that brought you to this course (step 1) followed by: Here’s how we’re going to satisfy that need (step 2). Then, before we get into the instruction proper, we let the learner reflect on what he already knows and how the instruction will fit in (step 3).

Learning objectives are certainly important for the learner, but they are not the first thing.

For the learner.

For those of us who design courses, however, learning objective serve a different function. They are out plan for course development. They help us determine how to apply most of the remaining steps in Gagné nine-steps of instruction. For each learning objective, there are materials to present to the learner, activities to design to get the learner to rehearse new knowledge and skills, coaching and feedback, assessment, and so on. While learning strategies may vary depending on the type of learning objective, an entire sequence needs to be implemented for each one.

Our project plans are built around learning objectives.

Creativity’s Muse Can Be a Project Manager

My wife and I met in a play writing class. We had different styles—at least at a macro level. I wrote outlines, she didn’t. Her work was organic, mine was planned. At a micro level, there was probably more similarity. Although I had my outlines, I improv’d scenes in my head, and sometimes (as they say) the characters in my plays took on a life of their own, thwarting my neat outlines. And although my wife built her plays up from individual scenes, she did have some sort of organizing principle—you just didn’t see it in a formal structure. . . .

Okay, wait a minute, What does this have to do with financial education?

Educational projects can involve many different people with many different perspectives. This presents a challenge in managing the project to a successful conclusion. Subject-matter experts know what they know, but they may not be able to articulate it. Instructional designers know learning theory, but they may not understand the subject-matter. They may take time to come up to speed. Executive have one sort of expectation for a project and production people have different ones.

Bringing a team of such diverse people together is a challenge. The effort can stifle creativity. My wife couldn’t understand how my outlines could help me write a work of drama or humor. But they did help. In my mind, they ensured that my creative vision was realized throughout the whole work.

In the same way, effective project management, which recognizes the particular value of each member of the team and their particular viewpoint, can orchestrate a superior product. It can create synergies by ensuring that the whole team remains focused on learner outcomes. By not working at cross purposes, a team can achieve its goal without waste and without rework.

“Portable Content” May Not Be So Portable After All

You invest a lot in developing course content, so you hope to get maximum use from it. Right?

This is the idea behind content management. You write a piece for insurance agents and you want to use it again for underwriters. You write a piece for financial analysts and you want to use it again for consumers. You write a piece for senior advisers and you want to use it again for support staff. You write a piece to help folks pass a test and you want to use it again to improve job performance.

Content management systems are designed to help you achieve your reuse aims. But does it work? Sometimes it does. But sometimes, content written for one audience goes over the head of another—or it flies beneath their radar. Why is that?

It all has to do with needs analysis.

To develop a superior financial education product, you need to understand what you are trying to accomplish and who your learner is. What is their need? What is their prior knowledge? What motivates them to learn? What engages them?

So a piece written for insurance agents may talk about customer needs. Agents are gregarious people-oriented folk and their purpose in life is to make a sale. Underwriters may have instructional needs that overlap the agent’s, but underwriters are more concerned with statistics and profitability for their company.

A piece for a financial analyst may focus on investment portfolios. Their resources allow for a great deal of research. Consumers may be interested in similar topics, but minutia may not get them where they need to be.

And so on.

I’m not saying that content management systems miss the point. They are great in making the intellectual property of a company available for reused. I’m just saying that it is not as automatic a process that we hope. That’s because learners are involved. And different learners have different needs and motivations.

It’s the learners that we are trying to reach.