"Tips for creating e-learning questions.
How to Create Effective and Efficient e-Learning Questions That Anyone Can Easily Develop
“To Those Considering or Hesitating to Implement e-Learning”
In e-learning, questions are generally used to assess the level of knowledge acquisition and achievement. The primary goal is to assess understanding of the course content, and questions should be selected based on that objective. However, this task can be quite challenging.
*This is because when implementing e-learning, it’s not enough to simply know the course content that constitutes the “correct answers” to the questions.
In this article, I’d like to explain “how to create effective and efficient e-learning that anyone can easily produce.” Providing learners with both “good learning materials” and “good tests” is the key to successful e-learning education. I hope you’ll find this guide helpful.
Table of Contents
1. What Constitutes “Good” Questions and Tests in E-Learning?
2. Start with the Learning Materials! The Process of Creating E-Learning Content
3. Once the Learning Materials Are Ready, Create the Questions and Tests!
4. Summary
1. What Constitutes “Good” Questions and Tests in e-Learning?
The Purpose Must Be Clear
First and foremost, “What is the purpose of this e-learning course?” Without a clear purpose, you cannot begin defining question specifications or creating questions and explanations. If you start production while the purpose remains vague, there is a high likelihood of having to backtrack during the production process. This can lead to delays in the question creation schedule and increased production costs, and may even result in the creation of unintended, irrelevant questions. As with anything, the start is crucial. To avoid regretting during production that “this wasn’t supposed to happen,” let’s begin by clearly defining the objective.
Validity and Reliability
The second condition is that test questions must possess validity and reliability. To achieve this, you must create questions that “can be answered using knowledge within the scope of what has been learned.” The purpose of a test is to assess learners’ understanding of the material covered in the course. Avoid using concepts or ideas outside the scope of the course simply because they are easier to create—this not only confuses learners but also leads to an increase in inquiries.
Ease of Answering
When creating test questions, it is important to think from the “learner’s perspective.” For example, if a learner is asked about something they haven’t studied, they will likely be confused and think, “I haven’t learned this!” Furthermore, even if a question tests knowledge within the curriculum, the specific content of the question can sometimes cause learners to feel unnecessary stress. Start with simple questions and gradually increase the difficulty. The more questions a learner cannot answer, the more their motivation to learn will decline, and they may stop working on the assignment altogether.
*Test questions must, above all, clearly and directly assess the content that has been covered.
Make the Intent of the Question Clear
When creating questions, make greater use of definitive, “assertive statements.” Wordy phrasing or sentences with double negatives can lead to misunderstandings or misinterpretations.
Use Images and Video Materials
We perceive and process “most of our information visually.” If you find yourself writing lengthy, disjointed text, don’t hesitate to make liberal use of images and videos.
2. Start with the Learning Materials! The Process of Creating e-Learning Content
Analysis
Consider the information necessary for creating the materials, such as “Why are we creating these materials and who is the target audience?”, “How much time should be allocated for learning and what are the learning goals?”, and “How will the system be used?”
↓ ↓ ↓
Design
Based on the information solidified during the analysis process, proceed with preparing the “course specifications,” “structure of the learning content,” and “production schedule.”
↓ ↓ ↓
Development
Based on the design concept diagrams, we actually create the learning materials. We have third parties review the completed materials to ensure the content is easy to understand and user-friendly, making revisions as needed while continuing development.
↓ ↓ ↓
Implementation
We begin rolling out the completed materials to learners.
↓ ↓ ↓
Evaluation
After a period of operation, we analyze learner trends based on their participation history and survey results to determine whether the initial objectives have been achieved. This information serves as valuable data when reviewing the educational content.
3. Once You’ve Created the Learning Materials, Let’s Make Questions and Tests!
Let’s take a look at the specific process for “creating questions and tests” in e-learning. Generally, questions are created following the steps below.
Determine what to measure and by what criteria
While the primary objective is to “verify understanding of the test content,” the breadth and depth of that understanding vary depending on the purpose of the learning materials that accompany the test. Let’s confirm the following three points:
▼ What is the purpose of the “learning materials” that accompany the test?
▼ What is the purpose of the test based on the learning materials?
▼ What are the criteria for measuring learners’ understanding?
*Set your question creation criteria based on these points.
Determine the test specifications (type, format, presence of explanations, passing criteria, etc.)
Once the purpose of the test has been established, consider the test specifications necessary to achieve that purpose. The main specifications to consider are as follows:
▼ Test Type
There are several types of test formats. Choose the most appropriate one based on the purpose.
Pre-test: A test designed to measure learners’ current proficiency.
Post-test: A test used to measure the extent to which learners have acquired knowledge by comparing results with those of the pre-test.
Progress Test: A test designed to measure understanding after each learning unit.
Final Test: A test designed to assess understanding of the entire course; learners are considered to have completed the course if they meet the established passing score.
▼ Test Question Formats
Commonly used question formats include “single-choice,” “multiple-choice,” and “true/false.” If you want to measure learners’ understanding more precisely, it is better to choose “single-choice” over “true/false” and “multiple-choice” over “single-choice.” Another way to increase the difficulty is through “random question order,” which changes the order of the questions each time the test is administered. In addition, there is “shuffled answer choices,” which changes the order of the options. *There are also essay-style questions that require students to write out their answers. Keep in mind that grading these essay-style questions after the test is difficult. ➡ Difficulty Level: “True/False” < “Single-Choice” < “Multiple-Choice”
▼ Need for Explanations
You must decide whether to provide explanations after grading. Explanations are unnecessary for questions that simply test the content of the learning materials. This is because if a student scores below the passing threshold, they will likely achieve better learning outcomes by reviewing the areas where their knowledge is unclear and then retaking the review test.
For practice-style questions, explanations are generally provided. Since practice questions are application-based problems that incorporate knowledge learned from the study materials, it can sometimes be difficult to understand why a particular answer is correct. Explanations are effective for helping learners develop applied thinking skills.
➡ In this case, the test is not merely a “measurement tool”; it inherently contains an element of “learning.”
▼ Passing Threshold
It is common to set a passing threshold for tests, but this threshold must be adjusted according to the course content and learning objectives. As shown in the examples below, a 100% score may sometimes be required.
70%: When you want all participants to acquire the minimum necessary knowledge
80%: When it is sufficient to meet a certain standard
100%: When perfect performance is required, etc.
Create the test script according to the question specifications
▼ Standardize Question Formatting
Try to standardize the question wording as much as possible. Having multiple styles can cause stress for learners and may prevent them from concentrating on answering the questions. Test scripts should be written in either the “desu/masu” style or the “da/aru” style. It is important to ensure that these two styles do not mix within a single sentence. This can also cause unnecessary stress for learners.
Creating Questions Using the Question Creation Tool
Once the draft is complete, you can begin creating the actual questions. Here, we’ll briefly explain the production process using our question creation tool, “Quiz Generator.” First, create a draft that complies with the file format required for importing into the creation tool. Below is a partial screenshot of the Excel and text files used to import test drafts into our educational material creation tool.
The educational material created by importing this file (draft) into the tool looks like this.
Sample Question ➡ Sample question created with our tool
*As shown here, as long as you create a file (draft) compatible with the question creation tool, you can create questions and tests in a short amount of time (about a few minutes). It also makes revisions much easier.
You can find detailed step-by-step instructions here:
➡ Creating Questions with a Text File
➡ Creating Questions with Excel
The process of creating fill-in-the-blank questions using Excel and publishing them online is also explained in detail in “How to Publish Fill-in-the-Blank Questions Online Using Excel.”
Analyzing Test Results and Making Improvements
While the learning materials you create will be uploaded to the LMS and put into use, simply creating them is not enough. It is important to use the analysis of test results and feedback from learner surveys to develop the next set of initiatives or to improve the tests themselves. Key points for analysis include the following:
・Were the objectives achieved?
・Were the questions appropriate for the learners’ level?
・Was the passing threshold appropriate?
・Did the learners feel stressed? etc.
➡ Continuously carrying out this analysis and making improvements will enhance the quality of the tests—and, by extension, the quality of educational training that utilizes e-learning—leading to higher learner satisfaction.
4. Summary
In this article, we discussed question creation for e-learning. Did you discover anything new?
Let’s briefly review the key points.
・First, consider the purpose of e-learning.
・Understand the workflow for creating e-learning materials.
・Once the materials are created, develop questions and tests.
・Use question-creation tools to streamline the process.
・Analyze test results and make improvements.
Finally, a test isn’t over once it’s administered. You need to analyze test results and learner feedback (evaluations) to inform future initiatives or test improvements. To do this, those responsible for test creation must be thoroughly familiar with the content of the learning materials. Creating tests can be challenging, but if you clarify your objectives and follow the proper steps, it isn’t that difficult. Please use this article as a reference and give creating questions a try.
