How Does Engagement Affect eLearning

by Chris Packert | Jun 9, 2023 | Learning Strategy, Uncategorized | 0 comments

Emerge ID: Award-winning learning design. AI-powered development. Smarter training at a lower cost.

Dashboards, badges, interactive activities, AI tools. Each one sounds like progress. In Episode 2 of our podcast, L&D Insights at Altitude, our hosts Nora and Ben walk through seven current sources (four peer-reviewed studies, a Gallup study, an OECD report, and a new Colorado tax law) and ask the same thing of each: what changes in how people work? This week's thread is simple. Features are not learning. Each one works only as well as the design around it. Here is what our team took from them.

Watch or listen to Episode 2 on YouTube

Enjoying the show? If you like our podcasts, be sure to like and subscribe to our YouTube channel.

Dashboards: a small average and a wide range

In The Internet and Higher Education, Rivan Gestiardi and Yuniawatika ran a meta-analysis of learning analytics dashboards, displays of learners' own data such as progress or activity. They pooled 21 of 22 controlled studies with more than 33,000 participants, and they left out dashboards bundled with adaptive tutoring or heavy automated feedback, so the result reflects the dashboard itself. For learner-facing dashboards, the average effect was small, about 0.29, and closer to 0.17 after adjusting for possible publication bias. The range across studies was wide enough to include zero. Effects showed up for grades and completion, but not for engagement logs, self-reported self-regulation, or motivation.

The authors' own point is that a given rollout may yield no benefit, so pilot with usage data and evaluate before you scale. The participants were mostly higher education students, not employees, and the authors rate certainty as low to moderate. The screen is not the intervention. What people do with it is.

Badges that backfired

In Behaviour & Information Technology, Sebiha Balci and Bradley Morris randomly assigned 162 online learners to three versions of a course: badges they knew were coming, surprise badges, or no badges. The hypothesis was that surprise badges would work better. It did not hold. Both kinds of badges reduced intrinsic motivation, did not meet learners' need to feel competent, and did not improve performance compared with no badges. It is one short-term study with online students, but it is randomized, and it is a good reason to test badges before you switch them on everywhere.

"Interactive" is a label, not a result

"Interactive beats passive" is a common pitch line. In Learning and Instruction, Christina Wekerle, Michael Sailer, Ingo Kollar, and Tim Fütterer tested the ICAP framework with 249 future teachers. After reading a text, participants reread it, underlined it, answered open questions alone, or answered the same questions in pairs. No condition won on knowledge: the authors found no significant differences in surface or deep knowledge. The pairs did report a stronger sense of belonging. This was one reading task with trainee teachers, and the authors do not say the framework is wrong. They say its evidence base may not be robust yet. Calling something interactive does not make it effective. The task has to make people think and practice.

How good designers work with AI

In the International Journal of Educational Technology in Higher Education, Fang, Feng, Yan, and Zhou watched 67 pre-service teachers build lesson plans with an AI assistant, then rated the plans. The stronger plans came from people who kept cycling between evaluating the AI's draft, revising it, and evaluating again. The weaker plans came from people stuck in loops of unclear back-and-forth. The stronger group also reported lower mental demand and frustration. It is one program of trainee teachers, so treat it as an early signal for workplace designers, and it is correlational: the groups were split by outcome, so it does not prove the habit caused the quality. Still, evaluate, revise, and re-evaluate is a habit you can coach.

Who gets value from AI at work, and who is left out

Gallup's first results from year two of the American Job Quality Study draw on a survey of more than 15,000 U.S. employees. Among employees who use AI, 63 percent say it helps them work faster and 56 percent say it helps them find more creative solutions. Regular use is concentrated, though. College graduates are more than twice as likely to use it daily or weekly (40 percent versus 17 percent), and so are managers compared with individual contributors (37 percent versus 25 percent). In healthcare support, manufacturing production, and retail sales, 70 to more than 80 percent never use AI in their role. Gallup is clear that these are employees' own assessments and that the link to job quality needs more research.

Our reading, not Gallup's claim: if AI training goes mainly to the people already using AI, it can widen the gap.

Most adults are not asking for more training

The OECD's Education at a Glance 2026, released September 29, includes a chapter on adult learning that draws mostly on the 2023 Survey of Adult Skills. Over 75 percent of adults report no interest in more non-formal learning beyond what they have already done. Participation is unequal, too: about 49 percent of employed adults took part in the past year, versus about 34 percent of unemployed adults. The OECD suggests testing interventions that target motivation, not only access. The data are self-reported and cross-country, and the U.S. is one country in the mix. For L&D, the message is plain. Another catalog will not fill seats. Learning has to connect to work people already care about.

Colorado's new software tax and your courses

In June, Governor Jared Polis signed HB26-1223. Starting January 1, 2027, Colorado's state sales and use tax covers computer software however it is delivered, including downloads, software you reach over the internet, and mobile apps. The Colorado Department of Revenue's Computer Software guidance, revised in October 2026, walks through what it means.

The state rate is 2.9 percent. Local taxes are less clear. The legislature's final fiscal note says counties, most cities, and districts like RTD only add this tax if their own voters approve it, and about 70 home-rule cities run their own sales taxes and make their own call. Sellers based in Colorado collect the tax. Out-of-state sellers start collecting once their Colorado sales top $100,000 a year, which was already the rule. If a seller does not collect, the Colorado buyer generally owes use tax instead.

Why does this matter for training? The law keeps two exemptions: software developed for a particular user, and software under an individually negotiated license that both sides sign in writing before or when access begins. Click-through terms and standard form contracts do not count, even if someone signs them. That means a custom course built for one client and a catalog course bought at a website checkout could be treated differently. The law does not mention courses, and the state has not said when an online course counts as software.

Before January, take two questions to your tax advisor. Was this course built for us alone? And is our license individually negotiated and signed by both sides? How you contract and deliver a course may change what you owe.

This episode is general information, not tax, legal, or accounting advice. Emerge ID is not a CPA firm or law firm. Colorado's rules for HB26-1223 are still being written. Consult a qualified tax advisor about your situation.

One contrarian take

The popular story is that better learning tech means better learning: add the dashboard, add the badges, make it interactive, give everyone AI. This week's sources say otherwise. Dashboards helped a little on average and maybe not at all for a given rollout. Badges cost motivation in one randomized test. Interactive activities did not move knowledge in another. Strong AI-assisted work came from judgment, not prompting. And most adults are not asking for more courses.

The feature is the easy part. The design is the hard part: what people need to do differently, why they would care, the practice and support on the job, and how you will measure it. At Emerge ID, Intelligent Design is what the ID in our name stands for. It is what we call our process for combining proven instructional design principles with the speed and production power of artificial intelligence.

Five takeaways

  1. Pilot dashboards and measure before you scale. The average effect was small, and the range included zero.
  2. Test badges before you turn them on. They backfired in one controlled study.
  3. Treat "interactive" as a label. Design the thinking and practice the task requires.
  4. With AI, coach the evaluate and revise loop. Have learners judge an AI draft, revise it, then judge it again.
  5. Aim AI and learning at the people who are not getting them yet. Start with why the learning matters to their work.

The practical challenge

Pick one feature you added to a program this year, such as a dashboard, a badge, or an AI tool, and ask what changed in how people work because of it. If you do not know, that is the place to start.

The ID Angle: what this changes on Monday

Start with the behavior, not the feature. Write down what people should do differently on the job, then pick the tool that supports it. Treat each feature as a test, not a default: pilot a dashboard with a comparison group, and try badges small before you roll them out. Build AI practice around judgment. Open with why the learning matters to people's work. And for Colorado, know how your courses are contracted and delivered, and take the two exemption questions to your tax advisor before January.

Let's design learning that changes what people do

As the episode puts it, "Anyone can add a feature in an afternoon. The hard part is designing learning that changes what people do on the job." That is what Intelligent Design is for: starting with the performance you need, then using proven instructional design principles and the speed of AI to build learning that holds up on the job.

Emerge ID is led by a United States Olympic and Paralympic Committee Coach Educator of the Year with 30+ years of experience in learning and development. If that sounds like your team, there is an easy next step: book a free 20-minute review with Chris Packert. By the next business day, you will get three prioritized next steps. No purchase required.

Planning a new course, onboarding program, or training pathway? Consider Emerge ID for your next project.

Find more episodes of L&D Insights at Altitude every two weeks.

Show notes: Episode 2 chapters

Runtime 13:01. Chapter times match the YouTube video.

  • 00:00 Cold open: do dashboards help people learn?
  • 00:24 Welcome to L&D Insights at Altitude
  • 00:53 This week's thread: features and learning
  • 01:17 Dashboards: a small average and a wide range
  • 02:33 Badges that backfired
  • 03:23 Interactive beats passive? Testing ICAP
  • 04:25 How good designers work with AI
  • 05:28 Who gets value from AI at work? (Gallup)
  • 06:25 Most adults aren't asking for more training (OECD)
  • 07:18 Colorado's new software tax and your courses
  • 09:31 One contrarian take
  • 10:25 Recap
  • 10:51 This week's challenge
  • 11:05 The ID Angle
  • 12:15 Next steps

Hosted by AI voices. Research by the Emerge ID team.

Gallup's American Job Quality Study is conducted with Jobs for the Future and The Families & Workers Fund.

Sources cited in Episode 2

Colorado HB26-1223 sources:

Engagement is a word that we often hear paired with eLearning. Instructional designers strive to create educational materials that learners can engage with. So what is engagement in eLearning, how does it affect the learning process, and is it actually necessary?

What is Engagement?

According to learnupon.com, “It's a willingness for the learner to interact consistently with the training provided in order to achieve a learning goal” (Quigley, n.d.). Engagement plays a crucial role in eLearning as it directly impacts the effectiveness and success of the learning experience. When learners are engaged, they are more motivated, attentive, and actively participate in the learning process, leading to improved knowledge acquisition and retention. Here are a few ways that engagement can enhance the learning experience:

Increases Focus and Leads to Higher Retention:

Engagement encourages the application of knowledge. Engaging eLearning experiences capture learners' attention and help them stay focused on the learning materials. Interactive elements, multimedia, and well-designed content can create a more immersive learning environment, reducing distractions and increasing concentration. When learners are actively engaged throughout the eLearning process, they are more likely to retain information effectively. 

Improved Performance:

Instructional designers focus on the why behind a course. eLearning courses should have a clear purpose that is evident to the learner. When learners understand the value of the course or training, they are more likely to be engaged and focused. “If you can untangle a complicated process, you’re going to solve a lot of procrastination in the workplace” (Cohen, 2023). Having a clear understanding of the value of a course will allow for a better transfer of knowledge, leading to improved performance on the job.

Increases Motivation

Engaged learners are more motivated to learn and explore the course content. They have a genuine interest in the subject matter, which drives them to invest time and effort into the eLearning activities. An investment in quality, engaging, training allows your employees to feel appreciated and appropriately challenged. For example, adding gamification to eLearning through a leaderboard, or a badge to reward learners will increase motivation. 

Overall, engagement is a critical factor in eLearning because it drives learners' motivation, participation, focus, knowledge retention, collaboration, and successful completion of courses. To maximize engagement, eLearning designers and instructors should incorporate interactive and immersive elements, foster learner interaction, and provide personalized learning experiences tailored to the individual needs and preferences of learners.

References:

Cohen, D. (n.d.). Five Rules of Engagement All eLearning Designers Should Live By. Five Rules of Engagement All eLearning Designers Should Live By. https://www.shiftelearning.com/blog/rules-of-engagement-elearning

Quigley, E. (2019, April 12). eLearning Engagement: A Formula for Success. LearnUpon. https://www.learnupon.com/blog/elearning-engagement/

Engaging students in learning - Center for Teaching and Learning. (n.d.). Center for Teaching and Learning. https://teaching.washington.edu/topics/engaging-students-in-learning/The Importance of Learner Engagement on Performance. (n.d.). TalentLMS. https:///ebook/learner-engagement/importance-learner-engagement-performance

This image shows the author of the blog posts.

Written by Chris Packert

Chris Packert is the founder and CEO of Emerge ID, a learning architecture and instructional design company that helps organizations design, build, launch, and improve meaningful learning experiences. With more than 30 years of experience across K–12 education, higher education, workforce development, military distance education, and sport education, Chris brings a practical, systems-based approach to learning strategy. His work focuses on helping teams clarify performance needs, create accessible and engaging learning solutions, implement effective learning technology, and measure impact beyond completion data. Through Emerge ID, Chris partners with organizations to turn complex learning challenges into clear, usable, and measurable learning experiences.

More Posts by Chris

Related Posts

0 Comments