Emerge ID: Award-winning learning design. AI-powered development. Smarter training at a lower cost.
Dashboards, badges, interactive activities, AI tools. Each one sounds like progress. In Episode 2 of our podcast, L&D Insights at Altitude, our hosts Nora and Ben walk through seven current sources (four peer-reviewed studies, a Gallup study, an OECD report, and a new Colorado tax law) and ask the same thing of each: what changes in how people work? This week's thread is simple. Features are not learning. Each one works only as well as the design around it. Here is what our team took from them.
Watch or listen to Episode 2 on YouTube
Enjoying the show? If you like our podcasts, be sure to like and subscribe to our YouTube channel.
Dashboards: a small average and a wide range
In The Internet and Higher Education, Rivan Gestiardi and Yuniawatika ran a meta-analysis of learning analytics dashboards, displays of learners' own data such as progress or activity. They pooled 21 of 22 controlled studies with more than 33,000 participants, and they left out dashboards bundled with adaptive tutoring or heavy automated feedback, so the result reflects the dashboard itself. For learner-facing dashboards, the average effect was small, about 0.29, and closer to 0.17 after adjusting for possible publication bias. The range across studies was wide enough to include zero. Effects showed up for grades and completion, but not for engagement logs, self-reported self-regulation, or motivation.
The authors' own point is that a given rollout may yield no benefit, so pilot with usage data and evaluate before you scale. The participants were mostly higher education students, not employees, and the authors rate certainty as low to moderate. The screen is not the intervention. What people do with it is.
Badges that backfired
In Behaviour & Information Technology, Sebiha Balci and Bradley Morris randomly assigned 162 online learners to three versions of a course: badges they knew were coming, surprise badges, or no badges. The hypothesis was that surprise badges would work better. It did not hold. Both kinds of badges reduced intrinsic motivation, did not meet learners' need to feel competent, and did not improve performance compared with no badges. It is one short-term study with online students, but it is randomized, and it is a good reason to test badges before you switch them on everywhere.
"Interactive" is a label, not a result
"Interactive beats passive" is a common pitch line. In Learning and Instruction, Christina Wekerle, Michael Sailer, Ingo Kollar, and Tim Fütterer tested the ICAP framework with 249 future teachers. After reading a text, participants reread it, underlined it, answered open questions alone, or answered the same questions in pairs. No condition won on knowledge: the authors found no significant differences in surface or deep knowledge. The pairs did report a stronger sense of belonging. This was one reading task with trainee teachers, and the authors do not say the framework is wrong. They say its evidence base may not be robust yet. Calling something interactive does not make it effective. The task has to make people think and practice.
How good designers work with AI
In the International Journal of Educational Technology in Higher Education, Fang, Feng, Yan, and Zhou watched 67 pre-service teachers build lesson plans with an AI assistant, then rated the plans. The stronger plans came from people who kept cycling between evaluating the AI's draft, revising it, and evaluating again. The weaker plans came from people stuck in loops of unclear back-and-forth. The stronger group also reported lower mental demand and frustration. It is one program of trainee teachers, so treat it as an early signal for workplace designers, and it is correlational: the groups were split by outcome, so it does not prove the habit caused the quality. Still, evaluate, revise, and re-evaluate is a habit you can coach.
Who gets value from AI at work, and who is left out
Gallup's first results from year two of the American Job Quality Study draw on a survey of more than 15,000 U.S. employees. Among employees who use AI, 63 percent say it helps them work faster and 56 percent say it helps them find more creative solutions. Regular use is concentrated, though. College graduates are more than twice as likely to use it daily or weekly (40 percent versus 17 percent), and so are managers compared with individual contributors (37 percent versus 25 percent). In healthcare support, manufacturing production, and retail sales, 70 to more than 80 percent never use AI in their role. Gallup is clear that these are employees' own assessments and that the link to job quality needs more research.
Our reading, not Gallup's claim: if AI training goes mainly to the people already using AI, it can widen the gap.
Most adults are not asking for more training
The OECD's Education at a Glance 2026, released September 29, includes a chapter on adult learning that draws mostly on the 2023 Survey of Adult Skills. Over 75 percent of adults report no interest in more non-formal learning beyond what they have already done. Participation is unequal, too: about 49 percent of employed adults took part in the past year, versus about 34 percent of unemployed adults. The OECD suggests testing interventions that target motivation, not only access. The data are self-reported and cross-country, and the U.S. is one country in the mix. For L&D, the message is plain. Another catalog will not fill seats. Learning has to connect to work people already care about.
Colorado's new software tax and your courses
In June, Governor Jared Polis signed HB26-1223. Starting January 1, 2027, Colorado's state sales and use tax covers computer software however it is delivered, including downloads, software you reach over the internet, and mobile apps. The Colorado Department of Revenue's Computer Software guidance, revised in October 2026, walks through what it means.
The state rate is 2.9 percent. Local taxes are less clear. The legislature's final fiscal note says counties, most cities, and districts like RTD only add this tax if their own voters approve it, and about 70 home-rule cities run their own sales taxes and make their own call. Sellers based in Colorado collect the tax. Out-of-state sellers start collecting once their Colorado sales top $100,000 a year, which was already the rule. If a seller does not collect, the Colorado buyer generally owes use tax instead.
Why does this matter for training? The law keeps two exemptions: software developed for a particular user, and software under an individually negotiated license that both sides sign in writing before or when access begins. Click-through terms and standard form contracts do not count, even if someone signs them. That means a custom course built for one client and a catalog course bought at a website checkout could be treated differently. The law does not mention courses, and the state has not said when an online course counts as software.
Before January, take two questions to your tax advisor. Was this course built for us alone? And is our license individually negotiated and signed by both sides? How you contract and deliver a course may change what you owe.
This episode is general information, not tax, legal, or accounting advice. Emerge ID is not a CPA firm or law firm. Colorado's rules for HB26-1223 are still being written. Consult a qualified tax advisor about your situation.
One contrarian take
The popular story is that better learning tech means better learning: add the dashboard, add the badges, make it interactive, give everyone AI. This week's sources say otherwise. Dashboards helped a little on average and maybe not at all for a given rollout. Badges cost motivation in one randomized test. Interactive activities did not move knowledge in another. Strong AI-assisted work came from judgment, not prompting. And most adults are not asking for more courses.
The feature is the easy part. The design is the hard part: what people need to do differently, why they would care, the practice and support on the job, and how you will measure it. At Emerge ID, Intelligent Design is what the ID in our name stands for. It is what we call our process for combining proven instructional design principles with the speed and production power of artificial intelligence.
Five takeaways
- Pilot dashboards and measure before you scale. The average effect was small, and the range included zero.
- Test badges before you turn them on. They backfired in one controlled study.
- Treat "interactive" as a label. Design the thinking and practice the task requires.
- With AI, coach the evaluate and revise loop. Have learners judge an AI draft, revise it, then judge it again.
- Aim AI and learning at the people who are not getting them yet. Start with why the learning matters to their work.
The practical challenge
Pick one feature you added to a program this year, such as a dashboard, a badge, or an AI tool, and ask what changed in how people work because of it. If you do not know, that is the place to start.
The ID Angle: what this changes on Monday
Start with the behavior, not the feature. Write down what people should do differently on the job, then pick the tool that supports it. Treat each feature as a test, not a default: pilot a dashboard with a comparison group, and try badges small before you roll them out. Build AI practice around judgment. Open with why the learning matters to people's work. And for Colorado, know how your courses are contracted and delivered, and take the two exemption questions to your tax advisor before January.
Let's design learning that changes what people do
As the episode puts it, "Anyone can add a feature in an afternoon. The hard part is designing learning that changes what people do on the job." That is what Intelligent Design is for: starting with the performance you need, then using proven instructional design principles and the speed of AI to build learning that holds up on the job.
Emerge ID is led by a United States Olympic and Paralympic Committee Coach Educator of the Year with 30+ years of experience in learning and development. If that sounds like your team, there is an easy next step: book a free 20-minute review with Chris Packert. By the next business day, you will get three prioritized next steps. No purchase required.
Planning a new course, onboarding program, or training pathway? Consider Emerge ID for your next project.
Find more episodes of L&D Insights at Altitude every two weeks.
Show notes: Episode 2 chapters
Runtime 13:01. Chapter times match the YouTube video.
- 00:00 Cold open: do dashboards help people learn?
- 00:24 Welcome to L&D Insights at Altitude
- 00:53 This week's thread: features and learning
- 01:17 Dashboards: a small average and a wide range
- 02:33 Badges that backfired
- 03:23 Interactive beats passive? Testing ICAP
- 04:25 How good designers work with AI
- 05:28 Who gets value from AI at work? (Gallup)
- 06:25 Most adults aren't asking for more training (OECD)
- 07:18 Colorado's new software tax and your courses
- 09:31 One contrarian take
- 10:25 Recap
- 10:51 This week's challenge
- 11:05 The ID Angle
- 12:15 Next steps
Hosted by AI voices. Research by the Emerge ID team.
Gallup's American Job Quality Study is conducted with Jobs for the Future and The Families & Workers Fund.
Sources cited in Episode 2
- Balci, S., & Morris, B. J. (2026). A comparison of how expected and unexpected badges affect the intrinsic motivation and performance of online learners. Behaviour & Information Technology. Advance online publication. https://doi.org/10.1080/0144929x.2026.2738364
- Fang, X., Feng, J., Yan, B., & Zhou, B. (2026). Human-AI collaborative patterns in GenAI-supported instructional design among pre-service teachers: An analysis of interaction sequences and cognitive load. International Journal of Educational Technology in Higher Education, 23, Article 52. https://doi.org/10.1186/s41239-026-00628-3
- Gestiardi, R., & Yuniawatika. (2026). Learning analytics dashboards in Internet-enabled higher education: A meta-analysis of effects on student outcomes, self-regulated learning, and engagement. The Internet and Higher Education, Article 101119. https://doi.org/10.1016/j.iheduc.2026.101119
- Jones, J. M. (2026, October 6). AI benefits at work unevenly distributed. Gallup. https://news.gallup.com/poll/714602/benefits-work-unevenly-distributed.aspx
- OECD. (2026). Education at a Glance 2026: OECD indicators (Chapter A5). OECD Publishing. https://doi.org/10.1787/b4968bbc-en
- Wekerle, C., Sailer, M., Kollar, I., & Fütterer, T. (2027). The effects of ICAP learning activities on learning outcomes. Learning and Instruction, 107, Article 102482. https://doi.org/10.1016/j.learninstruc.2026.102482
Colorado HB26-1223 sources:
0 Comments