Most conversations about generative AI in the workplace begin with productivity. A new three-month field experiment involving 133 patent attorneys offers a more uncomfortable lesson: the people who benefit most from AI on the job may also have the most to lose when the tool is no longer there. For HR leaders and managers, the findings challenge the assumption that short-term performance gains from AI automatically translate into a more skilled workforce.

The immediate boost is real, and juniors gain the most

The study put AI-assisted drafting and analysis tools in the hands of experienced and early-career patent attorneys. Junior practitioners, who typically spend years learning how to structure claims, search prior art, and anticipate objections, saw the largest immediate improvements in both speed and quality. The tool acted like a highly capable colleague, smoothing over gaps in experience and helping them produce work that looked closer to the output of a seasoned professional.

Senior attorneys also benefited, but their gains were more modest. Because they already possessed much of the reasoning the AI was supplying, the tool saved time but did not fundamentally change the quality of their judgment. For juniors, however, the AI often provided the missing logic, the standard phrasing, or the strategic framing they had not yet internalized. That is precisely why the short-term results looked so dramatic.

When the assistant disappears, a hidden gap emerges

The more striking part of the experiment came when the AI was removed. Senior attorneys largely retained the improvements they had shown while using the tool. Junior attorneys did not. Once the AI-driven scaffolding was gone, much of the apparent skill gain faded with it. Their performance reverted toward the level expected for their experience, suggesting that the AI had been masking capability gaps rather than closing them.

This pattern points to what learning specialists sometimes call shallow automation: a worker can produce an excellent result with the help of a system, but the underlying mental model remains underdeveloped. For junior employees in knowledge work, that is a serious concern. The early years of a career are meant to build the judgment that later allows independent, high-stakes work. If AI quietly substitutes for that struggle, organizations may discover too late that they have a cohort of professionals who are highly productive with the tool enabled but fragile without it.

Rethinking how organizations develop early-career talent

The implication is not to ban AI from junior roles. Doing so would ignore its power to accelerate learning when used deliberately. Instead, HR and learning teams should treat AI-assisted output and actual skill development as two separate metrics. One useful approach is to have juniors attempt a task independently before consulting AI, then compare their reasoning with the tool's suggestions. Another is to require periodic no-AI assessments so managers can see where genuine progress is being made and where AI has been doing the heavy lifting.

This matters even more in remote and hybrid environments, where managers often observe finished work rather than the process behind it. In an office, a senior colleague can overhear how a junior thinks through a problem. In distributed teams, that visibility is easily lost. Workforce platforms that combine productivity tracking with structured skill assessments can help separate tool-assisted output from durable capability. At XMF, we see a similar need among distributed teams: the systems that measure performance must evolve as AI changes what performance means.

For organizations hiring and developing junior talent, the lesson is clear. AI can make early-career employees look more experienced than they are, but that appearance is not the same as competence. The companies that thrive will be those that use AI not as a crutch, but as a deliberate part of a learning path—one that still requires people to struggle, reflect, and eventually work without the scaffolding.

Originally published by XMF, inspired by publicly reported industry news.