The Skills Gap Software Engineers Miss in AI

Marina Wyss, a Senior Applied Scientist at Amazon, divides software engineers looking at AI into two camps: those paralyzed by the misconception they need to master transformer architecture and calculus, and those convinced it's just another API integration. Both groups are wrong, she argues, but in ways that reveal something useful about the actual skills gap.

The paralysis makes sense given how AI engineering is marketed. The field wraps itself in the language of research—transformers, attention mechanisms, gradient descent—as if building production AI features requires understanding the mathematical foundations. Wyss dismisses this directly: "When most software engineers hear AI engineering, they immediately think about deep learning, the transformer architecture, and research papers. And look, that stuff is totally interesting, and it won't hurt you to know it. But that's not what this job is day-to-day."

What is the job? According to Wyss, AI engineers in 2026 work with foundation model APIs, design prompts, build retrieval-augmented generation pipelines, create multi-step agents, write evaluation frameworks, and occasionally fine-tune models. It's application layer work—exactly where software engineers already operate.

This is where the analysis gets interesting. Software engineers already know how to design systems, handle errors, manage latency, and ship to production. That infrastructure expertise transfers directly. But the engineers who think AI is just another API integration miss something fundamental about non-deterministic systems.

The Determinism Problem

Consider the mental model software engineers use for reliability: you write a test, define expected output, verify the match. When something breaks, you trace it. The system behaves predictably because it's deterministic.

AI systems don't work this way. "You can send the same prompt twice and get different outputs," Wyss notes. "You can ask your agent to complete a task and it mostly works, but sometimes it goes in circles and you're not entirely sure why. The inputs look the same, but the outputs don't."

This creates a question traditional software engineering doesn't prepare you for: Is this system good enough to ship? Does the response help users? Is it accurate? Is it safe? Most software engineers, Wyss observes, "don't have a framework for answering those kinds of questions."

The framework they're missing is evaluation. Not testing in the traditional sense, but evals—systems for measuring quality in probabilistic outputs. This includes machine learning metrics like precision and BLEU scores, human evaluation frameworks, and methods for assessing when "good enough" has been reached.

"Evals are to AI engineering what testing is to software engineering," Wyss says. "Ignoring evaluation is the single most common mistake I see from software engineers who cross over and it's the one that will limit your ceiling the most."

This matters because without evaluation literacy, software engineers can't communicate with the rest of the AI team. They can build features, but they can't participate in the conversation about whether those features work. Wyss puts it bluntly: "If you can't speak that language, you'll be treated kind of like a junior on the team, even if you've been shipping production software for a decade."

What Actually Needs Learning

Beyond evaluation, Wyss identifies several concepts without clean analogies in traditional software engineering:

How LLMs work conceptually—tokens, context windows, temperature, hallucination patterns, what fine-tuning actually does versus prompt engineering
Prompt engineering as a discipline—structured outputs, reasoning approaches, few-shot patterns, system prompts, experimentation methodology
Retrieval-augmented generation (RAG)—embeddings, vector databases, chunking strategies, retrieval quality measurement
Agentic design patterns—tool use, planning, multi-step reasoning, safety constraints

None of this requires going back to school. Wyss recommends building immediately—week one, create the simplest possible AI feature. A chatbot for codebase questions. A RAG app over internal documentation. An inbox organization agent. Use an LLM API directly or a framework like LangChain or LlamaIndex.

Then learn reactively. Why does the RAG pipeline return irrelevant results? Now learn chunking strategies and retrieval metrics. Why does the agent loop? Now learn planning patterns. How do you know if outputs are good? Now learn evals—and the learning sticks because you have a real system to evaluate.

This approach inverts the traditional education model. Instead of six months of courses before touching production code, you encounter concepts as they become necessary to solve problems you're actually facing.

The Positioning Strategy

Wyss suggests that software engineers already have the more valuable half of the skill set. Most people applying for AI engineering roles can call an API. Almost none can ship production software. A software engineer with even one or two solid AI projects is ahead of most applicants.

The transition might not even require changing jobs. "A lot of companies right now are actively looking for software engineers who can own AI features," she notes. The move might happen inside your current role if you position yourself as the person who can deliver.

That positioning requires visibility: writing about what you're building, contributing to open-source AI projects, talking to your manager about adding AI features to existing products. It's not enough to develop the skills privately.

What's notable about this framework is what it doesn't include. No master's degree. No six-month bootcamp. No requirement to understand the mathematics of neural networks or publish research. The path Wyss describes is practical, incremental, and grounded in the infrastructure skills software engineers already possess.

The question is whether companies hiring for AI roles share this definition of the job. If they're looking for research engineers who can implement papers, software engineers following Wyss's path will hit a ceiling. If they're looking for engineers who can ship reliable AI features to production, the advantage goes to people who already know how to build production systems.

That distinction—between AI as research and AI as engineering—will determine whether this transition path actually works at scale.

Samira Okonkwo-Barnes covers technology policy and regulation for Buzzrag.