We're Getting AI Agent Skills All Wrong

We're getting AI agent skills all wrong. But that's just my opinion.

Loading a skills.md file into an agent's context is just prompt/context engineering. It guides the model. It does not change its weights. This is quick and cheap. It stays limited to what the model already knows and slaps on a bit of extra knowledge.

True skills come from real training. Fine-tuning adjusts model parameters on targeted data. It creates deeper expertise. It delivers big gains. Fine-tuned models often gain 20 to 30 percent accuracy in tasks like code generation or medical analysis.

Humanoid robots show the real thing. They learn through imitation and reinforcement from actual experience or videos.

Tesla's Optimus, driven by Elon Musk, learns tasks from first-person human videos, even from internet sources. It sorts colored blocks. It folds shirts. It handles battery cells in factories.

Recent demos show walking, squatting, balancing on one leg, and household skills like cooking. Tesla uses vision-only training and neural networks trained on massive vehicle data. This scales learning quickly.

Anthropic's Claude offers code skills. These contain markdown instructions, scripts, and references for repeatable coding or debugging workflows. Claude Opus models lead agentic coding benchmarks with huge context windows. This still leans on prompting more than embedded training. Fine-tuning options do exist to boost specialization.

Agent teams take specialization further. You can build crews of agents with clear roles: research, validation, strategy. They handle complex tasks like analysis or automation. They mimic human teams. Most still rely on composable prompts instead of trained behaviors!

One serious problem exists. Prompt-based skills have weak security.

They lack hard permissions and granular access controls. Prompt injection attacks can manipulate agents. Attackers bypass safeguards. They leak data. They trigger unauthorized actions. These flaws turn helpful tools into real threats as discovered within the OpenClaw skills repo recently.

Calling prompt hacks skills confuses quick tricks with genuine learning.

AI is moving toward hybrids: prompting plus custom fine-tuning or reinforcement loops.

I'm excited to see optimized AI model utilization in complex workflows without the overhead cost that come from using frontier models for simple granular tasks.