What Actually Worked: My Open-Source Coding Model Comparison

What did I test?

I used OpenCode only as the common test harness. The real focus was the output from GLM 5.2, Kimi K3, DeepSeek V4 Pro, GPT-OSS, Gemma and Qwen 3.5 across four practical tasks: a colour-mixing tool, an OCR analysis CLI, direct image analysis and a space-shooting game.

This was not a synthetic benchmark. I judged whether each result was usable, whether the model stayed within my instructions, how much correction it required and what the observed session cost looked like through Hugging Face Inference. Scores are my personal ratings out of 10; costs are recorded test-session costs, not universal model prices.

What did the results show?

Model	Colour mixer	OCR workflow	Space shooter	My practical reading
GLM 5.2	9/10 · $0.07	7/10 · $2.00	10/10 · $0.36	Best overall coding control and finish quality
Kimi K3	8/10 · $0.50	8/10 · $0.44	8/10 · $0.48	Reliable for heavier work, with less polish than GLM in the game test
DeepSeek V4 Pro	7.5/10 · $0.70	4/10 · $0.17 (Libraries issue)	7/10 · $0.20	Cost-effective, but more tuning and correction were needed
GPT-OSS	4/10 · not recorded	6/10 · not recorded	5/10 · $0.02	Weak agentic consistency, with one useful OCR surprise
Gemma	3/10 · not recorded	9/10 · $0.02 for direct image analysis	4/10 · $0.10	Better suited to conversation and image understanding than coding
Qwen 3.5	5/10 · $0.06	Not recorded	6/10 · not recorded	Economical, but instruction control and customization need work

The Gemma OCR result used Gemma 26B with direct image input; the colour-mixing test used Gemma 31B. I have kept unrecorded values explicit rather than manufacturing precision.

How did each task change my view?

Colour-mixing tool

GLM and Kimi produced the most complete colour-mixing experiences. Both included detailed controls and useful information for the user instead of stopping at a basic visual demo. GLM scored 9/10 at an observed $0.07, while Kimi scored 8/10 at $0.50.

DeepSeek produced a respectable 7.5/10 result, but it did not give me the same sense of control. Qwen was inexpensive at $0.06, yet the lower score reflected the extra customization still needed. The useful lesson was that low token cost did not automatically produce the lowest completed-task cost.

Kimi K3: GLM 5.2: Deepseek-v4 pro: Qwen 3.5

OCR analysis CLI

The OCR task was more nuanced than its scores suggest. Kimi recorded the highest coding-workflow score at 8/10, but GLM and GPT-OSS stood out in how they approached the tool creation. GPT-OSS was unexpectedly capable here despite performing poorly in broader agentic work.

DeepSeek stopped short of a finalized result because of a package issue. That made its lower request count and $0.17 cost less meaningful: an inexpensive unfinished tool is still unfinished. GLM reached a stronger result, but its run required 55 requests and cost $2.00, compared with 21 requests and $0.44 for Kimi. For me, this was the clearest reminder to measure completion alongside cost and iteration count.

Test Image: GLM 5.2: Kimi K3:

Direct image analysis

Gemma was the clear task-fit winner when I supplied an image directly in context. It scanned the image successfully, scored 9/10 and cost $0.02 in the recorded session.

I do not read this as evidence that Gemma is the better coding model. I read it as evidence that model selection should follow the workload. For summarization, conversational understanding and image analysis, Gemma gave me useful output without forcing a coding-oriented model into the wrong job.

Gemma 4:

Space-shooting game

The final game test separated interface generation from actual software behaviour. Several models created acceptable-looking screens, but movement, controls and game logic failed to hold together.

GLM was the clear winner with 10/10. It followed the instructions, asked for refinement where decisions mattered and continued until the game felt polished, including sound effects. Kimi came next at 8/10. It completed the requested scope effectively, although it did not reach GLM’s level of refinement. The remaining models showed useful UI instincts, but their gameplay logic was not dependable enough for the final result.

Deepseek-v4 pro: Qwen 3.5: GLM 5.2: Kimi K3: GPT OSS:

Why did the cheapest model not always win?

My biggest takeaway is that instruction obedience is part of the cost model. DeepSeek and Qwen were among the most economical choices, but they also needed more tuning for customization. Every correction adds prompting, context and developer attention. I think of this as a control tax: the listed token price may be low while the cost of reaching an accepted result rises.

GLM and Kimi were more likely to stay within my intent and ask questions before making consequential decisions. That reduced rework. In my OpenCode setup, GLM also needed output-token capping and was effectively limited to 16K output tokens. Even with that constraint, it delivered the strongest overall coding result. I therefore treat the cap as an operational limitation to manage, not a reason to dismiss the model.

GPT-OSS needs a narrower conclusion. It did not work reliably enough for broad agentic testing, so my test coverage remained limited. Its OCR result was encouraging, but I would not generalize from that single success.

Which open-source model would I use?

For my workflow, GLM 5.2 is the default choice for general coding with clear guidance. It gave me the best combination of instruction control, refinement and final output.

I would run Kimi K3 alongside GLM for heavier workloads. It was consistently capable, completed the requested scope and remained easier to direct than DeepSeek or Qwen.

I would use Gemma for summarization, conversational work and image analysis, not as my primary coding model. DeepSeek and Qwen still have value when cost is the main constraint and I am willing to invest in stronger prompts, reusable instructions and task-specific tuning. GPT-OSS remains experimental for me until it behaves more consistently across agentic tasks.

I also would not lock this workflow to one model vendor. I prefer Hugging Face even with context or output limits because it gives me balanced control over model choice, spending and task completion. Provider flexibility matters more to me than extracting the maximum context window from one model.

Conclusion

I no longer compare open-source coding models by asking which one has the best benchmark or cheapest tokens. I ask which model reaches an acceptable result with the least correction work.

Based on these tests, GLM 5.2 is my strongest general coding option, Kimi K3 is the practical parallel choice for demanding work, and Gemma is the better specialist for non-coding and image-led tasks. My next step is to keep the same small task suite and track completion, score, cost and correction count whenever a model or inference route changes.