Beyond grep: The case for a context-rich AI coding harness

Augment Code has made a substantially different bet, although the two companies are not necessarily testing the same interventions or optimizing for the same outcomes; its product pre-indexes the repository using embeddings, a retrieval model, and a vector database, then retrieves conceptually relevant code. So to get the other side of that discussion, I spoke with Vinay Perneti, Augment Code’s VP of Engineering. Perneti briefly described Augment Code’s differing approach, offered his own team’s evaluations of the advantages, responded to some of the arguments for a leaner harness, and shared his perspective on some of the concerns developers have about AI tools and agentic workflows more broadly. This interview has been edited for length and clarity. Ars Technica: Can you explain how Augment Code’s context engine works? Vinay Perneti: So, if I take a step back, what is it that a particular developer is trying to achieve? They want a particular task implemented and they give it to an agent. And the interesting thing with agents today is they have a limited context window, and every time they need to essentially go get all of the context needed for that and then work on it. There’s two approaches to context. One is grep-based. Claude Code and Codex and other agents have done that. The second is the semantic retrieval bit. For Augment, we’ve always taken the semantic route, and I’ll describe the building blocks of that. I would say there’s two core pieces. One is, we have an embedding and a retrieval model pair that are working in the system, and then you have a vector database and an entire, highly optimized back-end system that makes it possible to retrieve in sub-milliseconds. Ars: Does it have an advantage in one kind of code base versus another? Perneti: Yeah, it actually turns out the advantage comes in large, private codebases. And here is why I say that. For all the public, open source repos where most of the benchmarks are run, every single model has basically memorized the repo. These models are large enough that they’re actually able to memorize the whole repo. So when you’re trying to get something done, the models kind of already know where to look so they can get to an outcome quickly. Whereas when you’re doing this in a private repo, a model has never seen that repo. And now the iteration loop for finding the outcome is much longer, right? If you have a semantic understanding of your entire private repo, you can ask a question, and you get to those outcomes much more quickly. Ars: What do you say to the notion that models are improving so fast that building anything with any assumptions at all makes no sense? That’s the argument for a super lean harness or not doing any of this, is just that you don’t know where it’s gonna be in six months or twelve months or whatever as the exponential goes up. Perneti: There’s a lot of truth to that, but I think you have to think about it from a couple of different perspectives. At Augment, the way we’ve been thinking about this is there’s two ingredients that you need for higher quality outcomes, intelligence and context. When models are getting really good, intelligence is going to get exponentially better, no doubt about that. But just because they’re more intelligent does not mean they have the context. Now, a person can get the context that they want by spending the tokens on it. And that’s where the second dimension comes, which is what all of the engineering leaders are asking right now, is cost. How much of your token budget is going towards context gathering and producing the right outcomes? Are you using the model the right way to get the highest quality outcomes? And that’s where I think harness design and context matter. So to me, it’s a combo of intelligence and context, and then it’s a systems engineering problem at that point: how do you put the right energy into each of these verticals so that you get the most optimal outcome with the least cost? The second thing… tech debt is actually very real, by the way. One of the patterns that we noticed internally at Augment—we are obviously very agent-forward—is agents are very good at duplicating code… so we found that we’ve had to do focused sprints on reducing tech debt with agents. The beauty about that is that you can say you can come up with a spec that talks about what it means to reduce tech debt, and they’re really good at executing that. So I think that is the way to tackle that part of it. And the second part, you talked about cost. I think that there’s an interesting point there. If you believe that this is the way of working moving forward, then how do you get the best outcome for your tokens? And that’s where I think things like context engine make a difference. To me, picking the right model for the right task makes a huge difference. The second thing—this is much more a philosophical, forward-looking bit—is, as these model capabilities continue to improve in exponential and open source models are going to keep up, there’s going to be a point where I think the proportion of tokens that’s going to frontier labs, to open-source models, is going to start tipping, where your most difficult problem, you’ll still throw at a frontier model, but the coding step that I described—like I know exactly what needs to get done and I know I described that in a spec—an open source model might be able to get that done for you. At that point, your cost drops quite a lot, right? So you need a system that allows you to work this way, and I think the cost will just kind of take care of itself, is how I see it. Ars: Thank you. I appreciate you making the time. Perneti: Yeah, I really enjoyed the chat, Sam. Both Wu and Perneti agree that they expect frontier models to continue to improve at or even above the current exponential, at least for the next year. (Neither explicitly addressed a longer time frame.) Both agreed that this is fundamentally changing the way software development is done for many organizations. The practical dispute, such as there is one, is whether context should be assembled ahead of time or rediscovered during each task—especially with large, private codebases. They’re also sometimes measuring and optimizing for different things. Wu specifically said that Anthropic did not find “a measurable improvement in performance” from “a few LSPs available.” It’s important to note that what Wu was describing when she said that (an LSP, or language server protocol) is not as broad as what Augment Code’s harness does. When Perneti and Wu describe evaluations of tools or systems that go beyond grep, they are not always talking about the same thing. That is one possible reason (among others) why they come to different conclusions. One thing Perneti was able to speak to that Wu did not is the possibility that frontier models like Anthropic’s Opus or Fable may become too expensive to remain the main or only models in developers’ and organizations’ agentic workflows, such that smaller or open-weight models might become more widely used. As the compute crunch continues, some open-weight models—including those small enough and sufficiently quantized to run on local hardware, or at least on hardware maintained in-house by the organization—now approach relatively recent frontier performance on some coding tasks. It remains possible that they, too, will continue to improve at a rapid rate, such that they become viable for coding agents in many teams and projects. Teams may increasingly reserve expensive frontier models for the hardest problems while routing more routine, well-specified work to these cheaper or locally operated models. Regardless of which approach is more dominant a year from now, more autonomous workflows do not eliminate the need for engineering judgment. If anything, it may be more valuable, not less, as the increases to the speed and scale of production make bad decisions carry higher costs.