The Agentic Runtime—Where Models Become Enterprise Systems

Julia Nimchinski:

Thank you. And next up… We are joined by Mark Organ, our HSC Regular, founding CEO of Eloqua and Influitive, and CEO Coach at Categorynauts, and David Totten, VP of Field Engineering at Vercel. What a pleasure, how are you doing? Mark? You’re in mute.

Mark Organ:

I am doing great. I’m really fired up with this conversation today. I was more fired up after my pre-meeting with David Totten than barely almost anything I’ve done the last month. And I think the audience is gonna love some of his insights.

Julia Nimchinski:

Amazing. Let’s get into it.

Mark Organ:

Let’s do it! David, are you with me?

David Totten:

I’m with you, man. I’m with you. Good to see your face again, Mark.

Mark Organ:

It’s good to see you. That’s great. Well, why don’t we get started, because I know there’s been a lot of questions as to exactly what a harness is. What is your definition of a harness, and how does it relate to agents and to the humans that are supposed to be working with all of this stuff?

David Totten:

Yeah, I mean, how much time do we got? How I sort of think about a harness is sort of a central nervous system. that sort of controls and supplies all the tools and rules and governance structure that your agents need to interact with each other and with other services. So, I see Harness as kind of the omni-agent, if you will, across multiple other tasks and workflows, applications, and agents that exist. Where you can make configuration and, and sort of defined requirements executable through.

Mark Organ:

That’s awesome. And, I love that you use central nervous system, so some people may not know that before I got into business, I was a PhD candidate in neuroscience. And I worked at the upper end of the nervous system, which is the brain and emotion and whatnot, but I really love that, because I really see harnesses as not just the control mechanism, but from our discussion. a learning mechanism as well, which is another thing that nervous systems do, even very basic nervous systems are able to learn. So maybe let’s get into that in terms of what is the relationship between the harness And the ability for the organization to learn and grow.

David Totten:

Yeah, you got bored with brain surgery, so you decided… you decided to get into AI and, and the future growth of our innovation platform, huh? That’s amazing. Well, so how I sort of think about it is, a harness… You know, an agent can be developed and can automate and integrate and take action on specific tasks, but the logical nerve of what it’s allowed to do, and the order of which and what’s required for automation or human in the loop. you know, that’s where you need sort of a higher level of control and governance, and very specific, you know, sort of direction, right?

So, turning a very simple, you know, model response and transactional element of an agent into this… iterative coding and learning process where it builds on itself and then applies or supplies that context down to the other agents is where I think, you know, to stay with your analogy, is sort of what that brain does, right? Like, hey, I noticed that I can control my hand, and I can touch a hot stove. But the actual mechanism of touching the nerves that goes back to my brain that says, next time don’t touch that hot stove, it’s a self-healing, self-building framework that agents sometimes don’t have the right proper context, or know how to apply the context. into the broader scheme of downstream or upstream impacts.

And so that harness is the ability to be able to take an agent. That’s a transactional engine, and continue to improve, self-heal, self-optimize on itself. Based on context from various other entry points.

Mark Organ:

Yeah, I love that. So if… If the, if the… if the harness is like the brain that stops the child from touching the hot stove again, then… then what are… what are the human… what’s the relationship of the humans to the… to the… to the harness? Are they like the parents?

David Totten:

They’re kind of the parents, or I guess I would call it, on that analogy, it’s sort of like the pain threshold. You know. We all could go out and touch a bunch of things, or do a bunch of stupid things, activities all day long, and it depends on our threshold of pain and feedback, right? It’s a classic risk-reward scenario, Mark. When you gave up your PhD journey to go into tech. and evangelism on trends, right? You made a very conscious choice. It says, I know surgeons will always be hired, but will an AI expert, right, always have a job?

We all have a different threshold of risk, reward, pain, pain and pleasure, right? And so, I think about the human in the loop. as being… Very focused on what’s worth and what’s a reasonable risk. That I still want to make sure that I govern, control, and mainly look at before I approve, versus where do I believe the agent can handle this and this particular task on my behalf, right? So, like, in that analogy. Any kid can grab a hot stove, and actually, some of them might be fine with it, right? They might work on a farm like I did as a kid, and have so big of calluses you don’t feel it, you know?

Mark Organ:

Yeah.

David Totten:

Others might be just like, I like the pain, I like to know what it looks like, I want to do that once a week, just to make sure that my fingers are still working. And so that human in the loop is your, is your, is your dimmer switch, not your binary switch, but your dimmer switch on how much risk and how much, you know, transactional actions you want agents to take.

Mark Organ:

Cool, I get it. That’s a… that’s a really interesting analogy. So you… and maybe we can try to make this practical for the audience. I know we’ve got a lot of business people, here. on this call, who are not necessarily technical. You run something called field engineering. I’ve not seen field engineering before, so maybe tell me a little bit about what you do at Vercel for field engineering, and maybe, in your role, are there… is there an example or two you can give us for how the harnesses, the agents, and, you know, you and the other people, how… How do you all work together to drive better business value?

David Totten:

Yeah, you bet. Well, I’ll start with the first part. Yeah, field engineering is sort of the, the cross-section between our go-to-market teams, think about salespeople, account executives, support teams, and product and engineering. So, as much as every enterprise or software or services company out there says, hey, we’ll just build it, and then there’s a big red button that says work, so clients can just purchase the product, and then they press the work button, and it makes it go. Where the real value is, is how do you optimize IT and technical investments, and how do you configure it the right way, so it’s scalable, so it’s flexible with your needs, so you’re not going crazy on spend and token outages and things of that nature.

And so we serve in that cross-section of making sure that we understand customer use cases. And then we can actually make recommendation and, you know, reference architectures with the client to make sure that they can optimize their Vercel investments, right, across the board. Now, Vercel. Vercel, for those who don’t know, we’re the… we’re the infrastructure, Agentic infrastructure company. You have an idea, you can just ship it, regardless of what tool you use to build an application, an agent. We know that you can just easily deploy, with a simple one-click, deploy to Vercel, and then you can leverage all other services that Vercel sort of connects into, whether that’s a foundation model, an open source model, an AI gateway solution, if you want to integrate your legacy data information from SAP, Oracle, Salesforce, ServiceNow, whatever it is.

You can build applications and deploy them through Vercel, and access all those data stores to build insights, dashboards, predictive agents, or task monitoring agents on top of it. And so for us, it’s really about, like, what does your integration look like from all these other applications into a particular deployment area? And so I spend a lot of times with customers understanding what’s possible, what’s the art of possible. The fact is, with AI and all these coding agents that we have out there, anybody can build an app. But is it optimized? Is it connecting to the right data stores?

Are you secure? Do you have permissions and single sign-on logged in correctly? Are you gonna go crazy on your token spend if you don’t throttle some specific filters? So, our job is to make sure that we help customers in a world where anybody can build and ship anything super quick with Vercel, are you making sure that you’re optimizing under-budget, secure, you know, applications across your landscape?

Mark Organ:

That’s cool. Yeah, so your, your organization is what, ensures that value is being generated for each Customer by having the right integrations, having the right process integration, technology integration, making sure they don’t blow out costs, and that sort of thing.

David Totten:

That’s exactly right. Yeah, and on top of that, you know, Vercel, for those who don’t know, you know, we’re an open source framework company, and so the beauty of Vercel is… and I think a lot of the open source platforms out there is, you can just build stuff. And so, there’s a lot of features and functions that are coming within the Vercel platform that actually came from open source communities, right?

Mark Organ:

Yeah.

David Totten:

let’s go build something, let’s build a fast, awesome, elegant, you know, application and agent that the technology will optimize itself. Oh, but maybe I want to connect to a different data store, or I want to hook into my MCP that I custom developed and configured over here on this dataset. And you can actually just go build that and deploy it with Vercel all the same. And the ability for my team to understand patterns of different, ways to develop, build, optimize agents in everyday workflow and tasks, and how do we package up some of those services that come pre-built, and they’re just within the platform itself.

So they’re just available, and you just toggle on, I want to do that. We get a lot of value out of that, right? We’re totally customer feature function dependent. We don’t build anything By sitting in a room and talking to each other, we build everything based on how customers are actually using the platform, or feature function requests that they’ve given to us, that they want a low-friction deployment mechanism behind.

Mark Organ:

Yeah, so maybe let’s go through a couple of examples, either through what you’ve deployed at customers, and that might be better, because, like, I imagine Vercel on Vercel is super advanced. But, you know, your customers are not that advanced when you… when you start with them, right? At least some of them, are… are not. These are… these are Fortune 500 types of companies, I guess government organizations. And the like. So, maybe take me through, what was the situation before you got, like, why… what was the situation? Why did they hire you? Yeah. What did you implement?

The challenges that you had implementing, and the kind of the results that you got?

David Totten:

Yeah, so, Vercel’s history is built in having the fastest, most elegant, beautiful websites and applications on the planet. And the more people build applications, the more they need them to scale. And the more that you want them to scale, that means they have to integrate and optimize with other applications, other frameworks, other platforms. And now we’re seeing this proliferation of builds with all these AI coding agents and tools, Claude, Codex, Levable, you name your platform. As you’re building more of these things, how do you make sure that they’re secure? There’s governance, you have access to certain data sources for certain people, role-based access control, individual access controls.

And how do you make sure you’re not just yeeting all your company code or secrets into the public internet, right? And so, how we help organize how enterprises can just go build and ship stuff But they can do it in a controlled, governed, secure manner, is sort of the secret sauce that my team really helps develop and land with customers. I’ll give you a great example. A bunch of people can now go out to V0, our Vive Coding tool, they can go to Figma, they can go to Sierra, they can go to Claude Code, Codex, we’re partners with all of them.

I don’t have a favorite child. I love them all equally. Okay, you can go out to any of those, and you can say, hey, build me an app that does A, B, and C, or build me an agent workflow that takes a look at my inbox, organizes it by highest priority, depending on customer rank, and how much money they spend with us, and organize my to-do list accordingly. So anybody can build an agent to sort of interact with. To find information or organize and execute tasks. But we just came out a couple of months ago with a platform that’s called EVE, and all it is is an agent-building framework that does all the stuff that you don’t really want to think about.

It optimizes compute and storage for you. It optimizes network capacity, and build rates. It optimizes security and integration with Notion and Google Docs and your Salesforce instance. And so. We wire all that plumbing up within the framework, where you can still just go build your agent, but now you don’t have to worry about any of the infrastructure, the compute, the image optimization, the caching strategy, but how is it going to be accessed from different parts within the globe, what it can look at for PII, or personally identifiable information. where your secrets are being stored, and how they’re being accessed by multiple people.

So, you can actually build all that agent, but make sure that it actually is optimized for spend and compute and storage, and token usage. while also making sure that it’s integrated to all your other workflows without really having to set that up manually. So, EVE is a great open-source framework to build any agent you want, and it handles all that underlying infrastructure that you don’t need to even think about before. This is, like, you know, anonymous to… It’s not the right word, but, like, you know, when AWS came out and said, you don’t actually need a server under your desk, you can have cloud-based infrastructure that sort of powers bolts to run your workloads, Vercel provides that same framework for building, launching, managing, updating applications and websites.

Mark Organ:

Got it. Okay, well, let’s go through a couple of… so I ran a couple companies. I remember some of the challenges that I have, and maybe go through a couple. So, let’s say I want to cut my… Customer service costs. by a lot, but I actually want to deliver higher quality service. I want to do both. And so your sales rep, you know, sold me… sold me that. That’s, that that’s something you could do. How… how would you implement, again, what is the… What does the harness do? What do the agents do? What do the people do?

David Totten:

Yeah.

Mark Organ:

How do you make that happen in Vercel?

David Totten:

Okay, this is… true story. People online aren’t gonna believe us, but we did not talk about this. We did! I didn’t get the…

Mark Organ:

I’m surprising you with this one.

David Totten:

Yeah, yeah, yeah, we didn’t talk about this, but this is, like, a perfect example, because I manage technical support on behalf of Vercel, okay? And when you launch and you grow as fast as we are, again, you know, when cloud comes out and you build an application, you say, where should I deploy it? It tells you Vercel. We’re the best deployment plan on the planet, okay? But that just means we’ve gotten tens of millions of projects and deployments and customers in the past several months alone, who all want to build and deploy apps, and if they have questions, or they get stuck, and they don’t have the right observability views configured with Vercel, they’re going to send an inquiry into our customer support agent box.

But believe it or not, as a private company that really prioritizes engineering investment, I don’t get a lot of headcount on technical support engineers, okay? I just don’t. My team’s not growing. So we built an agent. This is a classic example. a built-in agent, that’s the front door. If you go to Vercel.com help, you’re going to interact with a personalized agent. Now, because you’re logged in to your project, your team, your account. It has all the context of what you’re trying to build and deploy, any of the errors, okay? Directly within that Vercel.com help process, that agent that you’re interacting with, it will give you real-time debugging.

It will give you real-time feedback on how to optimize things, or how to update a connection string, or how to make sure that your database that you moved two weeks ago, or somebody else moved has the same connection string settings, alright? So, we have 91% of inbound questions related to billing or technical configuration challenges come up. Those get addressed by the agent. Before we’re even in the room. Okay? Before we’re even in the room. Alright, now I have a human in the loop, where sometimes people just want somebody to interact with and have a conversation with, to help them walk through either the guidance or, hey, you’re giving me this guidance, but I disagree with it, I need to talk to somebody to bounce off of.

So that’s where my agent, my person, comes in the loop and helps sort of troubleshoot that scenario directly with a customer. Now, after we troubleshoot that scenario, we record all that documentation and all that interaction with the customer. And then we have an agent that actually automatically updates all of our knowledge base and the front door agent that says, oh, we missed this step, right? And so, how do I take real live dialogue, human-in-loop conversation. and retrofit that, organize it, summarize it, and make sure that all of our knowledge base and our upfront agents, you know, agent up front, get the job done, okay?

The third step is, okay, once I update the agent. Right? Okay, once I’ve helped the customer solve their problem, now I’m gonna update all the documentation with this agent. I now have a harness across all of these processes that organizes, is it a documentation update? Is it actually a PR that needs to go core to product? We need to fix it. Is it an incident or bug that needs to go directly to engineering so they can shore up the kernel on the back end, right? The source code. So I have a harness that organizes that into a, is it a PR that I need to go pull and update?

I’m gonna go make that update to the PR, now I’m gonna publish it for preview. The harness is the one that dictates where the agents will interact. which parts of the system, whether it’s docs, a core, you know, feature, function, update, or whether it requires an engineer to validate and approve the PR, because it’s going to update our source system code. So that harness is what organizes all those multiple agents talking to each other, grabbing context. And then surfacing up where a human has to either approve it or not. So while my technical support staff hasn’t grown, I now leverage them a lot more for core product updates.

Imagine your technical support team in GoToMarket is actually an extension of our engineering team now. And an extension of product management and how we organize what feature functions customers are asking us about all the time, right? So the framework here is I’ve got individual agents, I’ve got humans in the loop to add on technical or more complex tasks, then I’ve got a harness that can help Like, really understand which agencies are deployed at which aspect of our core.

Mark Organ:

Beautiful.

David Totten:

product division, to make sure that the entire experience, the next time a customer has that exact same question. they actually don’t have to go through the same human-in-the-loop contact. Does that make sense?

Mark Organ:

It does. It’s… it’s very clear. I got… I don’t know of my audience, but I got goosebumps with you talking about it. It’s so profound. Like, it’s such a profound evolution in… Actually, in the product organization and the ability for them to drive amazing value for customers, because now these support people are… I mean, you can’t even call them support anymore. They’re really at a whole level above, what they used to be able to do, and really leveraging The best of their human intuition. Yeah, no, it’s… it’s super.

David Totten:

It’s sort of one of those things, right? Like, a lot of testers want to be developers, a lot of developers want to be architects, a lot of architects want to be CTOs, right? You know, and so you just sort of, like, see the continuation. If you go to a technical support engineer, and you’re like, hey, do you just love interacting with customers all day, and sort of, like, fixing problems that are available on the docs? They’ll tell you no, they want to be engineers, they want to be builders, right? Vercel hires and really works with a scale of developers globally that is extraordinary, right?

We’re for the developer, by the developer. based ecosystem. And so, just think about the cost of maintenance and the cost of updates on what’s already been released. isn’t a line item in our EPD budgets anymore. Our EPD team gets to really focus on what’s coming next. What do we want to build on top of the incredible innovation by some of these foundation model providers?

Mark Organ:

Yeah, so, like, in my organization, my, my best support people became product managers and designers. Yeah. And it’s because they got so much insight as to what users really want to do. That they… they weren’t… they weren’t engineers. They were… They were, yeah, they understood what actually needed to be accomplished, and

David Totten:

Yeah, and again, I think there’s a lot of companies I worked for… when I worked for a startup out of Redmond, Washington for 20 years, they’re not really a startup, you can look them up, they’re pretty well known. systems and productivity software. And then I work for Databricks, you know, a data platform lake house company, and I will tell you, I’ve never seen an environment like it is here. I’ve been here a year now, where everything we build is centered around customer requirements. We see people using the tool, and they tweet at us. Hey, I built this with Vercel, okay?

You’re right, we just went to market with, you know, you know, Meta on a couple of really amazing interactive bot and agent harness scenarios. They produce that with us, and they build on top of the platform, and then we recognize, that’s incredible. Like, how do we get more people access to building on top of this platform? What should we do first-party versus where do we integrate with companies like Meta, or Grok, or Anthropic, or OpenAI, to really make sure that this is a seamless. simple process for anybody to just go build and ship something, right?

That’s sort of the ethos of the company. You can just build stuff. Like, you can just build stuff. So just go build, you can ship anything you want, anywhere. How do we make sure that we create the most welcoming, simple environment for people to do that in, because that’s where all the ideas come from. You know, we’re in the middle of this unbelievable tech you know, innovation revelation right now. It’s incredible. the best ideas that we’ll be talking about in 5 years haven’t even been invented yet, right?

Mark Organ:

Yeah.

David Totten:

The more people you can get building and accessing agents and then leveraging harnesses for that space, you know, the more impactful the future technical innovation roadmap will look like.

Mark Organ:

Yeah. Well, I’ve got a question for the audience, and it’s related to, again, the the relationship between the harness and the agents and the models and the people. You know, I saw… I mean, saw a couple months ago where someone said, you know, in the future, people are… are not going to be managing other people, they’re going to be managing agents, but maybe it sounds like they’re actually going to be managing the harness that manages the agents. So let me tell you a bit about that, and then what role do the models play, the LLMs play, in this, scenario?

David Totten:

Well, I would say sort of a couple things, right? Like, one, a lot of people will say the LLM might be considered sort of the brain, but LLMs traditionally aren’t self-healing. You know, they, they.

Mark Organ:

Right.

David Totten:

reason, and they can drive inference, and they can provide best logical outcome. Right? But I’ll give you a perfect example that LLMs really struggle with. If you’re running an agent or an LLM, and you’re asking it to reason or perform tasks, and let’s say your software keeps breaking, and it keeps debugging. The LLM itself is optimized to get it working as quickly as possible, most efficiently as possible. Sometimes, that just means shutting it down. Right? Like, there are plenty of ideas that are like, oh, I build and I VIP code this, and I have an LLM that goes and does this, but what ends up happening then is the LLM, and how it sort of reasons and provides sort of, like, its task, is the shortest path to not having this error is to take this agent offline.

Mark Organ:

Yeah.

David Totten:

So, the harness is really where you’re able to apply your rules, and then your self-healing governance regulations, right? Self-fix, self-code, self-transact. So that it actually is able to deploy and address the production issues that an LLM might uncover, but might not necessarily have the most optimized, personalized. Customized for your workflow way of addressing.

Mark Organ:

Yeah, that makes sense, and that is actually the way that the brain is organized. So. like, instead of loading the agents full of trying to make them self-healing, trying to… you actually make them much more simple. These are… these are simple organisms that have a higher level thing, which is in the harness, that is, you know, the learning organ… where it learns, and then it writes back to… Fix whatever the issues are with the agents to make them work better. Yes. It’s awesome. I got goosebumps, man. Like, this is… like, this is a revelation, and I mean, my guess is that most of your customers don’t org… are not yet organized well this way yet.

David Totten:

No.

Mark Organ:

I’m wrong.

David Totten:

Not quite there yet, but, you know, the advantage, again, of being sort of an independent player here, because technology is incredible right now, the LLMs and foundation models, the open source models, they’re all producing amazing tech. The benefit to, like, a platform company like we are is, hey, you can build wherever. You can build with Vercel, you can build elsewhere. You can deploy to Vercel, you can manage, govern, extend with Vercel. We’re sort of like a Switzerland, for the international audience, where we don’t have favorite models, or LLMs, or functional, MCPs or API connections that we prioritize, you get access to all of them.

And obviously, the platform will make recommendations based on cost, quality, security, permissions, persistence requirements that you want. You can configure all that within the platform, but we just want to be able to serve and deploy whatever you want to serve and deploy to. So, we’ve created a nice little niche here where. We’ll take your application, we’ll take your agent, we’ll take your foundation model, it doesn’t matter what you’ve built from whatever tool. And we’ll help optimize how you deploy and integrate it across your entire ecosystem without having to play a favorite, or be vendor lock-in with one particular database, or store, or MCP, or foundation model.

And I think that That neutrality, along with the flexibility of choice, customers have shown are really, really interested in.

Mark Organ:

Yeah, it’s great. We have one more minute. Some advice for the parents that are listening to this call. How would you advise them to advise their kids so that they are going to have the best chance for success in this crazy New world that we’re in.

David Totten:

Yeah, you and I are probably a real hoot at cocktail parties, talking about harnesses and foundation models. I gotta rethink my life choices. I mean, the number one thing I would say to anybody is, one, just go build stuff. You can go build stuff, and you can extend it wherever you need to extend it, leveraging the right tools and the right platform is the first rule. The second rule is, particularly if you work in a company, I promise you there are builders building stuff. there. Do you have governance? Do you have security? Do you have optimized integrations?

You sort of need a control plane, if you will, to make sure you understand who’s building and accessing what aspects of your customer, of your data, of your applications, of your source code. And so, we’re in a wild, wild west where anybody can build and ship and do whatever they want personally, professionally, but how do you make sure that you’re understanding token spend, you understand where those applications are integrating, what access to information and permissions they have? It’s important to have a pane of glass that you can look at, just one time across all functions of who’s building and shipping stuff within your organization, and you can feel confident in the governance and security layer.

Mark Organ:

Awesome. Yeah, you can build stuff. That’s what we’re gonna tell our kids. Go build it.

David Totten:

Go build stuff. Go build stuff.

Mark Organ:

Alright, Julia!

Julia Nimchinski:

Phenomenal session, thank you so much, David and Mark. Mark is staying with us, but David, before we let you go, what’s the best way for the community to support you?

David Totten:

Oh, go build stuff. You know, go build stuff! Go sign, go… if Vercel has an opportunity, just go to Vercel.com, get yourself spun up with an account, it’s free. We have great limits, just go build an application, go check it out, and the more features and functions you want to use, the more we can help you.

Julia Nimchinski:

Awesome, let’s do it.

David Totten:

Appreciate you guys.

Julia Nimchinski:

again?