Agents at Work

Julia Nimchinski:

And next up, please welcome Praveen Akiraju, Managing Director at Insight Partners, and former CEO, Operator, and Managing Partner at SelfBank Investment Advisors. And he’ll be joined by Russell Sherwin, GTM Executive Advisor, educator, and former IBM CMO. What a pleasure to host you. How are you?

Praveen Akkiraju:

Great to be here.

Russell Scherwin:

rate.

Julia Nimchinski:

Russell, take it away.

Russell Scherwin:

Well, let’s just… I am so excited for an awesome session with Praveen. Praveen, why don’t we just get started? Introduce yourself to the audience, if you don’t mind.

Praveen Akkiraju:

Sure, I am a managing director at Insight Partners. Insight Partners is a growth stage, venture fund, investing out of our 13 fund now. Prior to Insight. As Julia mentioned, I have an operating background. I used to run product teams at Cisco for a long time. I was the CEO of a cloud infrastructure company, and also ran a startup.

And once my startup got acquired, I moved to the investing side, so… At Insight, I primarily, focus on our infrastructure, deep tech investments, focusing on early stage. I’m based in the San Francisco Bay Area, and, you know, deep in the belly of the beast of all things AI.

Russell Scherwin:

I love it, and I am so excited. We’re gonna get into a great discussion around agents. real or hype, and assuming they are real, which they are, what does it take to deploy them in the enterprise? Just a bit about me, some context. As Julie mentioned, former sales leader, former CMO for IBM Watson Marketing Commerce.

I teach in UGA’s MBA program, I teach companies how to sell more effectively, and most of all, I’m trying to teach my son how to be a boy, or a man. So that’s primary. Praveen, what I found most fascinating about your background is your past and your present. Presently, you’re deeply involved in the Valley.

In the past, you’ve sold to large enterprises and have a real great appreciation for what it takes, not just to productize technology and sell technology, but to drive enterprise adoption. And, you know, I got a kind of view where a lot of companies in Silicon Valley are selling to themselves.

When we talk about agents, when we talk about enterprise adoption, I think you’ve got a really unique perspective there. Before we get into the depths of agents, let’s just start, with the definition of an agent. I mean, I just built a custom GPT. Did I build an agent?

Praveen Akkiraju:

Yeah. So, yeah, no, a great way to start. I think, and the definition of agents has evolved quite significantly. I think the first initial proto-agents, if you were, were kind of launched back in February of 2023. two open-source projects called AutoGPT and, you know, Baby AGI.

And so that’s kind of, when I started tracking this whole notion of what is an agentic. architecture, and how does that evolve? And to go even further back, I think my interest in this was primarily driven by the fact that I’d spent a little bit of time working on automation in the enterprise.

You know, as you know, enterprises have a ton of workflows. Some of them are standardized, like, you know, functional workflow, like sales or finance, and some of them are non-standard. And so, prior to this sort of generative AI moment that we’re in.

You know, we had a technology called RPA, robotic Process Automation, and so I spent some time in that. And so, my interest in agents was essentially to kind of see if we could do better than what RPA delivered to the… to the industry, right?

Which is something Where you could, you could actually have workflows that were automated as opposed to, you know, all this kind of static, coding that we needed to do in RPA. So, that’s how I kind of got started on the journey.

So, you know, for me, an agent is really, you know, at a very basic, sort of the official definition is, you know, the ability for software, to understand A task, to be able to break that task down into a set of steps.

to use tools, whether they are, you know, external tools or functions, what have you, to execute that task and then be able to interact with the end user in a way to complete the task, right? So it’s basically, at a very high level, what an agent does.

Russell Scherwin:

Let me play that back to you, because I think that’s actually one of the first times I’ve heard a real robust definition of an agent. So, an agent is something It’s the ability for software to, first of all, understand the task. or understand the objective, then two, break the objective down into tasks.

It has the tools necessary to execute that task that might reach out… reach from outside itself, and then the ability to interact with its owner or its customer.

And so the bad news is, you just told me my custom GPT is not an agent, and I think you just told 90% of the people who are building agents that they’re actually not building agents, which is okay, because we know you gotta start somewhere, and we need to start start diving in. So… but that, that was, that was fun and games.

Give a breakdown, if you would may, of what would you say is real today? I mean, you’re seeing enterprises adopting AI, maybe not agents, and you’re seeing the cutting edge in some of your portfolio companies.

What do you see as the bleeding edge of being real today, and what do you see as, you know, the norm in what’s real today in agentic adoption?

Praveen Akkiraju:

Yeah, so, you know, as we were talking about a little bit earlier, the first agent’s, sort of, you know, first notion of an agent, was back in 2023. And since then, we’ve kind of evolved significantly, right? You know, the first agents were basically wrappers around a model.

Effectively, you had some kind of a user interface and some kind of a prompt ability to manage a prompt. I think the latest prototype of what truly unlocked agentic use cases was this open claw phenomenon, early this year, if you… you and your audience are familiar with this. So, open claw did something fundamental, right?

And I think, Just to kind of maybe… double-click a little bit to contextualize this. Today, the large language models that we have, are trained, you know, extensively, 1 trillion parameters, you know, tons of compute, tons of data, and so when they come out of the box, right, from one of these frontier labs.

They come with these well-established weights, and those weights essentially determine the parametric weights determine, essentially, the intelligence. It’s a compression of all the knowledge that they’ve been trained on is represented as weights.

Now, those weights are fixed, so when you’re using a model in a workflow, you are not able to change those weights. So, let’s say you learn something new, right? You can’t change those weights. So, the only way you can affect the behavior of the model, different from the weights, is to put that Information back into the context.

So the reason why this is important to understand is Opit Claw did something fundamental, right? OpenClaw essentially was able to capture the state of the workflow. So, as the workflow was executing, it was able to capture the state of what the workflow was. It was able to then compress that workflow… those… the state.

Pick the right, you know, signals to put back into the context. Right? And so that allowed the agent to now remember what it was doing, right? And be able to execute much longer workflows than ever was possible, right?

So that was the architectural change in OpenClaw that unlocked what, you know, the sort of the latest iteration of agentic architectures. The ability to remember state, ability to compress it, pick and prioritize what needs to be put back in the context, and continue to execute, right?

So, that… that is sort of where we are in terms of agentic architectures. The best representation of agentic execution today is coding engines, right? You know, today.

depending on who you talk to, if you talk to Anthropic, you’ve listened to some of the talks from some of the engineers at Anthropic, they will tell you that nobody at Anthropic writes code anymore, right?

It’s all agentic, in the sense that they’re able to define a task, be able to have the agent Fire up multiple sub-agents, execute those tasks in loops. Right? And be able to come back to the human for, for validation, right, or for being able to kind of approve the work that’s been done.

So that is the farthest, the cleanest representation of what a truly autonomous, agentic workflow might look like, albeit with still some kind of human in the loop.

Russell Scherwin:

I love it. I’m gonna take a right turn, if that’s okay, because you said something that hits a bone right now. Yeah, Anthropic, they’re not writing code, and Cursor, you name the tool, can write plenty of code for you.

Just like anything else, you need someone who deeply… has a deep appreciation for writing enterprise code to run those models, to create those models, hone those models, and do so.

your gut feel, just, when we look 20 years ahead, you and I know that we still have an apprentice system, which is, you don’t come out as a CS undergrad or a CS master’s and understand what it takes to write enterprise code. Where are those apprentice skills gonna come from, so 10 years from now.

the people who are at Anthropic, or the next iteration of Anthropic, can actually run Cursor and kind of oversee what these agents are doing and codify the agents.

Praveen Akkiraju:

well, you know, 10 years is like a lifetime, right? I think we’re living in weeks here, if not days, in terms of how fast the AI space is moving. Honestly, I think if you just kind of look at even 6 months ago, how coding and software was being written to where we are today, it is a dramatic, dramatic change. And I think so, you know.

10 years, I think we’ll all be on the beach, you know, sipping pina coladas or whatever, right, as AI runs our lives and gives us paychecks, I suppose. But, I… look, I think what we are seeing today, you know, in a lot of the ecosystem here in the San Francisco Bay Area. There’s a ton of new engineers.

I mean, I have companies in my portfolio that are desperately hiring engineers, right? We’re… in fact, the competition for talent is more intense than it ever was, right? However, the nature of what that talent is has changed. So today, you’re expecting You know.

graduates to come out not just with an understanding, like, a good theoretical understanding of computer science, but more, sort of, implementation, you know, skills.

So, a lot of the, the engineers that are now coming into these companies have worked on projects, have used Cursor, right, have played with Cloud Code, maybe have even sort of, you know, fine-tuned a model, and applied it to a certain use case.

And so, I think what’s really important is that, you know, I’ve gotten back to coding after, like, you know, a decade or so, right? We all now have the superpower of being able to, use these coding engines to, you know, build things that we want to build.

And so, I think the… this next generation, as we go forward, is really about, we all become a community of builders. We have the superpower of coding that is now, you know, where the engine kind of takes care of all the semantics and things that you need to get done.

And so, what becomes important is really the ability to sort of translate real problems into solutions, right? And using this latest set of tools that you have now, such as Coding engines and Agentic. Architecture is to be able to execute those.

Russell Scherwin:

I dig it. Going down to humanities basics. Find problem, solve problem, go on to the next layer of abstraction. So, let’s, let’s hit on, on the agents. We’ve defined what an agent is, and we’re starting to talk about what they’re doing relative to, let’s say, RPAs. Let’s talk… let’s talk about the enterprise right now.

Can we talk about RPAs, what we’re really talking about is deterministic workflows, and historically, anything put into production has been deterministic. Well, an agentic workflow, that’s probabilistic in nature, which Enterprises, they care about governance.

They, you know, most enterprise systems today are deterministic in nature, and business itself is deterministic in nature.

Talk about what’s different with Agentic from other forms of, let’s call them applications, and then talk about what do you see being the foundational requirements For enterprises, it’s not just build agents, but drive… The adoption needed to harvest its potential.

Praveen Akkiraju:

Yeah. So, maybe let’s start with a bit of, like, an example, if you will, of… How, agents might… transform, you know, an enterprise workflow, right? So, I’ll give you my own example. So, I was the CEO at VCE, we were growing super fast, so my team came to me and said, hey, we need a CRM, right? So I said, alright, let’s go get Salesforce.

So, you know, you write out the check to Salesforce. And then you get told, like, whoa, hang on a second, you can’t use the software, you need to, like, you know, have… you know, a BPO or an SI come in and build out these workflows and build out these dashboards.

And that’s going to take another 3 months, and it’s gonna cost you another million dollars. And oh, by the way, after that, you need to have, like, a maintenance contract, you know, in order for you to keep these workflows going, right?

So, in order for me to be able to kind of implement a CRM, I had to buy the CRM, I had to pay somebody to implement it. Now, if I had an IT team, they could have done it themselves, I suppose, right? Only then could I actually start, you know, engaging with my customer data.

So, contrast that with, I think, where we’re going is, today, you know, Salesforce already announced, you know, CRM is headless.

So, essentially, they’re offering the entire CRM system of record as an API, so you could take that API and be able to build an entire agentic workflow on top of it that Is able to, as we talked about, just, you know, be able to build custom dashboards. custom workflows.

You can go in there, you know, you might have… your sales force might have different salespeople wanting different looks at the data. Somebody might say, like, hey, give me my top 10 prospects. Others might say, like, I want to know, you know, which customers are coming up for renewal, right?

All of this information can be queried because you now have access to the data store, and you’re able to build this agentic layer, which essentially unlocks the user’s, you know, imagination. You can get whatever information you want. Because the agent interprets your intent, and is able to, you know, access the information.

and provide you back the information the way you want it, right? So that’s the transformation, I think, that is… that’s possible. Now, I think you raised a very important point, and I think it’s really central to everything agentic, right? In the sense that, large language models inherently are non-deterministic.

And so… what you… what you need to be able to do in an enterprise environment is, obviously, you can’t have non-deterministic answers, right? You can’t, you know, you can’t have a workflow doing different things at different times, based on… based on, sort of, the prompt or certain data sets.

So, if you go back to coding, right, why is coding so, you know, so amazingly, you know, so amazingly accurate today, right? What, you know, you’re able to engage in writing, like, really on-the-dot Python code and, you know, recently more, like, SQL code. It’s able to produce code very, very accurately.

The reason why is because it’s a combination of the model with what’s called a harness, right? And so the harness essentially consists of, you know, at a high level, it consists of all the context that you need, right, for that particular workflow. So what data you need, what are the integrations, etc.

It consists of, basically a set of task trajectories, right, of your workflow. It consists of policies that govern this workflow. permissions, etc. You know, guardrails, as well as evaluations, that essentially tell the agent, hey, this is what good looks like, right?

Coding is so amazing right now, because what Anthropic has been able to do is to take a really powerful model, combine it with a very effective harness that is. that gives the model all the information that it needs to write good code, right, and be able to sort of guardrail it, right, and evaluate it, right?

And so, in order for agents to be To get over this non-determinism, essentially, what you need is a combination of a model with the right harness, right, and the right, sort of. you know, governance incorporated into that harness.

So, I’m keeping this a fairly high level, there’s, like, a ton of detail, you know, you can double-click on it from a harness perspective, but effectively, what I’d say is you have to capture your workflows, your policies, your guardrails, your evaluations in this harness. And each of the… each harness is unique to that particular workflow, right?

So that’s… that’s basically the way you’d start to address it, because we have an example in that, in terms of how coding works today.

Russell Scherwin:

I love it. Let’s play this out in the go-to-market sphere, right? Imagine we have, a use case where I need an agent, I’m a rep, and I just want to, you know, spend my time in the bar or the beach, and I want to have an agent do the research and the outreach for me. And we can discuss whether that’s a good thing or not.

I think it’s not, but let’s play with this. And here’s what I’m hearing.

You say an agent needs to have the harness, which is context, tasks, policies, guardrails, and evaluation, so… if I’m playing that out for you, if I need to do an account research, well, perhaps it needs to have my Apollo subscription, so it can do research on the company and the people.

Perhaps it needs my Apollo context, or my Sales Loft contacts, so that way it can drive emails. policies, perhaps. Walk us through that, because when I’m thinking about that is, well, I want it to have my personality. I don’t want it to email people on weekends or off hours. I want it to take on the company’s brand as well as my personality.

I have an economic, constraint because I don’t want it spending countless tokens or even countless, credits on my Apollo usage. How do you think about the build of the harness in that kind of use case? But also, what kind of runtime is this agent or demon running in?

Praveen Akkiraju:

Yeah, so, so, great question. I think, the… The differentiation for you as a company, or you as a startup, effectively, is this combination of models and harnesses, right? Now we… if you kind of go back to our initial conversation around open claw, the other aspect of that that’s important is the ability to, to keep state, right?

To have sort of a memory architecture that allows you, to… to store state, to compress it, to prioritize it, and to keep refreshing the context. So, building that loop, right, of, you know, execution. Is what basically allows you to build an effective kind of agent, right?

So, I think if you… so that’s, you know, the way you build a harness is… and this kind of varies right now, a lot of startups will today have forward-deployed engineers who could come sit down with you and help you build that harness, right?

In certain instances, you know, if you’re buying, like, a functionally vertical software, or a functional software for, let’s say, sales use case, finance use case.

Those harnesses are already predefined for you, so essentially it’s just a matter of plugging in your integrations, and, you know, it understands essentially what a… like, if you’re doing, you know, a balance sheet or something like that, that’s a fairly standardized workflow, right?

So, in some sense, these predefined, it’s trained on predefined workflows, and you can continue to customize it. It learns from you, right? That’s the aspect of Agentech, is that every time you’re evaluating it, if you… if you’re able to capture this state properly and feed it back, it improves. And that’s kind of what you want to build, right?

It’s a learning system. So, you know, the first step is always the complex part, which is, like, either you do it yourself with your team, or you have a four-deployed engineering team, or you have, you know, one of the systems integrators or somebody help you stand this thing up, and then hopefully it keeps self-improving.

You know, the runtimes, on this, today, you know, essentially, you’re talking about, sort of, inference platforms today, right? We’re still, you know. we’re still early in the phase of, like, truly at-scale agent deployment. So, most enterprises today have some form of agentic applications running.

A lot of these are heavy human in the loop, in order for For the humans to both be, sort of, improve and train the agents, but also to kind of make sure that the agents are, you know, are kind of executing in the parameters that they’ve been given.

But the runtime’s essentially, you know, you’re… you’re running inference, your application’s running where your application’s running, right?

So, it’s sort of… still, we’re experimenting with these things, In terms of exactly what that architecture looks like, you’re clearly still in a, you know, a cloud provider or some form of a, you know, some form of a, you know, a data center, right, where the workload is running, but then.

You… models, depending on whether you’re hosting them, whether you’re using it externally, you know, the runtime kind of varies.

Russell Scherwin:

So, let’s now shift into the enterprise. So, I’m a Caterpillar, I’m a John Deere, I’m a Comcast, and, you know, my business is not agents, my business is, you know, serving entertainment, or serving earth-moving machines. And I’m highly risk-averse. And we got lots of policies, that have nothing to do with technology.

How do I think about… A getting started. Clearly, there’s lots of pockets that are already getting started, but if I’m corporate IT, and I’m responsible for the security of the organization, responsible for the risk posture of the organization. And I’m responsible for revenue outcomes of the organization.

How am I thinking about, A, getting started, and B, you know, optimizing, you know, for the flexibility of my users and my business folks, but also making sure I’m setting the right structure so we can use it securely and reliably?

Praveen Akkiraju:

Yeah, I think this is the… the billion dollar question, or maybe a trillion dollar question. These days, trillions are, you know, seems to be the currency of choice, but, because I think you hit on… hit upon a couple of things here, right?

Most large enterprises are, have, you know, existing IT applications, systems, data, systems of record, extensively deployed. And, you know, so the primary question is, okay, like, how do you… how do you… where does this journey start? As we talked about, it starts with, sort of, context.

So, you know, what we see enterprises doing is, firstly, prioritization of, okay, which workflows do we really want to experiment with, you know, an agentic architecture, where we get the maximum bang for buck, right? So I think the first step is prioritization of these workflows.

The second step is, okay, once you’ve got these workflows, what is it mean to bring the context together? Which means, you know, where’s the data, you know, for these workflows located? Is it, you know, behind nice, clean APIs? You know, do you need to kind of, you know, figure out what your data architecture looks like, right?

Do you need to build an MCP gateway for some of the connectors that you need in order for you to inform the agent? So there’s that architecture of preparing all the data layer and the services layer that you need to kind of provide to the agent. And then the next piece of this is, the policies and the governance, right?

So once you’ve defined the workflow, you probably have a good sense of, you know, what are the policies governing this particular workflow, in terms of, you know, what data does it have access to, what can it, can it actually… do you give it permission to write? Is it… are you going to always incorporate a human in the loop? Right?

How far do you let the, let the workflow run before you, you, you are verifying with a human?

I think those things, obviously, are trust-based, so you would, you probably would kind of, which is the enterprises, like, they run it, you know, they learn from it, and progressively give the agent more and more, you know, more and more run time, no, runway. Right? As they’re, as they get more confidence.

So, so you kind of start off, like, layer by layer. Now, the other part of this, I mean, there’s some more interesting aspect here, which is around the economics of, of running these agents. So, you know, software now has a variable cost, right? And, so there, there, you know, interestingly, there was this point in time, I think a few months back.

Where everybody was, like, token maxing, like, you were measured by how many tokens you used, right? And there was, like, all these, like, you know, ranking lists of, like, oh, who’s the maximum token user? And people quickly kind of got out of that because they realized, like, whoa, this is… This can get quite expensive, right?

And I’m sure, I don’t know if you’re playing with it, but I know my token costs, like, you know, show up in a bill. I’m like, what? You know, if you’re using an API, in some ways, you know, more than a subscription, I think you kind of run up these costs pretty fast. So, I think what now, you’re trying to figure out is.

Right, you know, in more sophisticated agents. where do I need to use a truly state-of-the-art model, like a 1 trillion parameter massive model that’s trained on the entire internet, versus, like, hey, listen, I just got this really basic workflow, so, you know, I want to use a smaller model, maybe cheaper, using lesser amount of tokens, etc.

So, those are the more, sort of, second-order things of optimizing the agent, but it starts with, you know, putting together the context, right? Putting together, prioritizing the workflow, figuring out what you did.

looks like, what do your integrations look like, figuring out what the policies look like, figuring out where you want human in the loop, right? And then you start to kind of get this ball rolling, and, you know, in some cases, you’re choosing a partner to go with, right? Where they may say, like, listen, we just do this soup to nuts.

In some cases, you have a sophisticated enough IT team, that, and, you know, an engineering team that’s kind of playing around with this stuff and is able to build that for you, right?

Russell Scherwin:

Cool. I’m gonna throw a perspective at you, Praveen, and I’d love for you to respond to this. Past and… past and future, past and present. In the past, I’m a CRO. I see an opportunity to use technology to automate. my go-to-market, how we’re doing research and outreach, which is… it’s a laborious part, and it’s a high-risk part.

And so, in the past, I’d say, go hire Accenture, go find a consultant, document, you know, the process, and then go build some. Today, the coding for that kind of workflow is not done with RPA, it’s done with English.

I can sit, and I could write all this out, I could write policy, I could write process, I could write at least process governance for how we’re doing outreach.

And I see… I know a lot of seros in my own work, where I’m a serial-level consultant, I’m putting hands to the keyboard where before I would have, you know, dictated requirements of the translating tools.

How do you respond to an approach where the business user is the programmer, because the programming code, we’re actually programming, not using Java.

but English, what are the benefits of that, and what are the, perhaps, enterprise-rich, or what are the risks that I may be short-sighted and not thinking about if I’m codifying my process and governance in English?

Praveen Akkiraju:

Yeah, no, I think that is… that is the future, right? What you just described, which is a business user being able to feel confident enough to essentially spin up their own agent, based on what they’re trying to… the problem they’re trying to solve.

In your case, you know, as a CRO, or as a, you know, head of sales, you’re trying to get a sense for how your team’s performing. etc, right? You get the right integrations, and you’re able to, you’re able to monitor those, and Get look… different looks based on what you’re, what you’re interested in.

I think the key here, and this is important, right? The role that IT teams play today is mostly to essentially set up the right infrastructure to enable business users to be able to go down that path, right? So, it’s all the things we talked about. Building the right integrations, building the right policies, the guardrails.

the evaluations, and so you as a business user don’t have to worry about, like, oh, do I’m using the right model, or am I using too many tokens, or am I getting access to, you know, different data sets that I should not be, etc.

Those, essentially, if you build the right harness and give you the agent to execute, you as a business user should be able to go execute that. I think, you know, there’s a difference between what, you know, how you’re using the agent versus actually building the harness.

I think the harness building stuff is kind of what your IT team or, you know, your partner or your vendor should be able to take care of. Ideally, as a business user, what you’re looking for is more like, how do I get work done, right? In your context, it’s sales.

In another context, from a CFO perspective, it’s a financial workflow, right, or a legal workflow, etc.

Russell Scherwin:

I see we’re up on time. Praveen, just… I enjoyed chatting with you, and I know the audience gained a ton. Julia, we’re gonna turn it back to you.

Julia Nimchinski:

What a fantastic conversation. Thank you so much, Praveen. Thank you, Brussel. And before you go, where should our community go to support you?

Praveen Akkiraju:

Yeah, I’m on LinkedIn, you know, people can reach out to me there, and, you know, happy to kind of engage, particularly with founders who are building interesting things.

Julia Nimchinski:

Thank you so much for being with Russell.

Russell Scherwin:

The same, Russell Sherwin, I’m on LinkedIn, want to talk all things go-to-market, or yeah, say hi.

Julia Nimchinski:

We’ll share your profiles. And… thank you. And now we’re joined… Back. Again.