Sovereign AI – Why It Matters, Why Now, and How to Build It

Julia Nimchinski:

We are transitioning to our next session. Next, please welcome Howie Xu, Chief AI and Innovation Officer at GEN, Stanford Lecturer, formerly SVP of AI at Palo Alto Networks. Super excited to feature you, and joining Howie is Randy Wootton, our HSC Regular Partner at CEO Coaching International, three-time CEO, board member, and advisor. Super excited to get to it again. How are you doing?

Randy Wootton:

Doing great! Thank you so much for including us. This is one of my favorite things to do, is help out on your AI conferences every time I learn something and get to meet someone cool. So, Julie, it’s great to be here, and I look forward to the conversation with Howie.

Howie Xu:

Thank you, Judith, for the introduction. I’m very much looking forward to the conversation with Randy.

Julia Nimchinski:

Amazing. Let’s get it.

Randy Wootton:

Awesome. Well, Howie, I think Julie did a great job introducing you in the high level. Do you want to maybe just give a minute or two of the depth of experience that you’re bringing to this question of sovereign AI and the difference between Davern, data sovereignty and AI?

Howie Xu:

Yeah, before I talk about sovereign AI, so just a little bit about myself and then the company. I’m the Chief AI Officer at Agent Digital. Gen Digital is one of the largest consumer technology companies. We own the brands like Norton, Vast, LifeLock, Moneyline. You can see that our brands span from cybersecurity to financial wellness. As a matter of fact, you know, I actually run the emerging products within Gen Digital. We actually do an AI browser that gives you safety, trust, you know, it’s a pretty amazing browser, you should give it a try. And then also some Agentic products, so that we help busy parents to manage busy schedules, things like that.

So it’s a kind of a consumer technology company. My background has been, you know, doing AI machine learning for the last 11 years. It has been… I saw the transition from the traditional AI technology to the, you know, recent large language model-based technology in the last 3-4 years. So, it’s the best time to live, so let’s get into it.

Randy Wootton:

Right on, and just for background, I was CEO of a public company, First Gen AI, and, you know, that’s when it’s really kind of… like, machine learning at its basics applied to big problems. It was called rocket fuel, and then most recently had been working with a bunch of early-stage companies that are trying to be AI-first or Agentic, and it’s just crazy how fast this is unfolding. But I think what was really interesting about this topic, and I’m so glad you’re here, is just this idea of sovereign AI, because I know with big companies, when I was running an enterprise company, we cared a lot about data governance. and data sovereignty.

Can you help us understand and just define for our audience what you mean by sovereign AI, and what are the two ways of thinking about it at this point? Oops, you’re on mute, Howie.

Howie Xu:

So, as you pointed out, there is a data angle, right? Sovereign data, we have been talking about it in many contexts in the past, or in the past decade or plus. Sovereign AI, to me, is, like, a more comprehensive. If you think about AI, it’s data, it’s compute, and then it’s the model. So we still need a, you know, part of the initiative, part of the motivation for sovereign AI is about enterprise, or government, or whoever, right? Whatever the entity, you control your data. Your data do not share with, you know, outside of the enterprise, or outside of your entity.

That’s very important for many entities, right? Less important for some others, but, you know, super important for the enterprise. Part of that is, you know, actually my CEO and I were just discussing, you know, a few hours ago, you know, in the future, application will be part of the model, and the model will be part of the application. So in that world, think about it, right? You know, the data model is a kind but more important. So there’s a data angle, and then there’s a model. We all know that there’s open-air, Anthropic, Gemini of the world.

However, why people even worry, you know, wanted to have a sovereign model? Because, look, you know, at the end of the day, you want to have the control of where the data sits, right? You know, you don’t want to just inference using the external model and then give all the data. You don’t want that. I want to say you don’t always want it, you often, you know, you don’t want that, right? So that’s kind of the model part. And then there’s a compute. In the longer run, right, you could argue there’s a value for the public cloud versus a private cloud, that there’s always a reason for you to have your own data center.

Again, you know, different companies will have different considerations. To me, it’s the three things. Sovereign data, sovereign model, and then sovereign compute. There’s one more angle, the operational aspect, right? You know, if you want to own all these three you have to have the sovereign operation staff. So it’s really the… Compute data model, and then the, the operation, around it. Yeah, I think it’s something they got.

Randy Wootton:

That’s great, and I think that’s super helpful, having those four layers, and being explicit about the decisions you’re making for each of them, and the risks that you’re incurring if you… if you go outside of your walls. But I think the tension that people are feeling is along a couple dimensions. One is, can you talk a little bit about capability versus control? Because one of the arguments would be, if you control it, that’s great, but you’re never going to move as fast If you’re able to engage the companies that focus on this and have invested billions of dollars in it.

So how do you think about that tension of solving for capability versus control?

Howie Xu:

I really like the framing of the tension, the word, because there is a tension. Let’s step back, right? You know, why are we talking about sobering AI today? You know, September 2026, not a year ago, not two years ago. There is a reason for that. A year ago, or two years ago, yeah, you can have your own, sort of, the AI model, data, but guess what? You know, your model is just way behind frontier model, right? If your model doesn’t perform, what’s the, you know, what’s the… what’s the point, right?

However, today, the open weight model, right, has gone into this tipping point that… I wouldn’t say it’s as good as open-air Anthropic, the latest model, but, you know, but for most people, for many enterprises, for many use cases, it’s good enough, or it’s not even more than good enough. It’s actually good Period, right? So in that world, we can talk about sovereign AI. It started making sense. When did we achieve that? I would say approximately in the last 6 months or so, we kind of got into this period that open weight model is viable enough for many or most enterprises, so that’s why we are talking about… So when you talk about the tension, yes, we have had this tension for years, right?

When I was at Palado Network, we discussed, talking about, you know, Palo GPT, you know, but at the end of the day, it’s… it wasn’t the bad thing to do at that time, because, you know, it’s… you know, you don’t want to compete with OpenAI, you know, for… for the… frontier model performance, but today, you know, we are at that point. So that’s kind of the tension. Now, is the tension going to continue to be there? Yes, you know, because a model does so many different things, right? OpenAI just released Astra, right, last week, and it does CAD, it does, you know, computer use so well.

I don’t believe, I don’t suspect, at least the frontier open weight model can do as a good job. Probably 3 months or 6 months behind. If you are so into that sort of use cases, you still need you know, the best capability. So the tension is always there. But what I emphasize is a vast majority of the enterprise use cases today, open weighted model is good enough.

Randy Wootton:

So that’s interesting. So really, the other tension that you’re sort of illuminating is the idea of innovation versus security, and kind of thinking about what type of company you are, how are you using AI on different dimensions in terms of either automating current workflows, or is it core to the offering you’re bringing to market? So we talked to a lot of people about, you know, the aggregating the data and putting an intelligence layer on it. And then as you step into the Agentic world, having peop… having agent models, doing work on behalf of people and substituting, it becomes… if you’re moving up at that level of abstraction.

The idea of how far on the edge do you need to be to compete, to create differentiation in the marketplace, and at what risk in terms of security, so cyber firms or others? So how have you thought about that? Because you’re right there in the middle of it, in terms of the elements of security versus innovation.

Howie Xu:

Yeah, cybersecurity is another big angle, right? OpenAI, Anthropic, started getting to this space, you know, from mythos model, OpenAI has their, you know, their similar things. There are a number of, you know, I want to just borrow your word, the tension, right, you know, going on here. What’s the tension? You know, does the model guys, the frontier model guys, who really wanted to give you the best models best performing models to everyone, you know, they have different considerations. Maybe it’s not so good ideas, because, you know, what if it’s in the bad guy’s hand? But, you know, on the other hand, what if the good guys do not have the best models?

You know, it literally happened, right? You know, when OpenAI, there’s security incidents, right? Things, you know, broken into hugging Face infrastructure. you know, accidentally, whatever you want to call it, right? Hugging Face guys are sort of a figure, trying to figure out, hey, I want to use the latest model to figure out what’s going on. Guess what? The latest model has this alignment, or has this boundary, has this limitation, so that they couldn’t do anything. They eventually had to resort to the open way model and, you know, figure out what’s going on. So that tension is going there.

Do you give me the best model. There are pros and cons, so I believe that we’re at the beginning of figuring that out. We are kind of having so many, you know, we are still learning what’s the best way. We don’t know what we are doing in many ways, but, you know, fundamentally because of the tension, you know, how to look at the tension is really the key.

Randy Wootton:

And at the end, in the last 5 minutes or so, we’ll come up with your 3 recommendations for operators to approach this problem, but I want to keep teasing it out a little bit. I think the other dimension you and I talked about was global scale versus local jurisdiction. And the idea, if you really are competing on a global marketplace, which all of us have ambition, I think, aspirations to do, like, how do you do that if your model is only locally organized or controlled versus the global reach. How have you thought about that with your company, and what do you advise other folks framing… To be able to scale globally versus control locally.

Howie Xu:

Yeah, look, first of all, Gen Digital is a global company. We have consumers, you know, using our products, 500 million users globally, right? You know, there are so many jurisdictions. I would say the following. I think, the… You know, we know, you know, there’s a data jurisdiction topic for a while, and that is getting even more and more complex. We are still catching up, you know, as an industry, you know, how to deal with that. Clearly, you know, that’s something that… why… part of the reason that a lot of our users are using our product, because they trust us to figure those things out.

Now, we’re talking about AI, sovereign AI here. I think it’s a new topic, right? The new topic is, you know, where do you do the influence? Where do you, you know, get the data, you know, sometimes even potentially polluted? That begs for a sovereign AI more than ever, because, you know, think about it, right? You know, I have an AI model from, let’s say, OpenAI Anthropic. On the surface, right, they handle data certain ways, but do you really, you know… Some enterprises, you know, the CISOs are telling you that they don’t necessarily trust the way they are doing things.

Some CISO says, look, you know, I give my you know, data to cloud, why I don’t want to do the same thing with Anthropic and OpenAI, it’s the same thing to me. I think it really depends on how you look at it. for a company like Gen Digital, we have, you know, global presence, we have to think about, you know, the different, complex, jurisdiction. We have to take a more, cautious, sort of the approach. That’s part of the reason why, You do, the local model, or the sovereign model, or sovereign AI is more important to us.

But for some startups, they probably care less for the right reason, because they just started it. They don’t have, you know, millions and millions of users. So I think, you know, then there are some government that they would have treated even more seriously, right? Hey, no open-weight model whatsoever over my dead body, right? That’s… there are policies like that in certain government, in certain departments. So it’s a very wide spectrum, you know, my legal people and myself, you know, we just reviewed it, you know, on the, you know, internally as well, you know, what is the policies?

We have to follow that. Even yesterday, my legal colleague was updating me, hey, this is, you know, on what it is, you know, what we need to do. So, there’s a lot of complexity in it.

Randy Wootton:

Yeah, I think, just a flashback. When we were at Rocket Fuel, which is a public company, we had data centers around the world, because AWS wasn’t even fully fleeted up yet, and we spent, I don’t know, $250 million billing on our data centers, in part so we could compute quickly in the local markets, and there was a huge issue around consumer data, and there was a whole set of technologies called CDPs, customer data platforms, and what we found, to your point, like, in the EU, they had very strict policies and laws about how data was aggregated and stored, and we had to have systems in Germany, for example, for us to be able to do business in Germany, and I think what you’re pointing at is a lot of that sovereign data and data governance policies and procedures are directly relevant to thinking about how to work with AI.

It’s just gotten a little bit more complex, because you have multiple levels. To your point, you’ve got the data, you’ve got the model, you’ve got the,

Howie Xu:

Where’s the compute, right?

Randy Wootton:

Yeah, the computer, right. And so, like, it becomes like, you know, Tetra. You gotta think about all these different dimensions, and I think what you’re suggesting is there’s, deliberation that needs to have at each, and if you’re a global country, a global company, understanding what the constraints are, they’re being put placed by different countries, you know, the EU, the US, and then Asia writ large. How do you do business there? Otherwise, you’re gonna get in trouble and, you know. There were big lawsuits in terms of consumer data violation and breaches. So, a great point. You were pointing at one other.

Howie Xu:

Sovereign AI is not for the faint part. That’s right, fair. It’s a lot of complexity, a lot of effort. It’s not just, hey, let me save money, let’s run it on my local GPU, and then be done with it. No, you know, initially, potentially, you know, you have a lot more complexity to deal with. It’s a cost increase, even, potentially.

Randy Wootton:

Yeah. And do you think… I know I’m putting you on the spot, this may be a softball, but is this the best argument for companies to hire a chief AI officer? It’s not just about thinking about the AI strategy, it’s not just thinking about how to create value for the data that you have to inform an intelligence layer. Should the expectation be that a chief AI officer also has depth in terms of legacy data governance and brings an approach in terms of risk versus opportunity. It’s not a legal officer, but it’s a business-oriented leader on the team that can help Figure out these super complex problems?

Howie Xu:

Well, you know, it’s actually more complex, right? Number one, you know, apparently my CEO has more, you know, foresight, you know, he hired me two years ago, as a CIO… sorry, Chief AI officer, right? You know, now there are a lot more CAIO out there. But on the more serious side, your question is, you know, is there a reason to hire a chief AI officer? I would say chief AI officer based on what I observed, there are many definitions of that, right? You know, some are more on the compliance side, some is on the sort of research side, some is about innovation side.

So, it’s actually all over the map. I would say that because AI is becoming more and more important for the company. whether it’s from the CEO, right, and the CEO has to be… by the way, when I was at Pallado Network, you know, Nimch always said, you know, I’m the chief AI officer, because, you know, I have to… I have to lead it, right? You know, even though my title was, you know, the Senior VP of AI, but, you know, he as the CEO, you know, clearly wanted to drive it, and, you know, I think that’s the right thing to do.

Same thing at GM Digital. the, you know, how we think about AI, you know, the COO has to drive it. So, to me, it’s not just a bad role. Now, personally, I do see that, you know. because of AI, the innovation speed will be different, right? The complexity will be different, right? The data sovereignty, or… data sovereignty or those things probably will be different, so it, you know, the companies… the companies need a chief AI officer for that reason.

Randy Wootton:

Yeah, I think so, and it needs to be more than just a CIO rebranded as a C… I… AIO, right? Like, AI.

Howie Xu:

Yeah, Dale City.

Randy Wootton:

Keep it right.

Howie Xu:

It’s a transformation. Unless you treat that as a technical task, then that’s a different story. But if you treat AI as a transform… you know, one of the, you know, most transforming technologies we’ve seen in our lifetime, you know, you need a COO to lead that. That’s how I see.

Randy Wootton:

Yeah. Yeah, I think you’re right, and as people have talked about, it’s not just a… the transformation we’re facing is once in a lifetime, it’s once in a, like, a civilization, right? And that, that, and this really… people have to lean into it. Just one other risk I wanted to go back to, What do you think, or can you help articulate and frame for the audience the greatest risks of dependence on one model cloud or agent platform. We used to talk about this with software, and, you know, going all in with an ERP, and it made sense to consolidate.

I mean, that was like Salesforce, Microsoft, you know, the ecosystems that they built. How do you apply that same sort of thinking and the risks in an AI environment where we’re laying in these other layers?

Howie Xu:

Right, hey, you know, Randy, you and I have gone through, you know, multiple technology, transformations in our lifetime, right? You know, whether this is the biggest one or not, we’ll see, but, you know, there are multiple transformations going on. The… at the end of the day, the technology is always a double-edged sword, right? You know, it can have negative, it can have positive. It’s all human. Humans need to, you know, drive it. Even this morning, my PR person sent me this article called something along the line that, you know, there’s a 10% chance that AI will kill all humans.

I told her that, you know, that’s not true. You know, at the end of the day, humans kill human. don’t, don’t, don’t put, you know, the AI as the scapegoat. I personally feel, strongly that we should be in the world. It’s not AI first. AI native, but human first. It’s always human first. Never, right? Human lived millions of years. It’s never going to be, you know, the AI first, in my opinion. The risk the risk, the specific risk you are talking about is a very human risk. I’ll give you an example. Even last week, right, or two weeks ago, Cursor announced that, oh, actually, not Cursor.

OpenAI announced that they are not going to give the model to Cursor after its acquisition by SpaceX, Elon Musk. We all know that Elon Musk and Sam Altman are not the best friends anymore, just because they are not the best friends. My guys, you know, my poor engineers no longer can benefit from using OpenAI models in cursor. That’s a risk, right? Just because I truly believe that people like Dario, truly like a Sam, they have the right intention. to do the good things, you know, oftentimes, but they have the right intention, doesn’t mean, you know, good… good outcome will always come out of it.

So I do want the choices. If I don’t have choices, you know, I will… I will be, you know, hostage of, you know, things like this, right? So… so I want to say choices enable less risk, for sure. So, from that point of view, I do see a risk. It’s not an intention. But I do see a big risk if that all of us have to, you know, beg OpenAI and Thropik to give us, you know, the next tokens. You know, I think we need to have choice. That’s another reason why sovereign AI is so important, or getting more and more important.

Randy Wootton:

Yeah, I mean, it’s the monopoly versus, you know, free markets, and not just on the risk side, but innovation, having other people being able to compete and push the edge. To that point, just one last question, and then we’ll go to your top three things for operators. How do you think about specialized and edge-based models? Their impact on this whole equation, the sovereignity equation, like, because that, you know, we’re geeking out a little bit, like, that really is the newest and craziest, and what’s going to happen there? How do you see that playing out?

Howie Xu:

Well, first of all, I’ll give you some technical insights. You know, whether we like it or not, the innovation on the edge or the device side of the model is not nearly as good as the large language model side. There hasn’t been major breakthrough, let’s just face it, but, you know, on the large language model side, the reasoning model, right, the coding agent, there has been… there have been multiple breakthroughs, even after ChatGPD moments. And then on the device side, right, you know, we haven’t seen breakthrough after breakthrough at all. However, it’s very much needed, because, you know, for Gen Digital, many of our software running out our users’ devices, or, you know, whatnot.

So, edge model performance is dear to my heart, so if you ask me, I want edge model performance to be, you know, 10x, 100x better from here on, so that I have more viable solutions, more interesting solutions for my users, so that they can enjoy AI while all the AI inference happening on the local device, which today is still a dream in many ways. I mean, you look at Apple, right, you know, of course, we’ll see what Apple is going to announce or whatnot in the coming, you know, keynote by the new COO. But, you know, I think so far, last few years, we haven’t seen an Apple device that works great.

Hopefully, that gives you some insights. Your question is about, okay, what’s going to happen with the edge device? I believe that the breakthrough will happen. The reason is very simple. We already knew, as a, you know, technologist, we knew what are the issues with the local models, right? You know, it’s a matter of putting efforts. It’s a matter of, you know… there are technical solutions for that, right? One example would be memory. You’ve probably heard so much about, hey, because of the Agentic workflow and, you know, memory pressure is so much, we need a lot more memory.

Memory price, you know, went up dramatically. But there are ways to actually do on the model side to leverage, you know, the device smaller, you know, the memory size for the inference. There are ways to do that. I think it’s a matter of, you know. we as an industry are putting more and more efforts on it. So I totally anticipate in the coming 2 or 3 years, we see breakthroughs on the device and the edge model side.

Randy Wootton:

Yeah, and so to your point, it’s a matter of when, not if. It will be something we need to think about in terms broadly, how we think about sovereignity at the different levels, and then I guess the other point is don’t ever bet against Apple, right? Like, they will come out with something at some point, which will blow all of our minds. And I’m not an Apple fan.

Howie Xu:

Well, either that, either that, or someone will disrupted, right? You know, they are lucky enough not to be disrupted over the last few years, because, you know, the mode is so, so, so, so good for them, but it’s not going to last forever. So, either they disrupt themselves, or they be disrupted, right? It’s one of the two things.

Randy Wootton:

Well, true. Alright, so in the last 4 minutes, 3 things that you would tell operators to think about, given this context, to work on tomorrow. Like, if we have CEOs, startup CEOs, software CEOs, and tech people online watching us right now, what are the 3 things you would recommend that they do tomorrow?

Howie Xu:

Number one, just like, a few topics, right, AI has become the board-level topic. I think sovereign AI is going to be a board-level topic in the coming 12 months. Every board meeting, people will start asking, hey, what are we doing with, you know, sovereign AI, with your own model? So that’s number one. So as a result, the first thing to do is I started, you know, started looking to this, exploring this, right? As I mentioned earlier, sovereign AI, local model, owning your own tech stack, owning your AI stack is not for the faint of heart.

It’s not easy, right? But not easy is not a reason not to get started. So, get started at getting, you know, your… whether chief AI officer or, you know, R&D, or whoever, right? BIO, my CIO, for the part is, you know, very versed in this kind of technology. It’s so great to have conversation with them on how to collaborate on these kind of things. I’m pretty sure that there are many CIOs out there starting about this, too. So, start looking at it. Second thing is actually looking at it from the legal compliance point of view, right?

Because, you know, it’s not just doing it from a technical point of view. you know, do you… are you… are you… is that okay for you to use XYZ open weight model that listed, you know, a minute ago, right? You know, the… you have to be… you have to pay attention to it, so… so it’s not something that’s just a technology thing, it’s also a legal, compliance thing. The third thing, I think, is, you know, looking closely from the data point of view. What I… what I mean by that is the following. Why do you want to own the AI stack fundamentally, right?

I think fundamentally is because you truly believe that your owning that AI stack is going to be a differentiation, is going to create, you know, a better business model or better applications for you. And I truly believe that’s true, you know. As an example, right, you know, in the last few years, we have used OpenAI Anthropic of the world, those models, right? But a lot of times, you need to do post-training tailored for your own enterprise needs, right? For your own kind of the user, for your own kind of the personalization. But how to do the post-training is not an easy thing.

However, that barrier is actually dramatically coming down. Not… low today, but still very high, but dramatically going down fast. My prediction is within 2 years, every enterprise should be able to do post-training. I wouldn’t say left and right, but, you know, for many of them, they should be able to do. So, as a result, if you are able to do post-training yourself, then you should look at, okay, what is the unique set of the data I have? What kind of workflow I wanted to, you know, distill, I wanted to sort of automate, I wanted to do something.

So you need to look at your data. Ultimately, it’s three things, right? You know, start exploring that, but from a technology point of view. Because operationalizing it, getting the right model to influence that at a reasonable cost is not easy, right? Learning it… the good news is, there are also, you know, third-party harnessing software company or inferencing shops that may help you, right? We are also looking into one of them, or, you know, some of them. The second thing is looking at it from the legal, the compliance, you know, the, you know, business point of view.

And the third thing is actually staring at your data, starting looking at it. Okay, if I’m able to do a lot of post-training. what sort of the post-training thing that makes sense, and then that would give me, a better business model… better business and a better application. That’s sort of how I think about it.

Randy Wootton:

Awesome. You didn’t put in, hire a chief AI officer, but I’ll throw that out there, because I will tell you that in the clients that we work with at CEOCI, we have about 550, Our conversation with our CEO clients is, you need to have an AI strategy. That’s something we’ve been bagging for the last 12 to 18 months. This idea of sovereign AI, to your point, I think is going to become a board-level issue, because they care about risk and opportunity, and they want to know their exposure, and they want to know your approach. So the AI strategy that you bring to the board now needs to have a module Which is around sovereign AI.

So, thank you, Howie.

Howie Xu:

You can go read.

Randy Wootton:

Appreciate getting to know you. Julia, we’ll hand it off to Mark in the next group.

Julia Nimchinski:

Thank you so much, Ben.

Howie Xu:

Thank you, everybody.

Julia Nimchinski:

Before we let you go, Howie and Randy, what’s the best way for the community to support your work?

Howie Xu:

Oh, you know. We have a lot of AI products, we have a lot of the safety and then various products, try it out. Not a new would give you the safe AI, would allow you to do a lot of amazing things for your browser. I truly believe that a browser needs that disruption as well, so that’s how you would save. How you would be able to help?

Julia Nimchinski:

Amazing, Randy?

Randy Wootton:

So, I mean, people can find me on LinkedIn, that’s where I’m doing writing, and I’m happy to chat with people about their broader AI strategy. This is something that we’ve been working with people to shape the set of questions you need to ask, the capabilities you need to bring, and also how much money you need to spend, and I think that is something that, Everyone needs to be spending some time on and happy to help.

Julia Nimchinski:

Thank you so much again.

Randy Wootton:

Take care.

Howie Xu:

Thank you, Julia. Thank you, everyone.