Retention Before and After the AI Revolution

00:00:04.120 –> 00:00:10.170
Julia Nimchinski: Second, and we are live.

00:00:11.090 –> 00:00:33.439
Julia Nimchinski: Hey, everyone! Thank you so much for joining us today in our live event brought to you by Hsc. And the men base. As we unveil the next evolution of agentic scaling, we’re excited to show you 11 practice sessions focused on frameworks for agent. First, st Gtm design, Roi benchmarks from systems in production.

00:00:33.540 –> 00:00:39.350
Julia Nimchinski: and next, Gen. AI. Stacks across sales, marketing, customer success. And Rev. Ops.

00:00:39.950 –> 00:00:50.750
Julia Nimchinski: This event is a little different. You’re going to learn concrete methods from real world operators. And hopefully you’ll get a real world, Roi.

00:00:51.050 –> 00:01:00.509
Julia Nimchinski: And with that I would like to welcome Tash Evans, Vp. Of customer growth at Hook and Sam Champion Hook’s founding. Gtm.

00:01:00.640 –> 00:01:07.030
Julia Nimchinski: they’ll be presenting the method retention before and after the AI Revolution. Tasha and Sam.

00:01:07.190 –> 00:01:11.790
Julia Nimchinski: Thank you so much for joining us today. And how’s sunny England.

00:01:11.790 –> 00:01:40.369
Tash Evans: Actually, surprisingly sunny today we can’t complain. But yeah, thanks so much for having us, Julia. We’re really excited to be here. And as Julia mentioned. We’re here to talk a little bit about retention pre and post the AI revolution here at Hook. What we do, we leverage AI and machine learning to predict customer revenue. So basically, we tell you what to do, how to do it and how to influence the outcome. And we’re going to tell a little bit of our story and our journey today through that lens of what does retention look like? Pre and post AI

00:01:40.370 –> 00:01:57.489
Tash Evans: and some of the things to maybe watch out for in the new world of AI. So you can make sure that whilst we’re all focused on getting our team to do things efficiently that we’re doing them efficiently in the right place and the right things on the right customers. So, Sam, if you wouldn’t mind sharing your screen, you’ve got our slides today.

00:01:57.640 –> 00:02:24.900
Tash Evans: I’m going to set a little bit of the backdrop, and I know that I don’t really need to tell this group this. But sas has been changing a little bit in the last couple of years in case folks didn’t realize there’s a lot of market pressure, there’s an increase in competition. Sales are slowing down, and it means a lot of businesses are turning to the post sales teams, and they’re inspecting our churn numbers, expecting more from Upsell and from Nrr. And whilst all of this is happening, budgets are tightening and teams are smaller and smaller than ever, and it’s

00:02:24.900 –> 00:02:34.360
Tash Evans: really creating this like boiling pot where everyone’s focused on. How do I get higher outputs? How do I get more revenue? But also, how do I do that in the most efficient way.

00:02:34.410 –> 00:02:45.179
Tash Evans: And in this interesting market we’ve then had the birth of AI. So AI was born, and we’re all looking to it. Going. Oh, is this gonna solve all of our problems really efficient. But great outputs

00:02:45.480 –> 00:03:02.180
Tash Evans: and the winners in this market are really focused on. How do I use AI in the right way. How do I use AI that’s trained on my customer data to make sure that it’s really relevant to my business and not just using AI for for the sake of it, but intentionally to solve real problems that me and my customers have.

00:03:02.520 –> 00:03:30.849
Tash Evans: and the ones who are starting to struggle are those that are looking to use one size fits all. AI treats every customer the same like, just create me a playbook, and I don’t really matter if it’s the if it’s the right one, and it creates this world where we’re focused on doing things efficiently, but not necessarily the right things. And that’s not good enough in this world. And so to see, how do we do it? Right? How do we not fall into that trap we’re gonna 1st go back into before AI was developed. So what was the world? Pre AI.

00:03:31.260 –> 00:03:46.539
Tash Evans: And so in this market we had totally different challenges. Right? We had all of this data. We couldn’t get any insight. Everything was really heavy and really manual, and we had no idea if a customer was going to turn or upsell until the moment it happened. And scaling was really really difficult.

00:03:46.720 –> 00:04:03.090
Tash Evans: And our idea of automation was very basic. It was as simple as if you click to the next slide, Sam. Hey? If my customer is red in my health score, send them to a cadence that says, Please use us more. And so a lot of area for improvement.

00:04:03.640 –> 00:04:07.179
Tash Evans: Then we have the birth of machine learning.

00:04:07.400 –> 00:04:27.949
Tash Evans: And so if you click the next slide now, we can get insight from our data. Now, all of a sudden, we can tell stories. And with machine learning, we can leverage this data to create really accurate health scores that are good at predicting churn, but still missing some of the signals that sit within that sentiment. Data which relies solely on Csms and gut feel

00:04:28.040 –> 00:04:50.890
Tash Evans: and scaling, is getting a little bit better, but still kind of difficult, and our idea of automation then becomes, hey? If a customer is red, if you click to the next slide the customers red, then find the thing that would make them green, and then push them into a cadence. That’s gonna tell them, hey? Here’s how to do that thing, and here’s why it would benefit you. So we start to get a little bit more mature in our processes.

00:04:51.730 –> 00:05:10.629
Tash Evans: and then, of course, AI gets more and more impressive. We get Llms. Large language models and agents. And now, all of a sudden, we can get signals from the sentiment rather than relying on gut feel we can start to automate tasks for Csms and post sales, professionals, and for most professions out there.

00:05:10.730 –> 00:05:30.430
Tash Evans: And AI can generate us literally anything that we want at the click of a button, a playbook, a Qbr. Deck, a health score. But is it really all that easy? Because how do we know that all these things it’s creating to make us super efficient are really the right things that actually work and actually impact our revenue number.

00:05:31.000 –> 00:05:37.169
Tash Evans: And here’s where we’re starting to see customers and people in the industry really fall into that trap

00:05:37.460 –> 00:05:42.370
Tash Evans: because AI is amazingly powerful if it’s pointed in the right direction.

00:05:42.490 –> 00:06:00.869
Tash Evans: And so we start to ask ourselves some of these questions that you’re seeing on screen. Because really, the last thing that we want to create is a world where our team can generate an AI playbook really easily. But it’s not the thing that’s actually going to help the customer, or it’s not the thing that we can prove has worked before and will work again.

00:06:01.210 –> 00:06:28.290
Tash Evans: And what if we’re creating something that’s a playbook that might work for one customer but doesn’t really work for the others, because every customer looks and feels a little bit different. What if we’re relying wholly on sentiment for a health score? But we’re missing some of this really clear data signals. And in this market that we talked about at the start, we just can’t afford to get this wrong like in a world where we’re focused on being really efficient. We can’t afford to waste resource. We can’t afford to point things in the wrong places, and we can’t afford to not influence the bottom line.

00:06:28.760 –> 00:06:43.889
Tash Evans: And so with that, I’m gonna pass over to Sam, and he’s gonna talk a little bit about how we’re thinking about making sure that our customers and us ourselves at Hook are leveraging AI in the right way to properly impact revenue and truly be efficient

00:06:44.650 –> 00:06:45.919
Tash Evans: over to you, Sam.

00:06:46.210 –> 00:07:00.840
Sam Champion: Awesome. Thanks for that tash. Yeah. So what we’re gonna walk through today is an overview of obviously our hooks helps with some of this, but we’re gonna try and keep it nice and framework based so sticking to the general concepts that we try and adhere to to our platform to give our customers what they need to boost retention.

00:07:01.130 –> 00:07:02.110
Sam Champion: And so

00:07:02.240 –> 00:07:25.650
Sam Champion: what we found on this journey of trying to help people get proactive, about which customers are at risk. And why is the kind of foundation of it? Is that machine learning that Tash spoke about? And so what this really does is give context, not just about like what a customer is saying, but actually, hard data around, are they using your product? How often are you meeting people? Are they raising severe tickets? Are they raising no tickets? What they spent over time?

00:07:25.670 –> 00:07:44.210
Sam Champion: We found that this structured data, so mostly numerical billions of rows of just like clicks, etc, is what gives a really good foundation for understanding. Customer bases. What type of trends lead to churn? What type of trends lead to renewal? What type of trends lead to expansion. Those 1 million dollar questions of what are my leading indicators

00:07:44.220 –> 00:08:06.259
Sam Champion: at Hook? What we use for this is something we call our engagement agent, which is basically a bespoke model that’s trained on this structured data. Obviously, we do it for ourselves. We do it for our customers where we’ve basically analyzed historically, what does actually cause churn? Is it that they’re embedded in this feature? And they tend to always renew? Or is it that they’ve never used this thing, or they have too many tickets, and they tend to churn.

00:08:06.260 –> 00:08:16.279
Sam Champion: That’s a massive oversimplification. But that’s the kind of questions that you want to ask on your data in order to start being able to build a model that gives context of what good looks like.

00:08:16.660 –> 00:08:18.370
Sam Champion: This is the foundation.

00:08:18.800 –> 00:08:36.480
Sam Champion: And now the next step is to go a step further and enhance this with the unstructured data. And so this is a bit more like the AI that you guys know, and love, which is chatgpt, analyzing, sentiment, generating outputs. And what a lot of people struggle with is. They find that this becomes overly generic

00:08:36.640 –> 00:09:01.000
Sam Champion: because it doesn’t really understand their business. What we do with this data is we layer it on top of that structured data and use the structured data as the context, we do something called the echo agent, which is basically analyzing this. So it’s like a Csm or an account manager that’s always watching basically picking out. Okay, well, what type of trends are people talking about in tickets, emails that have led to churn in the past in order to flag them to you and give you reasons.

00:09:01.570 –> 00:09:16.199
Sam Champion: and then a step beyond that is, we use something called play ibooks, so kind of killing the playbook and adding AI to it which we bill as the Csm that learns from your best. Csm, so it’s basically learning what interventions have worked in the past to kill the risk that’s been identified.

00:09:16.510 –> 00:09:28.649
Sam Champion: These 3 things may sound quite distinct, but the really key part of it is that this structured data is what informs these agents and actually keeps them super hyper, relevant to your business so that you can understand where the risk is

00:09:28.870 –> 00:09:38.189
Sam Champion: action, that risk actually understand the why behind it, and then also automate actions off the back of it, using generative AI. And that’s what we’re going to walk through today.

00:09:40.540 –> 00:09:42.550
Sam Champion: So if I jump into hook.

00:09:43.590 –> 00:09:58.390
Sam Champion: we’re going to walk through 2 of the agents today that we found to have results which is the engagement agent. And we’re going to talk through how we can basically analyze historic data to predict churn risk how you can do that, too, as well as the echo agent. So if I clicked on this engage agent here.

00:10:00.180 –> 00:10:09.550
Sam Champion: What we effectively do here is the 1st step to making any sort of model become actionable and relevant is we surface and rank, all of your historic data

00:10:10.080 –> 00:10:27.479
Sam Champion: so that you can see it trending over the past 12 months, for example. So if you were to log in, look at your customer. Dhl, that’s maybe got 2 product subscriptions. Today. We can see this trending over time. We can see how many na use, how many courses they’ve completed, how much meeting time you’ve had with them. And if I select all.

00:10:28.650 –> 00:10:52.089
Sam Champion: I think this will be a common theme today, this data by itself means basically nothing. If I’m a Csm. And I’m looking at this, I have absolutely no idea if 40% is good or bad, or if, like license, utilization is more important than Maus, or, of course, is completed, should be up there above support tickets. And this is really what causes churn. And so you can actually use machine learning to answer this, look back in the past

00:10:52.090 –> 00:11:00.689
Sam Champion: for every metric, and ask questions, to say when customers like Dhl spending about this much using this product have churned in the past. What of these data points

00:11:00.730 –> 00:11:11.139
Sam Champion: actually lead up to that event? So this is your leading indicators for renewal, basically using analysis to turn that data into actionable insight to tell you what features and usage patterns are sticky.

00:11:11.260 –> 00:11:33.790
Sam Champion: The second step, if we hit show benchmark, is you really want to try and understand. What does a healthy customer hit each one of these metrics? So, for example, we might find in the data that a customer about this size and spend tends to spend about an hour with your Csms each month in order to be likely to renew, or they tend to hit 88% license utilization. And this one’s actually only at 44%.

00:11:34.060 –> 00:11:44.209
Sam Champion: And what you’ll realize here is once you have these 2 answers of what healthy benchmarks that customers should be hitting based on my historic data. And out of all of these metrics

00:11:44.480 –> 00:12:07.270
Sam Champion: which metrics actually are important for my customers. That’s when you can start using AI to really make it impactful for your business. Because then, all of a sudden, we don’t have to look at all of this data we can just immediately see. Okay, well, this customer. Dhl, the elevate product. We’ve got a model in the background, predicting the risk of this one very, very low. But the learn product, this is the one that’s having the problem.

00:12:07.360 –> 00:12:21.389
Sam Champion: And this is machine learning. So this tends to be hovering around 80% accurate predicting churn even 6 months before. Renewal doesn’t tell the full story, which is the AI. We’re going to show later. But what this obviously enables you to do is when we zoom out to scale

00:12:23.320 –> 00:12:40.809
Sam Champion: the benefit of having this type of insight is suddenly we can go from not really knowing which customers are at risk, or even knowing what usage like health usage looks like to saying, Okay, well, 3 quarters in advance, which of my customers are going to cause me to miss my Grr target potentially impact funding down the line. And

00:12:41.050 –> 00:12:46.179
Sam Champion: what actions do I need to take in my base? So that’s what we’re going to cover. Now, which is

00:12:46.870 –> 00:13:03.950
Sam Champion: this is very much like solving the math problem. And this is a really cool foundation that we found with AI when rolling out to our customers is, people don’t want to solve math problems. They really want to solve revenue problems and just giving you a score to say something’s likely to renew or not is useful, but that it really is like a mathematical answer.

00:13:04.180 –> 00:13:15.329
Sam Champion: AI needs to be actionable. It needs to be explainable, or a people won’t trust it. And B people won’t actually know what to do, because 71% is just a number. It’s not actually helping me retain this customer

00:13:15.630 –> 00:13:20.099
Sam Champion: to help shorten that time from like searching for insight to getting insight.

00:13:20.360 –> 00:13:34.670
Sam Champion: Good models definitely need to be explainable. And so in our instance, we use shap values. So this is basically a way to reverse engineer the score to show you. Okay, well, what is actually contributing positively to this particular thing.

00:13:34.880 –> 00:13:43.779
Sam Champion: What’s contributing negatively to this customer’s health score. So giving you a immediate trust that this is something you can believe in. But B actually telling you what to focus on

00:13:44.230 –> 00:13:50.100
Sam Champion: in our instance, what to focus on is something we then turn into what we call suggested actions.

00:13:50.270 –> 00:14:17.710
Sam Champion: We basically run simulations against this score to say, Okay, well, let’s say there’s too many inactive Admins, or they’ve not had a content refresh in a while. How much do we need to change this metric by to make the account likely to renew again? Is it if we decrease in active admins by one? Is it enough to change the needle? Probably not. What about 2? What about 3? Then at 9, maybe it’s enough. So we’ll actually say, Okay, it’s in renewal, likelihood of 71%. But if you can decrease in active admins by 9, we might predict that get up to 87%.

00:14:17.950 –> 00:14:26.730
Sam Champion: Obviously, you then need to make it super easy for your team to action, and so surfacing, ranking all your users by engagement, too. So you know who to speak to is step one of that.

00:14:27.140 –> 00:14:38.059
Sam Champion: But step 2 is this is where the AI itself can come in. And this is why I wanted to focus a bit on the machine learning, first, st because what you’ll notice now is, we’ve got all the data in one place. We know what good looks like.

00:14:38.280 –> 00:14:42.180
Sam Champion: We’ve analyzed that data to pick out which customers are at risk.

00:14:42.290 –> 00:14:51.579
Sam Champion: And why so? AI needs to be insightful? It needs to be explainable. We’ve given some actions as to what should this customer do to improve this particular problem.

00:14:52.310 –> 00:15:01.299
Sam Champion: And what you can do when you overlay AI on top of this is now we have context, deep context about your business, your customers what they’re struggling with, what they need to get more value.

00:15:01.800 –> 00:15:07.509
Sam Champion: We can add what we call the echo agent on top of this and play ibooks. On top of this.

00:15:07.630 –> 00:15:19.190
Sam Champion: this agent effectively live, analyzes all your meeting, transcripts, your ticket, content your emails. And the play Ibook agent then suggests actions to take and can actually draft and generate those actions for you

00:15:19.340 –> 00:15:21.480
Sam Champion: in order to start mitigating risks.

00:15:21.600 –> 00:15:22.830
Sam Champion: For example.

00:15:23.160 –> 00:15:35.420
Sam Champion: we take Apollo here, a customer that maybe the machine learning is saying, super high. You know, they’ve got like a really high engagement score and super well adopted, but actually, Echo AI has overridden this particular thing.

00:15:35.580 –> 00:15:51.630
Sam Champion: And this is why it’s really important to have both like structured and unstructured insight, because in this instance looks healthy. But actually, we’ve seen they’re having issues backing up clusters. And in a recent meeting someone mentioned off the cuff that the Vp of data the main champion, had actually left.

00:15:51.740 –> 00:15:58.839
Sam Champion: And so in this instance, this is something that could have easily been missed. But obviously, AI doesn’t miss anything because it’s inhuman. It will just analyze everything.

00:15:59.600 –> 00:16:17.019
Sam Champion: And so this is adding some context to it. The next step beyond this is, if I’m a Csm, I want to immediately know. Okay, well, what is the support ticket? Can I see it? What should I send in return? And you can use AI to answer this as well. In this instance we cite exactly where these issues came from. For example, this Zendesk ticket.

00:16:17.020 –> 00:16:34.440
Sam Champion: this meeting transcript here, this Zendesk ticket, and we also suggest actions off the back of it. And this is something that could actually be quite actionable for you guys at home, which is obviously to understand how to mitigate these certain actions. So a champion leaving or someone failing to back up clusters.

00:16:34.470 –> 00:16:51.129
Sam Champion: There will always be something in your company’s repository. Always be some customer that’s had the same issue before some Csm. That launched a play that worked really well. In this particular instance. The issue is getting every Csm to act like your best. Csm, and that’s really where AI can come in.

00:16:51.130 –> 00:17:06.919
Sam Champion: which is, we can generate play. I books off the back of it, work out when people have issues with support tickets, when people have issues with champions, leaving what type of content, what type of plays have been activated, based on that? In this case we’ve got super simple demo one, which is you need to run this backups, one on one enablement.

00:17:06.970 –> 00:17:19.630
Sam Champion: AI can then generate a super relevant email based on emails that have been sent prior and had success. We can then generate a super personalized campaign based on this. And in this demo instance. Obviously, we want to copy and paste this in. But

00:17:20.740 –> 00:17:26.530
Sam Champion: we go to the settings page and the automations page

00:17:27.210 –> 00:17:45.950
Sam Champion: where we see this eventually going is that you’ll have enough context about your business to actually detect risks based on prompts that you can customize yourself in lots of different sources, risks that are super tailored to your data because we’ve got the machine learning that understands your base. We can not only create things for the Csm. To get flagged in action.

00:17:46.240 –> 00:17:58.130
Sam Champion: but actually off the back of these start letting the agent. Once you’ve manually with a human in the loop ticked off enough of these emails. Enough of these campaigns start letting the agent do what it thinks is best. And, for example, maybe it’s to

00:17:58.230 –> 00:18:27.799
Sam Champion: create a draft email for the Csm to tick off and send. Or maybe you actually trust it enough that you can also send these emails. Now, maybe it’s scheduling an internal meeting with the right stakeholders. For example, it’s a bug with backups. Maybe the Vp of engineering is the right escalation point. Maybe what you prefer is a slack bump or a support ticket being responded to directly. But the important thing to highlight is this only works because you’ve done that initial 1st step of building a bespoke custom model trained on your data that can give context about what your customers want and not.

00:18:27.800 –> 00:18:29.020
Sam Champion: And obviously.

00:18:29.200 –> 00:18:36.110
Sam Champion: there’s lots of other stuff that you could do with AI. But that’s the main things we’re going to focus on today, which is building custom models to learn.

00:18:36.280 –> 00:18:43.530
Sam Champion: making sure that they can give you some insight as to where the risk is, or opportunity is in your customer base

00:18:43.720 –> 00:19:00.029
Sam Champion: making sure they’re explainable, or people just won’t trust them won’t use them, and you may as well not have them. And then, also making sure they’re actionable. So you can actually have some business outcomes with it and obviously start automating it, going forwards with things such as like AI generated interventions.

00:19:00.280 –> 00:19:17.949
Sam Champion: So that’s pretty much what we’ve had success with at Hook, and what we help our customers do, using AI and data to know where the risk is and take action. There’s about 10 min left. And so obviously me and Tash Super happy to answer questions about either this or what we found and been hearing in the market and tash over to you.

00:19:18.650 –> 00:19:47.799
Tash Evans: Yeah, I think just to wrap the the point, here is, AI is here to to help all of us, and it’s here to help us hit our goals, but we can’t just rest on our laurels and go. Hey, yeah, AI, is the answer to everything. We have to have some of the data, some of the machine learning there to really back us up to make sure that when we’re leveraging AI we’re operating efficiently. But we’re pointing all of our resources in the right directions. And we’re giving our teams the things that they need to go and execute and to to hit the number on the bottom line.

00:19:48.070 –> 00:19:51.239
Tash Evans: So, yeah, with that. Julia, any questions for for me and Sam.

00:19:51.470 –> 00:19:54.890
Julia Nimchinski: This is great phenomenal work. Tash Sam and team.

00:19:55.660 –> 00:20:09.349
Julia Nimchinski: one of the aspects that keeps, just, you know, coming up in the community all over again and again, is in a gentic scaling is the transitioning piece, and I love how you started this session. And on that note

00:20:09.960 –> 00:20:21.410
Julia Nimchinski: early learnings, I guess, are using your customer base. Some use cases early roi people, just, you know, practicing this transitioning to this workflow.

00:20:21.410 –> 00:20:40.329
Tash Evans: Yeah, that’s such a good question. And this came up at a roundtable recently, I think, like part of the transition period is people people messing with AI and playing with AI on their own versus it, maybe not yet being an enforced workflow. That’s part of a part of a Cs team or part of a revenue team.

00:20:40.330 –> 00:20:52.659
Tash Evans: And so I think what really helps with that transition is allowing your team to get creative and to go and play with different pieces of AI tech, whether it’s in their space or just in their personal life, so that they get more

00:20:52.660 –> 00:21:17.280
Tash Evans: comfortable with the concept of agents and AI, and the value that it can bring to all of us. As part of the transition into perhaps having more of a workflow as a business that’s AI based or agent based. And so yeah, I think the answer is allowing your teams to get creative to go figure things out on their own, because then, when you’re handing them a more AI based workflow, then that they’re already bought in right.

00:21:18.070 –> 00:21:46.980
Sam Champion: And for me personally and obviously, Tash looks after our customers, and I look after our prospective customers. One of the big things that makes people feel reassured is you don’t just want to jump in and give AI free reign to message your customers and do everything for you, because you really, at least initially, want to definitely have a human in the loop that’s proofreading things that’s checking things. It’s giving feedback. So we can make it more accurate. It’s only once you’ve had enough of those past the sort of bar of yeah, I’m happy to send this. Then you can start removing some of the manual tasks by automating things entirely.

00:21:46.980 –> 00:21:51.380
Sam Champion: making sure it’s like a phased approach is something that tends to make people feel much more comfortable with rolling it out.

00:21:52.840 –> 00:22:04.320
Julia Nimchinski: Awesome. Let’s address a couple of questions from our community here, and one of them is, how do you make sure it’s not just spitting out generic advice or missing the context.

00:22:05.710 –> 00:22:18.839
Tash Evans: Yeah. Great question. Love that one. I think the the way that you do that is, you have to. You have to train it on customer specific data. But not only that, you need to create some customer specific rules in the background.

00:22:19.000 –> 00:22:22.080
Tash Evans: So part of what we’ve been working on with our customers is

00:22:22.320 –> 00:22:28.239
Tash Evans: training our Llms and training anything AI based or always based on customer context and customer data.

00:22:28.430 –> 00:22:45.709
Tash Evans: But what the AI doesn’t know until you tell it is what’s what’s acceptable as a suggestion? And what can you go do? And so then there’s like a human element where you need to come in and go. Okay, well, AI has told me where I need to focus whether it’s based on data, whether it’s based on sentiment.

00:22:45.966 –> 00:23:05.229
Tash Evans: It’s called out where I’ve got risk and upsell opportunity. And it’s told me how to go influence that based on what’s worked previously. And this is why the machine learning piece is so key, right? Because it’s all very well having AI create you a playbook. But how do you know that’s the one that’s worked in the past. When you’ve been facing the same renewal challenges or the same technical challenges, or whatever that looks like.

00:23:05.230 –> 00:23:29.489
Tash Evans: So it’s telling us how to go do that. But how do we know that those actions are appropriate to the customer? This is where the rules come in, and an extra layer of customer context. So you might have a customer who sits there going, hey? I’m never offer this or never suggest this step, or always suggest this step in these types of scenarios, or if I have a customer who spends more than X, then yeah, I’m willing to do XY and Z. But if they spend less than that, then actually, I want to take a different

00:23:29.490 –> 00:23:30.190
Tash Evans: approach.

00:23:30.220 –> 00:23:48.700
Tash Evans: And so I think it’s this combination of always training everything on customer specific context. But then adding in that, like human, sensible layer of rules into that, to make sure that what you’re what you’re showing back to your teams is acceptable and value. Add to them and to the customers too.

00:23:50.900 –> 00:23:53.750
Julia Nimchinski: All that next question

00:23:54.190 –> 00:24:00.880
Julia Nimchinski: since layering in agentic AI. What kind of results have you actually seen like churn? Nr, what’s more.

00:24:00.880 –> 00:24:28.130
Tash Evans: Yeah, so I’ll give you an example from a customer that we’ve been working with over the last couple of weeks. We we up until very recently have spent a lot of our time building machine learning models. It’s at the core part of of what we’ve always done at hook and for one of our customers we’ve built them a model that accurately predicts churn with 70% accuracy 180 days out, which is really cool, right? Like as a Cs. That’s the dream. You want to always be pro proactive in terms of getting ahead of your risk.

00:24:28.270 –> 00:24:53.809
Tash Evans: But then, by layering in some of the more agentic stuff, we’re now able to take that up to 90%, we have 90% accuracy of predicting their churn in advance. By layering in the data driven machine learning piece with that more agentic AI side that’s spotting and listening to the sentiment. So some real real life examples for how this is starting to impact our customers and their ability to view, risk and then go action. It.

00:24:55.970 –> 00:24:57.010
Julia Nimchinski: Love it?

00:24:57.680 –> 00:25:05.969
Julia Nimchinski: How soon can you spot a real upsell opportunity? And can sales or Cs actually run with it? Or is it still too early stage.

00:25:06.740 –> 00:25:22.560
Tash Evans: Oh, that’s a good question. So we apply the same kind of methodology to our upsell models that we do to our retention models. And so we’re trying to spot that as as far out as we can. And what we’re optimizing for is that 180 days out

00:25:23.001 –> 00:25:42.260
Tash Evans: can Csms run with it? Yeah, I don’t know if that’s like a philosoph philosophical question. But yeah, like ma, massive fan of Csms running with upsell, and I think you know part and parcel is giving them the confidence by telling them where where to go focus and the the steps to take. I think is is part of the battle, for sure.

00:25:44.186 –> 00:25:46.770
Julia Nimchinski: Question from Steven, or even

00:25:46.930 –> 00:25:53.840
Julia Nimchinski: they’re asking for your thoughts. Seems like a percentage of companies think they are a non-start on this fact.

00:25:54.030 –> 00:26:00.179
Julia Nimchinski: because they don’t even have their structured data world in order. Thoughts on this.

00:26:03.240 –> 00:26:07.329
Tash Evans: I’m like trying to formulate that in my head to make sure I understood it. So is the question.

00:26:07.830 –> 00:26:14.730
Tash Evans: Are there a set of companies that are total nonstarters for this as a route, because they’re not comfortable with how accurate their data is.

00:26:14.730 –> 00:26:15.360
Sam Champion: Yeah.

00:26:15.360 –> 00:26:17.149
Tash Evans: Oh, interesting! Sam! Do you want to take that.

00:26:17.400 –> 00:26:35.839
Sam Champion: Yeah, yeah, I think they’re talking right? Specifically about like, what if you’re not a Saas company, or even a software company? Or what if you are, but your data is just abysmal, which is not uncommon. This is actually one of the primary reasons that we built Echo. So the AI agent that can detect in more like meetings tickets. Because you don’t have this dirty data problem or lack of data problem.

00:26:36.067 –> 00:26:48.149
Sam Champion: I’m in sales, right? And so I need to like reach out to people to find out who could potentially use our service. And probably 3 out of 4 people would be Dq because their data is not in quite good enough shape in order to use the sort of traditional machine learning

00:26:48.422 –> 00:27:04.507
Sam Champion: going forwards. This won’t really be a problem because the unstructured data, I mean, it’s either kind of there or it’s not. Either you have your calls recorded, or you have your tickets there, or you have, for example, like emails that you send back and forth, or you don’t. And so you’ll lose a little bit of business.

00:27:04.780 –> 00:27:24.019
Sam Champion: business, what you call it like business specific context from not having the machine learning. But it’s still like tons better than not having a system in place, and the time to value is often a lot quicker, because you don’t need that big historical ingestion and build phase. And so the echo agent is definitely built specifically, almost, for those people who don’t have the product data.

00:27:24.840 –> 00:27:25.530
Tash Evans: Even if it.

00:27:25.530 –> 00:27:26.080
Sam Champion: It.

00:27:26.080 –> 00:27:53.249
Tash Evans: You time right to. You know, we we speak to customers who who really want to to sort out the data cleanliness and who want to get to this world of like being able to have a machine learning model. And I think the the beauty of this is like, as long as you can find a way of of giving it really good context, you can leverage the the the AI side of things to to get you a good bit of the way, and at least give you some some signal to Sam’s point, whilst perhaps you, you work on internal projects to to go and work on the data side.

00:27:54.910 –> 00:28:10.400
Julia Nimchinski: Let’s address one more question from Jack. How many months of previous customer data does the Ml. Need to start making predictions and to follow up on this. How do you determine data quality before implementing the AI or Ml tap.

00:28:10.890 –> 00:28:32.890
Tash Evans: Hmm, so the the months of data. So when it comes to the the aip specifically, or the the Llm we’ve been feeding. I think about 6 months of of data into there. I think you can put a little bit less in, but we we always try and push our customers to give it as much data as you possibly can, historically, because it’s how you help it, become more accurate and learn a little bit more

00:28:32.890 –> 00:28:56.979
Tash Evans: on the machine learning side, we tend to go for more. So when we’re looking at these really like data driven machine learning models, we tend to look for at least 12 months of data. And within that with with any kind of learning you want, like a set period that you’re using to train the model. And then a completely, distinctly set period that you’re using to test it. So you always need to factor that into to whatever kind of historical data that you’re pulling through?

00:28:57.357 –> 00:29:00.209
Tash Evans: And then, Julia, what was the second part of the question? Sorry.

00:29:01.080 –> 00:29:02.909
Tash Evans: How do you clean up your data?

00:29:03.320 –> 00:29:03.780
Julia Nimchinski: Yeah.

00:29:03.780 –> 00:29:05.169
Tash Evans: How do you assess your data?

00:29:05.580 –> 00:29:10.659
Julia Nimchinski: How do you determine data quality before implementing the AI or Ml on top.

00:29:10.900 –> 00:29:35.210
Tash Evans: Hmm! I think that with the with the AI piece, it’s a little bit easier, right? Because from the sentiment side of things you’re pulling in call recordings, email recordings, support tickets. And so we we tend to have like less of a data quality issue there. On the machine learning side of things. My technical team spend a lot of time with customers looking through things like, what kind of

00:29:35.720 –> 00:30:03.119
Tash Evans: what what kind of historical data. Do you have access to? How many metrics do you have access to in there and then? Are there any gaps in the data? So that’s kind of the 1st thing that we look at. And then the second thing is, how do you link all of that great usage data up to users and to accounts, and as long as you can do that in some way, shape or form, you have a way of attributing the product usage to the account itself, and therefore something that you can that you can build a model on. So they’d be the 2 things that I would focus on if it were me looking at that.

00:30:03.580 –> 00:30:10.870
Julia Nimchinski: Super insightful. Thank you so much. Dash and the questions keep coming. How can people test? Drive this.

00:30:12.318 –> 00:30:19.289
Tash Evans: Come find us on Linkedin. Come message me separately, and we will sort that out or message Sam happily.

00:30:19.910 –> 00:30:20.470
Julia Nimchinski: Awesome.