AI in the Real World: Balancing Innovation Risk in a Regulated Landscape

In this podcast episode, Dr. Beverly Wright, Vice President of Data Science & AI at Wavicle Data Solutions, is joined by Raghu Kulkarni, Chief AI Officer at Equifax. Together, they explore how regulated industries can drive innovation with AI. The conversation covers the differences between machine learning and generative AI, the growing importance of explainability, and how regulatory frameworks help build trust and ensure responsible implementation.

Speaker details:

• Dr. Beverly Wright, Vice President of Data Science & AI at Wavicle Data Solutions
• Raghu Kulkarni, Chief AI Officer at Equifax

Watch the full podcast here or keep scrolling to read a transcript of the discussion between Beverly and Raghu:

Beverly: Hello, I’m Doctor Beverly Wright and welcome to Tag Data Talk. With us today, we have Raghu Kulkarni, Chief AI Officer at Equifax, and we’re talking about AI in the real world, balancing innovation and risk in a regulated landscape. What a great topic. Welcome to Tag Data Talk Raghu.

Raghu: Thank you. Thank you for the nice introduction and thank you for inviting me.

Beverly: Absolutely. So, tell us, why are you so cool?

Raghu: I didn’t realize you would start with such an easy question. Well let me begin like this. It’s been 6 months for me here. I joined Equifax probably in July, August timeframe and it’s been a dream. And prior to this stint, I was the chief data scientist for Discover financial Services. And for the past almost 18 plus years I’ve been doing one or the other sorts of data science. In today’s world, it’s called data science. I started out being called as a statistician. I was called very many different things. But today it’s AI. So happy, humbled and thankful that there is this opportunity for me to pursue my passion and be paid for that.

Beverly: Excellent. Well, I love that you started as a statistician. That’s wonderful. And something that seems like it’s becoming rarer. So very interesting. So, when we talk about AI in the real world, especially in a regulated environment, when people hear the term AI, they think a lot of different things. You know, they think about robotics, they think of all kinds of things. ChatGPT. When you’re thinking about it in this context, what do we really mean?

Raghu: Yeah, thanks for again starting out by defining what does AI mean, right? And it means different things for different people. So again, this is my approach towards defining AI, broad moniker AI, we can divide them into machine learning based AI and generative based AI. Machine learning based AI are algorithms which are used for prediction, right? What’s the weather going to be tomorrow, right? What’s going to be the unemployment rate in the next 6 months, right? So on and so forth. Generative is more the ChatGPTs of the world where you type in a prompt question and you get an answer for that, right?

Those are based on more deep learning, transformer-based methodologies. Now within machine learning, when we talk about predictive analytics or in certain cases it’s called classification problems, those are the ones which are more used commonplace, especially in the financial sector or for that matter any of the sectors where we need prediction per se, right. And the good thing is in the last few decades, as computation power has caught up with the mathematical know how, we are at a stage where we can not only utilize them at scale, but also be able to explain them, right? So, each and every step of these algorithms can now be evaluated, looked upon, and the outcomes can be discerned back and reverse engineered back to how these predictions were made to begin with.

Beverly: OK, so you’re, we’re talking about two, if I’m hearing you right, you’re kind of categorizing them as two different classes. 1 is almost, if you want to call it the traditional, it’s kind of funny in a new field to call anything traditional. But this sort of traditional, you know, old AI, if you will, and that’s the machine learning algorithms that are responsible for classification, prediction, those sorts of things.

And then the second is more the generative AI. And this is where you’re creating new knowledge or new information, whether that’s summary of words, whether that’s an image or something like that. Exactly. Is it pseudo safe to say that it’s kind of structured on the left and unstructured on the right if you’re thinking about the one too?

Raghu: There are pieces of unstructured even in the machine learning side, but very broad strokes. Beverly, I agree with you. There is there is a class within machine learning called unsupervised learning which could take some of the unstructured data too.

Beverly: Ok, gotcha. And we’re talking about in a regulatory environment. And for those of the listeners that don’t operate in a regulatory environment, pat yourself on the back because it’s a lot easier. But what does this mean to work in a regulatory environment? I’ve been at utilities myself and financial services and I’ve kind of had to deal with some, but can you give us a sense of what it really means?

Raghu: Yeah. I can give an example of a financial sector utilizing a machine learning algorithm to develop models which are then utilized to underwrite an individual for a loan, right. And when we are taking a credit decision on, let’s say Beverly goes applies for a credit card loan, behind the scenes, there’s a lot of analytics which takes place, the financial institutions are going to look at your payment behaviour, your past credit behaviour. I’m sure you’ve heard about Ficos and Vantage scores, right?

So, there is a lot of analytics which goes in the background to decide whether Beverly can be approved or declined for a credit card loan, right? And on top of it, there needs to be a decision on the line, pricing. Now, if you’re approved, great. But if there a risk, for example, you must have, if you had missed some payments in your past or you’re running behind in certain payments or you had too much balance right, then the financial institution might say, no, we need to decline. So when an institution declines you, it’s regulated, there are laws which the financial institute need to follow like ECOA, FCRA, compliance so on and so forth wherein they are supposed to get back to you and say here is the top three reasons why you were declined.

Maybe you are delinquent on certain loan so on and so forth, right. To get those declined reasons which are called adverse actions, those might be very welcoming from a model. To understand the model and how those reason came about, that’s why we need what is called as explainability of the model.

Raghu: And hence, as you peel the onion, peel the onion, peel the onion, it becomes extremely important as to how you develop the model and how you’re getting it out in the real world. And very quickly, if it’s OK, I actually don’t see regulation as the word regulation, I see it as if the end goal of the financial institution is to get more customers, help the customers, these regulations actually make us strong to say, here are the guardrails with which these outcomes need to be you know worked upon so that any outcome which is used out of these models are well, tested.

Beverly: Got it.

Raghu: That’s the way I look at it. It’s a win, win situation.

Beverly: Got it. OK, so I heard a few different things. #1 people and consumers and borrowers and financial units, they’re seen as numbers in a sense. They’re kind of, you know, Beverly’s a score, you know, she’s some kind of number, and there’s some sort of reason for that number.

Lots of reasons for that number. OK. And then secondly, when it comes to regulation, it sounds like you’re saying models need to be simple enough that you can explain them very clearly to regulators, and sometimes it’s probably easier to use multiple models on top of each other instead of one very complex model. Would you say that’s accurate or?

Raghu: In certain cases. When I say simple enough, model needs to be explainable enough so that no matter how many layers of modeling you go through, the final outcome should be traced back to why the decision was taken.

Beverly: Got it. Ok.

Raghu: And that is why rigor and discipline is a must for a data scientist as they go through these models which are customer facing or regulated.

Beverly: Right, OK. And then the third piece I think is you’re saying that has to be strong explanation. You can’t just, you know, have this model. Let’s say it’s super explainable, but you still have to be able to explain why this person was or wasn’t accepted for the credit.

Raghu: At the end of the day, you’re more than a score.

Beverly: Yes, yes.

Raghu: So, we better be cognizant about it. And again, I truly feel it’s a win, win situation. If you’re able to figure out cohorts of population like Beverly’s, you know which can be approved. That’s what the institutions want.

Beverly: Right. Ok. So many ways it can help the company, it can help the regulators understand better. But I would imagine that models that are more difficult to replicate, like especially some of the generative AI models might be harder to explain. It has to be very explainable. It’s better for the consumer all around. So, but it does sound a tad restrictive, right? And so, in this world where you’re trying to leverage data and you’re making decisions and you’re building machine learning or generative AI models that are explainable, it sounds like it would be harder to innovate. OK, yeah, let’s talk about that.

Raghu: Let’s talk about it. Just taking a step back, as far as my knowledge goes, we typically use only machine learning algorithms in a regulated industry when it is customer facing for a credit approval or decline, right? There is absolutely no use of Gen AI type of methodologies in those places.

Beverly: Oh, interesting. OK. So, the two categories we’re really only talking about the first one.

Raghu: Very distinct. Because as a mathematician or whatever, we have a better hold on machine learning and within machine learning also we put in a lot of constraints within those algorithms, got it. So that they don’t go crazy, right. So, I’ll keep repeating the rigor if there is one, one word which keeps, you know.

Beverly: Rigor. Yes.

Raghu: Discipline, rigor, thought, and the objective of what we are using the model for.

Beverly: So, you’d rather have a really good, strong, well built, particular Occam’s Razor limited X variables regression?

Raghu: Exactly.

Beverly: Then like a deep learning model that maybe has a better fit.

Raghu: Maybe and you know, is the additional power worth the complexity? Right. At the end of the day, I actually liked your statement of simple, right? What does simple mean? Simple means you’re able to understand. It should have consistent behavior, right? And yet it should be able to pick up the elements of why your score is. Now that’s not easy, right? Right. That’s why you need experience. You need true, you know, hours being put in on developing these models.

Beverly: That’s going to be really interesting when we start seeing more people coming out of schools that maybe don’t understand the statistics behind it and they’re using auto ML tools, and they’re sort of pushing buttons without understanding what’s going on behind the scenes.

Raghu: Excellent segue, excellent segue. And that’s why, you know, again, my limited point of view and disclaimer this is mine is I would stay away. I mean, it’s good to know how these automated processes work. Click a button and there is a model. But at the end of the day, that’s not how the real-world works.

Beverly: Exactly.

Raghu: If you have to know how those were programmed, right? So that base foundation knowledge will always remain. And in fact, people think that, you know, in the world of automation, jobs will go away. I would say prompt engineering. I’m sorry I’m shifting on you.

Beverly: No, it’s good.

Raghu: But prompt engineering will actually be best utilized by people who are SMEs because it’s a two-way street, right. The more experience you have, the better questions you can ask, and the model learns back and spits back the right answer, right? Right. Now on the generative side of the applications, we are still learning as a community. So, you’ve got to be extra cautious, right.

Beverly: Without a doubt.

Raghu: Without doubt, right? I would say that’s fascinating for me, right?

Beverly: But the generative models are a lot harder to unpack. You almost have to iterate repeatedly to see what kind of behaviors they have. There’s a lab that, I believe it’s Harvard. It’s called PAIR, People and AI Research, and they have to iterate over and over and over again to make inferences about what the models are doing as opposed to just.

Raghu: 100%.

Beverly: Backing into it, which if you’re a statistician, you can always back into, say, a regression as an example. So, there’s a big difference there, right?

Raghu: And thanks for bringing up that lab, what they are doing is an iterative process based on the outcome, right? There are some recent papers by Anthropic, yes, mapping the mind of an LLM where they’re looking at how what phrases are activated when a certain particular prompt is given out.

So, we are making progress there and hopefully next time when you have somebody else on this podcast, you will have another breakthrough because the rate at which all the scientific community is trying to explain these LLM’s is out there, yes. So, once we reach that, then generative AI becomes stable stakes, right. Then it becomes a computational thing because these things are not cheap. And coming back to generative, it all depends upon what kind of usage we are looking at.

Beverly: Right. OK. So going back to the sort of first half where we’re saying, let’s limit ourselves initially to say machine learning models because we’re dealing with this regulatory environment, it’s got to be incredibly explainable. Looking at that half, if you’re trying to innovate, how do you innovate in that environment where you’re constantly having to think about the consumer and the regulator and the data? Like how can you recommend innovation happens in that world?

Raghu: So, first of all, so if you’re talking about the generative AI side of it, yes, in fact, ML also, this is a team sport. And what do I mean by team sport? This is not just a data scientist out there in their cave doing what they do, right. It actually starts with legal and compliance, right?

My best friends are in that department. And then you start by simulating how the final outcome is going to be useful for the end user. So, for example, let’s say I’m trying to create an agentic AI to which is able to answer on all the procedures out of 100 PDF documents. When a person is on a call they need to quickly respond, and this agent AI is trying to help. Role plays that, see what kind of output will be useful for the final person.

Beverly: Right.

Raghu: And to do that, the person who is actually doing the job becomes the centre of the problem statement. So, the more real we get, the closer we get to the solution, right?

Beverly: And the more collaboration it sounds like we’re going.

Raghu: To that’s the team sport.

Beverly: Right, I see.

Raghu: Right. So as much as I would say I’m the chief AI officer, it’s a true team sport. The final outcome is the real winner.

Beverly: Right. Do you feel like we’re ever having to compromise, though, in some ways with what we could do because of the regulation? Like, let’s be real, you know?

Raghu: Absolutely. And again, you know, compromise, yeah.

Beverly: But give up? No.

Raghu: Absolutely not, right? And I’m very, very hopeful of the rate and pace of change. But I’m also glad that we have these regulations which guardrails against hallucination, toxicity, right? Which are real things right now. There is momentum also to measure the accuracy of these LLMS, right. So, there is things happening, I think we are not going to give up for sure, Beverly, right.

But all these need to go hand in hand, right? And truth be told, we still have to reach a point where in I mean, what are we trying to do with these LLMS, right? That’s the key, right. So, there is multiple innovations happening. How best we can utilize them, how best we can explain them? How cheaply can we install them?

Beverly: Yes. Oh, we got to think about money too. Gosh. So, does it take a different kind of mindset, a different kind of person, different kind of attributes to be able to creatively innovate given that it’s a regulatory environment? Like does it take a different kind of person to think under those kinds of constraints? Or what would you say are some of the top attributes for your innovators?

Raghu: First thing is humility that things are way more complex in reality and curiosity to keep learning, right? Because by the time I’ve learned how to use one particular LLM, there is 10 more in Hugging face, right? So, a curious mind which is continuously learning, right? For me, I’ve been humbled every day, so it’s easy for me to say a humble mind to understand that it’s tough to solve real life problems, right? And again, patience to deal with all the things which are changing dynamically.

Beverly: Yeah. So, by the humility, let’s unpack that a little bit. Do you mean the acknowledgement that you really don’t know everything? That you have to consider there are whole pieces of knowledge you just haven’t even touched yet?

Raghu: Exactly. And when you go with that mindset, actually you overshoot the final outcome.

Beverly: Oh, I see.

Raghu: That’s the way I have seen it. Again, the best ideas come from the SMEs who are working in those areas rather than just the data scientist, right? OK. For example, if you’re trying to automate some call listening processes, let’s say a company gets 30 million calls in a year, we can’t listen to all of them, right? We need to summarize them and say here are the particular complaints which are coming out and, you know, just filtering them out.

What kind of filtering happens, what kind of complaints they are. They all come for people who are actually working today, not just Oh yeah, I will figure it out. That’s why I’m saying team sport, understanding the process. Focus on discipline to understand the entire end to end process and get the SMEs to design the problem with you as you’re going through this journey.

Beverly: Gotcha. OK, OK. When thinking about regulation, you know, I kind of have this lasagna theory. I’m part of the Emory Center for Ethics. I serve on that board. And the lasagna theory is that there are kind of layers. You know, there’s laws, there’s regulation, there’s policy, and then there’s ethics.

Ethics are things that you should do, but you don’t necessarily have to. Whereas regulation is different. You know, there’s more consequences with it. So do we need to just forget about ethics or we don’t have to worry about ethics because we have regulation? What’s that dance between regulation and ethics? How does that look?

Raghu: Oh, I feel it’s equally important, right? How you evaluate it is a big question.

Beverly: I see.

Raghu: Right. My way of looking at it is first try to solve the problem. Then as you’re solving the problem, see where the regulatory guard lines are there. And as you’re solving with directly regulatory guard lines, as you’re doing testing, never go into production. You know, you have to test rigorously.

That’s when you think about, OK, am I being too dominant in one particular cohort than other cohorts, right. What kind of outcomes are we reaching out to? OK, So if you’re an experienced data science person who works, again, it’s a team sport. People will point out that this is disproportionately impacting one place than the other.

Beverly: I see. OK.

Raghu: Right. So it’s it’s a natural outcome, yes. It needs to be balanced.

Beverly: Yeah, OK. A lot of people will say that I want to be responsible, I want to be ethical with my AI, but I feel like it limits my innovation. And then when you throw regulatory on top of that, I’ve heard people say things like, well, we’re regulated anyway, so I don’t have to worry about it. What would you say to that?

Raghu: Again, I have a completely, probably I have a different mindset. Yeah, I start by thinking about the guardrails because regulation at the end of the day are trying to make sure that we take care of the customers, right. So if my mindset starts from there, I don’t see regulations as a burden. I see that as a partner.

Beverly: Right. OK. It’s helping you in a way.

Raghu: Personally, you know there are much more smarter people in Silicon Valley. Regulations keep me in job too so.

Beverly: Very nice. What final piece of advice would you give for people really wanting to understand how to balance AI and innovation in AI in a regulated landscape?

Raghu: I think I I’m going to repeat myself. Yeah. But remember rigor.

Beverly: Rigor. Yes.

Raghu: And continuous learning and just understanding the process around which AI could be applied. Because, believe me, I would take a simple regression any day, compared to a complex ML, if we can solve that with the regression. So start with the problem we are trying to solve, see if it needs analytics and within analytics what kind is needed. And then go not start with Oh I have AI.

Beverly: Yeah, backwards. OK, gotcha. Always use the simplest solution. And you didn’t say this in the final piece of advice, but many times you talked about the team sport. So I think that’s also, it sounds very important. Excellent. Well, thank you so much again to Raghu Kulkarni, Chief AI Officer at Equifax, for talking to us about AI in the real world, balancing innovation and risk in a regulated landscape.

Raghu: Thanks a lot.

Regulation does not limit innovation; it helps shape it. As highlighted in this discussion, deploying AI in highly regulated sectors requires clarity, strong collaboration, and disciplined execution. With the right guardrails in place, organizations can confidently innovate while maintaining compliance and earning consumer trust.

Explore the full catalog of TAG Data Talk conversations here: TAG Data Talk with Dr. Beverly Wright – TAG Online.