2018 Conference

Carl Ryden:    We're gonna talk a bit today about using AI in banking to draw out behaviors. I got with me two guys who know a lot about this. You guys remember Jim from this morning. He's been around the world and seen what banks are doing, the different avenues around the world, and what's possible. 

Also, with us, Greg Michaelson from DataRobot. Greg is on the tip of the spear of making things real in this world. So, we're going to talk a little bit. We'll have a few questions. I'll ask these guys a few questions, then we'll open up the questions from you guys, if that sounds good. Okay?

My first question is, you see a lot of big dreams and you see a lot of hero stories, and all the things that Amazon and Google and Facebook and all these guys are doing, but a lot of time with a bank, it's how do you get off of zero? How do you get some positive momentum, start building the miracle of compounding, make things 1% better. What sort of things do you see as really the projects that get off of zero, or really good first use cases to kind of start building that muscle within a bank?

Jim Marous:    I think, first of all, we, at the Digital Banking Report, we've done two research studies now. We did one last fall on AI, and we did one just last month on machine learning, slightly different in concept, but both of them, went out to the marketplace and said, "Where are you? What is the maturity of your organization, what realm of AI and machine learning?"

What was interesting in both studies, it came up with a couple really key elements. Number one, the only organizations that are really doing very much in the AI and machine learning space are the bigger organizations. A generalist theory, but basically, it's the biggest firms. 

And I think this kind of reminds me of mobile payments, where five years ago, you could not read any kind of publication in the financial services area without it talking about mobile payments, and then you start looking at the reality of the world, and you saw that, "Oh, we're talking about penetration of 8-16%. It never got really off that until recently, but I think the buzz in the marketplace on the concept versus the implementation is vastly different. 

I think what we also found was that AI, as I mentioned, I think this morning, the application of AI in the marketplace right now is more of an evolution than a revolution. People doing a little bit better at what they used to do, and almost every organization, whether everybody's aware of it or not, are using AI for some purposes internally, but almost always regarding risk and fraud. 

So, we have a lot of AI learning around what are the things that indicate that a loan's going to do bad, a customer's going to become a bad customer. There's risk involved. But very little when it comes to personalization and development of products. So, what we're seeing is, while there's talk about it, the application into new areas, personalization, product development, innovation, things like this, really isn't starting very good.

So, that's not really answering your question, but it kind of sets the fear level that, right now, it's not all that advanced, I should say. On the other hand, and I'll refer to you, then, is that we're seeing that most organizations that are moving forward in the AI space, are not doing it internally, and this goes across all sizes of organizations. 

One major reason, and that is, there's just not enough talent in the marketplace, there's not enough experience in the marketplace, and people are more likely to feel more comfortable going out and testing some people that have already done it or done it for others than going out there and doing it themselves.

Starting from scratch right now is a tough way to go, even on the most basic functions. I actually think he'd say the same thing.

Carl Ryden:    That actually gives a good chance of, speaking of someone who does it for others for a living, particularly for banks, it'd probably work well for you to introduce yourself and then talk about doing it for others and how do you help them get off zero.

Greg Michaelson:    Right, so I'm Greg Michaelson. I'm a general manager for Banking at DataRobot, a data scientist. You can tell by my inappropriate choice of footwear. [crosstalk 00:04:12]

But yeah, I'm a data scientist. I started out building commercial credit scoring models at Regents Financial back in the day, back before the meltdown and everything, but over the last three years, I've been at DataRobot, really helping organizations figure this thing out. There are really two paths that organizations go down.

Well, three. One is, we don't want it, we don't care. Right? I'm gonna set that one aside, because that group of individuals at organizations is getting smaller and smaller, particularly as you see all these fintech companies come up into the lending space or the payment space, gobbled up little bits of the market. It's freaked people out, and so nearly everybody is in the market of saying, "Hey, we've got all this data that we're collecting. How can we use it?"

One path that people go down is the vendor path. There are core models that everybody needs. You need fraud models, you need credit scoring, you need loan loss forecasting type models. There are three or four or five core models in the line of business that are absolutely necessary. A lot of organizations buy them. That's an expensive way to go about it, but if you're not interested in developing that capability internally, then that's the route you go down.

The other route is the one that you should go down, and that is becoming AI driven. That's what we call it, where you're looking at every line of business, every organization, every functional area in your department, and saying, "How can I use my data to optimize, to automate, and to learn more about the way my business works?"

I've seen one cross sale use case, trying to identify customers for targeting for deepening of a relationship, generating $10 million a year in revenue for one line of business over the course of one year for one product, in a relatively small business banking unit. It's amazing where these use cases are, everywhere from even HR analytics to prospecting and sales to fraud.

One Canadian bank is using machine learning to reduce its investigation costs on AMO. If I can root out the ones I have to investigate, they actually cut their expenses by 60% by using machine learning to cut out the false alarms that they were getting out of their process. To me, the key to giving off zero is to teach the organization to spot opportunities, because there are literally hundreds of them in every line of business, and if your executives and your leaders and your line of business people, if you can give them the keys, as it were, the ability to spot opportunities to use AI, then that's gonna unlock the potential. I think that's the way you get off zero.

Jim Marous:    Thank you, because you have a good point that I kind of reverted to a little this morning, is that the best way to start is to find direct paybacks, is to say, "Okay, this is going to give me this much cost savings, or this much revenue, and it's almost a guarantee." It's one of these things that, you can fall upon it and it's going to do well for you.

The key, though, when you're looking toward the future, beyond just staying in the current, is to say, "How can you connect that with some kind of customer experience modeling that'll take it beyond simply loss reduction or cost reduction, and that, what's good is, you have money to start with. 

It's kind of customer onboarding, is that very few customers are ever going to say that a customer onboarding program doesn't make them money. The key is, how do you make that, then, a little more than that, and maybe use some of the revenue that's going to be generating from that, the reduction of losses, to apply towards better customer experience, because, at the end of the day, most of the things we're talking about in this room, or the room over there or any of the rooms, are going to be things that make us money. 
    
We have got to find ways that are going to make the customer experience better, that's going to make us more revenue, to get to know them better. One of the keys that you're bringing up is that most of these models take in outside information as well, to make the models more powerful. If we're going to be taking in outside information, let's not take it in simply for taking it in and building great reports, and great cost-reduction models.

Let's bring this in to apply even more insight into how can we make this into a product development or a customer growth tool, and it's the other side of the same coin. To be able to pre-approve somebody, to be able to go down the path and say, "You know what, this tells me enough that I can take some risk on the customer, because of the fact that I reduced the risk on other people," that's the key to the future. That's what the consumer's going to be expecting. 

They don't sit by and go, "Oh, thank goodness they saved some money on costs or they reduced their risk." That doesn't effect them. But I think it's important to make it that next step.

Carl Ryden:    I think it actually ties back a little to the last talk. You were talking to the last talk about removing friction and adding delight.

Jim Marous:    Right.

Carl Ryden:    And that kind of leads the relationship to a certain place. Okay, so getting off zero, there's some news cases and learn to kind of smell and find the value, and then there's inevitably, I think on thing that holds folks back is the unknown unknowns. It feels like we've never done this before, and it feels like, once I get into it, the way our IT organization works and the way this works, everybody who needs it doesn't know how to make it happen within the organization and all the stuff that's going to get thrown in their path. What are some of those? Can you reveal some of those, demystify some of those and turn them into at least known unknowns at this point?

Greg Michaelson:    Yeah. We run into these blockers all the time. Probably the biggest one that you're gonna face is detractors. Turns out all problems are people problems. That is no different in this space. 

You're gonna look across your organization and you're gonna see people that own servers that really, their number one priority is making sure that nothing breaks, ever, and if you do something that's outside the box, then that makes them incredibly nervous, and so they want to delay and stall and stop. 

You're gonna see compliance people that have forgotten, maybe they never knew what the rules were, but they're sure the answer is no. You can't do that stuff. They think that the independence is the opposite of collaboration.

Whatever you're looking at, it might be data scientists, it might be regulation people, it might be IT, or it might be business people, then they can say, "Oh, I don't really need help with credit scoring. I can spread a financial statement. I know how that works. Thanks but no thanks." Right?

So, you're gonna look out at your organization, you're gonna see detractors, and they can bring things to a halt, so it's super important to be prepared for that, and to be aware that there are people in your organization that are going to look at these kinds of new technologies and say, "That's not for us." And they're going to have interesting and good sounding reasons, but those may be the reasons kind of on the way down.

Carl Ryden:    You know I have one, and I'm answering my own question, so I apologize for this, but the one thing we've seen, is there's a lot of places in the organization where, really, folks are operating with a bit of a wet finger in the wind, is where, right now, they have to make a judgment or make a prediction, there's somebody that needs some number to fill in this application, and they don't know what it is, but they'll say, "You, Mr. Ryden, tell me what this is." 

And so, they're making a guess. They're going, "Eh. Here's what it is." And I always find it to be easy to look for wet fingers in the wind, and then, the whole story, you don't have to outrun the bear, you just got to outrun the other guy. It sets a threshold of, and even the compliance is, I don't think yours is accurate, it's not 100% accurate, because it never will be. It's a prediction, but let's rewind the tape here. How are we doing this today?

And having that good benchmark of, "Today, we're making this up. Today, we're just pulling this out of the air," yeah, it's 80% accurate, and then they start applying all of, because you're a thing now, they can apply all of their compliance and stuff to a thing, whereas before, there was nothing, people were just making it up. 

You said, "Point data, what we're doing now, and tell me we can stay there," and I think that's a good opportunity to find places where you can get off zero and overcome some of these things. 

Jim Marous:    Well, another-

Greg Michaelson:    Change management is a thing. I'm sorry.

Jim Marous:    No, you go.

Greg Michaelson:    Change management is definitely a thing. In the next five years, what's going to happen is that tools are going to become so much better, whether it's building models or deploying models or Andy's going to become so smart or whatever it is, that business people are going to have to become more technical or they're going to become completely irrelevant, and technical people are going to have to become more business-like or they're going to become irrelevant.

Tools are kind of blurring those polarized, specialized skills, so that people are having to be pushed towards the middle, and to be able to understand both the technical details and the business details. That's definitely what we're seeing.
Jim Marous:    One thing I saw, [inaudible 00:13:55] I just saw the picture again. About five years ago, I went to Poland and saw M Bank in the development of an online mobile app for consumers that basically results in a one minute online, 30 second mobile consumer loan.

The whole concept was, we want to make money available to firstly, every one of the customers, to some amount on a clickable button. The reality of that is, the way that got done, the only way it got done, was they put representatives from every interested party in a room and had the leadership of that organization say, "here's what your goal is. Here's what we have to achieve. You guys work together to find a way to do it." 

Which works a whole lot different than sequential turn downs, where everybody's saying, "Okay," but then compliance goes back. The second time it says, "I liked it the first time, and I said this, this, and this. You made those adjustments, but now I still don't like it." 

So, they have a mission that says, "I'm not going to like it at all, but I'm gonna find that one excuse all the way around the way." You get people in a room and you get leadership that says, "We need to accomplish X," whatever X may be, a mission, and you have everybody in the room saying, "How do we get to that?" That makes compromise a whole lot more likely.

Greg Michaelson:    Don't let them out.

Jim Marous:    Well, that's about it. I mean, the picture I have is 22 people in a room with white pages all over the boards on what had to be done, and so everybody had to move a little. So, it wasn't going to be, "Oh, I'm going to give everybody $10,000." No. Even giving $500 made the consumer feel like they were worth something. 

So, you had different components. You had contingency approvements, whatever it may be, but this goes with any mission out there. Because your point, we're going to be getting into very uncomfortable space as bankers. We're going to be asked to do things in the future that our consumers are going to want that they're going to say, "I don't care if you can or can't do it. Find a way."

We need leadership to say, "We're going to achieve this, we're going to put all the interested parties in a room and say, 'By the way, you have to get there,' so eventually it's like a jury." You can't leave that room with a hung jury, and at the end, everybody's got to give ae little bit, but at the end, they're going to come up with a solution. It maybe not be exactly what they went in there to achieve, but it's going to be a whole lot closer than doing it the way we traditionally do it, which is not agile. It's basically waterfall, and waterfall will not work as you're trying to move people into new areas.

Carl Ryden:    Well, quite a lot of organizations just do waterfall really fast. They call it agile. 

Greg Michaelson:    How many folks in the room have ever deposited a check with your phone? Yeah. Who knows the first bank to ever do that?

Carl Ryden:    It was USAA, right? And it was like three years between when USAA started doing it and Bank of America and Wells Fargo and some of these others started picking it up, right? You got to believe that the guy that said, "Okay, we're going to do this at USAA," that there had to be people saying, "What's going to stop people from taking pictures of fraudulent checks or multiple times or what if it gets lost and the person no longer has the check?" There's all kinds of reasons not to do that, but now it's industry standard.

Jim Marous:    Well, industry standards [inaudible 00:17:12] away, and this applies to every part of the business model. At Wells Fargo, where I have my personal account, and I have quite a bit less money than I have in my business account, they give me the ability to do remote deposit cashier up to $100000. That scares me, okay?

But they allow me to do it. PNC allows me $5000 a month, and I told them, I said, "You see the flow of money in my account, you see how much I take," and by the way, I have to take a picture of a check to transfer money between PMC and Wells Fargo twice a month. I've never written work checks, and I'm writing them now, because of the fact that transfers between institutions are just broken.

So, I go to PMC, and I say, "Okay, can you give me an ability to do more?" They go, "I'm sorry, but the limit's the exact same for everybody." I'm sorry. It doesn't take just a banker to say, "WTF?" What are you doing? What do you mean it's the same for everybody?

Carl Ryden:    Yeah, so the machine learning WTF detector algorithm is fairly easy to build. [crosstalk 00:18:18]

So, I got one more question and we'll open it up to the audience. Getting of zero, what sort of the unknown unknowns and there's compliance in IT, and that's the normal things, but they are overcomable, and I think bringing them into a room, getting them on the same page and giving them a charter where, for the compliance officer or the IT person, right now, failure is defined by the narrow scope of their job. Once you put them in that room and your mission is X. Now, failing at mission X is on their job, and they have to balance that, and I think that's really-

Jim Marous:    And they're risk adverse.

Carl Ryden:    And they're risk adverse, and they don't like to fail, and all that. So, hero stories. A lot of work to do to go through all that, and you mentioned a little about one, about the cross cell. What are the other ones where you go, "Gosh, this is where I'm gonna really unlock a lot of value" and ones [inaudible 00:19:16]

Greg Michaelson:    Yeah, there's mountains of them, so if you think of any line of business, you've got a huge opportunity to invest in machine learning and automate some decisions there. 

One example is fraud one that I mentioned, or the ML case that I mentioned. So, almost every AML use case that's out there today is rules based. If the transaction has characteristics A, B, and C, it raises a flag, and you can't miss one, or it's a big problem. 

So, the tendency is to generate as many as possible, and then throw people at them. Right? So you have 10, 20, 50 people that do nothing but investigate these fraud or AML cases and say, "Alright, is this severe enough to generate a SAR?" And if it is, you fill out this [inaudible 00:20:10] report, send it off to the Feds, and that's the last you ever hear about it, which is awesome, it's great feedback loop there. Thank you. Thank you federal government.

That system could actually work if there was a feedback loop, but what these guys found is that, if they could build a predicted model to say, "Will this flag, will this transaction generate a SAR?" So, just predicting the outcome of their investigation, and it turned out that they could, without missing a single SAR, eliminate 60% of the work in that entire process.

So, imagine the savings there, just from an expense perspective, so AML was a huge one. Fraud is the same way. So, most fraud detection systems, like point of sale fraud, transactional fraud systems or deposit fraud or application fraud or identity fraud, any of those are rules based. 

How many have ever had their card declined when you're traveling, like on vacation? I hate that. It makes me crazy. But now, I get an email from Citibank whenever I travel that says, "Oh, we noticed that you bought a plane ticket to Austin, don't bother calling us, because we've already made sure that your card is going to work." 

That's awesome! There's smart AI and there's dumb AI. How many have ever called their cell phone company on the phone? They treat you like they don't know you. The first thing they say-

Jim Marous:    Every time they transfer you, they also think [inaudible 00:21:46]

Greg Michaelson:    Please enter your phone number into, followed by a pound sign. They invented caller ID. My cell phone company knows everything about me. My bank knows everything about me. They know where I shop, they know when I shop, they know where I vacation, they know everything about me. They know what mail they send me. 

When I call a call center, they should be able to figure out why I'm calling and route me, and if I'm upset, they should be able to send me to the right person. I mean, if you read somebody's file, you can figure out where they are and who they are, and imagine how you could impact customer return, retention, customer satisfaction, all those things, just by being smarter about how you interact with customers that are reaching out to you. The technology's totally there.

Carl Ryden:    All the data and intelligence that exists anywhere within the organization to focus it into that moment, where you can actually create a better customer experience. Exactly.

Jim Marous:    And that, the challenge, also, is that we have situations where the difference between a digital first organization and a legacy organization. So, when I started my business, I had to accept credit cards as part of my payment process for subscriptions. 

Well, to get a credit card processing approved, you have to set up a reserve account at another company that takes a percentage of every transaction, builds this amount up, so that if anybody ever wants returns or has a problem, that they will pay back the money, and that's covered.

Well, this amount kept getting bigger and bigger, and it was a legacy organization. It got up to the amount of $40000. Well, we've never has any return, any subscription canceled, just by the nature of the business, that they don't understand. You had to put up reserve, but that was my only option. 

Today, I was mentioning to both of you, PayPal, which now, by the way, I don't get credit card payments anymore directly, credit card payments now all go through PayPal for me. No bank has any problem with that. We send it that way, we build the balances, I transfer them once a month to my traditional bank. 

But what's interesting is, PayPal has built a knowledge of what my business is. They came to me today with a capital funds business loan that said, "We have a business loan waiting for you, simply go through this short process, we'll approve it."

Now, it's a fee regular rate, as he mentioned, it's probably going to be usually limit type thing. But most small businesses don't run on logic, they run on simplicity and need, so if I need money, and if I need cash, I need it today, and I may not care how much I have to pay for it, so I can have it, and I'm only going to pay X amount of fee, a short fee. 

It's no different than a person that goes to an ATM that's not theirs and pays $3 to take out 100. That is not a really good interest rate for cash that you already own, but I think the reality is, is PayPal's using machine learning and advanced analytics to be able to determine what can they give me based on what they know about me, which is simply transaction value and amounts, while the other organization's saying, "We're going to go the legacy route. We know you have a credit card, and we know you accept this many a year. It's going to be this percent no matter what kind of business you're in." So, I think it's important to look at it that way.

Carl Ryden:    Yeah, I think a lot of those move more to the front of the bank, moving more towards experience and revenue. I think the takeaway is, a lot of the places start with an evolution from the place out, but at least there's places where they're comfortable with those, find the low hanging fruit where folks are gonna be maybe making just wet fingers in the wind, target those, because now you've got a good stocking horse to say, "Well, it's better than that," get off of zero, begin to improve. You can point to success stories, and then kind of grow into some of these, start building that track record of success, and then ultimately it's going to take over.

Let's break. I want to get questions. We got some questions. I think they're going to turn off my mic. Who is the mic runner? Is Scott around here? Yup, there's Scott. So, we only have three channels, so they're going to turn off my mic, so I'll just yell, or actually, you can go ahead and ask the question.

Jim Marous:    You yell.

Carl Ryden:    You yell, and then we'll go ahead and answer.

Speaker 4:    I think I can [inaudible 00:25:49] I thought I heard some [inaudible 00:25:52] is going on in other countries, and you said, I think, particularly the [inaudible 00:26:05] that part of the country. I forget the geography of it, but what regulatory restraints do they have? Is it similar to ours, is it as onerous as ours, more onerous, less onerous than ours? And if so, how does our regulatory environment impact the AI and machine learning, or does it impact it at all?

Carl Ryden:    So, the question I think is, is our regulatory environment disadvantaging folks here, and you guys have offices all around the world. 

Jim Marous:    Our regulatory environment is certainly more cumbersome than a lot of countries. It's not as cumbersome as the UK. The privacy regulations in both the UK and Canada are much more difficult than they are here, and they will continue to do that. We're lagging on privacy and fraud and all the other regulations, so while we get concerned about how much regulation there is, it's not as bad as the other guys.

On the other hand, it is worse than some of the developing countries, but because they are younger financial countries, younger financial environments overall, a Poland, a Chili, a Brazil, it's not the regulations that are making it so they can do more, it's the technology and the fact that they're not working with legacy systems. 

They really have a handle, they're much more digitally adept than we are, they built it that way. Even we're seeing this in the UK. Barkley's and Lloyds. They're doing some really amazing things, but what they're finding out is that they have to work within the law but not to love the law. 

I was in Rome, but heard a major US bank, top four US bank, say, "We no longer are going to wait for regulatory approval. We're going to build things with the regulations in mind, knowing that, if we ever have to defend them, we'll be able to defend them," and that's a bold move that can be made by the biggest banks, but they also do it with enough regulatory things that they say, I kind of know what they're trying to accomplish, and then you do that.

So, I would say that, in some cases, the regulations are lower, but the much bigger difference is their digitalization of financial services is much better. 

Carl Ryden:    And their lack of [inaudible 00:28:25] cuffs.

Jim Marous:    Oh, gosh, yeah.

Greg Michaelson:    So, I think there's some interesting stuff happening. Recently, the FDIC came out and said that the banks with over $1 billion in assets, they were going to have to follow the same SR 11-7 guidance that the big banks follow from a model government's perspective, which is a big deal. 

There's some infrastructure, some governance and everything that's going to go on around that, and we've all seen Zuckerberg on TV talking about privacy. I don't think the US is particularly far behind on the GDPR, the European regulations around data privacy.

The most interesting thing in GDPR, from the modeling perspective, is that GDPR actually requires organizations that use machine learning and AI, to be able to explain why the model's doing what it's doing to the people it's doing it to. And we already have that with Adverse Action type stuff in the US. So, somebody's credit is adversely effected by some piece of information, you have to tell people why. 

So, I think that's sort of being offset by sophistication in the tool set, so DataRobot is a machine learning platform. We offer a lot of stuff like transparency and the models, explanations of why the predictions are what they are, and so you can get a lot of that stuff out of the box.

We even automate model documentation these days, so one of the big pieces of SR 11-7, the model government step, is being able to document and prove that what you've done passes the regulatory bar, and so, generating that documentation is an automatic thing.

I guess what I'm saying is that technology, I think, is actually going faster than the regulation is, and so while the regulatory burden in the US and Europe is pretty high, technology is, I think, making it easier to keep up, at least from an AI and a modeling perspective.

Carl Ryden:    Well, and-

Jim Marous:    They're doing it in the UK that the privacy regulations are in tandem with their own banking regulations, which is one of these things that they're going ahead on in certain areas, and kind of making it a little stickier on the other. 

Carl Ryden:    Just one hopeful comment for those in the room, with enterprises, with banks in particular, they're probably been slow to use the data they have about [inaudible 00:31:00] to help their customers, but unlike the Facebooks and the other folks in the world where, in Silicon Valley, the statement is, "If you're not paying for the product, you are the product." They are forced into a model where they have to harvest your data and then sell it to others to kind of, almost the creepy stuff of sending it to advertisers to use against you.

As banks, I think there's an opportunity to take a position of, it's help not harvest. We use your data and we're a fiduciary of what we know about you, and there's actually a Yale law professor proposed in Zuckerberg's thing, was quoted as forcing folks to be fiduciaries of data, that you have to be responsible for the data, just like you're responsible for the financial fiduciary, and I think, for a bank, to stake out a case is, we're in the help not harvest business. We take your data and we use it to help you be better to your customers. 

I think would be a really positive place that you guys can stake out. The fact that you are where you are, you have an opportunity to do that, and so I think there is some hope. A really good opportunity, I think, for [inaudible 00:32:03]

Jim Marous:    Well, it's not even an opportunity, it's a community requirement.

Carl Ryden:    It'll be a requirement.

Jim Marous:    Because Facebook has now made people aware, and is going to continue to make people aware that their information, be it personal or business, is worth something. Unless we get something back in return, unless we, like I said earlier today, instead of good reports, you have to give great experiences for the data that we're harvesting. 

If we're not giving that back to the consumer or the business, they will have a right to come back against us in saying, "Somebody else will," and that's why all these Fintech companies are coming into being, in both the commercial and the consumer side, is because there's a break down of either utilization of digital transparency and digital technology, but just, more importantly, the better customer experience, being able to develop that better customer experience.

Carl Ryden:    Other questions? You got one, Seth?

Speaker 5:    For those of us that aren't digital or data natives, we're immigrants, a lot of times, I feel like I could move the ball forward, get off of zero, if I have some sort of human intervention or human oversight. Can you give us some examples of when you've seen human intervention, at least get some confidence until you get momentum to get off of zero?

Carl Ryden:    So a human in the loop type situation.

Greg Michaelson:    Yeah, yeah, so, I think that's where a lot of things start, particularly in the credit scoring space and in the fraud space. Being able to see what the model says does not necessarily mean that you have to do what the model says. The very first credit scoring model I ever built, everybody was very nervous about it. This was back in the day. It was a logistic regression model, kind of like the most vanilla thing you can do these days.

And all these crusty old credit guys at the bank were all super nervous, right? "I don't know about this model here." They were used to their score card with their cut points and you'd fill out your excel sheet, you'd figure out your risk rating and all that kind of stuff.

So, what we ended up doing was taking the model and then we gave them the ability to override it, but within a structured way. We gave 10 or 12 reasons, and they could really pick whatever reasons they wanted and change the rating however they wanted. 

So, they had complete flexibility to do whatever they thought was right for the particular account. Then a year later, we went back and took all the data and we said, "Alright, let's look at default risk on all the accounts that got rated, and see who did better, the model by itself, or the model plus the risk grading overlays?" 

And we were surprised to find that there was virtually no difference in terms of their discriminatory power in their ability to detect default, which, I'm not saying that those guys didn't add value, I mean, they added a ton of value from the perspective of knowing the financials, being able to work with customers and so on, but they weren't adding value as far as being able to detect default better from a modeling perspective.

But that's just one example. But the thing about it is, the way you deploy these models is highly flexible. If you don't want it to be automated, don't make it automated, make it human-assisted. If you want it to be just one input in the process of approving a loan. 

One other example is loan cycle times. So, let's say you're a commercial lender and you're writing loans to businesses and some loans take a long time to approve. They got to be touched by four or five people and signed off by managers, and if they're over a certain limit, or if he guarantor has certain characteristics, or collateral or covenants or whatever, right, the longer it takes, the more likely you are to lose that loan to a competitor.

So, cycle time matters. It's super important. But if you could take and build a model to predict how long it's going to take to process a loan, or how many people are likely to touch that loan before it gets approved, you could do all kinds of stuff with routing. 

If it says it's going to be approved in three hours, send it to the guys that can approve it in three hours, guys or gals. If it's going to take two weeks, just escalate it now, and see if you can get it done in three days or something like that, so it doesn't have to be just raw automation. It can be very human-assisted, very fit into your processes. 

The way I define artificial intelligence is you take your data, you take your business processes, whatever logic and constraints and everything, and then you take your predictions, your models, you combine all that stuff together and you build a system that can do things that would normally require human intelligence. And that can be as hands on as you want. 

Speaker 6:    Can you define machine learning and tell how that's different from artificial intelligence?

Greg Michaelson:    Yeah. Sure. Machine learning is basically using your historical data to build a model that can predict the future. You basically take all your loan data and all your file data, all your AML data. You know what the outcome was, and so you're going to use the machine. The machine is going to learn from those historical examples to build a model that can then be used to predict the future. 

AI is deploying that model. So, now I've got a model that can predict fraud. What am I going to do with it? Well, I could model those transactions in real time and block them if they're likely to be fraud. That would be an AI system. 

If somebody calls my call center, I could run some predictions to see why they're calling and then make some decisions about how to route the call based on what the models say. So, that's not the strictly theoretically accurate definition of AI. [crosstalk 00:37:57]

So, machine learning is stage one. I need the historical data to build the models. AI is stage two. I need to deploy those models and actually start to get value out of them.

Carl Ryden:    The more marketing flavor of the definition of AI is whatever machines can't do until they can. 

Greg Michaelson:    There you go. 

Carl Ryden:    It's always this kind of far fetched thing out in the future, so. Back there. Seth, get them. 

Speaker 7:    Hey, you gave us an example this morning, a demonstration of what you think the future looks like in communicating [inaudible 00:38:30] I'm curious, as we continue to go [inaudible 00:38:34] that interact with the firm instead of the branch, what are you guys' thoughts on the evolution of that branch network in banking and where do you see it?

Jim Marous:    I'ma be a little different than the speaker last year. I don't think [inaudible 00:38:52] go away, but I think that what is going to happen is branch networks as we know them today are going to get more sparse with regard to 95% of the organizations. I already know and it's very clear that Chase and Wells Fargo are going in the opposite direction, but that's simply put billboards in more areas, as opposed to making sure there's higher concentration.

They're actually reducing concentration but increasing breadth of distribution. The consumer does not care if their branch is five miles away or one mile away, 10 miles away or one mile away. The reality is, they want to know it's there in most cases, and that they could use it to open an account or to complain. 

Very few consumers say, "I want it because I want to do transactions," even in the most staunch of surveys. So, the reality is, the branch network and the people within the branch are going to be there not only in a fewer amount and farther apart, but most importantly, they're going to be there for different reasons. 

Now, I don't believe in the, "If you build it, they will come," so I don't believe in the organizations that are saying, "We're going to move tellers and we're going to increase more advisory people."

Just because you increase advisory people does not mean you're going to get more advisory business. Consumers don't all of a sudden say, "It's been a numbers game. I haven't gone to my bank for investment advice because there haven't been enough of them." No. They don't trust banks to make that decision.

Now, that can change with AI and machine learning and things of this nature, but that's going to be a hard one, so I think that the branch will be there in some way, shape or form, but on the other hand, most people now, there's a lot more people that do not transfer their financial relationship because they've moved. They realize it just doesn't matter. 

I've been 11 years away from California, I still have my Wells Fargo account that I kid about all the time, but the reality is, I don't need a sign to know that Wells Fargo is going to be there, and they've always performed when I've had a problem, very quickly, so, again, and some thing thing with the commercial loan officer, okay?

I don't believe you're going to need to have them in the location of all the people that want a commercial loan, but I think you have to find a way to make them available, and the reality is, technology's going to make it so these people are actually more available because the future's going to be where a commercial loan officer can be in their office all the time, seeing three to four times more people on a more personalized, better basis, because they're going to have more information at their disposal, going to be able to meet with them personally and interact, because again, we want humanization, and they'll be able to bring much better resolution and advisory services because they're going to have that technology at their disposal.

Carl Ryden:    Alex, up front.

Speaker 9:    [inaudible 00:41:50] So, Elon Musk is sensationalizing the dangers of AI. Specifically in the financial services industry, do you guys see concern that it could be put to bad uses, so to speak?

Greg Michaelson:    So, Elon Musk, whatever. We're a startup. He also recently famously came out and said that people are rushing to enter in AI, he said, if your competitor is rushing to build AI and you're not, it will crush you. That's a direct quote. 

I think the robot takeover stuff is a little crazy. Having said that, there is risk? I mean, that's why there are model risk management departments. I know the guy who was running Frannie Mae when it melted down, largely due to model risk. I mean, the whole financial collapse was due to problems with risk rating and how that whole system worked.

So, I think there certainly is risk, and part of it is the Black Swan problem, you can't predict things that have never, ever happened before. It's just not possible, and part of it is just you have to be responsible and careful in the way you do things.

Jim Marous:    That's why the humans have to be involved in the process still, because you need that common sense. We all know what could happen. The balance, though, is to say, "Let's not let that be put in the way of moving forward," because it's a great excuse. I see it every day in decisions made by bankers that say, "Well, [inaudible 00:43:30] our policy says we can't do that."

Yeah, but there's a lot that's changed since yesterday, even. Elon Musk, I bet you there's some AI, machine learning in those rockets that can take off and land themselves. I'm just taking a guess, though. 

Carl Ryden:    And the Tesla, too. So, this is a personal view, and this is why I'm having purpose and vision values is right. When we train a machine learning model, you have to have an objective function. What are you trying to maximize or minimize? What's the outcome you're trying to produce?

And it's just as trivial to have a machine learning model of who could we open an account for and them not notice, and optimize that problem. You can do that, this trivial, who can we slip a basis point by and they wouldn't notice? Who could we stick a fee to and they wouldn't notice, or have a machine learning model that predicts, "How could I better help this customer achieve their goals?"

Ultimately, I think having the right leadership, having the right folks to choose the proper objective functions, particularly as it moves closer to the customer, is absolutely vital, and often times, a lot of times where it starts out, you don't know. Anakin Skywalker in the first movie was a cute little kid, later he turned into Darth Vader, right? He went through a phase where he was a good guy, and you got to kind of keep your eye on that to see how they evolve over time. 

Greg Michaelson:    [crosstalk 00:44:52] catch phrase.

Carl Ryden:    Yeah?

Greg Michaelson:    What is Google's company slogan?

Carl Ryden:    Don't be evil.

Greg Michaelson:    Don't be evil. Right? There are a lot of evil banks out there. A lot of great banks. Payday lenders are super into machine learning. I read about a bank that is building a machine learning model to predict how far they can let somebody overdraw their account before they can't climb out again. It's pure evil, right?

Jim Marous:    Unless you're the consumer that manages their money that way. I mean, there's two ways to that equation, because we had the Twitter thing went on and on and on, but basically, the bad parts about NSF and overdraft fees. You can tell people all day about how we can make it so that's bad, but for many people, that's cheaper than going to payday lenders. 

And for many people, they manage their money because they need to have that overdraft at a certain point. Now, is there a better solution? Yes. But the obvious, there's people do not think that's a human process.

Greg Michaelson:    Well, I mean, yeah, you're right. I shouldn't say it's evil or not evil, but the question is, what kind of an organization do you want to be? That's not a machine learning question.

Carl Ryden:    That's a purposeful answer, and you got to make that choice, are you going to wake up one day, [inaudible 00:46:11] Or you don't like it when it's on the front page of the Wall Street Journal and the New York Times, that, "Oh my god, this is who we became."

Jim Marous:    Well, I mean, Wells Fargo's a great example. [inaudible 00:46:22] process. Now, what's interesting is they have to move away from that legacy. I was presented with a person at Wells Fargo two weeks ago, and she said, "You go into Wells Fargo office now, everybody's aware of what they will not do, and as a result, their sales numbers are up higher than they were before these people had quotas." 

Why? Because now they're able to work on behalf of the consumer to bring better solutions, and because their mindset and their leadership, by the way, changed over night, they did not become one of those people that they're now better at their job than they ever were before, so we have to be fearful of not testing new things that may work better than anything we've done in the past. 

Carl Ryden:    Yeah, right here [inaudible 00:47:07]

Speaker 10:    Can you give a specific example of how a bank is using AI right now to empower the RMs to look forward and give forward advice to the entrepreneurs that they serve.

Greg Michaelson:    Yeah, I can do a couple. Let's say, so we're working with one bank now that is using machine learning to forecast pipeline for setting sales goals. So, right now, the way it mostly works for our aims is that their manager kind of gets the number and divides it up, and you can even look at how they did last year compared to this year, and I don't know how equitable that system is for your top performers or for your laggards, right?

So, what if you could predict how much pipeline's going to get generated and use that as a baseline for your sales goal setting? I bet that would have a positive impact on sales. So, that's on the operational type side.

As an RM, let's say I wanted to deepen relationships. We're working with a bank right now that is looking at trying to improve its conversion rates and trying to cross sale products within their commercial learn accounts. 

Let's say I have somebody that has a commercial loan and I see that they make cross border payments. Maybe they'd be interested in buying a conversion rate hedge, and effects product. This particular bank had about a 2% conversion rate in trying to sell that product. We built some models to predict who would be a good target and they increased it to around 8 or 10%, which was worth about $10 million. That's one product in one line of business, $10 million a year.

You can do the same thing with prospecting. So, that same bank is actually looking at third-party data sources to say, "Alright, here's a list of every company in America form DNB, and we know the data's sort of questionable, but can we use this list and combine it with the accounts we already have to see if we can model credit quality based on just their public data.

And if you can pivot credit quality and maybe growth potential, which are the two models they're trying to build, could you conceivably build a target list of customers that fit the risk profile that your bank is trying to cater to? 

And all this said, instead of having the list of a million opportunities, you've got a ranked list of high quality versus low quality opportunities and we kind of target more accurately there. There's lots of things you could do.

Carl Ryden:    So, I think we're three minutes over. These guys will be around. Greg will be around-

Greg Michaelson:    [inaudible 00:49:44]

Carl Ryden:    And Jim, you're here today, but gone tomorrow?

Jim Marous:    Yup. 

Carl Ryden:    Right? So.

Jim Marous:    Here today, gone tomorrow. 

Carl Ryden:    Here today, gone tomorrow. So, thank you all for attending. I think we've got refreshments outside, ice cream break, and then back in the main salon is Mikey Trafton. If you haven't seen him, you absolutely should. It's about hiring a really badass team and how to go about that. So, thank you guys.