2017 Conference

Dallas: The way we wanted to structure this was to have Brett come up here and do this, kinda open our eyes to what’s there. But then we wanted to talk through some of those issues, and talk through … Okay that’s the future, that’s what looks like it’s in front of us, how do we advocate this? Kinda what’s next and what do we do in a very tactical way to figure this out as an industry?

So to help us do that, we’re gonna bring up Carl and Justin, so if you guys wanna make your way up here. So Carl Ryden, CEO of PrecisionLender, you met him earlier today. We’re also going to be joined by Justin LaFayette. So, Justin is a managing partner at Georgian, who’s been partnering with us for the last year or so? That about right? So, Justin is an expert in AI, machine learning has been a big help to us, a big resource for us. And I think his job, the reason we put the big guy in the middle is to try to corral the conversation here. So Carl and Brett, I don’t know that you’ll get a word in edgewise, but you can try to help keep him on track.

Brett: I can look up to him, though.

Speaker 1: So, anyway, these guys are gonna talk through this. We would love to take some questions, so we’ll have some microphones running around here. We’ll let you guys get it kicked off a little bit first, and then we’ll throw it out there and see what folks wanna talk about.

Justin: Great, thank you. That was fantastic, by the way.

Brett: Thank you.

Justin: Very, very good. I thought we could start with some questions grounded on some more pragmatic things about artificial intelligence, specific to commercial banking, and then take it back up to specifically lending where things might evolve. So, maybe we start with just some questions you can both chime in on.

My firm invests only in business software companies and we have a very specific investment pieces around artificial intelligence, so we see a lot of real businesses. We do bigger, mature software companies that are driving real revenue, and all kinds of verticals, many in financial services. So, this idea, you know, where it is, it is very real.

All day I see companies dealing with the pragmatic realities of actually rolling this into businesses. And I know spending time with PrecisionLender, some similar things come up. Which one of them is artificial intelligence is a fascinating new technology not well understood by a lot of people, and one of the fundamental things about it is it’s not programmatic. You don’t describe what you want it to do, it learns. And that’s very troubling for some people, financial service is a great example, where there’s concerns when there’s effectively a black box doing things and no one can explain exactly how it works. How do you test that? How do you attribute that? How do you do QA on something like that? Yet, it has tremendous power, so it’s rolling out. So what has your experience been, having people with that issue?

Carl: So, I always take a really practical approach to things, and so this is the black box problem. A lot of folks are worried about it, and probably worry more than they should about it. Particularly financial services, where things feel like they’re controlled and specified, and those things. Other industries don’t have the problem with it, particularly medical. For 85 years, they didn’t know how Aspirin worked, it’s a black box. Yet, they still used it.

Yeah, so they’re used to not specifying the process, but specifying the outcomes and measuring that the outcomes are being achieved, and you qualify on the outcomes. You train on the outcomes. The whole thought of Bobby and AJ, and Bobby’s different than AJ but they should … Something really cool about it, [crosstalk 00:03:34]

It’s really cool as an engineer, but kinda scary to folks who are used to, “We specify a system to work this way, it has to work.” Scared of regulators. But ultimately, I think there’ll be some learning that takes place across industries, and eventually we can point to real things where, you know … Do you know why Watson is 96% accurate at cancer? Can you explain that? It learned it from viewing all those things. I think folks will be able to point to other things that are actually occurring in our lives and being able to say, “Well, if we can do it here, why can’t we do it here?” And we’ll see that cross pollination kinda take over.

Brett: No, I agree. I think … I was sitting at breakfast this morning listening to some guys talk about quality of credit data, and being able to do a credit risk assessment, and whether it casts their clients three or four or whatever in respect to that, right? And this conversation that you probably would hear in an event like this quite frequently, in terms of how someone differentiates that part of that business. But, ultimately, I can throw a whole lot more data at this in terms of financial data on the company, extended data in terms of who uses the client for their business, all of these things that a human in terms of analyzing that data and putting it into a black box and turning the handle, I mean it becomes hard to program all of that.

But a machine will be able to learn some of those nuances in terms of those behaviors, which is really where the differentiation will come. So, I think if you don’t submit to that process in terms of machine learning, ultimately the way we’ll cross check it is not having to understand it. But we’ll compare the accuracy of the black box to the way we’ve done it in the past, and once we trust that the results are coming up better and smarter and give us an advantage, then I think, you know, ultimately it’s going to be a differentiation, can be an advantage.

Carl: I think there’s … your historical timeline you drew of we used to be in the business of growing things and making things and doing things, and then we had machines that helped us grow things and make things and do things faster, and that gave competitive advantage and changed the nature of the economy. And when suddenly the machines take over the doing things and the making things, then we differentiate only on our ability to learn things. And now we have machines that can actually speed up … That can learn things faster, you know? We went from we do things, the machines do things faster. Now we differentiate by learning things, now machines can learn things faster. I think there are places still though, where humans definitely fit into the equation. This is something I talked about our cab yesterday is that, in any value chain, arithmetic became very cheap with computers, prediction becomes very cheap with AI machine learning, but there’s still judgment and empathy. You know, Watson says-

Brett: You’re gonna have cancer.

Carl: 96% certain you might have this sort of cancer or accurate.

Brett: You don’t want Watson telling you that on a pop-up on your smart phone, right?

Carl: You still need a human to sit down with that person and your family circumstance, what’s the right … There’s three courses of action, lets figure out the one that fits to your particular situation. And I think what you’ll see is in economics whenever any piece of a value chain becomes really cheap, the thing right next to it become very valuable. So I think uniquely human skills like empathy and understanding and those things that are harder for a machine to learn, and actually humans at least now, maybe one day don’t want a machine to fake. I’m fine with you predicting that I have cancer but I want a human to tell me a discuss the treatments with me.

There are places where I think humans and machines together get to a much better solution.

Brett: But this is actually one area where I think if you see the investments taking place now alongside AI, to how AI will integrate into society, those creative disciplines like behavioral psychology and user experience design. All of that is going ahead leaps and bounds, and there is huge amounts of investment happening parallel because of exactly that fact that if we are going to leverage off of AI and put it into human society so that from a psychological and behavioral perspective it makes the most sense. Then we are having to redesign that interface. So there’s actually huge amount of work that has to be done to integrate our AI into society along that creative experience path.

Justin: So sticking to another more pragmatic question of things that a lot of people are dealing with right now, as this technology rolls out, banking is a great example where we see this all the time even other companies coming at it from other directions, providing services. And that is, you know you mentioned in your presentation the data pools, the real time data that can be accumulated that represents human experience and business experience is what fuels artificial intelligence. So companies are starting to look at the data they have and realize, as they have for some time, but in the AI world that that is the encapsulation of the intelligence that they have and the differentiation they have-

Brett: Its an asset.

Justin: It’s and asset. So how do you protect that asset when business propositions come along that say if you pool that asset with other companies or, third parties from the outside. If you share signals from what happens in your world with other parts of the ecosystem, other places that use that in that smart city plane where things might be happening. If you are willing to share signals and representation of your intelligence and get something back to provide better service or better decisions, are you willing to do that?

Google and Facebook have proven in the consumer world that the marketplace with tolerate a lot of sharing, a lot of very personal information. There are many who think that that is going to come to the commercial world. And yet as you see companies roll out their first examples it comes off like how is their valuable asset protected?

What are your thoughts?

Brett: Well, let me start with one area for example, which is identity. KYC type data. Whether that is a corporation or whether that is an individual. The reality is right now we spend a lot of time identifying our customers before we do business with them. But that’s a great example of data that we shouldn’t really invest in as financial services, it really doesn’t make and sense for FI’s to be in the identity business. And if you look at technologies like block chain and you couple that with AI, I think in ten years time FI’s won’t be invested in the identity KYC business. I think that will be something we’ll happily outsource because someone else can do it better. Whether it’s a state agent or whether that is a platform like Amazon or Alibaba from a business perceptive. I think they’re going to have better data than an individual financial institution could ever have.

Carl: So to take you back to kind of hard core finance theory, I guess. There’s a guy that teaches a course at MIT, Andrew Lowe. Teaches finance theory. First day of class he says “what is an asset?” And you get all sorts of answers, it’s a thing it’s a whatever it’s usually data as an asset. His answer is “the financial theory, finance theory definition of an asset is a stream of future cash flows.” The asset is the stream of future cash flows that this thing generates. So if you hold your data close to the vest and you don’t ever do anything with it, it’s not an asset. It’s a liability. Its something, its negative. That storage cost is going really low, you pay to move it around, clean it up. But if you are so scares of sharing it with someone else that you never do anything with it, it’s not an asset. Its not worth much.

Now there are places, I think there’ll be in many ways a certain hybrid vigor. Almost the same thing you get when you cross pollinate different species and the hybrid becomes better than the two. The old saying from a computer scientist Alan Kay “a perspective is worth 80 IQ points”. Your data has your perspective and its baked into it, its almost like you’ve got Bobby and you’ve got AJ and if we can put them together do we get something that’s better that drives faster, the right corners without the risk of missing any other corners. Just like cross pollination of two corn, you get a better answer. And I think there is probably going to be a bunch of science and research around how do you blend these things together and how do you avoid the downside of AI machine learning is over fitting. Right?

Is where if you train on the same data long enough it will match. It can predict that set very well but it doesn’t translate to the rest of the world because you’ve sucked your own exhaust. You fed off your own inputs and I think that hybrid vigor that comes from the cross pollination, the different perspectives, the different ways of harvesting the data actually will almost always lead to lift. But I think its something you’ve got to look at on each case by case basis.

Brett: Well this is another way of first principals thinking I think really is important. So if you look at, for example, Uber. Looking at offering leases to drivers, what do they have to know about you to determine whether or not you’re qualified for a lease. Not much because if you are going to drive for Uber, that’s the main qualifier you need.

So is there’s data like that from a lending perspective with the clients you have that from a behavioral perspective you can find data that is a better data point than three years of tax returns? As an example.

Carl: Well

Brett: And you may not own that data.

Carl: Well I also think its back to the talk earlier, the one about leading versus lag indicators. Uber is looking at leading indicators. If Uber’s under riding a loan to a driver of their cars, they know how often you work, the trend in the market. They know the utilization of that car, they sit in the payment stream between you and repaying the loan. They’re actually seeing leading indicators of what’s going to happen. When you get financial statement in a rears of this is what happened, at least as of this morning that was the result of what you were doing 12 months ago.

Brett: They can see if you’ve stopped driving for a couple of days and it’s going to affect –

Carl: Right, you won’t see that until you get the next quarters financial statements. And so I think there’s a lot of places where, in almost every industry, Justin and I talked about this, there are companies, whether it’s Uber in this vertical or suppose there’s a company that runs … there’s a company here we’re friends with that runs great SaaS business that built softwares and service for home healthcare providers and they actually, Kinser he’s on the board there, they actually run your business for home healthcare providers. They track all your visits, they track the patient treatments. They can predict the probability of readmittance, they can do all these great things.

If you’re ever going to lend to those folks … you’ll never have better data than that. In almost every industry, tool rental and everything there’s going to be a run your business application. And I think finding ways to partner with those folks and pick off the best farm logs in the Midwest. Farm logs, something like 60 percent of the nations row crops are run through farm logs. You see what they planted, you see what their yield was you see when it went in the ground, when its coming out. It tracks weather, all those things. If you were an [ag 00:15:11] lender, I would partner with farm logs and be the lender on the farm logs platform. You could pick off all the best credits and fine tune the exact solution based on exactly what they put in the ground to match theirs.

I think it takes what Lisa said and what Bob said and what you’ve said and puts it all together and makes it into something that would work. And this is the “is banking a product or a feature?”. And you know a feature is something that hangs off the major product and in the past its been a separate thing. But Tim Shanahan actually, from Citizens was talking to me yesterday and he said “when I first moved into banking, went to work for the bank, they asked me who do you bank with?”. And he says “I don’t know what you mean by that, I’ve never thought of the word bank as a verb, I make payments with this company I do my kids savings over here, I do these things” and I think that’s where we kind of put together the things we’ve seen throughout the morning about the job they’re trying to do, the progress they’re trying to make, the experiences they’re trying to create and it all comes together.

Brett: Well you know actually I sort of strip it back to something even more primitive. If you think about financial services whether its in commercial space or whether its in retial we really only have three basic products. We have the ability to pay, move money around. We have the ability to store value and the ability to access credit. And every product that we have built on top of that core utility of banking has a different flavor but it has core element. Whether it’s a commercial loan under riding a piece of farming equipment or whether it’s a mortgage or a car lease, you have the value of the asset that you are lending against, you have payments you have maturity data, whatever right? The structure is pretty similar. The underlying utility of access to credit is the magic.

Now what we’re seeing with this technology is, whether its AI or behavioral data or whatever it is, you are taking in a new way to surface that utility, but ultimately the behavior data, for example, that drives that decision making in the future is going to result in a very, very different experience of the way that credit service or utility service. And its not going to be that product, per se, anymore delivered through a new channel like mobile. Its actually going to be about how the behavior triggers access to the utility. And that’s, I think, where we’ve sort of fundamentally got to think about the redesign of this and that’s where when you combine all of this stuff it’s that experiential layer that we have to start building.

Justin: So Let’s talk for a minute about how folks in the audience can think about projecting what they do in the services they offer into that alternate kind of consumption plane. And its interesting, I was thinking about examples of how far out that was, how some of these things were and I realize that in our own portfolio we have a company that has done just that. We were early investors in a company called Shopify, which went public and is now the largest provider of merchants, online eCommerce services to small businesses.

Now they are scaling up though, if you buy Tesla equipment or Patagonia clothes, they are working with bigger and bigger companies and one of their fastest growing services they offer is lending. They lend to these merchants, and they didn’t partner with anybody so they did it first principles. They didn’t partner with payment companies, they didn’t partner, I mean they designed their own system. And they felt they can do that because they can see all of the activity, the entire business is in their platform, so they can see everything. And when they lend money they even see where they’re spending it because they spend it on building up their platform.

But your point about Kinser is a great example. There are thousands of software companies out there automating the day to day operations of entire industries and they themselves are not that big often. They’re 50 million dollar a year companies, they’re 100 million dollar a year companies, yet the business taking place on their platform in many cases is billions of dollars. And I’m interested to hear what you think. How would a bank go about approaching those companies? How do you? I mean most of those companies are not as Shopify will build it themselves. But if somebody came and said would you like to offer lending within your platform to your massive customer base would you see the entire business unfolding? Its an interesting proposition.

I mean, we’ve seen examples of that kind of partnership.

Carl: I’ll give you a … I’ll make it even more specific. Two of our clients, one of them is here, one of them is normally here but couldn’t make it, are two of the top four ag lenders in the US. I actually put together a long email with this and said you aught to farm logs. You aught to go partner with these guys, and both of them, both of our contacts thought that was a great idea. And they kind of went thought the channels of the bank and it kind of died. Now, if you’re within one of those banks you go “holey, we should be doing something like this” what would you do? Like how do you, how do you bring folks over to it and how would you set something like that up?

Brett: I mean, when you look at startups, how they think about this the startups’ sort of start from scratch. So I’d probably, I’d say lets pick one area like for example you said farming equipment as an example. What’s the data we need to determine where people are going to be buying farming equipment and where do they buy it from? And how do we get access into that platform or network to sort of building in that synergy.

So I’d be looking for the data points and the existing platforms that I could integrate with. Or if its in the case of something like Uber, and I’m a leasing company, I’d be building my business so I could be acquired by Uber. I’d be building the API that hooks into Uber, I’d be developing that off my own back so that I’ve made it as easy as possible to be acquired or integrated with them. Because that’s the other thing, is these technology companies are going to start to … they’re not going to build all of their financial services capability from scratch, particularly if it’s access to market place lending and things like that. They’re going to just “say well lets acquire something” and its what Alipay did with Moneygram.

Carl: Well this is, I guess this is what I see some within banks is that they’re existing legacy systems aren’t designed, not just legacy systems IT, but policies and other things are all built around. We get quarterly financial statements and what- [crosstalk 00:21:27]

Brett: Yeah, yeah.

Carl: You can say “well that’s a lagging indicator”, there’s this better data that’s a leading indicator of predicting credit loses and other things that under ride the loan. But our systems aren’t built to handle that, they don’t have API’s or whatever and all of a sudden it becomes the tail wagging the dog that the systems you have don’t allow you to go where you need to go. This is something we see a lot, right now and we talked about a lot yesterday is a lot of banks right now are rebuilding the brain of the bank. Top to bottom and really thinking how do we rebuild the brain of our bank so we can operate where we need to be. And its hard work, particularly moving from those legacy systems onward.

Brett: Well you look at any retail bank for example, that’s looking at a core system replacement is looking at a billion dollars to start with. They’re a big bank. But 70% of retail banks in the US don’t have their own IT architecture. They rent it from Jack Henry and stuff like that, so you’re going to be dependent on those sorts of capabilities.

But in the space that we’re talking about here, if you look at the document handling as an example, that’s definitely legacy behavior. And we can see things like block chain distributing legend technologies that’s going to absolutely revolutionize that. For example if you are going to a client and you’re offering a facility and then you’re shopping it out to the market to do that, that’s all going to be on the block chain with in a few years. Because that’s going to be the ability to move very, very quickly. In fact if you want to automate that, you’re going to have to have those sort of capabilities. So then you’re going to have to be participating in a marketplace that is block chain enabled for the back end learning facilities and getting access to marketplace credit.

Carl: To scare folks I think.

Justin: So I think we’ve got time for one more question then we’ll ask some questions from the audience.

So I don’t want to put you on the spot, because we talked about some really interesting things to the board about the future and you were like a fountain of ideas for the future. And this is not a product announcement, but you’ve talked to me in the past about how the tape plays out on the augmentation of knowledge workers and things like commercial lending. And you guys right now have an offering around loan officers that’s specific to part of the function and its very cool but how far can it go? When you play the tape out and you look at AI stating to augment people, you’ve used the term with me of the iron man suit, Jarvis talking in your ear give you super powers. How far does that go and how quickly, do you think? That all workers and everybody that has to get something done at a bank is augmented?

Carl: So I think there’s, well, I’ve spent a lot of time thinking about this and … so one of the things, your slide about humans working with AI, I always come back to this idea of the chess analogy.

We talk about this a little bit in our cab yesterday, client advisory board … Many years ago Deep Blue beat Kasparov [inaudible 00:24:34] moment in AI computers beat humans for the first time at this cognitive game. But you can brute force that, but then many years later now they compete as centaurs they have these games where a human with a computer competes in the same tournament. And what happens is the human competing with the AI beats both the best AI and the best human.

And the reason is, is there are some things that humans are fundamentally better at right now than computers, and kind of will be for quite some time. Like the power, the energy consumption of the human brain is many multiple orders of magnitude less than what it takes to power these huge computers. And that is and idea of the humans and machines working together in exactly the right way. Now we’ll see that manifest itself in a couple ways. We built Andy to help commercial [enters 00:25:29] to help give them guidance because Andy can see all the loans at the bank, she can see all the relationships at the bank, she can see what’s winning, what’s losing. And the idea is to help that relationship manager in the moment, allow them to focus on the more human side of it, the interaction, delivering the diagnosis to the patient, talking to them, building trust and honesty.

What folks have asked us to move Andy outside and deliver intelligence all across all enterprise apps, and be able to [Andyfy 00:25:59] all applications. And that’s something we’re working towards right now to help folks with. We have other companies, and we focus on providing intelligence and automating intelligence and augmenting intelligence to humans in that commercial value chain that still has a lot of human need in it. But there’s other tools that will be complete robotic process automation that will take out the mundane stuff.

If you’re doing stuff … Our goal is to do everything a machine can do well, so that humans can do the things they do well. That’s kind of, been part of our DNA from the beginning. There are other companies that are just, find things that machines fundamentally do better than humans and take them out. And that’s the robotic process automation where folks are pulling from this document, pushing this button, sticking it here and you’ve got a human who does that. Their work fusion is one of your companies that’s really good at just automating that and taking it out. So you’ll see these two things as stuff that shouldn’t be done by humans being pulled out. Stuff where there’s a big piece that shouldn’t be done by humans or is better done by machines but there still is a huge human component still in there, and I think that’s the area with Andy we’re going to focus on a lot.

Justin: Excellent.

Brett: So I think that, just quickly to emphasis that, I think the next five years is going to be about augmenting revenue and relationship. So we use technology to speed up our ability to execute to our relationship with the customer. And so that’s really where we’re focus is going to be. Making a quick decision, giving them the best service possible, keeping that relationship sticky and not giving someone else the opportunity to attack that with better level of technology service.

But looking sort of seven to ten years out, this is where were going to start to see increased automation of delivery of services where we take the human out of it where ever possible. Where humans just are friction rather than a benefit. So where humans differentiate over the longer term, that’s going to be a lot more tricky. And we have to start training our people differently for that seven to ten to 20 year timeframe. But over the next five years we’re going to be using that technology to augment revenue and relationship in terms of our speed to market and speed to revenue.

Justin: Do we have any questions from the audience? There’s a couple people with microphones on either side.

There’s one right in the middle here.

Brett: Can you keep your hand up?

Thanks.

Stan: Hi, Stan Sleuter.  So you talked in your presentation about delivery, but is the real threat in our undervaluing of the data that we have and really not in the delivery. I think we as an industry are probably under utilizing our data more than-

Brett: Oh yeah, Now I Guess this is a threat. The threat is, you’ve got all this data and its under leveraged, you know you’re not using it today. I mean if you look at the behavioral data that you have on your clients or access to data you have, in particularly in terms of predicting a default as an example. You probably have all the data you need there to do predictive analytics on that but we still take a very traditional approach to looking at financial statements. But if you’ve known a client for ten years, you don’t need to ask them for more financial statements when they do a new application. That’s sort of a little crazy.

So we definitely have under leveraged that data but part of that is the problem that we have this very defined legacy process that you put in these inputs, you need these pieces of paper, then we turn the handle then we get a credit rating for you and then we decide whether to issue the lending product. That process we need to break down because that’s the process that’s sort of restricting our ability to be creative in terms of use of that data. So how do you do that? Well, I guess that’s the challenge.

The other problem is you’ve got data all over the place. And so you probably need to start thinking about what is the technology you need to put over those disparate data pools to bring it together. And again this is the benefit of machine learning because machine learning can sort of pool that disparate data together for a client across different data bases and sort of make sense of it.

Its much more difficult for humans to do.

Carl: Not to be the negative, there’s actually more … its worse than that. There’s more problems and because a lot of times the systems … he talked about the devices on this table would have cost 80 million dollars, most of the systems that banks use, particularly smaller banks, the [inaudible 00:30:40] not to pick on those guys, but they were built 20 years ago when one gigabyte of memory was 500 thousand dollars.

Brett: Yeah.

Carl: You know, or half a million or even more, 2 million dollars and a raised floor. So its down to, they only stored the outcomes “where did we end up” and I need to put it in a two digit code right? So you only see the outcomes, and its like trying to train a self driving car by “okay, you left at eight am and arrived here at 8:30, but I don’t know all of the behaviors and decisions that were made a long the way”. Whereas a Tesla actually sees the road that you see, sees the decisions you make and learns from that behavior. The data you have, that only captures the endpoint is the shadow of what you need, you need the experience data, the behavior data to learn from. So having systems that actually capture the sequence of events that folks follow through, so you can actually see and build that story of how they got there, make those calls and links. If you don’t have that, you just have shadow, you just have the outcome. Right?

Brett: Yeah, I agree.

Justin: Do we have another question?

I’ll see when I think we’re right.

Brett: That’s one there.

Justin: Oh, we have one there.

Carl: Oh, over there.

Justin: We’ve got this big spotlight right now

Speaker 6: Very interesting topic, thank you all. How do you build into others in the financial services’ realm or other genres, how is ethical behavior and decision making built into AI robotics and those kind of things? Just kind of talk about that if you would.

Brett: Yeah, this is a great question. Right now, we don’t actually codify ethics when we build machine learning algorithms. We assume that the machine learning algorithms that watch what humans are doing is learning the ethics from the humans its learning from. Which in itself is probably an issue, a problem. But there is a growing school of thought that we actually need to build ethical algorithms and sort of distribute these in terms of AI’s. Particularly when we start getting into AI agencies, you know where we use agents like a smart assistant to do things for us. Or, you know you heard the trolley problem with the self driving car, the trolley problem is should the self driving car save the passengers of the vehicle or should it run into pedestrians crossing a crossing. And this is the ethical consideration we have to build into these algorithms. Now when you’re talking about a lending facility to a corporate client, you know the ethics of that is not going to be as acute as a self driving car deciding if it should run someone over.

And yet there is ethics in this, you know for example maybe we can lend some money to someone but should we? is this ethical to do so when we know that in 12, 18 months time, based on the data they won’t be able to afford to pay for this facility. And it may jeopardize their business. So there’s things like that are on the boundaries of these decisions, which we’re not really, we haven’t as yet started to codify and that’s definitely an issue for us. But this is part of the learning curve as a society that we’re going to have to make in terms of how AI integrates into the world around us and where there are going to be issues with AI’s is just that their purely logical in terms of their decision making.

Carl: So, I think it’s a great question to and … Part of when we put the program together, having Lisa talk and having Bob and those guys talk and then coming to here, is financial leverage, which is the business you’re all in, right, financial leverage doesn’t take a bad asset and make it a good asset. It actually takes a bad asset and makes it a worse asset. It takes a good asset, it’s an amplifier, right. AI is massive technological leverage. If you’re fundamentally, it will amplify who your business is, so you better be sure that you’ve got a strong sense of purpose and mission, vision and values of what you’re doing. SO for example, most AI, all AI’s are trained with an objective function, like “what am I trying to do”. And if I give it solely back to Lisa’s talk earlier for its solely to maximize profit and you set it free, an AI can easily learn “heres how I slip a few basis points by customers when they’re not looking, heres how I slip a fee through when they’re not looking, heres how I open an account for them when they’re not looking”.

All these things which leads to short term goodness but very long term badness. And by the way, this is a foreign to banking, most of what banking is, is the trade off between “I can boost my net income next quarter, but I’m pay the price down the road on the credit risk I’m picking up, or the reputation I risk I pick up are those things”. You need to have a strong sense of who you are and what you are, you can’t leave this solely to the technologist. You really have to say “what’s the objective function, what’s the goal that we’re going to solve for in this?” And as much as you can, I think, tie it back to “how do we create value for our customers, how do we build an AI that create value for our customers” and then we can participate in that value. Or else you kind of end up in the … The easiest problem to solve is, identify all the customers where I can slip by a couple dollars and they wouldn’t notice, until they do. So …

Justin: Very good.

Speaker 1: How we doing on time?

Carl: He’s walking up here.

Justin: [crosstalk 00:36:43]

Speaker 1: This is the big hook, now.

Any more questions out there before we move on?

Okay this was great, I appreciate it from all three of you guys.