How do you scale something 10x and then scale it again? The answer is surprisingly simple, and yet very few organizations have the discipline to do it. My guest today is Elan Mosbacher, Senior Vice President of Strategy and Operations at Spothero. In this conversation we talk about how Elan and his team made went about growing SpotHero to the largest mobile parking platform in the country.

Elan has a ton of experience around building marketplace businesses, pursuing strategic partnerships and more. He’s also very thoughtful about the future of mobility in general, and we get into a pretty fascinating conversation around autonomous vehicles and where things are headed.

• How to scale quickly
• How to scale yourself
• How to figure out who your best customers are
• Selling a big strategic change to your board
• Why you shouldn’t copy tactics
• Channel mix when trying to 10x
• Strategies for scaling marketplace businesses
• How to pursue strategic partnerships
• Mental models
• Succeeding early
• The future of mobility
• Challenges to autonomous vehicle adoption
• Scooters

How to scale quickly

DI:  I know when you first joined Spothero, you were primarily in a marketing capacity and your explicit mandate was to help them 10x. And then to do it again. And I remember you talking about how the things you did to 10x the first time were very different than the second. What were some of the lessons you learned trying to scale something that quickly?

Elan: Early on, the most important thing was to figure out who our best customer was. There have been probably 200 different companies in our space, and I think the thing that really set us apart was our understanding of our customer. We figured that out and aligned ALL of our initiatives around acquiring that best customer and making the experience as good as possible for them.

“The thing that really set us apart was our understanding of our customer. We figured that out and aligned ALL of our initiatives around acquiring that best customer and making the experience as good as possible for them.”

Fast forward several years later, we see our best customers are still buying from us. And that’s what made us really big relative to our competitors who didn’t focus on the best customer. Today they might be acquiring roughly the same number of customers every month, but don’t have, three, four, five, six, seven years of people who joined their app, who are still buying.

Another big lesson is just about scaling yourself and scaling your team. As a leader of a start-up, you have to scale yourself as fast, if not faster, than your company. It’s hard to do that when all you’re doing is working on your company, you also have to work on yourself.

“As a leader of a start-up, you have to scale yourself as fast, if not faster, than your company.”

For some people that might be a part-time MBA, for others an executive coach, or reading, or podcasts, or mentors, or board members. But it’s really important to invest in yourself to make sure you’re at least one step ahead of where your company needs to be.

How to Scale Yourself

DI: What did that look like for you? What were the areas you identifying as needing growth in and how did you go about doing it?

Elan: I think I probably used all of the above. I started an MBA part-time just before joining SpotHero, so I focused a lot of that time on how do I apply it to the job. Everything from, how do we figure out who our best customers are, to how to lay out a business plan to, how to manage our team, how to recruit and hire.

I also read a lot. I have a busy life so I’ll go a month or two without reading, but then I’ll read four or five books in a weekend.

SpotHero also had the very good fortune of assembling a pretty remarkable board. So having an opportunity to learn from board members, and during board meetings, and from other mentors has been really instrumental.

You have to find what works for you. You have to find a way to learn, outside of just doing your day-to-day job.

How to figure out who your best customers are

DI: Going back to the finding your best customers – it seems like that’s a pretty common problem for start-ups. They take in some money, they have this mandate to grow really quickly, they end up putting a lot of that money into paid acquisition, and either it takes them a little while to figure out those users aren’t sticking around, or they’re not measuring the right things. They’re mistaking top-line growth for product/market fit. How did you avoid that and figure out who those best customers were?

Elan: By the time we turned into SpotHero, we had a product that people used to find parking. We had decent product/market fit, but I asked Mark, our CEO, who is our best customer? I actually asked everyone on the team, who do you think our best customer is? No one really had a good answer.

So we started looking at who bought the most times, actually interviewing them. From there we sent out a survey, did some statistical analysis. We found there’s a certain segment of customers who use us for a specific reason, and were like 13x more profitable than anyone else. In fact they were the only profitable segment at the time.

From there we focused on building out our parking inventory around the needs of that customer. We focused on channels to reach that best customer. Our sales and customer service focused around the best customer. With all that focus, we didn’t necessarily acquire the most customers, but we acquired better more profitable ones who are still with us all these years later.

Along the way we acquired one of our competitors who was not part of our SpotHero family but had a very different approach. It was interesting for us to look at how do we do things and how do they do things, and the differences. One thing was we had enough capital to focus on maybe more expensive channels where our best customers could be found. They were more capital constrained, so their approach was to find customers who were profitable on the first transaction. And when we looked into it, they did a really good job of building out inventory, and aligning their company around that customer.

So there are multiple ways to attack things. But whatever approach you choose you need to align all your efforts toward that customer archetype. I will say though that I think our focus on that profitable customer, even though they were more expensive to acquire, allowed us to become much bigger than them over time because of the repeat business. And now we have the best of both worlds because we have the customers who are really sticky, but we also have this great channel with customers who maybe only buy once, but they’re profitable on the first transaction.

DI: Are you targeting those people in different ways with different messages? Or are you using the same messaging and you just realize that some of those people are going to look like this, and some of these people are going to look like this?

Elan: So the tricky thing there is the highest volume bottom of the funnel channels most start-ups use, like search, actually acquire the worst customers. And the trickiest channels, the most expensive channels, the riskiest channels, are the ones that are probably the best customers.

“The highest volume, bottom of the funnel channels most start-ups use, like search, actually acquire the worst customers. And the trickiest channels, the most expensive channels, the riskiest channels, are the ones that are probably the best customers.”

So that was a challenge for us. But by having that insight into our best customer, we had the conviction to spend on offline, and on other more expensive, harder to measure channels early on. We still do a ton of SEO, paid search, those types of things. But those customers for our business don’t repeat as often.

Selling big strategic changes to your board

DI: That sounds like a big turning point for the company. When you’re bringing that to your board, I would imagine things like your total addressable market (TAM) look smaller. It might impact the projections that were made earlier on. That pitch of “we think our most profitable customers are this sliver of the market, and acquiring them is going to be a lot more expensive.” Aside from having great board members, are there strategies you used to paint that story in a compelling way?

Elan: I think there’s been two periods of time where we went through that. One was on the advertising side – how do we invest in these more expensive channels?

The other was on the product side. We actually developed a product called SpotHero for Business, which honed in on a customized version of our product for a really strong audience and very strong customer segment. In both those cases we had that conversation.

On the advertising side it was pretty simple. A simple formula of, stacked rank, ROI by channel is really helpful. Basically what’s the lifetime value, what’s the customer acquisition cost, what’s the payback by channel? Where are we going to get the most efficiency? Where is the LTD impact ratio best? Where are we going to get the most volume? If we realize that the highest LTD customers are going to come from slightly more expensive channels, that’s fine as long as the ROI is there. There are some pretty proven frameworks to at least have that discussion.

The other piece we did is have a set amount of our budget, say 80%, allocated to proven channels. And then 10% to 20% was to try new things. Some of those would be expensive, and some of those would not be that measurable, and some of those would be kind of crazy. We didn’t really count that money against ourselves, because we knew at any moment we could shut that off if it wasn’t working.

On the product side when we said, we want to make a major investment to improve the product for our best customer, the feedback from the board was, “Why didn’t you do this six months ago?” So that was a pretty easy sell as well.

I think the main lesson there, in terms of incubating a start-up within a start-up was having a separate P&L. Operating as two different entities.

If you are always looking at the trade-off between a really big thing and a small thing, the ROI on the big thing in the short term is always going to be better than the ROI on the small thing or the new thing in the short run. So bifurcating those two projects was definitely a big thing that we’ve done to help us set ourselves up for success when trying new things.

Why you shouldn’t copy tactics

DI: You’ve talked about the need to avoid copying what other people do. In the marketing or “growth hacking” community, there’s a ton of content around tactics, lots of “Uber did this one trick, so you should too.” And I know you feel like that’s probably a misguided approach.

Elan: I would say start with the why. It’s perfectly acceptable and expected to learn from others. To look at who’s done it before and figure out what they done that could work for us. The mistake is copying tactics without thinking about the strategy behind it.

We often look at analogs of companies in mobility or that are further along in their journey, looking at tactics to evaluate if they fit within our strategy. And that’s okay. But I’ve seen companies that just copy the competitor. If you’re not in a leadership position and you’re trying to catch up by copying what the leader does, it isn’t going to help you very much.

“If you’re not in a leadership position and you’re trying to catch up by copying what the leader does, it isn’t going to help you very much.”

We target drivers. At one point we hired some folks to stand at the exit to one of the highways in Chicago with a big sign saying “Use SpotHero” with a promo code. A few days later someone doing marketing a totally different vertical told me their CEO saw it and was wondering why we don’t do sign flippers too. Well, we’re targeting drivers a few minutes away from parking and we want to remind them. This is a great tactic given our audience and our strategy, but that’s not your target audience. Why would you copy that?

One other example. In 2015 or 2016, there were all these on demand apps for everything. There was on-demand valets, on-demand food. All these apps popping up. And they were spending a ridiculous amount of money on Facebook app installs. The cost of app installs is crazy now. But we had all sorts of people saying we should do it too. But we kept trying it, and it kept not working, and at a certain point we just stopped.  It’s just not the right channel for us.

So understanding what you’re trying to achieve and aligning the tactics to what you’re trying to do is important.

Channel mix when trying to 10x

DI: When you do have a mandate to scale aggressively, it does seem like one of the perceived advantages of Facebook, Google, etc. is speed to scale. You can scale quite a bit before saturating that channel. So with some of these other channels like outdoor, that seems a lot harder. How does that fit into the mandate to 10x something – are those channels able to get you all the way there, or do you still have to supplement with some of the more common channels, to make up the deficit?

Elan: I think it depends on what kind of business you run. There are really successful businesses that spend almost nothing on marketing, growing through the product and word-of-mouth. There are other businesses that rely entirely on one channel, so it depends.

Back when we were in a crazy growth phase, we were trying to decide whether we should spend or not on a channel. And that that point, the decision was if we don’t spend we lose, if we do spend, we might lose. So we spent.

In terms of channel, it really depends on your business. In most businesses, certainly marketplace businesses, there’s usually one or two primary channels. Often that’s Google or Facebook, which is why they’re so valuable as companies. But other companies have different channels.

There are things you can do inside of the product as well, refer-a-friend or conversion rate optimization or whatever. But there’s rarely one silver-bullet. Usually it’s one, or two, or three primary channels, and then a whole bunch of other tests and optimizations. Things that work for a day, a week, a month, a year, and then kind of go away.

Our business is a little different because we’re a marketplace business and we’re also based on geographies. Sometimes we’re advertising to customers, but it’s actually less focused on volume of consumers and the profitability of those consumers. Sometimes it’s actually about filling the merchant or the supply side of the business.

The more customers you get the more inventory you can get. If you’re starting a new vertical or geography, you want to acquire a bunch of customers and enough supply to keep them happy, even if it’s not super profitable. It’s similar to Uber – when they started a new city, they paid drivers a minimum amount until they got enough demand that they didn’t have to do that anymore.

Strategies for scaling marketplace businesses

DI: Let’s talk a little bit about marketplaces. That’s a challenging model for a lot of people. What are some of the other lessons you’ve learned trying to build out two sides of the marketplace at the same time?

Elan: Ten years ago there wasn’t a ton of information about this, and the chicken and egg problem was the big question. But these days it’s actually pretty well documented. One of our investors, Insight Venture Partners, has this document called the Periodic Table of Marketplace Businesses which talks about all the metrics you have to look with example benchmarks.

One of the things that served us really well was focusing on liquidity over geography, meaning start with one geography or one vertical before you try to scale. I like to use Facebook as an analogy for this. Facebook started at Harvard, got saturation, then moved on to Ivy League schools, then east coast, then colleges, then high school, then adults, then businesses. Whereas some of their competitors were open to everyone from day one.

“Focus on liquidity over geography. Start with one geography or one vertical before you try to scale.”

We were only in Chicago for several years, and in fact our business was bigger in Chicago than any competitor who was national. We only really expanded beyond Chicago once we got the playbook down, and as a result we were able to move much, much, much faster when we launched in new markets.

It’s very easy to spread yourself too thin if you try to go too broad in the beginning, so that’s probably the biggest lesson learned.

In terms of chicken or egg, in most businesses you want to focus on the supply side. The merchant is the one who is more patient because a merchant. They realize it might take a while to get a ton of value out of your platform while you build up demand. But consumers will go to your site once and you don’t have what they want, they’re going to leave pretty quickly.

Partnerships

DI: One of the channels that is a little less commonly talked about is partnerships. I know you’ve been pretty aggressive about pursuing those. How do you think through which types of partnerships make sense? There’s opportunity cost in getting those things set up. What decision making criteria do you use to evaluate what makes a good partnership versus a bad partnership?

Elan: In the last decade or so of doing this I’ve often started as the first marketer in various start-ups. Inevitably after two or three years, you get to this point where you’re like okay, I’ve got Facebook, I’ve got Google. What’s next? And it always ends up being partnerships.

I mentioned earlier the importance of mentors and finding ways to learn outside of the day-to-day. An old mentor of mine who spent a lot of time doing business development recommended a book called The Sumo Advantage. Really good book, and it basically outlines the business development process for start-ups.

There’s a few ways I think about this. The difference between sales and business development or partnerships is this: with sales, you generally go to someone and say, I’m going to do this, and you write me a check. With business development or partnerships it’s generally we’ll do this thing together to serve our mutual audience. So you have to think through who the customers are whose lives we’re both trying to improve with different solutions. Where’s the symbiosis?

Another thing to think about – when you’re really big everyone wants to do a deal with you. But when you’re small, how do you get attention? In a partnership, the smaller company is usually looking for customer acquisition and the bigger company is usually looking for a stickier product or more retention.

If I’m Salesforce, I’m looking for all the other software companies to plug into my marketplace, so that when people use Salesforce they also have all the other tools they need. If I’m really small business, I want Salesforce’s customers, so I plug into Salesforce in order to get a small fraction of the companies that use Salesforce to also use my product too.

When you’re a start-up, the approach I’ve found is to say “Hey BigCo, there’s a feature set missing in your product. I can make your product stickier, increase your LTV and differentiate you from your competitors.”

For SpotHero, we look at the segments or verticals of companies targeting similar audiences. Who has a big audience of drivers? Well, car companies and navigation and mobility type apps and eventually autonomous vehicles. Then we look at who the leaders are in that space. Then we work to figure out the mutual value prop for our customers. Then we pitch and see where things go.

Most recently we did a partnership with Waze in the City of Chicago. One of the challenges Waze had was their GPS works really well when there’s cell reception, but if you’re driving in a tunnel or an underground road, you don’t have cell reception, or GPS doesn’t work. The problem for us was there were drivers in Chicago who were trying to park in garages that were underground and it was really hard for our customers to find those garages.

So we partnered with Waze and introduced them to the City of Chicago. We installed Waze beacons in all the underground roads in Chicago. And now they have a better product for their audience, and our customers are able to find the garage they want to park at, and the City of Chicago has better traffic flows and all the benefits of having navigation work.

Another partnership was with Google Assistant. That was announced at CES, and Google announced that though Assistant and eventually Android Auto, you’d be able to buy coffee from Starbucks, or buy a parking spot with SpotHero. That’s another example of more of a product innovation.

We’ve done distribution deals where we partner with folks who plug into our API so they can offer parking to their audience, and we’ve done deals with smaller companies where we take their technology and plug into our app to it stickier and better for our audience. Those are all examples of partnerships and ways we’ve approached it recently.

Mental Models

DI: Changing gears a little bit, you and I have talked several times in the past about this idea of mental models. You talked earlier about reading and ways that you’ve tried to level up yourself, and I know you’ve tried to cultivate a different way of thinking, or thinking about how you think. How have you gone about doing that?

Elan: There are a lot of frameworks I like to use depending on the situation.

One of the questions that I like to ask myself is “What would Elan five years from now do in this situation?” That really forces you to step outside of short term thinking and I’ve found it helps me operate at my best.

A slight variation on that question when trying to evaluate two different things to do is “what will matter in five years?” Am I going to remember being at that event, meeting that person, closing that deal, or am I going to remember that I was Inbox Zero today? So that’s two variations of a question I use on a personal level.

In terms of team decisions, the framework we use there is to try to think in extremes. On one hand do we shut this project down? On the other hand, do we have every single person in the company focus on this project? On one hand, do we have no resources, on the other hand, do we hire the best person in the world to go do this? By pushing the boundaries initially, you tend to get comfortable with some kind of decision in the middle. So that’s often how I think about strategy and where we want to spend our time.

On a company level, one of my favorite frameworks is based on a book called Predictable Success. Once you’ve done this a few times, you start to see patterns and the evolution of a start-up, and you’re like oh, yeah, it’s always this way. Predictable Success is a framework that talks about that evolution.

Imagine an arch, like a rainbow, and at the bottom there’s survival. Call it trying to get to product market fit. And many companies never make it past there.

If you do, phase 2 is called fun. When you’ve figured something out and you’ve got funding it’s fun. The founder has some money in the bank, they maybe get a salary, they’re in the press, their friends are like, oh, maybe you’re not as crazy as we thought. You hire a few people, you get an award, all that fun stuff that comes with getting funding or actually getting paid by a customer. You have a handful of people and everything is great.

But then as you work your way up the rainbow, at the top of the rainbow is this thing called predictable success. That’s where you set a goal and you hit it, and you set a goal, and you hit it, and you can think about some of the world-class companies today that are probably there.

There’s this middle stage between fun and predictable success called white water. Imagine you’re rafting and you hit these rapids where there’s bumps in the road and things get a little hard. And maybe in life maybe that’s your teenage years where you’re struggling with who you are and what the world looks like.

You start to get big enough where the people you had early on who worked hard and learned fast but maybe hadn’t done it before aren’t necessarily the right people anymore to lead a team. Sometimes they are, but often they aren’t.

Things start to break. The process that you built for communication when you had two people, or five people, or 10 people doesn’t work for when you have 100 people, or 200 people. The technology you built when you had a few customers doesn’t work anymore when you have millions of customers.

White water is really about getting the right people, processes,  an technology in place so that you can get to predictable success and operate like a world-class business. The reason I like that framework so much is it provides context. It’s a really good tool for communicating to your team about why things feel like they do.

When you start to hit white water, it can be tough. Team members say things like “It’s not as fun as it used to be.” Or “I don’t get to spend as much time with the CEO anymore.” Or “Why’d you hire this experienced person to be my boss?” Those are all real pain points that can kind of kill a culture. But if you’re really transparent and communicate with your team about where you are on the journey, giving people context can help a ton.

The other thing it’s helpful for is communicating to new team members. You bring someone in and they look under the hood and ask why this process in place, why things are such a mess. And I can just say “If I had optimized my process for communicating with 200 people when we were two people, I would have never earned the right to be here. I would have never gotten through survival and fun, into this white water phase. I’ve hired you to build the process to get us from white water into predictable success.”

Succeeding early

DI: Most of your recent experience at least has been operating in hyper growth mode. But I know in a previous life you were involved at an incubator. For people in a corporate innovation group or a similar incubator type situation, any advice you would give to increase the chances of success there?

Elan: Yeah, I was at Sandbox. It was really a neat place at the time because on the same floor you had people in venture capital, and these entrepreneurs in residence starting businesses, and people running the businesses. So it was this really neat mix and it was a really great opportunity to learn.

When you’re first getting a company off the ground, it’s really hard to do a lot of things well. You have to pick one. You live and die by speed, momentum, and traction, and change, and testing.

All these things that are really the antithesis of a big company, where you want to be slow and careful, and not make a mistake, and not lose your job, and not mess up, and not upset your customer, not ruin your reputation.

Like I said before, putting a wall between those two with different P&Ls, different teams, different goals, different evaluation criteria, is really helpful.

The future of mobility

DI: At SpotHero you’ve elevated your thinking, or broadened your thinking. It was about helping people find parking, you’ve executed on that really well. And now a lot of your thinking these days is about thinking into the future. About what the future of mobility might look like, and how changes in transportation impact a business like yours. What you think the future of transportation looks like? And secondly, how do you make decisions at a strategic level with something so fuzzy? How do you place bets?

Elan: We’re in a really interesting time for transportation. If you think about the tech world, the 2000s were about the web and SaaS and user generated content and social networks. The last decade has been focused on mobile. I think there’s going to be a pretty massive revolution in the next decade or so around transportation.

SpotHero started at the beginning of the mobile revolution. With the thought “Why do I have to drive downtown and circle around the block, and not know where to park, and not know how much it’s going to cost, and have to pay cash?” All these things felt very outdated for a mobile world.

Our business has evolved from selling parking on a website, to a mobile app. But as we think about the future much of parking will be driven by API. You’re going to use a variety of interfaces – a computer in your car, a smart speaker in your car, another mobility app you use to get around, or perhaps even the car parking itself. So that’s what we’re working to position ourselves for.

In terms of transportation in general, there are a lot of challenges. Cities are getting denser, and if you look at data around population and urbanization, cities are going to be way more dense in the future than they are now.

Traffic has gotten way worse over the last few years. Ride share is a huge part of that. There are so many cars now that commutes are 10-30 minutes longer in some cases than they used to be. That’s another major challenge.

Another challenge is consumer expectations have changed. People are no longer willing to stand at a bus stop, not knowing when the bus will come. People aren’t willing to deal with a cab not accepting a credit card. Expectations around how transportation should be have changed. But because of the infrastructure, and government regulation, and all sorts of crazy things it takes longer to change transportation then it does to change our expectations.

In terms of the future, I think it’s important to remember that just because something is said in the press doesn’t mean it’s going to happen. When people started to talk about autonomous vehicles the headlines were like “Parking’s dead because autonomous vehicles will be here next year.”

But autonomous vehicles aren’t going to be here next year – it’s going to take time. And when that does happen, are they going to be in perpetual motion, driving around the streets 24 hours a day? No. That would only cause even more traffic and congestion. The biggest cost associated with autonomous vehicles will be them being on the road and the wear and tear, so that doesn’t make any sense.

Then the headlines became, “I’ll take my autonomous vehicle to work and then it’ll just drive back to my personal garage.” So then traffic gets even worse – instead of one leg back and forth every day, there’s two legs back and forth every day.

If you’re actually an inside of the industry, and you talk to car companies and you talk to autonomous vehicle companies, they’re all highly interested in a few things.

They’re interested in navigation. How am I going to figure out where the car should go?

They’re interested in entertainment. When someone’s in the car or the car’s driving itself, are they going to be entertained during that period of time?

And they’re interested in parking. Where’s the car going to park, where’s it going to get charged and where is the fleet going to get cleaned and taken care of.

Going back to ride share, it has become so popular and so cheap that it’s actually not pulling that many people from driving. It’s pulling a lot of people away from public transit. Why take a subway in New York which is hot and crowded if for an extra dollar you can be in a Via for example.

Waze carpool announced they’re expanding nationally. Their thesis is the cost to improve the infrastructure to accommodate all the people and cars to fix traffic is insane. But what if you just rode to work with your neighbor, or someone in your community, and had people share cars when they’re going from work to home and back and forth.

So I think you’re going to see a lot more innovation around how we get smarter about getting around, whether that’s on a scooter or car autonomous vehicle.

Challenges to autonomous vehicle adoption

DI: In terms of autonomous vehicles, you mentioned it’s not a year away, There was a ton of press around pilots, and it seems like a lot of those have either gotten more modest, or they’ve been shelved entirely. For folks that aren’t following the space, what are some of the challenges that have to be addressed in order for the autonomous thing to become a legitimate reality from what you understand?

Elan: One is the technology. There’s a reason that autonomous vehicles are becoming relevant now, because there’s a handful of technologies that have emerged to make it possible. But there are still some technology things that have to be solved.

There is infrastructure and insurance and regulation, For example there is this concept of “duty of care”. As a driver you have to do your best to keep everyone else safe. But what if you’re a machine? What if you’re a car being driven by an algorithm, what does duty of care look like?

Ethically, or morally, how do we program these algorithms so if you have to make a trade off between hitting person A or person B, how do you make that decision? So there’s some things that as a society, we have to figure out. That’s a big one.

There’s others. The average car is on the road for 11 years. So let’s say tomorrow, autonomous vehicles are everywhere. What are you going to do with your car? If you still have your car, it’s going to be cheaper and more convenient for you to just use your own car, because you have it. And if everyone tries to sell their car at the same time, there’s going to be an amazing amount of supply of used vehicles and low demand, so there’s some things that are going to happen there. There’s a natural evolution that’s going to take time.

I think there’s going to be use cases where autonomous vehicles make a lot of sense, but there’s some where it doesn’t. If you’re a senior citizen, part of a retirement community in a suburb of Arizona where there’s not crazy things happening on the road and maybe you can’t drive, autonomous vehicles are amazing. The risks are relatively low. But what happens when there’s three feet of snow in Chicago? Or you’re in these places where there’s real world challenges that are pretty complex. So those are some of the things people are thinking about.

Scooters

DI: What about scooters? You mentioned urbanization and how cities are going to get a lot more dense. How do those fit into your picture of the future of mobility?

Elan: In the last week I’ve been on a plane, I’ve been in a car, I’ve been on the train, I’ve used ride share. Chicago doesn’t have scooters yet, but I’ve tried them out in other cities. So I think we’re entering a multi-modal world with transportation. The more options or ways to get from place A to place B is really a good thing for the world.

It seems like everyone has their niche. If you’re 50 miles out of the city, you’re probably going to want to take the commuter train in. If you’re 10 miles out of the city, you’re probably going to want to drive. And if a neighbor or two or three are down at one time you’re probably going to want to do ride share. If you’re going half a mile to a mile, scooters are really interesting. They’re not going to take you 15 miles away, but they might give me five more minutes in the office every day if I can get from my office to the train faster. So I think there’s a place for all of them.

One thing many high growth startups have in common is the growth team – a multi-disciplinary group tasked with moving bottom line product metrics. Unlike traditional marketers that focus on acquisition, they are tasked with iterating on the product to improve things like retention, referral, and average revenue per user.

Mike Rome leads growth at Eat Street, and in his career has helped acquire millions of users for a number of startups. In this discussion Mike shares how he approaches growth, his methodology for prioritizing experiments, the components of a good growth team and more.

• The 5 levers for growth
• Growth process
• Looking for growth opportunities
• Baking growth into your organization
• The risks of “faking” growth
• Innovation accounting vs. traditional accounting
• How to execute the process
• How to run experiments faster
• Good enough data
• Building a growth team
• Getting developer resources
• Growth models
• Creating a growth culture

The 5 levers for growth

DI: You’ve talked about the intersection of growth and product. When you say things like “the product is the marketing,” or that a lot of the levers that you can influence from a marketing perspective actually live inside of the product itself, what do you mean?

Mike: One problem with all the data out there now is the noise degrading the signal. What you really want to do is operate above the blizzard. You want to figure out the metrics that actually put you on a path to sustainable growth.

I like Dave McClure’s “Startup Metrics for Pirates” which talks about 5 levers for growth: acquisition, activation, retention, revenue, and referral. We can just run through the definitions really quick.

Acquisition:

Acquisition is essentially marketing. How do we get people to this thing that we build? How do we get people who we think we might be solving a problem for with this new solution to this thing?

Lots of people do this. When they are thinking through growth, they’re thinking about the channels they could potentially tap to acquire customers.

When I’m looking for success in an acquisition channel, the two things that always matter are scale and unit economics.

Scale means “can we find a sufficient amount of customers relative to whatever the goal is?” Unit economics is the cost to acquire them relative to the value that they create. We talked about that generally as LTV.

Like I said, most teams do this. But many companies lean too heavily on acquisition or marketing as an engine for growth.

Activation

Activation means once we get people to this thing that we have, how do we get them to do what we want. Depending the size of what you’re selling, it might be a purchase if it’s an easier sell. If it’s something way more expensive, it might be a sign up or even watching a video or something like that

Retention

Retention is how to get them to continue to come back and do what you want. Probably the most important of them all, because it’s the most indicative of a good product.

Revenue

Revenue is how you get them to engage in some sort of monetization behavior. And there’s ways to do that beyond bringing people to your site and letting them buy what they think they want.

Referral

The last one is is referral. How do you get them to have such a great experience that they become the marketers for you?

The really big takeaway: of these five levers, only one of them is marketing. All of these other things happen once people get to the product itself.

People don’t spend enough time trying to study, reflect on, and experiment with those product levers, and ultimately they’re a much more ROI-friendly way of growing. If you already have people coming to something and you’re able to use the resources you have in-house to build a better product to raise order values or encourage them to become marketers for you, you get a lot more like bang for your buck.

Growth Process

DI: I would imagine that a lot of people who do marketing would say “I stand up referral campaigns. I build funnels to activate.” They might think this sounds the same as what they do now. What are some of the process or competency differences that you think distinguish growth from marketing?

Mike: Sure. I think you’re hitting on this idea of putting growth process before tactics. Which is one of the bigger differences I’ve seen.

Early on when I got into tactics, a lot of it was informed by who’s just yelling the loudest in their content marketing on social media. You’d read something and be like, “Oh, that sounds interesting, right? We should go try that.”

The danger and putting tactics before process is unless you have a way of continually starting and finishing tasks, it’s very easy to get left behind. Everyone who’s listening to this podcast is going to be resource-constrained. If you don’t have a process to thoughtfully prioritize and have conviction around what you’re doing, you’re just going to waste too much time.

Our process at a high level is to start with a brainstorm around those five letters that we talked about – acquisition, activation, retention, referral, revenue. We try to go for unbridled ideation and think of things hat you don’t even think will work.

Maybe they’re just marketing channels that are less intuitive. Sometimes that stuff gets really interesting because they’re generally less crowded less saturated stuff. Once you do that brainstorming to get all the ideas down, I would pair it with some sort of quantitative audit to assess potential impact.

It’s okay if you’re just starting up and you don’t have any internal data to reference to inform what tactics within those levers you might hit first. Secondary research works – there’s stuff out there, maybe future competitors in the space or general insights that have been written about either the problems you’re trying to solve or even solutions that exist.

Looking for growth opportunities

DI: If they do have data, what are you looking for to assess impact—things like bottlenecks? Is that what you mean when you say doing an audit of what they already have? What should they be looking for?

Mike: I think one of the dirty little secrets of trying to grow something is that often the best things are just fixing stuff that’s broken.

A lot of people hear this term growth or growth teams, and it’s a sexy notion. But the reality is a lot of times we’re just finding hiccups in the product and taking things away or just making things work as we thought they would work, across all platforms or all browsers.

Outside of fixing stuff, we’re looking for where inputs and outputs not equal. How do we spend a low effort on high-potential, high-impact, high-confidence tasks?

We use something called ICE scoring to prioritize. ICE is just an acronym for impact, confidence, and effort.

So after brainstorming and doing that audit, you have all these things you want to test or these new product ideas as it relates to activation, retention, etc.

For every single idea, you should start with impact. Ask, “If this thing were to work, what’s the impact? What’s the payoff?”

You can put together a quick and dirty model. It doesn’t have to be perfect, just go through the exercises and figure out what’s important.

Next is confidence. It’s great if you find an idea and you’re like, “Wow. If this works, the impact is huge.” If you don’t have a lot of conviction around the impact though, and there’s not a lot of primary or secondary data to support what you’re saying, it’s probably not the first thing I would do. Confidence matters just as much as impact.

The last thing, the most important thing when you’re starting up, is effort. What’s it going to take to actually test this new idea? Can I do it myself? Can someone on my team do it? Do I need a developer? Do I need a designer? Do I need multiple developers?

The Holy Grail is the low-effort, high-impact, high-confidence ideas. And that’s where you want to start.

The reason I would weigh effort so much is in the early days, momentum is everything. If you think about the formula for growth, it’s number of experiments you can run multiplied by impact multiplied by success rate.

So you have these three inputs and really only one of them is in your control, and that’s the number of experiments you start and finish.

And the reason starting and finishing so experiments is important is because that’s how you learn. Most of the things you’ll do don’t work. You don’t get demoralized. You ask yourself ask yourself “Why didn’t this work? What was our hypothesis that was disproven and how does that re-inform our priorities? How do we move stuff around based on what we’ve learned?”

The more tests you can start and finish, the quicker you learn, and the quicker you learn the higher your success rates. And that’s when you get that flywheel spinning.

Baking growth into your organization

DI: What would you say to somebody that says, “Hey, this all sounds awesome. I would love to try to implement something like this for my team, but I have folks who come from a different sort of mentality?” How do you try to make something like that happen inside of an organization?

Mike: It’s really important when you have those conversations not to think that that you have all of the answers. Starting from a place where your path is the only path is a bad idea.

I recommend this path just candidly because I’ve seen it work for me. But I’m always mindful that our process is a living process. It’s broken. We are changing it all the time and trying to get better.

And so I would ask questions. “Why do you think that the best approach?” It’s important to try to persuade people that you’re just trying to develop a better mechanism for how to move forward.

Everyone thing every realizes is there are a thousand different things they could be doing. So I’ve told people that process is piece of mind. It gives you confidence in what you’re doing and not doing. And looking at it from that lens is really helpful.

The other thing is to realize it’s only part of the equation. There’s still the need for brand marketing. In my job today, I spend a half my time on marketing and customer acquisition and the other half on product. But my side is very data driven.

The other part of customer acquisition for our company is brand-focused. I’ve learned they complement each other super well. If you execute on creative, whether it’s TV or radio or out of home, they can really lift up direct response and what my team does. So they’re symbiotic.

The risks of “faking” growth

DI: It seems like if you were to start on the product side, get some wins, demonstrate the process works, and then ask for a seat of the table around acquisition, that might help.

Mike: I think that’s a great idea. It’s also important to stress that they have their job, which is to get people to the site, and you have your job which is to make sure the product does what you sold.

One of the big mistakes I’ve made and I know is common is that it’s really easy to fake growth. The number-one way to do that is to find channels where you can acquire lots of inexpensive customers without being mindful of the unit economics on the other half of that equation of successful customer acquisition.

It’s equally important to start with the product and to worry about levers and retention to make sure you’re ready for marketing. If you haven’t built something that enough people want and ideally are willing to pay for, it won’t work.

You want to get enough people using the product to see if some of those activation retention metrics are improving; you just want to know if you’re steadying the foundation. A lot of times the foundation isn’t steady, and when people get into customer acquisition it’s intoxicating to see lots of app downloads or lots of views or even purchases but, you know, those are one-time purchases. If people aren’t coming back, that’s the biggest mistake.

DI: You see that a lot with startups who have investors setting expectations around consistent growth period over period, and now you have to keep plowing increasing amounts of money into it to continue to show what they’re expecting, even if your foundation is shaky underneath.

Mike: Absolutely. You have to acknowledge that even even if you have really great investors and backers, your incentives are never perfectly aligned.

If you look at the unit economics of how most investors or funds make money, it’s not because they have lots and lots of wins. It’s usually one big whale that carries the fund.

So they have an incentive to figure out who’s going to boom or bust as fast as possible. They want to figure out if they should spend their time and energy with a business or even put more capital into it. As a result there is a lot of pressure to spend money to drive scale, even if it doesn’t make the most sense for the business.

I’m lucky, in that we’ve got really awesome investors, and we can have these kinds of conversations about what’s responsible. But there’s always pressure to spend faster and you really want to make sure you’re spending in a way that’s sustainable.

Innovation accounting vs. traditional accounting

DI: We run into that in an innovation context where organizations are used to measuring success the same way they measure the success of the core business. There’s a strong tendency to want to show immediate results and to make it look bigger than it’s ready to be. that whole that concern around premature scaling. But it takes time to iterate and find product-market fit.

Mike: One of the best pieces of advice I would give to folks who are part of larger organizations and working on an innovation teams is you have to have buy-in from the top that this is going to be a long and demoralizing road and, if people don’t have the stomach for it, it’s just tough.

It’s very exciting at the beginning. But what really drives success is the people and teams who can persist in the trough of sorrow. It comes down to determination and commitment because of all the startups that I’ve studied or been a part of, there’s always that slog. You have to have that buy-in from the top.

Someone told me the other day that one of their best qualities was they take bad news very well. I think that’s actually an amazing competitive advantage. You’ve got to be excited enough about the problem you’re solving to keep going. If it can’t hold your attention span, or the attention of your organization, it’s really tough.

How to execute the process

DI: Getting back to process – what does that look like concretely? Can you think of examples of focusing on things that weren’t acquisition related and were able to generate meaningful upside for the organization?

Mike: Sure. So in the early days before we had good language around this, again we were really resource constrained like most of the people probably listening to this.

We were hitting the database and seeing where users were getting hung up. Or going to a coffee shop and user testing a signup flow with people. Talking to users and digging into the product. You can talk to customer service people and figure out where people are getting frustrated.

You obviously have to have good tracking in place to mine for good inputs in the database. All the steps mapped out. In our case it was medical fundraisers. So activation was getting people to start fundraisers and make some money for the fundraiser up front.

We had enough events in place within the products to look at all of those steps and just see where it was broken, and those were always great examples of just not having enough innovative ideas for how we could improve the flow. Mostly it was things not working as we’d communicated or intended for it to work.

How to run experiments faster

DI: Talking about the experiments and speed and number of experiments that you’re running is the one metric that you’re able to really control. Are you running a bunch of experiments at the same time to move one metric? How can organizations move faster and control that one variable?

Mike: It’s a million-dollar question. I think tracking is really tough and it’s hard to know what the optimal things you should be doing at once are.

I think sometimes people get hung up trying to start and finish an experiment and definitively know what it means.

Take acquisition. In most cases you just need to know if there’s something there, or if it’s a hard pass. Assuming you get some results, and you know there’s scale, that’s probably sufficient. So you don’t always need to have absolute certainty about something – often directional certainty is sufficient.

That’s really important when you’re resource constrained. Even for me – I work for a later stage startup with hundreds of employees and tens of millions of dollars in revenue – I’m still super resource constrained. I don’t always have the luxury of running 5 product experiments at once.

DI: People that think of testing probably think of Amazon’s hundred shades of blue. But unless you have scale, it’s going to take way too long to find wins that way, especially since most experiments fail.

Mike: Yeah. I think early on that’s good advice. If you’re only changing this one thing and it’s very early and you have a small subset of customers, you’re probably not testing ambitious enough things.

Don’t worry about this idea of not knowing your goal; you want to acquire twice as many customers or you want to improve the signup flow by 50%. You’re not looking for a five percent relative improvement, you’re looking for a 50% absolute improvement.

If you adopt that mentality, you have the freedom to change more things. You just have to get content knowing that even though you’re better off, you might not be exactly sure why things are working and that’s OK sometimes.

DI: I’ve told our team before – lacking causation, I’ll take correlation. I don’t know if this is actually causing it, but it seems to be correlated and that’s good enough for me for now. Let’s deploy and move forward.

There’s also a team member education piece to this, where you’ll run the test higher up in the funnel – on your registration page for example – and if it works and you see a 15 percent lift, they’re thinking that’s going to trickle down to a 15% bottom line bump.

But it usually doesn’t work out that way. Any change that you make changes the way people interact with your product. Their behavior is going to be different.

Mike: Yeah, there are people a lot smarter than me on A/B testing. Evan Miller writes some really thoughtful things around what to pay attention to when you’re testing different experiences.

I think you have to be OK with some of this ambiguity and not knowing if you want to move forward fast enough.

It’s a delicate balance though. How do you make sure you’re using data to have conviction around what you do next, while operating above the blizzard of data?

Good enough data

DI: For us, often clients have the reverse problem where they don’t have any tracking set up, at least not at the event level. They track sales or registrations, but not all of the events that lead up to that. So we usually we usually have the reverse problem.

Mike: Well I think the larger takeaway here is, when it comes to starting something even if you don’t have data, that’s fine. Go make data. Go talk to potential customers. Don’t sit in a conference room and ideate with a bunch of the executives who aren’t going to be the user of the product.

I don’t think this whole idea of “we don’t have analytics set up” or “we don’t have users” should get in the way of putting in the work. Go talk to talk to people. Even if it’s bad news. The best feedback you can get when you want to build something is figuring out it doesn’t work. Figure out why and improve.

DI: I was having a conversation with our creative director about design thinking. We often do “how might we” exercises that are similar to the unbridled ideation you were talking about.

He said a lot of UX is about pattern recognition, and is dependent on the inputs you have. Someone cold off the street can throw out ideas, and it’s better than nothing. But they’ll be less informed than if you do some research in advance of a workshop. If I know what my customer says and how they use my product, you have much better inputs and the session is more productive.

You mentioned the potential risks of becoming prescriptive and looking at what other people have done and just copying it. At the same time, those are inputs I can bring and say, “Hey, this product or completely unrelated industry solved a similar type of problem by doing x.” Does that make sense at all?

Mike: I think it’s a really powerful point. I think again, especially when you’re starting, don’t get hung up on not having certain inputs. And don’t get hung up on this idea that you need hundreds of pieces of input.

There’s plenty of reading out there around picking up UX patterns. And from a user testing perspective, it takes a small subset of users to figure out what things would satisfy 80 percent of the product needs.

Don’t get hung up on the idea that your inputs aren’t good enough. Just go find any inputs. If you’re curious and you’re bringing inputs to the table that are rooted in actual user feedback, not just from your head but from talking and listening to a potential customer, that’s useful.

Paul Graham talks a lot about how users have lots of the answers. Nine times out of ten, I can’t think of a better use of time than just going and listening to some users and studying them, either using the product or reacting to you talking through a product.

Building a growth team

DI: Let’s talk about team. I know it looks different depending on the stage of an organization, but in order to execute on growth, it seems like you need more than just a person with a marketing mind. What skill sets are necessary?

Mike: Curious people are a big deal. You want to find people who are comfortable with bad news. They’ve got a lot of persistence and determination—the whole idea of a rapid rate of learning is the reward and that’s enough. I think all of those are really good things.

In terms of the makeup of an early team, you should have people on the team who are essentially the customer. It’s hard to build something if you’re not the customer and you have to guess at what the customer wants. Be the customer because then you can move a lot faster. You can just think about how you would create value for yourself.

When it comes to actual skill sets, it’s good to have a developer to execute on experiments. And you need some design help. Design matters.

I think technical marketers are important—people who are really into direct response marketing. They’re good because they’re generally detail-oriented, so they can double as product managers.

They also have empathy for developers and designers to respect what they’re asking them to do. It’s really easy for someone to say a couple of sentences and create five weeks of work for a developer. Knowing what you’re asking and why that’s hard is important. It’s just good for the morale of the team.

You also need someone with some analytics background. Have someone who is a truth seeker and is going to keep you honest.

I think if you just find people who are excited to build shiny new things instead of building things that actually solve a real and deep problem, that would be difficult.

Getting developer resources

DI: One of the issues with growth teams has been getting access to developers because so many teams are resource constrained. How do you recommend advocating for getting resources when you’re just getting started?

Mike: When it comes to getting developer time, you have to find someone who’s excited about the opportunity. You need to find a developer who gets fired up about the work. You can’t understate the importance of being excited about the problem. At some point it’s going to get hard. And if you’re not excited about the problem, it’s tough to keep going.

When you’re trying to get buy-in from the top, explaining the value the work could create for the business is a good approach. Just let the facts do the talking. It forces you to be really well researched and prepared. It’s not very hard to show that working on x versus y could have a financial benefit to the business.

Figure out who you’re trying to get buy-in from. What are the things that make them look good? What do they care about? It’s tough, but politics matters. Especially if you’re in a larger company.

Growth models

DI: One of the tools that we’ve been using a lot more in the last year is a super granular growth model. Trying to visualize and quantify all of those levers, figuring out the six steps during activation here and the three different referral loops inside of a product.

Now, instead of saying let’s spend three months on this feature because is’t next on the roadmap, showing how something will influence a variable and doing a sensitivity analysis is really helpful in making that case.

Mike: The reason detailed models are are very helpful is it opens your eyes to the road ahead. Getting a sense of awareness before you start and a degree of humility is going to help you. You want to make sure everyone has the stomach for what’s ahead and then is still excited.

I took a look at it. You sent it over to me a little while ago and I was super impressed. I mean, a lot of that stuff was more savvy than some of the stuff I’m using, so hats off.

DI: Everybody gets it from other people. That was that was all Brian Balfour. It’s incredibly helpful for vetting potential investments as well. Founder says, “I’m raising $1.5M and our plan is to get to 50,000 customers over the next twelve months.”

And I ask, “How do you plan on getting there?”

And they’re like, “Well, there’s this bucket of money and we’re going to do word-of-mouth and we’re going to do paid marketing.”

When you show them the model, they start to realize that getting to 50,000 is going to take a lot more work than they thought. When they talk about doing content marketing, you have to find out how many articles you’re going to have to write to rank. What’s the monthly search volume on all of those and your expected click-through rate, etc. They start to realize they need to really buckle up and plan on hustling.

Mike: I couldn’t agree more.

Creating a growth culture

DI: You talked about process and you’ve talked about team. I know you’re a big believer in culture too. Once you’ve got the team together, what are some recommendations for instilling a culture of growth to make this this stuff stick?

Mike: A great shortcut is to find people who fit the mold. There’s certain things you can’t teach. You have to find people who are determined, who are okay being wrong a lot, who are curious and truth seekers.

Setting the precedent up front helps too. Making sure people really understand how difficult the road ahead is going to be and are bought in. I don’t mean to talk about it negatively because I love it. But the learning curve is just ridiculous. You grind it out for six months and hit some hard times. But you come up for air you just you’ve built this new skill set and that high rate of learning never stops. There are certain people who just have an appetite for it.

Ask some tough questions in the early days to make sure people are doing this for the right reasons. Get people who aren’t just here because it seems like a cool idea. It needs to be something more—people really need to believe in the idea.

DI: I know you have a reputation for keeping people’s enthusiasm levels high throughout the slog. And you also have a reputation for modeling behavior – getting your hands dirty instead of just barking orders. Were those deliberate choices?

Mike: I’ve always appreciated people that are still in the trenches. That carries a lot of weight for me.

The other thing is that lately I’ve just been trying to get out of the way a little more. We’ve been at it for a while; the team’s been a formal team for several years now. And I think sometimes you need to get out of the way.

I think about all of the best growth ideas that have happened on growth teams that I’ve been a part of and not one of them was my idea, and so getting comfortable with that idea is something I’ve been working on.

The thing I’m always focused on bringing to the table is just making sure there is a culture of meritocracy, where best idea wins and it doesn’t matter who it comes from. Use ICE scores, make sure people have the tools and time to execute, and get out of the way.

The financial services industry impacts us in broad sweeping ways, perhaps more than any other industry. And technology is forcing legacy institutions to adapt while juggling massive compliance, security, and legacy platform concerns.

In this discussion with Jason Henrichs we talk about how financial services companies can adapt more successfully, leveraging unique frameworks that keep the existing business moving while enabling rapid tests and decision making. We also discuss how various emerging technologies are going to impact financial services going forward.

• Innovation and Regulation in Financial Services
• Bimodal IT in Financial Services
• Using Innovation Accounting
• Innovation sandboxes in financial services
• Measuring the success of innovation initiatives
• Unbundling in financial services
• The rise of robo-advisors
• Leveraging data for financial services innovation
• Blockchain in financial services
• AI in financial services
• Basic income
• The opportunity for fintech to create social good

DI: I know you’re working on several things almost always. What are you up to these days?

Jason: The main thing is I’m co-founder and managing director of a company called Fintech Forge. We’re a managed service business that helps financial institutions extend their innovation capacity, and we’ve developed a proprietary way of looking at innovation in a highly regulated environment.

Related to that, my partner JP Nichols and I are the co-hosts of Breaking Banks with Brett King, which is the largest fintech podcast in the world. And we use that platform to talk with innovators and push the evangelism of the need to evolve and better serve our customers with financial services.

Lastly, I am the chairman of FinTex, which is a non-profit industry association growing the financial technology ecosystem in Chicago and the broader Midwest. We try to push both incumbents and startups to build better financial service products that are more inclusive. 

Innovation and Regulation in Financial Services

DI:You mentioned regulatory issues being one of the more unique aspects of financial services. What are some of those challenges or problem, and what are some ways organizations try to navigate that?

Jason: When we think about regulation, it really has three components.

First is intent. What is the intent of this new thing that you’re trying to do?

The second thing is the policies and procedures around how it’s delivered. And the last are the outcomes.

We’ve avoided measuring intent because that requires personal judgment. And outcomes are hard to measure – it often takes a long time to develop.

So the emphasis from a compliance point of view has been to audit policies and procedures. This leads to the people, processes and technology being optimized around scalability, reliability, and compliance. And as a result they’re very rigid. They’re expected to deliver with 5 nines.

It turns out people don’t like if you send a wire transfer and it doesn’t arrive. Doing your job 80 percent of the time isn’t going to cut it.

When we think about innovation – doing something new to create value – if you’re doing something new, you have to learn. And learning requires actually having things go wrong, which is the antithesis of optimizing for the highly reproducible five-nines.

So how do you actually build these two organizations that can work together simultaneously? We would never suggest throwing out all of your policies and procedures, regulation be damned. You can’t just treat your existing business like a startup – that doesn’t work.

When you’re executing your existing business for the types of results that you’ve come to expect, you can predict with high accuracy and high precision – just keep doing what you’re already doing.

But when you’re looking at something new, you’re probably going to have dramatically difference results. You might have a 50% completion rate, and you’re looking to get it to 80. You need to go do experiments and do things very differently.

And that’s where our frameworks around doing experiments comes in. We call it FIRE building on the forge as an analogy. How do we do Fast Iterative Responsive Experiments?

• Fast: shorten the time between the discussion and actually having a result that can be debated.
• Iterative: take the result and you continue to build on it.
• Responsive: you need to respond to what you learn and it needs to change your mind. 
• Experiment: at least half the time it isn’t going to work as you expect. 

Bimodal IT in Financial Services

DI: Do you recommend that the team responsible for the core business is also executing on the innovation stuff? Or do you recommend carving out a separate team, like a Bimodal IT methodology, where they have a very different mandate to move fast and break things, they’re measured differently, etc?

Jason: For the majority of institutions, if you’re looking to actually transform your mainline business, disconnecting it completely from that customer-facing experience actually makes it a bigger challenge because you need the customer to be driving what innovation looks like.

If you’re creating a separate group that focuses on innovation where the mandate is about complete reinvention, we think it’s totally appropriate to create an innovation team that’s completely separate.

In most organizations, we’ve found it’s very successful if they work in that bimodal mode. They work in the core business, but then they jump in and participate on FIRE teams with very different objectives. It actually becomes a badge of honor to spend time on a FIRE team.

As an interesting byproduct, people often carry those practices back into the other parts of their job. We’ve found over and over that job satisfaction actually goes up as a result of feeling like they’re doing something new. It gives them a special sense of purpose.

DI: Once you get this team carved out and they adapt to this way of working and they’re moving quickly and they fall in love with that approach, and they want to bring it back in, how do you how do you maximize the likelihood that the organization, when they do kind of get folded back into the mothership, welcomes that change and adapts accordingly?

Jason: One of the important things is the fire team doesn’t actually leave its day job.

The fact that we’ve asked people to take on more work than they’re already doing, and they consider it a badge of honor and job satisfaction goes up is not what you’d expect.

But they’re feeling purpose because they’re helping drive change, and they find some other areas where they can either free up time or use time that was being misallocated anyway.

But remember, these experiments are meant to be as incremental as possible. And the results are being published continuously. The whole idea is to quit doing things that aren’t working sooner.

So that’s part of how we change the cultural stigma around stopping a project. It isn’t failure; pursuing it even though we know it isn’t going to work or meet our expectations would be the failure.

So by publishing those things along the way we can adjust as we go and work with the body of the organization.

It’s like using your own stem cells to grow your replacement organ. It’s not like a fancy consultant can come in and successfully say, “Here’s your innovation strategy, go execute it and call us when you’re done and successful.” That’s just not likely to be adopted.

It’s our perspective that the best ideas are by the people who are closest to the business, but what they haven’t had is a toolkit and a governance process for how to actually take ideas and evolve them into what will ultimately be successful.

Using Innovation Accounting in Financial Services

DI: When you’re teaching a team to think this way, they are probably used to measuring success in a certain way and they’re used to measuring things of a certain size and a certain degree of scale. But when you’re talking about some of the more disruptive stuff, they probably look really small. How do you teach them to understand the definition of success with an initiative like that is going to look a lot different than how you may be used to evaluating success or failure?

Jason: We’ve created several lenses for the executive leadership teams to create innovation portfolios around. Those lenses need to take into account each organization’s individual strategy.

For some, innovation might be focused on new revenue growth. For others, it might be around expense reduction. So they’re going to be heavily focused on the tactical and the back office and what they’re doing for others. It’s transformation.

We work with a small bank that is in the Midwest that is not growing. In fact, it’s shrinking. Your ability to take share against competitors in a shrinking population environment is relatively limited. So they’re trying to create a new business unit that is going to act as the banking rails for fintech startups. That’s a transformational lens.

A lot of what we do is also give them the concept of building a portfolio of innovation. You’d never have your retirement tied up in a single place. Likewise, you need to tailor your portfolio to what your strategic objectives are in your portfolio.

You’d never allocate, the same amount to every single thing. You don’t say, “Hey 25 percent fixed income, 25 percent large gap, 25 percent in small caps, 25 percent in emerging markets. And boom, I’m done.” That might work at some point in your life. But as you get closer to retirement, you’re probably shifting things, but you keep small allocations.

Artificial intelligence is a great example, especially around doing some of the natural language processing and the ability to translate that into advice, etc. For most institutions, you can’t write a really good business case right now. If you’re dealing with things that are emphasizing learning, you’re either deluding yourselves or it’s just never going to pass the hurdle. So you need to redefine what the hurdle is.

So using AI as the example, don’t make a huge investment, but begin to play with it a little bit. Invite some companies in that are working in this space, whether that be IBM or a start-up, to just begin to open your eyes. Say, “We’re in the exploratory phase. It’s a small investment. It’s an investment of time.”

Or you take the next step which is “Hey, you know, is there a platform that we can redraw what the hurdle, redraws what the risk is we’re willing to take?” We call that in our nomenclature your FIRE break.

And so what’s the risk appetite? What’s the investment appetite? Let’s run an experiment and see what happens. Do customers engage with it? Do they not engage with it?

You can easily convince yourself to just be a fast follower and wait until someone else is successful. But that is actually really dangerous. The likelihood you will catch up is very low. And you can’t just take this same technology and assume your customers are going to use it because someone else did it. You need to tailor your innovation efforts to your unique strengths, weaknesses, and vision of what you’re trying to become in the future.

DI: You and I have talked in the past about the Volcker Rule and how that adds some complexity for banks in terms of being able to execute on things that would be considered speculative. Do these innovation initiatives fall under that and, if so, how do people mitigate that?

Jason: Volker was specific to investments in terms of financial returns, and some of that’s been repealed. But if we speak broadly around regulation, my view is deregulation isn’t the answer and over-regulation isn’t the answer.

Regulation is monumental, and here’s how I define a monument.

• It’s normally done in response to something really, really bad that happened.
• It takes a long time to build.
• The only thing that tends to visit the monument afterwards are the pigeons.

When I say monumental, I really mean it’s like building one of these monuments that becomes overgrown. The problem with that is deregulation says “let’s hold back for as long as possible and see what happens”. Well that tends to be really bad, and so then we overreact and go to the other extreme.

Dodd Frank’s a great example of this. By the time the bank’s got everything implemented, the world moved much further and faster past it, so we’re creating this backlog of systemic risk.

We work with a state bank regulator. And the regulation they’re using for cryptocurrency is from the pre-civil War era that was created to regulate the interstate transfer of private currency via steam boat. But it’s the only regulation on the books that is applicable in the same way.

We talked about testing and learning in our approach to innovation. I think we need to take a very similar approach in how we think about regulation. We need to not be afraid of trying something not working and taking it off the books and saying, “Nope. No longer applicable.”

The UK did this with the FCA when they decided they’re going to be the fintech capital of the world. They actually merged all of the agencies, put them into one that only had two arms, and one of those arms is really around looking at how it helps pursue innovation, and cleared the rest out.

We’re not going to do that in the US any time soon. But there are ways that we can take a more engaged, proactive approach.

Arizona just passed a fintech sandbox bill. There’s lots of discussion with this idea of an OCC – the office of comptroller of currency – doing a national charter that says not every bank needs to do all of the functions of a bank. This is an important first step.

It was challenging when the CFPB first came about and I was running one of the first Challenger banks in the world called Perk Street. One of the things they actually did well for as much heat as they take is they would be willing to sit down and have conversations. Historically, regulators have been sort of “Well, we’ll let you know when we come and do an exam and tell you whether what you did was right or wrong.”

In the post-meltdown world, some of the other financial implications of this – where your auditor isn’t supposed to give you guidance in advance of the audit – is like, it’s almost like having your doctor say, “I can’t give you any advice until you’re sick.” That doesn’t make any sense. Wouldn’t it be better if I didn’t get sick?

Innovation sandboxes in financial services

DI: You mentioned the innovation sandbox. I don’t know if they’re using the definition of sandbox the same way that you have used it in the past, but for folks that maybe don’t know, can you talk about why you believe having an innovation sandbox inside of an organization is important? And then to the degree that it is relevant and that there is overlap with legislation? How is that either enabling or making it more difficult?

Jason: You hit the nail on the head with part of this when we talked about the financial accounting versus innovation accounting.

If you take a new endeavor and try to hold it to the same standards of your existing business, it’s not going to match up. This is the innovator’s dilemma by Clayton Christensen — the incumbents have a tendency to over-invest in what they already have because the incremental investment is a lot more certain than pursuing something new.

I like to say the competitive advantage of startups is their desperate. If I don’t figure out how to sell you stuff at a price point that I can make money on I before I run out of money myself, I will go out of business. So they will very quickly develop products that customers want, versus the incumbent will continue to optimize, get that 2 to 5 percent growth, cut out 2 to 5 percent of cost and it works very nicely.

You need the sandbox to say, “OK, we can’t throw all the rules out. We actually just need a different set of rules.” And those are the rules are going to govern the boundaries and expectations.

I think you’ve heard me use this phrase before around innovation theater. The purpose of innovation isn’t to be like, “I have an idea that. There’s going to be Legos on the tables and it’s going to have all the slogans.” How many companies have you been in that that’s their innovation effort. They use Macs versus PCs, and they don’t actually have any outcomes.

That’s why our definition of innovation is “doing something new to produce a tangible result”, which is really important because when economies take downturns, that’s when those programs quickly go out of favor, or even before that when the CFO says, “We spent how much on this lab?? What has it produced?”

It needs to actually have an end in mind in terms of the problem you’re solving, and needs to be making steady measurable progress against it.

Measuring the success of innovation initiatives in financial services

DI: Do you ever run into a sunk cost issue, where even if they identify success criteria, they run experiments and they don’t hit that goal, but they got 80% of the way there?  How do you navigate that gray area there around success and failure?

Jason: I mean, it’s a great question. There is this tendency where no one likes to be wrong, particularly in financial services. If you have a loan officer that has a 5 percent loss in their loan book, that’s going to be an ex-officer.

But if you have an innovation department that has a 95 percent success rate, I will tell you you don’t actually have an innovation department there. Just a group executing on things that are already there.

The flip side is, you can’t just celebrate every failure and say, “Yay, we failed fast.” You actually need healthy tension in the organization to have debates on getting results, even if they weren’t what you were expecting.

There are three outcomes to one of these experiments.

• I’ve identified another experiment that needs to be done to de-risk this.
• I’ve learned all I can and we should just table it.
• It’s ready to graduate into the mainstream business and we want to actually roll it out much more broadly.

That should be a spirited debate, and it’s one of the reasons that we take a very strong view on governance. The fire team comes up with what the experiment is and then presents it to the governance group. They’re the ones proposing which of those three outcomes we’re at.

That’s why we don’t want an over-funded project, because that’s where the inertia comes in, and the sunk costs. It’s one of the reasons we find the CFOs become some of our biggest advocates.

One of the things we do to help teach this is we’ll take an executive leadership team, and we’ll sit them down to play poker.

DI: Bankroll management.

Jason: It is. Put them on teams instead of individuals. They have to place their back before they see any of the cards. They only get to see what’s in their hand, like typical Texas Hold’em, before they get to see any of the incremental cards. They place their bets and then they’re done.

Well guess which team wins every single time? It only takes a few hands. But the one that gets to see the incremental cards and can then decide to fold or not fold, they win.

And then we turn around say, “Then why do you run your business this way where you fund a two-year plan to go do this innovation thing? Instead, as part of your standard annual capital planning process, put aside a pool of resources—those are dollars and people—for your innovation budget.”

Don’t allocate it to a specific project, because that’s going to create the inertia you talked about. If it’s fully funded to specific programs, surprisingly it always gets spent.

My wife works in advertising. Success for her is on time in 98 percent of budget. If they do that, the client is thrilled, except they know what they’re actually spending it on.
If your innovation program is like, “Hey, we spent 98 percent of the budget. We spent it on time.” Well that isn’t necessarily success. So you need that ability in-flight to course correct.

Unbundling in financial services

DI: From a startup’s perspective, where you’re dealing with, not even 800-pound gorillas—

Jason: Trillion-pound gorillas?

DI: Yeah, exactly. By definition they probably have to take an approach where they take off a small piece of something and try to just focus on that, and maybe over time if they’re successful they can layer additional things. Do you feel like the trend around unbundling happening with financial services customers? Do they want to interact with fifteen or twenty different apps all loosely connects through APIs? Are the days of getting the majority of services through a single monolithic institution over?

Jason: I think this is one of the biggest impacts the iPhone had, other than keeping us glued to the screen for ten hours a day. It’s conditioned us to tailor a whole set of point applications into a single device. People don’t care if they have to flip between five different buttons on a phone to get what they want because it’s highly personalized, right?

Call it the “app genome” on your iPhone or your Android. It’s highly tailored to you. And you don’t care because it exactly matches your needs.

You want to tailor your music? Hello, Spotify. You want to personalize your mutual fund? Hello, Betterment. That propagates all the way through our financial lives, particularly as digital allows us to take so much of the friction out.

That’s actually where too often we just put digital lipstick on the analog pig. We don’t solve the underlying actual experience issues. We just put a digital interface in front of it. So let’s think about your mortgage application. I’m not sure the last time you got a mortgage.

DI: Two years ago. So still relatively fresh.

Jason: Yeah, and were you shocked that even though it was more digital, it was still excruciatingly painful and it never made you want to buy or sell property?

DI: Yeah. It was still eighty-seven steps. It was lots of e-mails that didn’t tell you anything where you had to log into a special interface due to security concerns. You couldn’t access it via your phone, so I had to wait until I was in front of a laptop. It was tough from a friction perspective. If you talk about BJ Fogg’s behavior model—my motivation was sufficiently high where I was willing to overcome all of those issues. But it was definitely tedious and the efficiencies gained from the tech were marginal at best.

Jason: I’m sure someone there was very proudly talking about their all-digital experience. But if you said, “I don’t care about my digital experience I care about my user experience,” it was horrible.

But look at someone like Rocket Mortgage. They didn’t just make it all digital. They re-plumbed the system and the processes they go through. I don’t care if my existing bank offers me a mortgage unless it’s at a rate that is significantly better if they’re putting that much friction in the process.

Why do I have to give you my bank statement? You’re my bank. That was my experience two years ago. It makes no sense to have to keep explaining these things. You didn’t actually solve the problem. I’m going to gravitate to whoever can actually solve my problem.

Now am I going to pay a tremendously higher rate for that?  No. But whoever can deliver the most value and just get me through the process as fast and as easy as possible is who I’m going to go to.

The rise of robo-advisors in fintech

DI: You mentioned transparency. I think about robo advisors and the sarcasm I would imagine financial advisors probably had when it came out that consumers are going to trust a robot more than they’re going to trust you. And then it turns out “Yeah, actually, I do believe that they’re going to do a better job than than Joe who took however much training.” It’s got to be disruptive for them.

Jason: Well, look at the three biggest custodians, Schwab, TD, and Fidelity, that sit behind the majority of advisors now all offer their own robos. I don’t think the advisor goes away completely, but their role is changed dramatically.

How do you build wealth?

• Well, you need to invest consistently, regardless of market going up and down.
• You need to do with a market exposure that is cost efficient from both the fees and the taxes.

Nothing does that better than a robo does. Except the robo doesn’t have human empathy and understanding the human condition to help you go solve these other things. It’s a joke that John Stein at Betterment and I’ve kind of had as an argument for close to a decade now, which was no one wants to talk to their advisor until they need to talk to their advisor, at which point the machine doesn’t cut it.

DI: When you need to be talked off of a ledge. The robo advisors haven’t dealt with a severe market correction yet. It’ll be interesting to see how that unfolds.

Jason: I think we’re going to see the future is going to belong to the cyborg. It’s to be no longer this environment where it’s either all people or all do-it-yourself. Those two things need to blend.

The ability to charge 1% fees as a financial advisor—those days are quickly waning. They’re going to need to figure out how to do it efficiently. And as compliance and technology costs go up for them. They either need more big clients or to serve a lot more smaller clients. The only way they’re going to be able to do that is through technology.

Leveraging data for financial services innovation

DI: Have you seen examples of organizations that are leveraging data well, not just sending me a more tailored to e-mail based on my interests, but actually improving customer experience. Is anybody doing that really well? 

Jason: There’s a huge advantage to the incumbent because they have the data. They just need to choose to use it differently than they have before, because right now too often they spend all of their time mining for the wrong things.

I like that they do my fraud protection, but they do a horrible job of learning my actual behaviors. They do very little to experiment with what they can do differently.

Imagine Amazon. They have a whole host of data about you and they’re really good at mining it to deliver non-intuitive offerings to you just based on the experiences they’ve seen. I don’t know about you, but Amazon recommends something and it wasn’t even something I was searching for, but it’s something I want.

DI: With financial services, it seems like it would be harder because I’m not trying to change checking accounts or add credit cards on a regular basis, and active investing isn’t necessarily even a good idea.

Jason: Let’s just take “active” and cut “investing” and this is where I think so many banks can exist at the user experience level. They’re used to selling you products. If they were to think about selling you actions or outcomes, then suddenly the host of this data and what they can be able to provide you as a service that you’d be willing to pay for is very different.

Instead of saying, “Hey Sean. Here’s a new credit card for teachers or a checking account that has incremental little value compared to what you had, but I’ll give you 500 dollars to do it.”

What if they were to say, “Hey, Sean, we’re going to help you live a better financial life. We’re going to use this data to help you save more and save in the right places to use credit when you need to, and there’s a monthly fee attached to it. But we’re going to show you based on your history, here’s your trajectory. And here’s how we bend that curve.”

I bet you would pay that, and I bet that you’d be a much stickier customer, and you’d be a much longer term profitable customer, and then they could find products that they could layer in that are better for you.

DI: We’ve talked internally about some of the relaxing around in accredited investors and some of the things that would theoretically open up. It’s been a little surprising to me that you haven’t seen more offerings tailored to those types of investors through some of the traditional financial institutions. Maybe there are good reasons for that.

Jason: I can tell you what some of the good reasons are. I don’t know if you looked at what bank profits are lately, but they’re at record highs when the cost of borrowing for the bank is practically zero. Slightly above that now, but they just got a massive corporate tax cut and the economy is booming, so you’re not seeing default rates go up and the prices that they can charge are up, whether it’s their commercial or their consumer customers.

That’s one of our problems with banking—the profitability is driven by debt. What is the spread on how I can get money in versus what I landed out at? We need to rethink that model.

Blockchain in financial services

DI: Let’s talk about blockchain. In the hype cycle it seems like we’re probably in a trough of disillusionment. There’s been a lot of pilots, but it doesn’t seem like there’s a ton that’s happened in terms of in production applications, either in the private blockchains or otherwise. What are your thoughts on the underlying tech with distributed ledgers? How do you feel about folks’ interest in either providing exposure to tokens or even leveraging their own tokens?

Jason: Great question. Lots of parts there—let’s start with the first one.

Distributed ledgers are not new. They’ve been around for a really long time. And the answer to everything is not always blockchain. Why use a blockchain when you could use the traditional distributed ledger that is cheaper to operate and works just as well. And why you a distributed ledger when it’s actually a database you need?

It’s so easy to play buzzword bingo. When’s the last time you hired a hammer company? You don’t hire a hammer company, you hire a construction company, and the type of the company you hire is different if you’re building a high-rise or you’re building a house. So why do we throw around the idea of “blockchain companies?” If the problem you’re solving is blockchain, that’s a circular reference.

Is there a lot of legitimate use for blockchain? Yes. I had the CEO of Currency Cloud on Breaking Banks a couple months ago and we’re talking about the last mile problem.

The episode is actually titled “The Last Mile” on Provoke.fm. The challenge being if you’re solving the middle part of the problem, it’s really painful to get the information in or the information out. If I’ve accelerated the middle part, have I actually solved any of the problem?

So where I’m seeing the most interesting applications, in particular with blockchain and distributed ledgers, is either being wholly used within a single organization or within closed networks.

A great example is there’s a very large bank that we spend a lot of time with that is putting all of their treasury management on the blockchain. It is a global bank. So every day they have to reconcile what their risk capital is and what they’re holding in reserve by putting it on an internal version of blockchain. They have to share pieces of this with regulators and with outside audits, and blockchain allowed them to share pieces of it versus everything, so they freed up hundreds of millions of dollars that they already had on their books that now they can actually use for lending.

Now another company, they use Bitcoin to move money from one country to another and instantly change it back into fiat currency, but it is faster and cheaper for them to do that internally then it is to do a traditional bank transfer.

DI: Seems like there would potentially be like in our even an arbitrage opportunity for them there too. Just because I know that there’s fluctuations in price across borders—that doesn’t sound like the primary reason why they would be doing that, though.

Jason: Yeah, that’s one of the things we’re trying to get out it. In the three days that it currently takes me to be transferring money around I’m taking currency risk whether I like it or not. This a way to take the currency risk out as well as the time factor, and they control the transaction completely.

AI in financial services

DI: What else are you interested in from an emerging tech perspective, either specifically as it applies to fintech or just sort of in general?

Jason: I’m simultaneously fascinated and scared to death of where we’re going with AI. Not in an Elon Musk sort of way, although I can see his point. But I start to begin to question and worry about the moral implications of when we’ve completely abdicated our own sense of ownership over it.

Yet at the same time, one of the reasons I can’t let it go is I look at the financial situation of the majority of Americans and it scares me and it scares them.

There’s Pew research that shows 84 percent of Americans say their number-one or their number-two stress is running out of money. And that’s something that’s getting worse, as we take away social safety nets and that stress level goes up.

With AI, we could have a person on your shoulder who answers every question for you. Can you afford that vacation, Sean? Should you be getting this mortgage versus that mortgage?

But the question is, who’s providing that? Facebook reminded us that if you’re not paying for it, you are the customer. I would say I accept that they own all of my information and will be monetizing it some way and I’m fine with that because of what I get out of it.

But if I’m looking at the ability to bring great advice that alleviates stress to the masses, what are the implications for who owns that and how are they monetizing it?

What does that mean for us as the individuals—is it being used for good or is it being used for bad? Is it actually taking our own sense of ownership away from us? At the same time, it’s here and it’s not going away.

DI: Are you concerned that we might be widening the income gap? Now the people who are going to have access to the best information or the best machines will have the most data to train on and are able to learn the fastest. We’ll concentrate wealth more than we already have. Is that a legitimate worry or is that less of a concern from your perspective?

Jason: I actually think it might go the opposite.

One of the most exciting things about financial technology broadly, not just AI, is what used to be either unprofitable market segments or unprofitable products I can now actually bring to the masses.

The potential for fintech to be one of the greatest drivers of financial inclusivity and hopefully to shrink those gaps—well, I don’t know that you ever shrink the gap completely. But can we close the disparity between the 99 percent up to the last 2 percent in a much better way than we ever have before.

Basic Income

DI: Do you get involved in conversations around basic income?Are people talking about that with any level of seriousness? Economically, it seems like there’s some problems with that idea.

Jason: Are there?

DI: Logistically, how much money do you have to create to do something like that?

Jason: There have been some economists that have said that it actually could end up saving us money.

DI: Really? Interesting. Where would you recommend reading about that?

Jason: Oh now you ask the challenging question. I’ll have to find it for you.

But this is this is what I would point you to. Do you know that the UN publishes an annual report on happiness—on global happiness. It looks at if you really want to look at the well-being of a country, GDPs are the worst possible measure. They’ve been to several countries with economists who have been pursuing this idea of “gross domestic happiness.”

One of the things they found is that the countries that do the best on gross domestic happiness are in the Danish region. And what do they have in common?

• They all have universal health care.
• They all have universal free education that’s top-notch.
• They put in place a whole bunch of social safety nets including a version of UBI, and happiness goes through the roof.

Now you could also argue that the US is hands-down one of the most productive countries and one of the most innovative countries. Don’t we need that pressure to cause innovation?

Yes, to some degree. I don’t think that necessarily goes away.

Jeffrey Sachs is the economist from Colombia that leads that UN effort, and he talks about looking at the trade-off. Would you rather have universal misery or trade off some of the fast pace and the innovation—which, by the way, some of it might be moving faster than it should—and generally raise the level of life satisfaction. Is that a trade-off we’d be willing to make?

The beauty of fintech and other technology is the ability to do some of this is going up, and the question will become, “So where you we actually derive the purpose that gives life satisfaction,” which is another key ingredient in the happiness index.

I think personally if we were to shift the stress people feel in their financial lives and replace it with another purpose like family and community and creative endeavors, whether that creative endeavor is something like invention or art or gardening, not only would general happiness go up, I actually think our overall productivity as a society would go up.

Let’s think about some of the things that are completely unproductive. Well from a financial point of view, the amount of money spent on treatment of illnesses, many of which are based on bad lifestyle choices like obesity, is a lot.

If people actually had financial incentives to live better lives versus feeling stuck in what they’re doing, or if people had more time to be with family—you know, one of the signs for being at risk is actually the absence of a father figure in the home.

Can we act on what is one of the biggest drains on society? We incarcerate so many of certain parts of a population that if we can address some of the grassroots issues of this—let’s not talk about prison reform, let’s talk about how we need fewer prisons.

The opportunity for fintech to create social good

But there’s the opportunity within financial technology to promote financial inclusion and to bring more people into the system.

With those resources, they can do other things that make the other parts of their life happier and more productive and to me that’s the promise and the number one reason I’m excited about fintech.

We need innovation, to bring it full circle. If we maintain the status quo of how we approach the world and people’s finances and companies’ finances, that isn’t sufficient for me.

DI: So for folks who want to learn more about the stuff you’re working on, how can they find you?

Jason: Listen to Breaking Banks at Provoke.fm or you can check out what we do at Forge.

DI: Awesome. Jason, thanks for being super generous with your time. I really appreciate it and always learn a ton from you. Thank you for taking the time.

Jason: Thank you, sir.

Chris Durbin is a product designer for Amazon. In this discussion we discuss what she learned leading the design for the Apple Watch app, why Amazon’s culture of writing is the key to its success, and why Amazon believes the phrase “Minimum Viable Product” should be outlawed.

DI: You’ve done a bunch of things in your career. When someone comes up to today at a networking event or a party and asks you what you do, what do you say?

Chris: It depends on the audience. I usually keep it simple and say “I like to make stuff at Amazon, and I watch people use the stuff I make, and then I go back and make it better.”

DI: You’re mostly known for the user experience side, which has evolved quite a bit over the years. It used to be websites, and then it was mobile, and now it’s almost device agnostic, especially at Amazon. How do you as a UX professional navigate that, designing and optimizing for every different type of form factor?

Chris: Once nice thing about Amazon is all the internal talks and events and just opportunities to immerse yourself in another team’s world.

But any time you approach design at Amazon, you have to remember it’s worldwide. You always have to be thinking, “I’m building this right now for my audience today, but in one to two years we will have to scale.” And scaling at Amazon includes Alexa devices.

I usually start mobile first. I think about if someone has a phone and an Android phone that’s a couple of years old, how are they going to use it?

But all the while I also have to think about the distilled experience. I have to ask myself if someone is using an Echo device and they want to use my product, what are the prompts and things they will say? So you have to really be crisp in what you want the person to do and how you keep them moving through the flow.

DI: Does that lead you to focus and remove anything extraneous? Or do you still visualize a full featured product, and as you’re working on identify use cases for a voice interface?

Chris: I like to start with the customer and work backwards. I try to design as robust as system as possible. I go through every use case, and then leave it up to either the product manager or developer to say either we can’t do this or I’m glad you thought about this but we’re not going to do this right now. So there’s an organic push and pull.

There’s the concept of the MVP or Minimum Viable Product. It’s a bad word at Amazon. If anyone says “We have an MVP we want to show you,” it’s frowned upon. We can’t ship something that’s just viable anymore because the landscape is so competitive.

We say MRP or “Minimum Remarkable Product.” It has to be memorable. It has to be lovable. Making it easy is good, but those are the table stakes now. Now it’s about making this memorable.

When people think of Amazon they usually think of customer experience. People talk about how they love Amazon. You put the logo in front of them and their mood and emotions change. I have to bottle that up. It’s my job to push the PM and engineer to build the best thing possible.

DI: If you were giving advice to a team that didn’t have the resources of Amazon, but wanted to hold themselves to the same standard remarkable products, are there principles or patterns that would help them figure out what qualifies as remarkable and not just simply viable?

Chris: It starts with your vision statement. Before you think about the tech or devices this is going to be on, you have to boil it down to a very simple straightforward statement.

At Amazon, before you can get any traction behind an idea, you have to be able to codify what you want to do for the customer in a one-page document. And from there. If it’s got legs, that grows into a six-page document which goes more into the implementation how it might work.

You have to be willing to change your vision based on what you learned in that process. If you never change your vision statement, then you’re not really doing any work. You’re just pushing your same vision forward.

When I worked in the concept lab we had five really really basic questions we asked:

• Who is the customer? If you don’t know who you’re building for and who you’re not building for the vision statement is going to be way too broad.
• What is the customer problem? You need to be able to prioritize? This is the person that I want to make a thing for. And this is the first thing that I’m going help them solve in their lives. There might be secondary and tertiary problems that they have. But you just have to focus on one when you start.
• Is the most important customer benefit clear? Again you have to focus and make sure you understand which benefit you’re focusing on.
• Do you know what customers need or want? Have you done your research or have some data have some signals?
• What does the experience look like? When you’re starting out just because people are very very visual, we started with prototyping first. Answering what the experience looks like does not mean it can’t change. It actually should change.

It also doesn’t mean throw some quick sketches together. It means thinking through the entire customer lifecycle. So when I am brand-new and when I’m onboarding, how do you talk to me? What’s the messaging? Now I’ve been a customer for three months. That’s a lot different – what are the things that you’re doing for me now? What about one year? Five years?

DI: Are those prototypes being created at the same time as you’re fleshing out that document? Or did you need to have that six pager down first?

Chris: It depends on the team. What I saw was the tenure of the team matters. Newer teams, if you show them something shiny they love it. They’ll go right for it. But someone tenured at Amazon is going to ask a very Amazonian question. “What are the dogs not barking?” You’ve told me all of the great things about this product, but what are the risks? What is your contingency plan?

DI: The press release and the FAQ document are probably the things that Amazon most famous for in terms of their product development process. What other aspects of the process are unique relative to what you’ve seen in other organizations?

Chris: There’s not a fear. We know this is really hard work, and we know it’s not going to always be right. We just want people to push everything as far as they can, and know that if this whole thing blows up, it’s not going to reflect badly on you.

The other thing is going back to the big bold vision statements. My favorite favorite vision statement is was for the original Kindle. “Every book ever printed in any language, all available in less than 60 seconds.” Most companies wouldn’t be that bold.

DI: One of the things I know you did while you were in the concept lab was the was the Apple Watch app . What that process was like?

Chris: Imagine if you can before the time before everyone had a smart watch. We didn’t even know if people would care that they could use the app on their watch. We didn’t even know if watches were going to last or if it was going to be a fad, or where it was on the hype cycle.

But again, this being Amazon, they said who cares if this device doesn’t go anywhere. If it does, we want to make that bet.

One thing I learned was the importance of function-driven design. Being as concise as possible while trying to make an experience make sense on the wearer. With Amazon, you can theoretically buy millions of products. You could buy an Xbox on the watch. You could buy a bicycle on the watch. What’s the catalog that we shoud put on this device?

I started with research, and I went out to customers homes and showed them storyboards. I tried to figure out how customers would invision using this device, both early adopters and people who didn’t have watches yet.

I did tons of storyboard sketches for different user stories that we imagined might be successful. Let’s say you go into a grocery store and you have a list and you just want to know where in that store your items are. I made a storyboard for that scenario. When you are at home or you’re in the car and you need to do a task what types of tasks would you want to do, both shopping and non-shopping related?

That’s how we decided on our initial set of use cases are initial catalog. And then from the design process from there is actually very simple because Apple does a very good job with all their documentation. The bulk of it is revising copy, because most of that experience is just text.

DI: if you were to go work for a company that doesn’t have the reputation for Innovation that Amazon does and you were tasked with trying to help them make things happen, how would you how would you try to go about doing that?

Chris: Writing more documents. I had never written a document like these before I joined Amazon. Now it’s second nature. The amount of crap you can cut through when you can just write your idea down and pass it around and hone the idea is fantastic.

I’d also focus on the culture of giving candid feedback in a way that’s constructive and direct. Sometimes you’ll get a PDF and you look through it and not really want to engage or give feedback or even look at it. But at Amazon it’s your job to say help identify the holes in this argument, to help fill gaps.

One amazing part of this culture where I don’t have to wait until my idea is refined enough to share. I can share a very very unrefined idea and know that I can share it with my inner circle and they’ll fill the gaps and the more people that I share it with the more quickly I’ll be able to rewrite my document and get it really crisp and ready for the next phase.

DI: Now you’re at now you’re at Twitch. What are you doing for them?

Chris: I’ve mostly been studying what makes Gamers different as an audience. I wanted to understand what makes gamers unique, and how do we design for them. I’m calling it “gameful design”, which is different than gamification. We’re not building for users or customers. We’re building for “players.”

It’s important to make the experience feel more game like. It’s a departure from function over form because with games, they reward free-spirited exploration. So how do we do that within the prime experience?

DI: The whole E-sports world has blown up. For people who are confused about it – watching other people play video games – how would you describe it to them? What is responsible for the rapid growth?

Chris: I think it’s the age of the population. If you were to ask someone if they played football as a kid or played sports, that’s probably part of the appeal of watching sports. Now you have people who have grown up playing more video games than sports. It’s that same sort of nostalgia. So just like you’d go to a Super Bowl party, it’s like that.

I also think that with E-sports, there’s a level of polish. The graphics are amazing and they look beautiful, and there’s an extra layer of imagination. In games you could be on any map in any world anywhere and it’s just beautiful and imaginative and immersive and even if you don’t even understand everything that’s going on, the commentators are there bringing you along, making you feel excited.

DI: Facebook and other sites have gotten a fair amount of bad press in the last year around people becoming addicted to their devices, and how some of these these sites are sort of complicit in that. And they’re leveraging a lot of the same kinds of game mechanics that video games have gotten really good at. When you’re designing an interface that is not a game, and you have the opportunity to leverage similar types of mechanics to build a habit loop or tap into a psychological trigger, how do you think about that? What do you think is responsible use of those mechanics?

Chris: This is a awesome question, because from my point of view as a designer I have this intrinsic motivation to make things to the best of my ability. I want to make things that people want to use. And in order to tap into those emotions, it’s sort of in my duty to use those loops and in some way shape or form at some point.

So how do you use them responsibly? There’s one worldview that says people are gonna be addicted to some product’s feedback loop. It might as well be mine. But that’s not ethical necessarily.
There’s also this issue around privacy and data. Right now the dominant opinion is it should be easy for me to opt into license agreements and make it as easy as possible. But I’m not sure they should be designed that way. It should be a difficult process and the fact that. I should know exactly what I’m optin into, especially as we come upon this age of machine learning and data democratization, where our data is now being used in many more ways than we ever thought possible when we opted in.

DI: Any other technologies you’re particularly excited about?

Chris: Machine learning and AI. I want better design tools. Autodesk has an amazing tool where, if I want to design a motorcycle and I want it to be optimized for speed not durability, and these are the materials I want to use and these are my constraints, it can recommend all the ways I can build this. and now I’m curating. It dramatically speeds up the design process.

DI: Are you in the camp that says that you can’t automate creativity. Or are you in the camp that says this is going to take my job away from me in 10 years?

Chris: You can’t automate creativity. Creativity comes from the juxtaposition of two opposing unrelated ideas. Machine learning works by teaching it all the things that are related to each other. Here’s all the trees. It doesn’t do a great job of creating new or interesting or creative things. I can’t say “make this bike pretty.” and have it know what that means. So I think there’s going to still be the human element of creativity and creating things that are emotional.

Data science has infiltrated most enterprise organizations. But it’s also being used to improve the efficacy of organizations in the social sector.

Andrew Means is the founder of Data Analysts for Social Good and Big Elephant Studios, and the Co-founder of BrightHive. Previously he was the founding executive director of Uptake.org and Associate Director at the University of Chicago’s Center for Data Science & Public Policy.

Andrew is an internationally recognized speaker on issues of data and philanthropy. In this conversation we talk about how smart non-profits are taking advantage of data science to improve transparency and impact, how they’re overcoming issues around lack of data through unique collaborative models, and how organizations can grow a data competency in house.

DI: I know you sort of had an interesting sort of path. How did you get into the world of data science in the first place? How did you end up doing what you do now?

AM: My path is quite unique in the sense that I’m part of the “data for good” community, whatever you want to label that. And I come from the “for good” side. So I was always really interested in nonprofits, really interested in social change, and I came to data because I thought data science is actually a really good way to create change in the world.

DI: Why is that?

AM: Because a lot of organizations in the social sector I think struggled to know if they’re actually achieving the goals that they want. And they struggle to to actually know are they making the impact that they set out to make. And data was a really nice way to validate that.

Part of the way I think about how the social sector even works is that in some sense, nonprofits are selling their ability to create change in the world.

“Nonprofits are selling their ability to create change in the world. Nonprofits exist because there’s something about the world as it is today that we want to see different for tomorrow. And funders, whether it’s me, just an individual donor, or the Gates Foundation, exist to buy that change.

Nonprofits exist because there’s something about the world as it is today that we want to see different for tomorrow. And funders, whether it’s me, just an individual donor, or the Gates Foundation, exist to buy that change.

When you give your money to a non-profit, what you’re doing is saying “I like the change that you’re creating the world and want to see more of it.” I’m buying that.

The issue is far too often that transaction relies on stories alone. It’s an organization saying, “Well I’m doing this thing. I promise.” And they have no proof.

Data is the thing that can actually validate whether the change that we think is occurring is actually occurring or not.

DI: It seems like even on the problem side, asking whether or not this is a problem worth solving, is it able to help on that side as well?

AM: Absolutely. This is the thing with data, and data science in particular. Just just like it can transform the ways that we get around, and that we do commerce, and the ways we do logistics, and the ways we consume and create entertainment, data has the power to actually transform the way the social sector operates.

Whether that is identifying which problems we want to solve, changing how we solve those problems, or evaluating whether we solve those problems or not, data has a role to play across the entire value spectrum.

DI: Is that something you feel like at this stage of the game most change-based organizations have realized, or is this still early days? They don’t totally get it, and you have to kind of convince them of the need for this stuff.

AM: I think it’s early days. It’s not as early as we had five years ago. I mean five years ago, there was very little work being done around the use of data science and the social sector. I think there was a group of us that were trying to demonstrate that it was possible.

Today we have organizations that have Chief Data Officers and data scientists on their staff, and very little of that existed even five or six years ago. But when you think about the breadth and depth of the social sector, it’s still at a pretty nascent stage.

And I think part of that is for many nonprofit organizations and change based organizations, especially established ones and ones that have existed for some period of time, they’re run by people that come from a a certain background. Often its human services or social services kind of background.

They’re not technologists. They they got to their position running large human services organizations because they started as a social worker, working with homeless populations for example, and they just have built their careers and gone up in this direction.

And so convincing some of the these kinds of organizations that data and technology can help you solve the problems that you care about in new ways is a challenging endeavor sometimes.

DI: Is there a fear component to it, where if data gives me visibility into the efficacy of my program – or not – is there a piece of this where maybe they realize what they’re doing isn’t working as well as they would like it to?

AM: I absolutely think so. Data is transforming the way the social sector operates.

If we think about the private sector, what data essentially did was make more efficient businesses. But businesses were still competing on the same thing. They were competing on their ability to create profits. Data just changed the way they created those profits.

In the social sector, the organizations that win are the ones that tell really compelling stories, and have well-connected boards, and convinced donors really well to give them money. They have great marketing machines. They’re not necessarily the most effective.

So data comes in and it begins to actually change the way that donors operate. It begins to shed light onto what’s working, and what’s not.

I think there is a fear among many organizations that they’re going to look less effective. No one ever looks as good as their marketing campaign. And today largely the only data we have about nonprofits is based on the stories of their choosing to tell us.

So as you see this shift to a more data-driven social sector, it’s going to radically change the winners and losers.

DI: Is donor awareness what’s going to ultimately drive that shift? As they become aware, it’s almost like being a more informed consumer. Are they going to force that level of transparency on these organizations, where even if they don’t want to be held accountable that way, they don’t really have a choice?

AM: I think to some extent. There’s a challenge here – when it comes to individual giving the research actually shows we’re driven by stories most of the time. It’s the story that really compels us. That money I think will largely still be determined by reputable brands in the social sector, or I have a friend who started this organization and I care about my friend so I’ll support this organization.

If you step up a level and look at institutional donors. Foundations, whether it be the Kellogg Foundation, or Gates Foundation, etc. They are becoming more and more data driven in their philanthropy and requiring more organizations to share data. But even there you have a disincentive to some extent.

The corollary to this is finance. 20 years ago, you had your investment broker who was supposedly smarter than the market, and was going to make all of these decisions because they knew better than you.

A program officer in a foundation acts in much the same way. They’ll say “I’ve been giving away money to homelessness organizations or human rights organizations or whatever for my entire career. I can do this really well.”

If I come in as a data guy, and say I can actually set up the the technology where we can get all of the workforce development organizations in the state to share data, and I can tell you which ones are most effective, that program officer’s job is fundamentally changed.
So there are some that are starting to see the importance of data. But I think there’s still some disincentives there.

I think we’re actually surprisingly seeing some of the most changes at the federal level. Federal and state government are the largest donors to the nonprofit sector. In many ways, the nonprofit sector exists to provide state-supported services outside of the state. And they are increasingly requiring more transparency from organizations and and requiring that they share data. It’s not always the right data. It’s not always done in the right way. There’s a lot of problems there. But they’re actually seeing the needle move.

DI: You mentioned something I think sounds quite a bit different from how the enterprise environments think about approaches to data, in that maybe because of different philosophy around competition, it sounds like these organizations realize that they have a very small piece of sort of the pie. And their ability to have the kind of visibility that they would need to make effective change is somewhat limited, and by actually coordinating and actually sharing all of their data with each other they can all end up being more effective. Is that accurate?

AM: Yeah. One of the challenges facing the social sector, the nonprofit sector in particular, is that’s very fractured. Since there’s no incentive for a merger or acquisition in the social sector, it’s very rare that you have super large institutions.

In every city, you have dozens of afterschool programs that are all essentially doing the same thing. You have dozens of homeless agencies or food pantries or workforce development organizations – many organizations doing the same thing.

We don’t have a Netflix that has that kind of market penetration. And the reason Netflix and Amazon and Google and all of these services work is because they have massive market penetration, massive amounts of information that they build their systems on.

DI: Because you need a ton of data for an algorithm to be useful.

AM: Exactly. For Netflix to know that I actually really want to watch the Great British Bake Off, it needs lots of people that look like me that watch The Great British Bake Off. In the Social Circle, you don’t have that kind of user base.

So the only way to get there and gain the kinds of insights that you can gain using that kind of data is to share data.

I think there’s a moral imperative to share data. Nonprofits are public institutions. They’re part of a public trust. We need to know if they’re effective or not. And the only way to do that is to require some kind of transparency when it comes to the impact that they’re actually creating.

DI: Who practically does that? Is it is it one of these organizations that says “This is important to us and we have the resources to do this on the on the data side. Will you help we partner with us and help?” Or is this more coming from the federal or state level. Who’s driving the push to get all of these organizations to share their data? And I would imagine there’s a normalization part to get all of it to match up, etc.

AM: Yeah, you have to normalize the data. You have to make sure that when you’re talking about graduation rates, you’re talking about the same thing.

But the there are two main places where I’m seeing this happen. One is some funder is requiring it – whether that be the federal or state government or a large foundation of saying “You all need to play nice. To have access to this kind of funding, this is now going to be required of you.” And that’s one kind of lever that we can pull.

The other is that there are many institutions.The nice thing about the nonprofit sector is that while there are human incentives that’s sometimes hold us back, but these are people who actually care about the work that they’re doing. Some of them will say “Look, if I’m not the most effective maybe I should go out of business and somebody else should get the dollar.”

If you can find enough of those kind of leaders and organizations, I often them see them driving these conversations. This is really a time for great leaders to step up in the social sector and say there is something more important than my institution lasting another year. I look to be successful not just by raising more money next year, but by being more impactful next year. And for me to do that, I need to know how I compared to my peers and know how well I’m doing. And I think there there are leaders that are leading the sector in that direction.

DI: what are some examples of initiatives that have been successful. Where by being able to do this, this is something I can now do that I wasn’t previously able to do.

AM: So I’m the co-founder of this organization called Bright Hive, and we help facilitate some of this large data sharing work. And we do a lot of initiatives around workforce development? One of the challenges of workforce development is that I train you for a job, and then you go out and get a job, and then I lose track of you. I might have helped you get the first job. But did you lose it in two months? Where are you a year from now? How much money are you actually earning?

So we’re doing some work in the state of Colorado, where we’re working with the state and we’re able to get wage and employment data down to the individual level. And we’re able to connect that to training providers, and say who are you serving? Let’s find them and find their wage and employment data, and then give you some of that aggregated data back so you can begin to understand what kinds of jobs are we helping people get, and what kind of wage bump are we seeing.

What’s great is that then we can begin to say who’s serving what kind of populations well? Who’s actually doing a good job increasing family income? And then funders can come in and say we want to support organizations that are effective according to this criteria. Whatever that criteria might be.

DI: You said earlier that a lot of these organizations are fundamentally driven by story-based marketing, and there are obviously holes with that. But I would imagine the reverse of that is true.

I’m reading Steven Pinker’s book right now, and he was saying one of the most surprising things to him was he thought by sharing yet 500 examples of how this is objectively the best time we’ve ever lived – by far – he thought that that the data would be so compelling that it would sort of do its own job, and he was very surprised to see that wasn’t the case.

Do you run into that? Is there still a need for for story? And if so, how do you connect the the rational part of the brain with the emotional part of the brain and do it in a way where I’m able to turn the data into a compelling story that is more effective?

AM: Absolutely. There’s constantly this kind of question “data vs. story”. And they aren’t necessarily on opposing sides the field. There are ways for them to work together.
Data is a raw resource. It’s like a block of stone. And there’s this great Michelangelo quote that says “inside every block of stone is a statue. And it’s the job of the sculptor to uncover it.”

For those of us like working with data, our job is to take this raw resource and use our insight and experience and tools and methods to uncover something really valuable, and compelling, and true.

I think there are ways where this kind of scientific methodology can meet with our ability to tell stories. Stories are how we make sense of the world. Data is actually not how we make sense of the world. Data plays a role in the stories we oftentimes to tell. But we don’t make pure analytical decisions most of the time.

I think there are ways where we can use data to validate whether our stories are true or not. There are things that are true and there are things that are untrue. And I think the role of data in helping us identify what’s true and what’s not is important. And then I think we can find ways to tell stories that resonate with that truth.

I also think sometimes we think of data is just giving us something. Like the data will tell us the answer. And most the time that’s not true. Data should inform the answer but data doesn’t oftentimes tell you the answer.

There’s certainly places where data is automating decisions, and I think those are really exciting opportunities. But I think where data often adds the most value is in assisting our decisions.

It’s the doctor standing in front of a patient, looking at the results of a bunch of different algorithms, and then using their own experience and intuition to interpret and make a call for a patient. Or it’s the organization that’s trying to help kids graduate from high school getting a list of the 20 kids that we think are most likely to drop out and then intervening with them and then using their own mind and creativity to intervene.

I don’t think it’s about computers just automating all of our decisions. It’s about augmenting our ability to make really smart decisions with with data.

DI: For organizations, seeing a lack of bodies to do the work, or at least the perception there’s a lack of bodies, but for organizations that are wanting to to start making better decisions informed by data that don’t necessarily know where to start, what kind of advice do you give those types of organizations?

AM: So I think there’s oftentimes this idea that the only way to do something with data is to hire some really expensive nerd. But any organization that does data science has a team. It’s almost always a team. Because there’s such a breadth of skills that are necessary to turn raw data into valuable product. So one “data person” for your organization is rarely the answer.

I also think that there’s this kind of reaction that we need to hire somebody and invest in technology, when I think often the best place to begin is culture. If you don’t have a culture that is driven by evidence, you’re gonna hire a data person or invest in technology and then never use it. If you’re not a culture that’s concerned about your performance, or is rigorous about how you understand whether you’re making progress or not, you’re never going to utilize the data that you have.

DI: If you’re not that way now, but you want to be that – if you’re a leader who wants to create that culture – how do you change the culture to get it to be more evidence-based?

AM: There’s a couple things I’ve seen that I think work. One is it’s important to have some champion of this kind of way of thinking who’s at the table, who’s senior enough to sit at the executive table, but junior enough to still get along with “the people”, and who can really kind of be the the nagging voice in the corner. Why are you making this decision? What’s the evidence?

And I also think thinking in terms of “evidence” rather than just data is really valuable. Because evidence is a term we use that encompasses a much more broad array of things.

When we talk about data, we tend to simplify it into numbers and ones and zeros on the computer. And there’s a lot of organizations that don’t have a lot of data, that don’t have a lot of ones and zeros on servers somewhere. But every organization can make an evidence-based decision. The evidence might look different if you’re a one or two-person small nonprofit vs. a five billion dollar company. But everyone can make decisions based on evidence. And so I think having a an evidence champion is really helpful.

Another thing that I sometimes do with with my nonprofit partners is – because everyone in the nonprofit sector is really nice and we just want to get along and don’t want conflict – I’ll sometimes assign somebody in meanings to be the “skeptic”.

The tendency is somebody throws out an idea. We all agree with it and think it’s great and think of all the reasons why the work. But I’ll tell them I want you to be the person to poke holes in all of our ideas. I think that frees everyone up to think a little bit more critically.

And then the third thing is setting up rhythms. In my first role as director of research analytics at the YMCA Chicago, we would sit down every quarter with our different business leaders, whether they are running programs, or running facilities for us. And we would do these “planning with data” sessions.

Literally all I would do is sit down and have a dashboard of some important metrics that we collected. And I’d say, “Why did this spike happen here? And why did this happen here? Which of these numbers do we want to see changed and improved in three months? And what are three actions you can take to change this number?”

And then three months later we come back and see did that number go up or go down like we thought it would. And it gets people in a rhythm of realizing that the data is not some ethereal, out of body thing that you have no control over. It’s actually just a mirror. It’s a representation of your work. And the decisions that you make can actually change the metrics. So getting people in that rhythm of looking at some information, interpreting that information, making some actionable decisions about it, then evaluating whether they worked or not, that rhythm can be really helpful.

DI: So let’s say that I’ve done that. I do have a culture that is more execution oriented, and that’s not my hurdle. What what do I do next? If I want to get into legit data science stuff, and I don’t know where to start. Where do I start?

AM: So I think the first level is what I call a “data analysis”. And that’s about how do we get an insight? How do I learn something that I didn’t know before that I can make decisions on? And if your culture humming around all of those things, the next kind of thing is “how do I build products?”

How do I move from from a mental thought to a technological product? Where data is actually not just giving me more information, but where we you move into that decision supports or automated decision space.

DI: And by “products”, you don’t necessarily mean something that I then turn around and sell to external people. It’s an internal tool that does something on a repetitive basis.

AM: Yeah, exactly. So for example, a lot of nonprofit programs are oversubscribed. More people want to get into them than you’re able to fit. And for a lot of organizations the way that they make that decision is “who signed up first?” We’re just gonna let who signed up first into the program.

If you’re thinking about it from an impact perspective that’s a really dumb way to decide who gets into your program or not. You could make the decision based upon “who do I think I’m going to have the most impact on, or who needs my services the most?”

So by “product”, I mean you could actually build an application engine that has people who applied, and you rank order them. And you then decide based on that rank ordering who to let in your program. That’s a product. It’s something that plugs into your operational process and helps you do the work that you do in a slightly different way.

So there’s a there’s a few ways of beginning to move in that direction. There’s a guy named Jake Garcia at the Foundations Center in New York who I think runs the best kind of data science team in a non-profit I’ve seen. And what he talks about is a lot around “skilling up.”

There’s some people in the social sector who think “If I could just get somebody who used to work at Google, all my problems will be solved.” And I don’t think that’s actually true.

We have tremendously skilled workforce that we can begin to scale up, and give them new abilities and new opportunities. As you’re having natural turnover and your organization can up the technical skills that are required to fill that role. Don’t just fill it at the level it was at. Say “We want you to be able to do this, but we also want you to know a little bit of coding.” Scale up those people those people.

And with the people you already have, give them opportunities to learn new technologies and test those out. Give them projects and time to stretch themselves. This can actually be really challenging at nonprofits because you’re under-resourced, everyone is strapped and wearing 13 different hats. But as much as possible carve out a little bit of time for them to go and learn.
If you go to any enterprise company using data science, they they carve out time for the people just to learn, because these technologies are changing so quickly. We also need to do that in the social sector.

And then the other kind of aspect to this is really starting to to ensure that the the tech folks or the data folks much more attached to the mission of the organization. They’re not in a service of IT, but may be more connected to the program staff or the executive office where they have a broader mandate where they’re seen not just as technology folks, but as folks that are there to help us achieve our mission.

I’ve seen many organizations who have actually moved a data science team out of IT and made them their own standalone team, reporting to the CTO or a program person. And I think that’s also a good next step.

When it comes to like the the challenge of staffing, I actually think to some extent social sector organizations and have the ability to give people more meaningful work. I know a lot of people in the data science field that are really driven by the problems that they’re solving. And there is the opportunity to say “You could go in and improve click-through rates at a big tech company and make $250,000 a year. Or you could come and help us eradicate Malaria in Eastern Africa. And we’re going to pay you less, but you’ll save millions of kids lives. Up to you.”

I do think there’s a portion of the workforce that’s highly motivated by those kind of causes. Now if you’ve been in graduate school for 12 years, you’re not going to give up two hundred thousand dollars of income. But you might give up a portion of that to do something that’s more meaningful.

DI: You’ve been in very different worlds, on the consulting side, and pure nonprofit. You’ve been sort of corporate social responsibility. Have you seen differences in those worlds in terms of how they’re approaching the problems they’re trying to solve?

AM: Absolutely. They’re definitely different approaches. To some extent though I think we over-focus on organizational form. The only difference between a non-profit and a for-profit is that a non-profit doesn’t have owners. It’s a public trust. So any any revenues that it generates beyond its expenses go back into the organization. There are sometimes nonprofits and for-profits that look almost identical in their their work, they just have this different legal structure.

But there are some some differences. Sometimes given the management structure of many nonprofits, they sometimes moved a different pace. There’s sometimes a lot of board ownership, and so things move at the pace of board meetings. There’s a lot of effort coming up to a board meeting and then it gets maybe a little bit quiet.

But I’ve also been I know nonprofits that are actually very agile and very fast moving, just like any tech startup. There are some fascinating nonprofit organizations where if you walked into their office, you wouldn’t know if they were a venture backed startup or nonprofit. There are places that are doing really interesting work. And I think it comes much more down to the leadership and mission of the organization.

DI: What are you most excited about like over the next five years? What do you think the world of data science is going to look like?

AM: I’m jazzed for the hype bubble to burst, and to actually get to the real value. I’m excited for like data literacy to increase, where a broader range of people know what data science can and can’t do, and leverage it really well for what it can do.

From the social sector perspective, I’m very excited at the kinds and scale of some of these data sharing initiatives. For the first time we’re beginning to see some pretty large scale work being done there. And what that will unlock and enable is very very exciting.

On the technology said, as we start to see a more instrumented world, I’m very interested to see how that affects the social sector – the ways that that we take advantage of the internet of things and connected technologies, all the emerging tech, I’m interested how that begins to trickle down social sector.

There are some great organizations doing cool stuff with drones on anti-poaching or disaster relief efforts. There’s really exciting stuff in the medical field.

I was talking with a very large foundation that funds tens of millions of dollars or hundreds of millions dollars worth of studies every year. And we’re in talks about how data science can predict the results of a clinical trial based on other existing trials you’ve already done. And this can radically speed up the way in which we discover what works. That’s an area that I’m very excited about.

In general I think we’re going to get much more efficient. We’re gonna be able to say “You don’t need all of these different medicines. You’ve eradicated these two diseases, but you can actually eradicate these five others because those branch off of the other two.” Or “If we train the future of our workforce in this kind of way, we’ll actually have better employment outcomes or wage outcomes.”

So I’m excited for this kind of like optimization of the social sector to occur.