App Marketing AI in an era of Data Scarcity

Launched in 2018, App Marketers Unplugged is an event series for advertisers to connect and discuss challenges and trends with their peers. This year, we’re introducing curated video podcasts with a new selection of industry experts.

In this episode, Diego Meller interviewed Lomit Patel, VP of Growth at IMVU and best-selling author of Lean AI.

Watch the video to hear Lomit Patel’s expert take on:

• Leveraging the right partners to scale with Lean AI
• How to gain user insights more efficiently using machine learning
• Key success metrics for effective customer acquisition
• Moving from IDFA to probabilistic attribution
• What app marketers need to sail through the automation journey

Or read the abridged transcript below.

Diego: Welcome Lomit, we're very happy to have you. Lomit wrote a great book, called Lean AI, which is part of the Eric Ries Lean Startup Series. We actually bought a bunch of copies for the Jampp team, because it summarizes everything that we're trying to do, which is basically to provide tools that automate things that marketers need to do. So, why don’t you start by telling us what compelled you to write this book?

Lomit: I started speaking a lot about this at conferences in the last couple of years, and what I came to realize is that a lot of companies aren't' really fully leveraging AI and automation to power their growth strategies. As you and I know, your biggest superpower now is data and how you can leverage data to surface insights, and how quickly you can act on those insights.
The inspiration for the book is to outline the digital transformation that a lot of growth teams are eventually going towards in the next 3-5 years, but can start taking advantage of today.

Diego: In your book, you talk about this sort of a Lean AI autonomy scale. Where do you think the average performance marketing team is now?

Lomit: The Lean AI scale point zero is the rudiments where everything is being done manually. The other extreme of that is where you fully automate everything and have AI and machines looking at data and making decisions in real-time, and acting on those decisions.

Right now, most of us spend our money, whether it's Google or Facebook, on different partners that have already integrated some form of AI and automation. I would say, most companies are at stage 2 or 3, where they are really dependent on the AI of their partners. In order to go all the way to stage 5, where everything is fully automated, it requires either building a system like this, like we did at IMVU, or working with a partner that can get you to that.

Diego: As you were going through your journey of implementing a lean AI strategy, what was sort of the first "oh wow" moment? Can you share an insight that you discovered that you wouldn't have discovered otherwise?

Lomit: One of those was user onboarding, so I'll use that as an example. What we used to do was, for the most part, treat all users pretty similar. We’d take them through an onboarding experience between day 1 and day 7, to try to get them engaged, but once we started having our AI looking at the data, we were able to get more granular about personalizing based on what the user was either doing or not doing in the first 24hs.
So every user that came into IMVU would get a certain number of free IMVU credits, which is a virtual currency that they would redeem to customize the look and feel of their avatar and, for the most part, it's enough to give you a basic look, but not to give you a great looking avatar. So we looked at the people that were redeeming those credits and if they weren't spending any money in the first 24 to 48hs, we knew they were not going to be monetized through in-app purchases. So, we started showing these users how to earn those credits with managed-offers or offer walls. Without AI, we weren't able to predict that so quickly, and in mobile, most people don't stick around for 5 days if they don't know what your product's about. So that helped us to get better at retaining customers, but it also got us better at personalizing the right experience.

Diego: Performance marketers are quite obsessed with acquiring new users, but as you mention in the book, what's important is the whole lifecycle of the user. Very often the downloads and installs are kind of vanity metrics, on their own, they don't tell you much. So what are the 2-3 KPIs that a sophisticated lean AI marketer should be looking at?

Lomit: The premise of Lean AI is to try and get more done with less. Part of that is really understanding the entire user journey and what is the right metric that's going to enable you to measure success. We got pretty disciplined to optimize towards ROAS and cost to acquire a customer.
And then, the third thing that we look at is the payback period: how long does it take for us to redeem that money. When I started, it took an average of 6-12 months. Today, whatever we're spending, we recoup that back within 30 days, which is amazing. Once you're able to do that, most CFOs in the world are going to say: “fine, you can get an open budget.”

Diego: And I think making the payback period shorter requires a real sort of marriage between marketing and product.

Lomit: Yeah. you're right. Growth teams are kind of the intersection between product and marketing and customer support, and we're responsible for all of those key metrics. With our AI machine, we're constantly testing different audiences, creatives, bids, budgets, and moving all of those different dials. On average, we're generally running about ten thousand experiments at scale. A majority of those are based on creatives, it’s become a much bigger lever for us.

Diego: It's the last line of defense. It's the thing that is going to impact your user and is going to decide if they click or not.

Lomit: Right, so you need to get that piece of the story right. We do a lot of video ads, but we're always experimenting with different parts of the video. For static creatives, we try to create shells of creatives and we do RSS feeds so we're always optimizing images, taglines, calls to actions, and all of that is happening seamlessly in the background.

Diego: So I'm an AI optimist, I believe that the role of AI is not to replace jobs or people, but to replace tasks that people do and let people focus on the things that people are good at. What are those tasks that marketers should be looking to automate when it comes to performance marketing in particular?

Lomit: I would say machines are way better than humans when it comes to processing data and surfacing insights. You want to find the right audience to target, and then automate the process to push those insights out to your different partners, in real-time. You also want to automate, based on lifetime value, how much you want to bid in the different exchanges for a certain user.
Whether you're buying on Google or Facebook or on DSPs, all of these are like exchanges where it's all based on supply and demand. Ultimately, you want to get in at the right time and get out at the right time. We keep our budgets very fluid, so it's based on performance at any given time. So for example, if Facebook is experiencing more competition in the exchange, then our machine will start looking at Snapchat, TikTok, or whatever and it will pivot the budgets. With Lean AI, the machine does a lot of the heavy lifting, and with more data, the accuracy rate continues to go up.

Diego: There's obviously been a shift in the tools that are available, how is that changing now?

Lomit: Right, a lot of processes are being moved into the cloud, especially with data storage. Now, there's a lot more uniformity thanks to SASS tools, and a lot of them have API integrations in place. So all of this data now is able to flow in real-time, which is important, and people have a lot more confidence to work with cloud storage solutions than before. This is kind of the start of a golden era.
The other thing that I would add is that the computing power continues to go up too and the bandwidth around wifi, with 5G coming in as well. Now what is possible was probably not possible like 5-10 years ago.

Diego: Of course, data is the main ingredient in machine learning and AI. Now, a significant chunk of the data that mobile marketers are using today is actually under threat with the IDFA debacle and the potential consequence of that in our context. So what's the impact on this whole way of thinking around Lean AI when you lose that, or when you have a lot less of that?

Lomit: IDFA is a really important way to attribute user data and build really high predictable algorithms. Fortunately, we have over three years of good historical context on a lot of data, but what's going to happen moving forward is that instead of being really focused on deterministic attribution (which is what the IDFA was giving us), we're going to have to build a model that's more probabilistic.
What we can control is to try and get as many people to opt in and share their IDFA. Some people are saying it could be as low as 10% and some people say it could be as high as 30-50%, who knows? What we've been trying to do is to get our AI model to start training on targeting people that already have limited ad tracking (LAT) on, and to try to figure out how that compares to the model we have. This is something we've started doing in the last 2-3 weeks. And what we've found is: yes, there's a little bit of inefficiency, but it isn't drastically worse.
We are also trying to identify what's the right way to incentivize users to increase our opt in rate for the IDFA, and we want to try to put in some A/B tests in place.

Diego: I feel the media and consumers have a very unfair perception of what the advertising industry does with their data, right? At the end of the day, what we do is no different from what Spotify does when they recommend a song, or what Netflix does when they recommend a show to you. They're using machine learning based on your behavior. So what do you do about that?

Lomit: Yes, you bring up a really good point. Part of it is just educating users on what they get when they share their data. As an industry, we have not done a good job of that. For example, everyone loves recommendations and relevant offers, and all of that is at risk without data. What I think we should be doing as an industry right now is educating users.
The other thing is everybody loves things for free, but it's never free, right? There are so many apps that are free, the only reason that they're free is because they're being supported by advertising. If users start opting out of advertising and these apps aren't able to support a lot of their free services, one of two things is going to happen: either they're going to cut back on all of the features and functionalities that people love and make it very basic, or they're going to start turning into subscription, where you pay to get access to it.

Diego: Right, but it's not about finding a shortcut, it's about finding a way to be fair to the user and to be fair to the people building technology. If done properly, it is a win-win-win situation, let's hope we get a fair shot at explaining why it's a good idea to be tracked.
To finalize, a lot of the stuff we talked about is quite theoretical. What is a piece of practical advice that you can give to performance marketing leaders to start their automation journey? What would be the first thing any sophisticated app marketer should do to go in the right direction when it comes to this?

Lomit: The first thing is just to make sure that you have all your data aggregated in one place and that you've got good data coming in. Even if you don't really build this personalized AI, you can still leverage a lot of AI with different partners that you work with, as well as Jampp. The idea is you set your partners up for success. You've got to make sure that you're giving them the right data so that their algorithms can optimize towards your outcomes and clearly define what success is. A lot of people don't want to share data with different partners.

Diego: That's a massive challenge. I get it, especially for example in gaming. When you have a user dynamic where a very small percentage of your users are spending money, I understand how that is very coveted data. But if you are worried about giving away this data, there's a bunch of things you can do: you don't even have to give us the IDFAs, you just flip a switch and the MMPs do it anonymously. And it's a very big responsibility, I cannot speak for every company, but most of us are very careful with that.

Lomit: Exactly. The other recommendation I have is don't try to build a huge data science team in-house and try to build this technology, because this technology is being updated so quickly, it will also get dated very quickly. There's a reason why partners have been around for a long time. Whether it's Google, Facebook, or Jampp, you guys have the engineers that are focused on this, right? So it doesn't make sense to compete with these companies.

Diego: But also, we're solving this problem for you and fifty more companies in your vertical. You cannot get the learnings from your competitors and apply that to your platform. We can because for each vertical we have a number of advertisers, and that is all filling in how we do things.

Lomit: And that's the other thing I was going to say, if you try to build this, you cannot just look into your own ecosystem. If you work with a partner that's looking and working with so many other different partners and verticals, the benefit there is learning from all of these other partners. It's going to be able to outlearn you and the algorithm is going to be able to outsmart you anyway. The second part of that is it's going to be able to provide really good benchmarks, it'll be able to tell you if you are on par, above, or below, right?

Diego: Exactly. Lomit, is there anything else you want to say or you want to share?

Lomit: First of all, thanks for having me on the show. I feel we have the same philosophy around the benefits of AI and automation and the role it's going to play in performance marketing. At the end of the day, the key is data. For the most part, I wrote the book to help other growth marketers and it all starts with getting the executive team excited. So, learn from my experience on how to get the people at the top excited, how to mobilize cross-functionally with your team, and most importantly, how to work well with all the different partners—and obviously Jampp is one of those partners that you want to work with. You'll be able to move much faster and further collaboratively with the right partners internally and externally.

Diego: I strongly recommend that everybody gets Lomit's book: "How Innovative Startups Use Artificial Intelligence to Grow". As a manager, I learned a few concepts and a few things about the technology and its limitations that I wish I had known before. Lomit, I want to thank you a lot for participating in App Marketers Unplugged, thank you everybody for listening and we'll see you very soon at the next App Marketers Unplugged.

Wrapping up

Thank you for joining us on this session of App Marketers Unplugged 2020. If you enjoyed this episode, check our event series agenda for more information on upcoming sessions:

• Founder Stories
• Programmatic Media Buying and iOS 14
• The State of Supply in 2020
• Creative Concept Testing in Games