Creative concept testing in mobile games

App Marketers Unplugged is an event series dedicated to discussing the latest trends and challenges in our industry. This year, we’re introducing curated video podcasts with a new selection of industry experts.

In this episode, Philip Weiskirchen, Head of Sales, APAC, at Jampp interviewed gaming experts from GoodGame Studios, Twitter, and Luna Labs to hear their takes on the latest trends and changes in mobile games.

Watch the video to learn...

• How COVID-19 impacted creative strategy
• The benefits and challenges for different ad formats and upcoming trends in creative development
• How to leverage technology and automation for better creative performance
• What practical advice and lessons learned our panelists can offer to advertisers

Or read the abridged transcript below.

Philip: Welcome to App Marketers Unplugged. In this session we'll be talking to three industry experts focusing on the mobile games industry:

• Dawn Hosie works as a Content Producer at Twitter, where she's heading up the production of playable ads.
• Pau Quevedo leads Programmatic Trading at GoodGame Studios. Prior to that, he was in charge of programmatic advertising at InnoGames.
• Steven Chard co-founded Luna Labs, a platform that helps game developers to create playable ads directly from the Unity engine.
  

COVID-19

Philip: The quarantine has affected how a lot of us structure our days and, as a result, playtime has changed. Pau, have you adjusted your creative strategy since the beginning of the pandemic? Have you seen big differences in terms of ad performance since then?

Pau: We haven't been using any variation specific to COVID. What has happened is that we were able to scale and test new things. It was more the fact that it allowed us to push the spend. We haven't seen a difference before and post COVID, but we were able to test more different themes.

Dawn: We have had clients request we use COVID-specific language on a banner. However, we ethically decided that that was not really the best road to take. So, the same answer for us, it's all about the scale: more testing, but nothing specific to the pandemic.

Ad Formats

Philip: Do you have a must-have creative format? Do you think that there is one format that is more engaging than others?

Pau: At GoodGame Studios, we have games that are based on farming games, which are more casual, and also simulation and strategy games. I'm saying this first because you’ll probably use different types of creatives for different games.
We focus on banners and videos. If you're running UA you'll most likely be using rewarded videos of 15 or 30 seconds on networks or Facebook; if you're running banners, you'll be running native or retargeting.
Video especially is dominating the industry right now, at least our section. What we are seeing is that bringing human characters into it works really well. Even if your game doesn't have them, we've seen good results by humanizing the game and generating emotional attachments.

Philip: In the case of rewarded video ads, do you see them work for certain types of users, or do you use them across the board?

Pau: For retargeting, we normally use banners because they are cheaper, unless we’re targeting a very valuable user or a user who stopped playing some time ago and we want to show them some new gameplays or new levels.
For UA, we use rewarded video for payers and non-payers, they both want to get rewards. If you're showing a banner, most likely it's gonna be an interstitial for the non-payers. Payers don't want to get those, you have to take that into account.
And then we'll also have differences in the type of videos, for instance, if they are more focused on the gameplay or cinematics. There'll be differences if we believe they are hyper users or not, and depending on the CPMs that we see.

Playable Ads

Steven: When it comes to playables, I think there's a question mark about how they are constructed, how they are used, how they're tested, how much can be tested, and how you’re going to tap into the kind of metric. Also, how to tap into emotions to make it fun and bring users in. Once you get the right formula, you can be very successful. But in the past, it's been quite tricky to get access to good playables, and it does take quite a few playables to get where you need to be.

Dawn: It's not necessarily the genre of the game that’s the main blocker for the success of the creative. I feel like the reason why playables in general fail is just because they are a lot more expensive to build than video and banner, and that makes it harder to iterate. If you build one playable and it fails, that’s the cost of 3 or 4 videos. There's a lot of opportunity for playables to succeed for any genre, it's just a matter of finding the accessibility to build them.

Steve: When it comes down to playable production, you need a bunch of different concepts to start up, test, and learn from. So you’ll be switching things like characters, sensitivity, difficulty, palettes of colors, different end cards, optimizing the exit points, optimizing the entry points. But once you get there, it can have strong longevity and outperform a video for a long time. It’s also very dependent on the traffic you’re running. Every channel is different, so that needs to be taken into consideration.

Philip: The conception of playable ads sounds quite complicated. Do advertisers have a concept in their minds or do you need to guide them through it?

Steve: It's slightly different at Luna because we have software through licensed studios for them to be able to go ahead and build out playables or create more video. We help them optimize that kind of workflow, the CTA, the engagement. Getting the first 5 seconds right in a playable is absolutely mission-critical, and then it’s also key to get the end card right. You need the right incentive for people to actually go on to the store. We definitely advise and do what we can to make sure that our clients are successful.

Dawn: A lot of clients think of playables as if they were a video, but they are a very different medium. You don't have to tell the user what's in the game. You need to show them, and you show them by driving a really fun experience.
I'll create a really quick fictional game: we want a character to beat this enemy; after you beat the enemy, you open the chest; after you open the chest, you get three cards; after that, you pick the one that you want; and then it does a cool effect. On paper that sounds great, but in a playable (1) that is like a 50-second experience. So, it's not gonna perform; but (2) it's focusing on so many different things, and this is what we need to educate clients on: it's about short, sweet, simple experiences. If you don't get someone into the action very quickly, and that action is not clearly communicated, then your playable is going to fail.

Philip: Interesting. So it sounds like there needs to be lots of help and advice.

Steve: It doesn't need to be too scary. It's something you can be successful at across all genres, it just takes time to understand. You will get a lot of stuff wrong at the beginning, but once you do get it right, it can be very successful.

Incrementality and A/B Testing

Philip: Steven, Dawn, is incrementality a topic for your clients? Is this something they care about? I saw, Steve, that Luna Labs also provides analytics for creatives. Is incrementality something you provide feedback on as well?

Steve: Everyone wants to have incrementality. In playables, you can set up all these different trigger points in the creative. You've got every possible variation you could make in that ad because of the framework in which it's built. And then we really look at engagement, where people are dropping off, how they are dropping off, and how many go through to install. We're going into a world of a bit of uncertainty with iOS 14 and I think it becomes even more important to understand how people are interacting prior to the install. All these points become extremely valuable.

Philip: Let's talk a bit about the importance of A/B testing. Pau, do you feel like at the moment you've got A/B testing sorted out for you?

Pau: For us, the challenge is the following: we allocate most of the spend on Facebook and Google, and unfortunately in Google you cannot A/B test creatives because of how the algorithm is set up. It's similar to Facebook. It's a completely biased system where the algorithm is eventually gonna decide after a couple of first impressions which creative performs best.
In terms of A/B testing, we are trying to automate it, like everyone else. I think automation actually starts with creative production. But in the end, it's always the same: when we test, it has to be tested on Facebook, and Facebook is the biggest challenge for the gaming companies who are looking at A/B testing.

Philip: When A/B testing creatives do you focus on banner A vs banner B, or do you look at certain ad components, like this picture vs that picture, or this CTA vs that CTA? How granular are you?

Pau: We divide the creatives we are testing and then we iterate those, so we're trying to test within the different streams; and yes, that's constant. Normally we do A/B tests. I think that more and more, Product is realizing how important the role of Marketing actually is.

Steven: It's really hard to run a proper A/B test. The way that we approach it is we influence the elements within the playable. So you upload maybe 3-4 different playable versions and run those for a couple of weeks, maybe a month, to see what the data looks like, and then change the elements and repeat.
Let's say there are 50 different elements which can be changed within a playable concept, you can let the machine run and see what is driving the best CTR, lock that into 50% of the traffic and then continue to change the other elements and see if anything can challenge that.
There's a lot of room for that to grow and let the machine really test a lot of elements across the different platforms. Since they all perform differently, you can't just look at them as a whole, you have to be very granular on each channel.

Dawn: On our side, we have A/B testing pretty well figured out, we build parameters that can be changed on each playable, and we actually use machine learning to determine the best value for each, just from iterative testing. So we just skip the manual A/B testing process and as long as we design the parameters with intent, we just plug them in, watch it go, and see which one wins. We then remove the parameters that we know are of certain value and add new ones. And we test videos and banners the same as everyone else: A to B, over and over.

Philip: When you do creative A/B testing, are you focusing on the actual performance of the creative, or are you most looking at post-install performance?

Dawn: It's a little bit of both, and it also depends on the clients' KPIs. So the client tells us what is most important for them. We do have some clients that are purely focused on the click-through rate, the hyper-casuals of the world. However, most are focused on the post-install. So, our algorithm is not just optimizing for the creative itself but also what happens down the line.

The future of creatives

Philip: I think it's fair to say that Gaming is driving innovation for creative development for the whole industry. What do you think will be the next big thing in mobile games as regards creatives?

Pau: So Dawn just mentioned the different elements within the creative that we can optimize through machine learning, I think that will be coming to jpgs at some point. Like with different psd elements you can add in. I know a few DSPs that are already working on this. In fact, we're building and automating creative laboratories with those tags within the creatives to start optimizing more and more from that meta information. We'd love to have it for video too.

Steve: We're going to move to things with more authenticity, better quality, and narrative. An understanding of the creative and tracking to enable the machine to test at a much higher volume is fundamental, especially on the video side it's very hard to do that now.
We definitely see playables at the end of videos as end cards—it's massive. Right now it's just everywhere. All of our clients are creating nice playables and a lot of video assets, but just having those playable end cards is a real thing.

Dawn: When people think innovation nowadays, they think augmented reality and VR, but the way that we currently serve ads, that's never going to happen because we’ll have to request permission from their camera to do anything cool. If we can't evolve by using the camera, then how?
As Pau said, we’ll be adding parameters to video and banners. How would we add parameters to video? The format as far as I know does not allow for it. Because it's a one-file export. So, is there a new format to invent? With videos performing so successfully across all mobile games and genres, it would revolutionize the industry.
Regarding banners, we actually have to catch up with the branding world. Normally branding follows gaming, this time gaming is following branding. I am a big believer in HTML5 banners because you can do that sort of testing.

Practical Advice and Lessons Learned

Philip: I have two open questions. The first one is what's one piece of practical advice you can offer and the second is what's one mistake you've learned from that you wish you’d known about before.

Pau: I started doing browser advertising on desktop some years ago and moved into mobile about 5-6 years ago. I used to think: "oh, mobile is the new thing, don’t listen to these desktop dinosaurs", and I realized that it was the opposite. Desktop was ahead of mobile, and now I would say: go back to basics, go back to desktop and you’ll learn more for mobile than you thought.

Dawn: One piece of practical advice I would offer is make sure that data and creatives have an equal seat at the table. I think a lot of people tend to favor one over the other, but data doesn't necessarily breed innovation, and creative intuition doesn't necessarily create optimal performance. You need both.
As for a mistake I've learned from, I'll just go personal here. I wish I had stopped building logos and brochures for chiropractors and banks and just got into gaming sooner. Chase what you enjoy in life, don't be afraid. Gaming was a risk for me, and it paid off.

Steven: In terms of mistakes we’ve learned from, I first embarked on this kind of grandiose idea that if you can basically take an exact replica of a level of a game, that was gonna work really well as a playable experience. We got proven to be pretty wrong. You need to really think about these experiences as playables. And we could have learned from that mistake quicker, but mistakes are about learning... and that's the beauty of this, you get to then go onto the next mistake, and the next one.

Philip: We’re always learning, especially in such a fast-changing industry like app marketing. I’d like to thank all of you for being on this podcast and sharing all these insights. This was App Marketers Unplugged, thank you.

What’s next?

If you enjoyed this episode, check our event series agenda to learn more about the upcoming sessions.