Having a mobile app means constant experimentation. Experimenting with targeting, with personalization, with A/B testing, with everything! This article by Apptimize features in-depth case studies about experiments (and results!) that were done by Runtastic, busuu, and HotelTonight.

Check out their tips below.

1. Use advanced targeting and allocation across digital platforms to offer personalization and to stay relevant to your customers.

To ensure healthy growth and continuous learning, your organization should have a clear understanding of user preferences to be able to serve them individualized content and keep them coming back to your mobile app for more content.

Below is a case study from Apptimize Customer Runtastic on how they use advanced targeting and segmentation to deliver a consistent personalized experience to their different users.

Runtastic Case Study

About Runtastic

Runtastic has a portfolio of more than 15 health and fitness apps. which combine traditional fitness models with mobile technology for over 95 million registered users.Their messaging is based on several factors, including the app that the user was on, the user status, the frequency at which the user uses the app, and their location. By understanding these user segments, they are able to delight their users with extremely customized experiences.

Challenge

Runtastic redesigned their premium communication within the app and web and found that too much content and too much information on the screen was overwhelming their users and they found that upselling a huge list of features was not converting their users very well.

Goal

At Runtastic, personalization is crucial for their strategy, as the company has several apps and offerings. So they wanted to offer a more personalized experience for their premium subscription that would result in more upsells in their premium subscription product.

Testing

Runtastic ran a multivariate test across different user segments to deliver a more personalized value proposition for their premium membership against their original variant which had a large list of features that was not personalized to the user. Content was differentiated and personalized by user values such as Nutrition, Guidance, Feedback & Motivation & No ads or benefits. Through this they were able to uncover which messages worked for various users.

Results

Thanks to a number of A/B tests ran by the Runtastic team, the three different personalized premium messages delivered to specific user segments across platforms resulted in a 40% uplift in premium sales compared to their original experience which included a large list of features.

Takeaway

At Runtastic, personalization has become a key benefit of our product within our ecosystem. Testing enables us to use data to prove the user is reaching their goals through a truly personalized experience while enabling us to fulfill our business goals, says Benedict Bleimschein, Product Manager, Runtastic.

2. Try moving from in-app purchases to a subscription-based model.

According to App Annie, 8 out of the world’s 10 top grossing apps of 2016 were freemium, subscription-based apps (Spotify, Netflix, Tinder etc). Until 2014, paid apps or “in-app purchase” revenue was higher than subscription revenue. Since then, revenue generated from subscription based apps have increased 10x according to latest research by Google Play. Consumer attitude toward mobile app subscription is shifting as users are more willing to switch to a subscription if they find value in your app content.

Below is a case study from Apptimize Customer busuu on how they were able to successfully move from an in-app purchase model to subscription based model with the help of experimentation.

busuu Case Study

About busuu

Based in London, busuu builds language learning apps that have been used around the world by over 75M people to date. Their dream is to empower anyone, anywhere to learn a new language through personalised courses, enriched by interactions with native speakers.

Challenge

Until summer 2014, the busuu app offered in-app purchases that would unlock a specific level in a specific language. They switched to subscriptions after 2014, offering monthly, 6-monthly and yearly plans, thinking they would see a drastic increase in their purchase conversion rate and thus revenue. However, with rigorous testing they found that this wasn’t the case-Purchase conversion rate decreased by a whooping 75% after introducing subscription payments!

Goal

Due to the recurring nature of subscriptions they knew that over time they would see a much higher, compounding revenue but they needed to test their different user segments to understand how to increase purchase conversion rate.

Testing

The product team at busuu, led by Antoine Sakho, approached the problem scientifically by thinking of subscription revenue as the following equation:

busuu was able to increase their subscription revenue by running experiments on each driver for subscription revenue including number of installs, conversion rate, average order value and renewal rates.

One such experiment, aptly named ‘content lockdown,’ was to restrict access to all languages to premium users only. Previously on busuu, free users could switch at will between any language, but considering that they would encounter locked content in each of them.

The hypothesis was that some hardcore free users who learn several languages at a time would upgrade to continue.

Result

The conversion increase for the participants (around 20% of our user base learned >1 language) was 83%.

Takeaway

If you have a limited group of free, power users who exhibit a very particular content consumption behaviour, consider locking access to that content and you might see significant improvements to your conversion rates.

3. Speed up feature releases through release trains.

Your team can have incredible product ideas and awe-inspiring UI/UX designs but without a quick development cycle, your app users may drop off before those ideas come to fruition. Mobile teams are moving away from agile and toward 2-week release trains to iterate quickly. Using feature toggle release tools, like Apptimize, allow you to push these changes live with the ability to turn them on and off.

This solution decreases development cycle times and gets your ideas to your customers faster! Below is a case study from Apptimize customer HotelTonight on how they moved to release train intervals.

HotelTonight Case Study

About HotelTonight

HotelTonight used a feature release train method to move fast, which has enabled them to ship a better mobile product. HotelTonight is the leading mobile only solution for last minute hotel bookings, with over 20 million app downloads.

Challenges

The key issue that initially kept HotelTonight from moving quickly and executing on their roadmap was their existing release cycles. They had 3 main issues:

1) Releases were massive – The team would wait until a cornerstone feature was complete to release any new code to the app store which could sometimes take up to 2-3 months. No new releases or fixes would be made for months.
2) Releases were stressful – This method caused great stress and lots of debugging before, during and after the app store release.
3) Releases were limiting – making it difficult to launch experiments.

Goals

The team established clear goals:

1. Cut the time it took to develop features
2. Create a predictable/consistent schedule
3. Make it easier to test new features on different users

Testing

HotelTonight adjusted their release cycle to release a new version every two weeks through a release train. Each application at HotelTonight has a release train that happens at a regular 2 week interval. If a feature is not complete when the train leaves, the Apptimize feature flag is used to shut it off. HotelTonight used feature flag toggle to turn their code or feature on or off depending on the environment or user. This allows them to introduce phased rollouts, quickly ship out new code, and turn off incomplete portions. Throughout this process they learned what worked and what didn’t. Most importantly, this made their releases safe to fail because they could simply shut off code that caused them problems.
They featured improvements to their Referral Program with the help of Release Train that resulted in 50% increase in conversions

Result

HotelTonight was able to ship 57 new versions and released 88 new features on their app. Their first test resulted in 50% increase in conversions largely due to this rigorous release train and evolution.

Takeaway

Establishing the right mobile release process will have a big impact on your products quality, says Doug Suriano, Lead iOS Engineer.

About Apptimize

Apptimize enables organizations to create amazing customer experiences, improve acquisition, engagement and retention for their native mobile apps. The Apptimize platform empowers product teams to efficiently run AB tests, make instant updates and personalize user experiences through staged and segmented app releases. Top apps including The Wall Street Journal, Vevo, HotelTonight and Strava use Apptimize to continuously drive mobile growth and retention.

Source

https://www.appnext.com/blog/mobile-app-experimentation-2018-tips-apptimize/