Mather Economics’ Listener Intelligent Paywall, a 2018 Global BIGGIES Award winner

Mather Economics Listener Intelligent Paywall won a first place in the best in data-driven technology in the Global BIGGIES Awards competition.

Big Data & AI for Media:

Define the problem to be solved or technology disruption to be addressed 

Arvid Tchivzhel:

To help newspaper and media companies transform into the digital age, we developed the Listener™ data platform. Listener is the foundation that collects data from online audience, ad servers, payment platforms, other online interactions (like commenting engines, login systems…etc.) and offline data sources (such as demographics, subscriber payment history, newsletter usage…etc.). What makes our data collection unique is the granularity of data collection and ability to connect all the user interactions from dozens of platforms used by media companies. This removes data silos and eliminates the need to install multiple tracking tools that slow down site performance. We have submitted a patent in 2017 on the technology, design, and methodology of the data platform.

A robust data foundation is the first requirement to help publishers transition to advanced audience and content analytics in the digital age. We had attempted to utilize other tracking tools and widely-used dashboards but found limitations which did not provide a sufficient 360 customer view to address essential analytical questions, and more importantly, the ability to execute recommendations based on the analytics in real time. Thus, the Listener infrastructure not only collects data to build holistic datasets for analysis but also enables users to apply recommendations with direct integration into fulfillment tools such as email service providers, paywalls, data management platforms, and ad servers.

We built the Intelligent Paywall™ algorithm on top of the Listener data platform to solve one of the most critical questions for media companies. As many newspapers struggle to offset losses from their print subscribers and advertising and find challenges with declining CPMs in digital advertising, the need for a sustainable paid digital subscription business is apparent. However, publishers must balance the paywall configuration to limit potential advertising risk while still converting enough anonymous users to paid subscribers. Without a clear understanding of the tradeoff on a per-user-basis, publishers are “leaving money on the table” when applying a one-size-fits-all paywall. The Intelligent Paywall triggers the optimal user experience where net revenue is maximized. Media companies have multiple levers that can be adjusted to maximize net revenue. Levers available to publishers are:

• The timing of the call to action (commonly referred to as meter setting);
• The content used to encourage a call to action (what articles should be “behind” the paywall or accessible for free);
• The price and discount being offered to incentivize the user to take action;
• The product offered (digital-only, print+digital, or simply a free registration or newsletter signup in exchange for more content access);
• The messaging and the creative proposed to the user (for example, focusing the value proposition on quality journalism, exclusive sports coverage, or community engagement based upon user preferences and behavior).

Knowing which user should be offered the right combination of these levers is the future of sustainable data-driven audience development and acquisition strategy. The Intelligent Paywall assigns each user the right combination of these levers to save advertising revenue and grow incremental subscription revenue in real time.

In summary, the Listener data platform solves the data capture and implementation problem while the Intelligent Paywall solves the analytical and business problem.

Big Data & AI for Media:

Describe time frame for project, and milestones that took place, details on planning, testing, experiments, rollout, review, improvements

Arvid Tchivzhel:

Listener and the intelligent paywall are the culmination of four years of ongoing development in collaboration with multiple publishers. Key milestones for the Listener data platform include:

• Standardization of Listener configuration with publisher content management systems and data layers. Most paywall systems, ad servers, authentication systems, commenting engines, e-editions, and email systems are now supported or require minimal development resources.
• Software development kits for Android and iOS platforms to apply the same principles and intelligence to mobile apps.
• Deployment of Listener Tag Manager to simplify and expedite client onboarding with all the above. LTM has reduced onboarding and maintenance significantly allowing us scale most of the custom JavaScript development for each new publisher.
• Quality control and standardization of all data definitions and pipelines. The “downstream processing” of the data captured by LTM transforms raw unstructured data into usable tables and metrics for reporting and analytics.
• Setup of infrastructure to deliver recommendations into fulfillment systems. This is the piece of the puzzle to connect the data and analytics into the fulfillment engines common to publishers. The Listener Data Platform delivers recommendations at a user level into email systems, paywalls, ad servers, data management platforms, retargeting engines and almost any marketing fulfillment tool available to publishers.

Key milestones for the intelligent paywall include:

• Development of the User Database – an internal processing and analytics engine to build user profiles, engagement metrics, and custom user segments (for example: sports fan, politics fan, engagement level, cart abandoner…etc.). Almost two years of development have been invested to build a robust user segmentation engine which continues to be refined with additional features and functionality.
• Over one year has been invested to develop a conversion propensity model. The model uses discrete choice and clustering methodology to identify users who are likely to pay for content. This model is part of the core of the Intelligent Paywall segmentation and recommendation engine.
• Currently, the team is working with multiple publishers to test the Intelligent Paywall and measure the impact of various levers to maximize user value. After testing, the Intelligent Paywall algorithm will be adjusted based on actual results.
• Future milestones are to improve the propensity modeling and intelligent paywall algorithms using machine learning techniques to further improve predictive power.

Big Data & AI for Media:

Describe the resources used for the project, including budget, technologies, team members, skill sets, time spent

Arvid Tchivzhel:

Significant investment in capital and acquisition of talent were completed over four years and are ongoing to refine and scale the data platform and analytics. Key technologies that are utilized include:

• Setup and configuration of a robust content delivery network to capture over 20 terabytes of new data each month with a 30% growth rate year-over-year utilizing Amazon and Google cloud services.
• JavaScript code and a proprietary tagging engine (Listener Tag Manager) to configure site-specific data capture.
• S3 buckets in Amazon Web Services to store unstructured raw data.
• A Java-based ingestion process to transfer unstructured data into semi-structured tables in a distributed Hadoop file system (HDFS).
• Utilization of a variety of coding languages to structure and process/automate data into usable datasets, analytics, and dashboards such as Bash, Stata, Python, Hive, Spark, PySpark, Pig, Oozie, MYSQL, and Tableau.
• Github to manage the codebase and maintain version control.

The collective toolkit noted above has been implemented by a mix of data scientists, economists, developers, and data engineers. The team members involved have grown from a handful of economists to dozens of team members involved in various aspects of the product.

Big Data & AI for Media:

Describe challenges with management, resources, audience, technologies, etc.

Arvid Tchivzhel:

The primary challenge has been the alignment of distinct departments and stakeholders to test and execute the technology and analytics end-to-end. Most publishers are still evolving their organizational structure to catch up to the rapid pace of applying data science in the media industry and developing efficient workflows between key stakeholders and departments.

• Onboarding often requires coordination between web developers, paywall vendors, authentication systems, and advertising operations. Often there are multiple stakeholders within each of these functions who must be engaged and aligned.
• Publishers also have overlap between departments who are responsible for audience development, newsletter strategy, registration, digital subscriptions, and marketing automation/execution. Though a single department may be responsible for digital subscription revenue, implementing the Intelligent Paywall still requires considerable project management and coordination from within the organization to combine functional roles who have finite tactical goals under a single strategic umbrella. Incentives for one distinct operating group may be indirectly affected by others which also may prevent willing teamwork towards a broader goal. Thus, marketing, audience acquisition, audience engagement, and audience retention must be coordinated to ensure the successful implementation of the Intelligent Paywall and a holistic customer lifecycle management strategy.
• Reporting hierarchies may also be vague which lead to perpetual deferment of decision-making and accountability of project success. The workflows between marketing, technology, and analytics to meet business goals is still elusive for many publishers.

Big Data & AI for Media:

Describe specific progress made with the project, such as increased revenue, engagement, audience growth, teamwork, subscription increase, etc.

Arvid Tchivzhel:

When applied to a known user base (users who have already registered for online access but have not yet paid for a product), the Intelligent Paywall targeted a select audience that generated a 0.34% conversion rate. A randomly sampled audience of the same size generated a conversion rate of 0.10%. The targeted audience by the Intelligent Paywall returned over 3X the revenue compared to the randomly sampled control group. Though not part of the test, the audience deemed “low propensity” would be left unaffected by any paywall tactics which enables them to continue browsing and generating advertising revenue. Though the revenue saved is difficult to measure, it is expected any page views beyond a meter setting of 5 (the meter at the tested media company) read by the “low propensity” group would have been lost without the Intelligent Paywall. Another test was implemented using the algorithm from the Intelligent Paywall but was integrated into an advertising server (DoubleClick for Publishers). The randomly sampled “Run of Site” group showed a conversion rate of 0.02% while the Mather High-Propensity audience showed a conversion rate of 0.04%. The conversion rates overall were low since the targeting was in an ad position rather than dedicated paywall lightbox but still returned 2X the value for a Sunday+Digital $1 offer. Additional testing across multiple publishers has proven the reliability of the propensity model in 2018. We have proven that the audiences identified as “high propensity” by the Intelligent Paywall convert 2-3X greater (including 2-3X the revenue) than a randomly sampled audience across multiple sites of varying sizes and regions. This has enabled publishers to ensure they don’t restrict content to users who are not going to subscribe (thereby not risking their ad revenue) and only offer the paywall to users who are likely to subscribe (getting the same or greater conversion volume than an across-the-board approach and limiting the advertising risk).

The project has evolved to test the application of different levers (outlined above) to the high, medium and low propensity audiences. Multiple publishers are launching A/B testing to measure the impact of different meter settings, messaging, product offering and other characteristics. We are eager to see results this year.

Big Data & AI for Media:

Identify any long-term issues, such as plans for growth, change of technologies, projection for revenues or audience, impact on other projects, impact on company’s resources, management, strategy and/or resources.

Arvid Tchivzhel:

There are long term challenges to consider:

• Open source technology is changing at a rapid pace and requires a unique skillset to manage. Data engineers must keep abreast of new tools, methods, and emerging languages that are being used. However, they must also be thorough in evaluating if new technologies are short-term fads or if they will become mainstays in Big Data. One such example is Pig, which had a surge of activity several years ago but is now not attracting many submissions to maintaining this language. Spark is an example of a technology that has become a mainstay after some debate within the community about its viability as a standalone technology.
• The media industry continues to transform and adapt to new technologies like virtual reality as well as constantly deciding what platforms to embrace or shun. Facebook Instant Articles, Twitter, Instagram, and Google are vying for valuable content and have built-in audiences. Publishers will dictate how large of a role these platform providers will play in relevance and revenue. We must be ready to integrate and adapt Listener and the Intelligent Paywall to the direction of the industry but also be wary of short-term enthusiasms as publishers search for new revenue sources in the digital age.

Big Data & AI for Media:

Summary of recommendations for duplicating success and avoiding challenges for other media companies

Arvid Tchivzhel:

There are several lessons learned over the last four years:

• Publishers should pick the right vendors who can execute a user-specific strategy. Email systems and paywall systems have different features and functionality so finding technology that can execute user-level personalization is important.
• We have seen only the largest and most advanced publishers with significant resources and national audiences be able to build the data, modeling, and execution with internal resources. Metro, medium and smaller publishers should look to partners who can provide some or all parts of the toolkit required to replicate the success from larger publishers.
• Reporting key performance metrics should be measured holistically. Often publishers are focused on a singular metric (for example, online conversions) so may forget about the importance of nurturing and engaging audiences first as well as measuring success across multiple channels such as email and referrals. For example, capturing a significant volume of email addresses via newsletters and then remarketing directly via email with subscription offers may not improve on-site conversion rates but is a stronger audience development strategy.

Almost all user engagement metrics show acceleration about two weeks prior to the conversion event. The propensity model underlying the Intelligent Paywall models this acceleration and identifies users who exhibit this type of behavior.

Propensity model shows strong correlation with actual historical data. Users with a score greater than 90 show an inflection point of conversion propensity. These users are combined into a high propensity segment for targeting specific marketing messages. Users with scores below 40 show low propensity and are unlikely to pay for content. Users with scores between 40-90 show moderate conversion propensity so may not be ready to pay for content but may be open for free registration or newsletter signup.

Case study

https://www.mathereconomics.com/case-studies/#data-science-as-a-service-acquisition-audience-package-subscriber-conversion

Enter the 2018 BIGGIES EMEA competition, with 12 categories focused on Big Data- and AI-driven products and processes at media companies. Deadline is Sept. 7. Enter now!