Capturing Value from Big Data at comScore Through Platform, People, and Perception

Many companies are struggling with how to capture value from big data, but there is a type of firm that has no choice but to be good at it: analytics providers. These companies’ business models require the ability to effectively and sustainably deliver data-based products and services for which customers will pay. 

In this briefing, we will share how one great analytics provider—comScore, a 14-year-old company that rose to be the #11 US Market Research Organization in the 2012 Honomichl Top 50—achieves value creation from big data via three key assets: a cost-efficient, scalable platform; an analytics-savvy workforce; and a deep understanding of its clients. 

comScore, Inc. 

Headquartered in Reston, Virginia, in 2012 comScore generated annual revenue of $255M and achieved an EBITDA margin of 17.3%. comScore predominantly serves customer-facing companies, including those listed in figure 1, and interacts primarily with marketing, advertising, and media departments, or with IT departments servicing those functions. 

At its core, comScore measures digital consumer behavior to discover information about unique website visitors, their web traffic patterns, and device choice; user demographics, attitudes, lifestyles, and offline behavior; and comparative consumer website and mobile behavior across competitors. Clients use insights gleaned from comScore products and services to set or adjust advertising strategies, manage marketing campaigns, and optimize digital content delivery. 

In 1999, Magid Abraham and a small group of technologists and entrepreneurs set out to measure buying behavior on the Internet. The challenge was that at the time, less than 3% of Internet users made online purchases, which meant that Magid and his team had to amass data from two million people globally to assemble a sufficient sample size for any kind of reasonable statistical analysis. Over the subsequent fourteen years, Magid’s group grew into a publicly traded company called comScore that accumulates data at a rate of 50 billion events each day, 1½ trillion events per month.[foot]These numbers reflect activity from June 2013.[/foot]

A Cost‐Efficient, Scalable Data Platform 

The foundation of comScore’s offerings is fourteen petabytes of diversely sourced online data, collected real time from around the world—what companies now refer to as big data. 

In July 2013, we had almost 1.8 trillion events, an all-time record. Our peak daily volume was reached on July 23, 2013, with 64.8 billion events. Usually we collect about 20 terabytes of compressed data each day and deliver over 150,000 reports, spiking to 45 reports per second.

Michael Brown, Chief Technology Officer 

The data originates from four primary sources. The first source is panel data, collected from two million Internet users, one million each within and outside the US.[foot] In 1999, comScore began with a 50,000 person panel, which at that time was considered a very large sample. Within a year, comScore increased the panel tenfold to 500,000 members, and another decade later it reached today's two million panel member threshold.[/foot] Panel members grant comScore permission to confidentially capture passive measurement of user behavior and demographics. Census data is second, gathered from sensors placed, with permission, on approximately 90% of the Top 100 US digital media properties.[foot]comScore Media Metrix® Top 100 digital media properties[/foot] A third data source is perceptual data collected from panel members using proprietary surveys. And the fourth source is data obtained from strategic partners. For example, comScore uses loyalty card in-store purchase data from dunnhumby to help tie online advertising campaigns together with offline in-store purchases.

These data are processed and integrated in what comScore refers to as its “data factory.”

We are a data factory. We take raw pieces of data and organize it, cleanse it, divide it, subtract it, filter it, [and] contextualize it to create our product.

Michael Brown, Chief Technology Officer 

In the late ‘90s, a mainstream commercial technology solution to manage comScore’s data factory did not exist. comScore developed an efficient, patent-protected technical platform to store and manage big data. 

Our technology team was…really smart in doing [big data] in a very incremental, scalable way. That is something that had to be learned because we were a start up, and it turned out to be a competence that we have been able to leverage and continue to leverage. 

—Linda Abraham, Co-founder 

By 2013, the technology team had evolved the platform from a proprietary solution into a services-oriented architecture that supported three key systems running on MapR’s Hadoop and Pivotal Greenplum Database, each having unique workload and scaling requirements. comScore needs to be relentless about staying on top of its platform; in the last twelve months, data volumes have grown by about 80%. 

An Analytics‐Savvy Workforce 

comScore relies on exceptional people to master the art and science of big data. It has grown to a 1200-person global workforce that possesses varying levels of “data scientist” abilities. To select and develop these people while keeping pace with the company’s growth, company leaders put in place a number of talent management strategies. 

One strategy is to hire analytical people, primarily from university business schools and math programs,[foot]comScore also invested significant time and resources to establishing a presence at the universities from which it hired. This translated into providing free educational access to comScore products, speaking in classes, and supporting marketing and IT professors who run semester-long student projects.[/foot] and give them even more analytics training. comScore has an ongoing company-wide training program—comScore University—attended by the majority of employees. 

For topics that are core to the work we do— basic regression analysis, market mix modeling, digital ad delivery—the majority of employees attend, regardless of their team or position. Subject matter experts from across the company lead the sessions. Our CEO has even led a few sessions on regression analysis and market mix modeling.

Carson Green, Senior Manager, Telecom and Technology 

Another practice that has fostered a broad, analytics-savvy workforce is establishing groups across the company with different levels of skill but enough skill overlap to ensure that the groups can communicate. Teams that need to communicate regularly are co-located. 

Leaders also use matrix organizational structures to bring together people with diverse skillsets. For example, comScore has concurrently running scrum teams, each with a business product owner who understands the product road map and analytics representatives who monitor data quality and methodology. This approach is considered particularly important for the product development process. 

Deep Client Perception That Leads to Actionable Insights 

comScore delivers about 150,000 online reports per day. The challenge is to deliver reports from which clients can not only extract insights, but also incorporate insights into a work task. Customers get value from comScore’s products and services when they act upon insights—such as by changing an ad campaign or targeting a new customer segment—but as an analytics provider, comScore has limited control over how customers use the data. 

What comScore can control is its own understanding of its clients’ needs. The company focuses on developing deep domain expertise and understanding the problems that its clients are trying to solve. Armed with this knowledge, comScore makes big data “consumable” and proactively helps clients identify actionable insights. It recognizes that data can quickly become overwhelming, and encourages clients to focus on answering a few key questions and then to iterate. 

The objective is not to show everything to the client—but to give them four or five insights and help them digest those.

Serge Matta, President 

comScore provides its clients with software tools that have graphical interfaces, incorporating visualizations and graphics; supports self service through extensive use of wizards and template reports; offers real-time access to data through dashboards when “in-flight” decisions are required; and creates knowledge portals to support knowledge sharing. Further, the company has an organizational unit dedicated to supporting the consumability of its products and services. 

We designed an “experience” group, and their mandate is to really understand what the user flows need to be as people try to understand the problems that they need solutions for—and to take that to the nth level. 

Naresh Rekhi, Vice President Product Management 

In the End, It’s All About the Data 

Constant adaptation is required to keep pace with turbulence in the big data space. 

If you have been at comScore for any length of time, you feel in your bones that this market is a moving target. We're never going to get it right and just slap ourselves on the back and say we're done. Less than three years ago, iPads didn't exist.

Mark Donovan, Senior Vice President for Mobile 

Years ago comScore was expected only to measure the number of visitors to a webpage with a client’s ad; now it must report on whether that ad is actually viewed. Early on, companies did not have much interest in non-US data; the of globalization has created a market for the worldwide representation in comScore’s data. 

Additionally, comScore has needed to keep abreast of technology-related changes. For example, customers need to analyze emergent data types, such as streams from video and smartphones. Another example of an important change is the increase in customers who visit online properties with a large number of devices. As a result, companies need to identify unique users across multiple devices. 

comScore’s platform, people, and deep customer perception empower the company to respond to turbulence commonly associated with big data environments. Its cost-efficient, scalable data platform allows comScore to adjust to rapidly increasing data volumes and to ingest new data types. Overlapping teams and skillsets, combined with a workforce that possesses deep analytics skills, enable fluid adaptation to changing needs. And fine attunement to clients’ use of analytics offerings permits the company to anticipate trends and align products and services. CISR believes that companies interested in leveraging big data would be well served to consider comScore’s approach.