When did web analytics officially die? Perhaps earlier this year when the Web Analytics Association renamed itself to the Digital Analytics Association in recognition that there is a lot more to the digital world than fixed web, and that social and mobile channels need to be added to the mix.
Phil Kemelor, Vice President of Strategic Analytics at Semphonic, highlights that from a measurement perspective, even this was a bit of old news. There has always been multiple ways to gauge the effectiveness of online initiatives such as user experience testing, surveys, focus groups and market research. Add the recent social media measurement tools and mobile analytics and the data starts to pile up pretty quick. Phil provides guidance on the tools and techniques to use to untangle the data deluge.
How do you know when to use each of these tools and techniques?
During a recent engagement at the National Institutes of Health I introduced a methodology that I coined “triangulation.” In the “real” world, triangulation is a term used in surveying and social sciences. Read the dictionary definition and I think you’ll agree that we can apply this to the use of disparate data sources:
“A surveying technique in which a region is divided into a series of triangular elements based on a line of known length so that accurate measurements of distances and directions may be made by the application of trigonometry.”
Social science researchers have adopted this term to describe methods for validating the results of their work. This is precisely how we can use this technique to use different data sources to both inform and cross check.
Triangulation and the Web Site Development Process
First we need to describe how digital channel project development occurs. Basically, there are two types of projects:
- Content, with text pages with links; this can include campaigns, such as email blasts.
- Applications, with interactive elements, such as forms, shopping carts, search engines, and data selection wizards.
The only difference between these two is in size. Some projects are small, such as creating a micro site, and some are large, such as doing a major site redesign. However, the steps are the same for all projects. In Figure 2, we illustrate five basic steps in Web development projects and how to incorporate site measurement into them.
Now, let’s see how we can use the right measurement technique at the right point in the development process to inform what we’re doing.
Step 1: Requirements: Requirements state what the digital project will be and what elements it will include. Developing requirements may include preparing use-case scenarios to suggest how visitors will interact with the site.
How to triangulate site-measurement methods for requirements:
- Use analytics for historical data to understand how visitor segments currently use content and applications to get ideas about the audience’s interests.
- Use focus groups, surveys, email, and call-center contacts to stimulate ideas for brainstorming about development.
- Use market research to understand how your target audiences are using similar Web sites.
It is recommended that by the time the design phase is complete, the development team has prepared research plans that consider all the elements to be used for measurement and testing in conjunction with the new initiative. The team should emphasise mapping how the qualitative and quantitative data would be used in tandem.
Knowing your visitors’ thoughts, comments and interests is the most important data asset you can have. Many organisations are becoming more aggressive about collecting, consolidating, analysing, and using customer-sentiment data as this valuable data can help achieve online business goals.
From online surveys, comment cards, and call centres to focus groups and social media, organisations have access to enormous amounts of customer feedback and opinions. It’s challenging to capture and use the data, and it requires new systems and analysis techniques that are different from traditional Web analytics.
Further challenges are expected to materialise with the advent of social media tracking. However, the same practice of goal setting and clarifying how the data correlate with other data streams should help decrease the confusion about what source to believe and how much credence to give one source versus another.
Step 2: Design: Once the requirements are defined, the information architecture and page design can begin. How to triangulate site-measurement methods for design:
- Use Web analytics for historical data to understand any site-navigation successes and challenges experienced by different segments of visitors.
- Use Web site performance monitoring reports to avoid performance issues that can arise with large graphics files.
Step 3: Development: Once the design has been finalised, the development work can begin to include creating functional specifications and coding.
How to triangulate site-measurement methods for development:
- Use Web site monitoring reports to avoid performance issues that can arise with densely coded pages.
Step 4: Testing Conduct all quality-assurance tasks related to development.
Step 5: Launch: When the content or applications “go live” you can collect data on their effectiveness. This is also when you begin to evaluate how to improve the site or application and how to enhance strategies for driving more traffic to the site.
How to triangulate site-measurement methods for the launch:
- Use analytics to compare new content or applications to your business goals or navigation ease and campaign effectiveness.
- Use customer-satisfaction surveys to assess ease of use and value.
- Use post-use surveys to assess the potential behavioral impacts of new content or applications.
- Use market research to evaluate whether new content has changed the positioning of your site or applications relative to similar or competing sites or applications.
A closer look at measurement techniques
Let’s start off with basic definitions of quantitative and qualitative tools and methods:
- Quantitative Tools and Methods
– Record performance data generated by the operation of a fixed, mobile or social presence
– Provide information about actions, events, and trends are happening.
- Qualitative Tools and Methods
– Record usage data generated by current and potential visitors.
– Provide information about events and trends are happening.
Here’s a list of the quantitative and qualitative methods and tools used for analyses.
- Web and Mobile: Analytics software is necessary for collecting information on
- visitor activity
- content and function usage
- site promotion and marketing
- task completion
- site usability
- external search
- Internal search
- Social: Social media has its own set of metrics that are similar but different than fixed web. We’ve listed most common types below:
- Traffic (visits, impressions/views)
- Interactions (likes, comments, posts, tweets, impressions)
- Channel ad campaign cost
- Share of Voice
- Value (or other research based metrics)
- Referred traffic from social to web or web to social
- Referred conversions (or other online successes) from social to web
- Site Quality and Performance: Website monitoring is necessary for keeping track of Web server performance, such as server availability and response time.
- Market Research: Internet industry or market research and business intelligence companies recruit users and use Internet service provider (ISP) data to define their Internet and Web users’ demographic and behavioral characteristics. This provides a clearer understanding of who is using your site, as well as your competitors’ sites. Market research organisations include Experian Hitwise, ComScore, Alexa.com, and Google.
- Usability Measures—User-Experience and Usability Testing: This kind of testing is used to evaluate website design for usability (for example, ease of finding information, reading a site, using site navigation, following workflow processes, filling out forms, and using search engines, multimedia, and graphics).
- Customer Sentiment—Email and Call-Center Contact: Direct contact and feedback from site users can raise red flags about issues with site content, function, design, and navigation. For every person who makes a comment, there are many others who have the same issues but do not comment.
- Customer-Sentiment Online Information Gathering—Surveys: Customer-satisfaction surveys that deploy during a visit may be for understanding visitors’ points of view as they relate directly to navigation or tasks they are trying to perform. Direct online surveys sent to visitors after use of the Web site provide insight into whether using the Web site elicited a change in behavior based on information received or on a specific task completed.
- Customer-Sentiment Focus groups and interviews: Focus groups and interviews can provide information that you can use for improving and planning outreach campaigns, site content, and site functions.
Digital analytics and the other measurement methods should all be used together through triangulation. I believe that the concept of triangulation makes more sense when it’s viewed within the context of the Web development lifecycle.
Understanding the role and value of each type of measurement within the lifecycle helps determine how the information from each measurement method informs the lifecycle activity.
To discover more about how to be sure you engage with the right people in the right way and optimise the resources you dedicate to each channel, Phil is taking part in a joint webinar Thursday Oct 4th 18:00 GMT, titled: Smarter Web Engagement with Advanced Targeting.