Help Center Article Ratings: Do They Matter?

You've partnered with product managers and marketers to create benefit-driven documentation on a shiny new feature set. Support gurus have given you feedback on your troubleshooting tips and screenshots. Maybe you've even done some usability testing on your instructions with a customer.

But a month later, when you check in to see how your article is doing, your heart drops when you see a 23% customer satisfaction rating. What the heck went wrong?

I've been there. Content strategists at Google, Twitter, SurveyMonkey, and Airbnb have been there.

We're all facing the same struggle: how do we determine if our help center content is working?

Most out-of-the box help center setups (like Desk.com or Zendesk) include an article rating feature. Sounds great, right? Let your customers tell if you if your content is good or not!

Except here's the thing:

• Votes skew negative. Anyone who's been on the Internet knows that angry folks are far likelier to air their opinions than satisfied people. On top of that, users who get their answer by step 4 of 10 have little incentive to scroll to the bottom of your article, where voting modules typically live. 
• Most article voting systems are binary. Thumbs up or thumbs down. Smiley face or frowny face. Most rating systems are a simple yes or no vote (although some companies, like Facebook, have incorporated scaled satisfaction systems). This limits what you can learn about why content works or doesn't.
• It's hard to achieve statistical relevance. Even with hundreds of thousands of help center sessions every month, at Eventbrite we still only see single-digit percentages of customers who vote on our content. 

So what's a support content creator to do? 

 

Don't require login for article voting.

An authenticated help center can drastically improve your customer's experience with features like content personalization and recommendations.

But requiring a login—or worse, requiring a login to a special support-only account separate from their product account (I'm looking at you, Squarespace)—severely limits the amount of valuable customer insight you could be using to improve your content.

Opening up your feedback modules to any visitors will help you get more of the feedback you need to understand article improvement opportunities.

Ask the right question.

There's a very real (and important) difference between these seemingly similar feedback questions:

• Was this article helpful?
• Did this article answer your question?

The first option is the most common phrasing, but the second option will help you understand your content's success better.

Why? Put yourself in your customer's shoes—just because an article was helpful, doesn't mean it gave them what they needed or wanted. 

As content creators, it feels good to get those positive votes from customers—but it robs us of really understanding if our content leads to customer resolution, which is what should really matter.

 

Structure how you're collecting customer feedback. 

Even if it's skewed to the negative, customer feedback is an absolute goldmine if you're collecting it the right way. The problem with simple voting, however, is it provides zero insight on why an article has failed.

Sometimes you can use your best content judgment to know what to change—but you'd be surprised how often it's hard to know what's wrong (for example, 5-10% of our customer feedback is about product functionality and feature requests, so we pass that customer feedback to the product team). Why guess if you can know?

View fullsize

Open feedback fields are great for getting specific feedback in your customers' own words—but buckets of feedback types can make it easier for customers to give more insight with minimal effort. 

Use directional signals.

Maybe you have an article that has always just sat at a sad 31% customer satisfaction rating. You've eventually come to accept that this rating is going to stay pretty low unless your product changes—something you may or may not have control over.

What you can do, however, is understand how this rating fluctuates over time. At ratings this low, you should pay attention to even small fluctuations if there have been product changes, seasonal trends, or big updates to content.

In other words, don't worry too much if this article is at 31% month after month. Raise an eyebrow if it all of a sudden drops to 17%.

Compare content performance against support cases.

A full-picture view of the customer experience is more valuable than a single article rating metric.

The Eventbrite team uses article ratings primarily for insight on what needs attention and for specific customer feedback so we don't spin our wheels guessing what to fix.

But when it comes to understanding bottom-line success, we cross-reference article metrics with inbound support cases.

• At a high level, we compare overall help center sessions to the number of contacts our support team gets.
• For more specific content, we compare article views to cases tagged with a specific contact driver classification.

We also look at the next page path from the article—specifically, what percentage of users on the article go to our contact form next—as a sign of success or failure at resolving customer issues with content.

This triangulated way of examining article performance—directional success rates, structured and open-ended customer feedback, and performance against support trends—is how my team looks for self-service success.

What methods does your team use to determine article success?

Why I Left Journalism and Never Looked Back (Mostly)

A few weeks ago, a friend of mine asked me about my transition from journalism to copywriting and content strategy, and whether she should consider a similar move. At the time, I simply told her to go for it, but such a significant shift probably warrants a bit more explanation.

I graduated with, among other things, a B.A. in English—something no one thought to tell me typically leads to either graduate school or teaching, neither of which I felt strongly about (both of which I considered). Fortunately, I'd worked for the university newspaper since arriving on campus and my experience in that tiny newsroom gave me sufficient résumé fodder to get paid slightly more in a slightly larger newsroom in almost-middle-of-nowhere Texas.

I lasted longer than most—about a year and a half—before I realized I needed a change. Here's why the transition made sense for me:

1. I realized I hated forcing people to talk to me (and then questioning whether to trust their word). I worked the hard news beat, often covering crime and politics, and that meant people didn't always want to talk to me. On more than one occasion, people charged with crimes came into the office looking for me, and only now are the mugshots not the first image to come up when you Google image search my name.
   Moving into copywriting and content strategy for a digital agency meant working with people who were actually paying to talk to me and were passionate and excited about what we could create together. I was able to keep my reporter's curiosity in getting to know their business while helping them translate that into the most authentic voice for their online presence
2. I decided I wanted to focus on quality more than quantity. When you work in a small newsroom, you eventually wear every hat: on-scene reporting, writing, copyediting, proofing, photography, layout. And while that does keep things interesting, it didn't take long for me to feel like there was more focus on getting a paper out every day on as few resources as possible as opposed to taking the time to create really quality work less often (the result of a die-hard daily).
   Copywriting appealed to me because it gave me the chance to think about a single headline for a few days or explore the proper tone for a site over a week or two. I wanted depth in my craft, and I still got the breadth I loved  by working with clients in many different industries.
3. I fell in love with architecting information online.It started with hyperlinking to previously written stories when I posted the oft-maligned web edition of our paper (the stories I could tell you about our commenters...). When I realized the potential for connecting  just the right information at the  just right moment, I recognized the power of digital content.
   I adore the printed word and I'll be the first to defend journalism as an important craft in today's world regardless of medium, but I wanted to dig deeper into how people experience stories and information they're interested in online, and content strategy lets me do that every day.

I'm privileged to love the work I do and to feel really at home with my chosen career. While it's not right for everyone, it's absolutely where I want to be. It's a relatively young field (let's just say this job didn't exist in any recognizable format when I was doing career research back in middle school), and it's still defining itself.

But that's just one more thing I love—there's a chance to do something really special in a field so new. I hope more smart, capable, information-loving people join us, and I know journalism is one of the places we'll find those next content strategists.

Why Your Client Hates You Right Now (and How to Prevent It in the Future)

This time last year, I was working in client services as a project manager with a great digital agency I loved. I originally got into project management because I was tired of seeing creative professionals being taken advantage of by their managers, their clients, or both. Now I work in-house as a content strategist, and I have personal experience on why clients can come to truly loathe their vendors (often to their complete surprise). In the words of the infamous Ice Cube, check yourself before you wreck yourself. Here's why you might be hated.

• You deliver sloppy work. Part of solid project management is making sure any deliverables you present for client acceptance meet quality requirements and generally using your noggin to look at them through a client's eyes. If you're simply passing on work from creator to client, you're not doing your job (full disclosure: I learned this lesson early on from a great client who was so detail oriented he could catch something that was one pixel off.)
• You miss deadlines (especially early on). Do not be the first to miss a deadline in a project; it sets awful expectations and does the opposite of building trust. Set reasonable deadlines that you can hit—and if things get tight, double down to make it happen on time. When you go above and beyond to keep your word, your client will be encouraged to do the same.
• You don't help the client understand what they don't know. Part of your job as a hired hand is to help your client understand something that is not their everyday. Help them think through their choices (focusing on defining a problem rather than specifying a solution, for example), and let them know what to expect during your specific process.
• You don't negotiate. Scope changes, budgets get blown, and deadlines are missed—and usually this isn't the fault of one side alone. If you've been part of the problem, be part of the solution. It's okay to acknowledge when you're doing more than you may be contractually obligated to do, but use this as a "we're on the same team" opportunity to keep the relationship positive and professional.
• You make the client look bad to their team. There's nothing more frustrating than having to explain to your boss (or the people who work with and for you, for that matter) why a project went off the rails. Don't make your client go on the defense—help them look good to the people that matter to them.

None of these issues means we should abandon project management. In fact, I'd argue that excellent project management can absolutely make the difference between a project I can't wait to get done with and one that I rave about for months.

It's totally possible to build a love-love relationship.

• Deliver excellent work, especially early on. Prove that you know how to listen to the core of what a client is saying and that you can bring your smart expertise to the table. Show this early on and you'll build trust for the rest of the project.
• Talk about scope at the right level. They say children love limits, and it's true for clients as well (although neither would likely ever say that). Clients don't hate scope restrictions—they hate running up against restrictions they didn't see coming. Set a scope that is broad enough for them to actually understand but distinct enough for them to see for themselves when they're butting up against scope creep.
• Set them up to manage their stakeholders internally. Everyone has an opinion, and nothing brings out ego like creative projects (we've all heard "well my wife thinks this font is too small" kind of comments). Help your client manage their internal stakeholders by giving them ample time to collect feedback and a clear process for presenting it (e.g., ask them to provide one document with cohesive feedback within 5 business days of presenting a draft).
• Have a contingency plan. Even the best laid plans can go haywire with one weird stakeholder jumping in last minute or a ship date change out of the blue. Be ready for this by providing a contingency budget you can draw from if your client needs one more round of changes or a rush on this version. You'll all hope to never use it, but you'll be so happy that it's there if you need it.

When you manage a project well, you'll have a team that feels good about the work they've done and a client that's eager to work with you again—that's how you'll know you did your job well.