Customer satisfaction survey: why the score may hide more than it tells

Tell us about your project

Please reach out to us about your project and see how we can help you accomplish your goals.

We value your privacy and will never share your contact information

Please note that submissions without a valid business email address or business phone number may not receive a timely response

Healthcare Solutions

If you're in healthcare, you owe it to yourself to learn how you can make your everyday business processes more efficient and save money at the same time. We can help in automating many of your routine and repetitive tasks, including Patient Engagement surveys.

Contact us to learn more

Transportation Solutions

If you're in the transportation business, you can automate many of your routine tasks like package notifications, surveys, collection calls and more. Improve your customer satisfaction by extending your service hours without extending your costs.

Connect with us to learn more

Practical checks utilities should run on sampling, channels, and language access before benchmarking

A customer satisfaction survey can look like an objective yardstick – a single score to summarize years of work. What often gets missed is that the score is a product of who answered and how they were reached. A customer satisfaction survey that feeds a national award is worth auditing early, not after the press release lands.

Recent coverage about the Public Power Customer Satisfaction Awards is a reminder that awards based on survey instruments only matter if the underlying measurement represents the full service population. The operational questions are not glamorous – they are about response bias, channel mix, language access, and how open comments are handled.

The sampling gap most programs ignore

Most survey programs capture the people who are easiest to reach. That skews results. If a program leans heavily on email or web links, it overweights younger, more tech-savvy customers; if it uses only SMS, it misses people without text-capable phones. The first operational check is: who is not in the sample?

Worth asking your team or vendor: How do you build the contact frame – billing records, account contacts, or another source – and how are duplicate or shared phones handled? What channels are used for outreach and what are the completion rates by channel and by demographic group? How are non-responders followed up and are callbacks tracked and attributed to the original request?

Those are practical, testable items. If the answers are vague, the score is fragile. For more on how sample and channel choices skew results, operations teams often start with a targeted review of channel bias and coverage – see our notes on how network mix skews customer satisfaction surveys and NPS.

Language, open-ended feedback, and the work that follows a low score

Language coverage is not a nice-to-have. Customers who are less comfortable in the survey language either drop out or answer less precisely. That both reduces completion rate and biases the verbatim feedback that managers use to diagnose problems. A sensible program offers the survey in the customer’s preferred language up front and captures language preference in the contact frame.

Open-ended comments are where useful detail lives, but they are the operationally expensive part. Collecting spoken or typed comments in multiple languages without a plan for translation and analysis turns this into a reporting headache. One practical approach is to ensure comments are transcribed and translated into a single working language for analysis, while preserving the original text for auditability.

Also think beyond the score. A dissatisfied response should not end in a spreadsheet row. The workflow that follows a low score matters: who gets notified, by which channel, and what is the documented resolution? Closing the loop is what turns measurement into improvement.

Vendor and procurement checks that matter in practice

When a utility participates in a benchmarking program or subscribes to an industry data source, the procurement checklist often misses operational items. The critical, non-glamorous items to confirm are data ownership, reporting transparency, and how the vendor separates local-site data for audits or regulatory review. Those are the questions a regulator or awards committee will ask if the methodology is challenged.

Worth confirming with any vendor before committing: Are raw responses available and in a usable form for internal analysis? What channels are supported (phone, SMS, web) and for how long are recordings or transcripts retained? How are non-English responses captured, transcribed, and translated for cross-language analysis?

Large benchmarking programs also create a temptation to optimize for the score rather than for representativeness. An honest vendor will show completion rates by channel and language, and will make the distribution method explicit. If you want a fuller checklist for enterprise programs, the enterprise CSAT and NPS program notes several of the governance items operations teams typically ask for.

Reaching a statistically useful cross-section of customers usually requires a multi-channel approach. Phone-based surveys still reach people who never open an email or a web link, and offering a voice path matters for older customers, limited English proficiency households, and low-bandwidth areas. If language access or inclusion is material for your community, plan for multilingual delivery from the outset; our guidance on multilingual survey delivery highlights the operational tradeoffs most programs skip.

Finally, document the workflow so the score survives scrutiny. That includes a clear data lineage from the account frame to the final report, a record of who had access to the raw responses, and an auditable trail that shows how low scores were routed and resolved. That kind of documentation is what separates a score you can stand behind from a score you regret later.

There is no single technical trick that fixes a broken measurement program. The work is operational: pick the right mix of channels, capture language and contact preferences, plan for open-ended translation, and make clear what happens after a low score. If you are preparing for national benchmarking, bake those checks into the vendor contract and the program acceptance criteria so the score reflects community experience and not convenience.

Operators thinking through these items usually start with a short diagnostic of channel coverage, response rates by subgroup, and the vendor’s translation and reporting practices. For leaders responsible for a community-facing utility, that small piece of diligence is what makes national recognition meaningful rather than misleading. For more practical guidance on reaching customers who do not answer web forms, see phone and SMS survey options.

Related coverage: Customer Satisfaction Awards recognize top-performing public power utilities – American Public Power Association