Testing evidence completeness in health score without a convenience sample

Purpose and scope

A grounded customer success note for customer success leaders: health score with explicit evidence, decision rights, and a verified close.

What the protocol tests

A grounded customer success note for customer success leaders: health score with explicit evidence, decision rights, and a verified close.

Report denominators and unresolved records.

Keep the customer decision with the owner.

Limits and reporting discipline

Methodology

The protocol does not estimate an industry benchmark; it produces a local description that customer success leaders can review. Exclude a reopened case only when the reopening did not create a new requirement; keep the exclusion count and reason. Distinguish an administrative pause from a failure: a pause waits for a named decision, while a failure leaves the next action unowned. If the study is exploratory, label it exploratory and do not present the result as a stable baseline. Record this decision in the study log with the case key, the review date, and the person who approved the classification.

The scope is one defined population of health score cases observed from first request through verified close in the onboarding tracker. If the population is too small to sample, say so and report the complete set rather than implying a larger study. Distinguish an administrative pause from a failure: a pause waits for a named decision, while a failure leaves the next action unowned. Name the population, the observation window, and the systems included in the extraction. Keep the evidence reference beside the coded value so a reviewer can test the same case without rebuilding the queue.

Test the unit definition on a case that crosses owners, a case that is still open, and a case that was reopened. A case that moves between owners stays one case unless the process defines a new requirement at the handoff. The stop event should be the verified destination update, not the last message in the thread. If the queue was archived during the window, report the loss and exclude the affected period rather than assuming the cases were resolved. If the source changes after the observation, preserve the earlier value and add a dated note instead of replacing it.

If a reopened case creates a new requirement, count it as a new case and retain a link to the earlier record. If duplicate requests are merged, keep the duplicate count and the merge rule in the dataset. A correction should be classified by the original case and by the correction event, with both dates retained. If a request was reopened after the window, keep it in the original population and report the later event separately. At Customer Success Managers, the reviewer should be able to explain this step to customer sponsor from the case record alone.

Define how a correction after close is linked to the original case and whether it creates a new case. A blank field, a not-applicable field, and a confirmed negative field should have different codes. A duplicate request is a separate intake event even when it points to the same underlying need; record the link. Write the start and stop events as observable actions, not as opinions about whether the work felt finished. Use the same note in the case file and the study extract; do not create a second version in a personal document.

Name the owner who can declare a case complete, and record that role in the data dictionary. Keep the original text and the coded value together so a reviewer can check the classification. Define whether a case that waits for a third party is open, paused, or excluded, and apply that rule to every owner. Keep a change log for every definition adjustment and the date it took effect. When a case needs an exception, record the reason, the owner, and the date before continuing the review.

Record the version or date of the usage report and the QBR template library value so a later reviewer can see what was known at the first action. Use a controlled list for exception reasons; do not let free text become the analysis category. Keep the source reference and the coded value in the same row, and do not overwrite the original with a later correction. A case key should remain stable if the case moves between owners or changes status. Keep the measurement note short enough to read with the case and specific enough to support the reported health score freshness.

Test the dictionary on five cases before the full sample; ambiguous fields should be fixed before collection, not after. Keep the identity crosswalk in the approved system and restrict it to the review owner. Use one row per case and one column per required field; do not bury evidence in a long free-text note. Archive the dictionary with the study so a later reviewer can reproduce the categories. If two reviewers disagree, preserve both classifications until the definition or the evidence is resolved.

Use a pseudonymous case key and keep the identity crosswalk in an approved system with restricted access. Version the dictionary and report which version was used for each observation period. Store sensitive evidence in the approved repository and keep only the reference in the analysis file. Pilot the dictionary with two reviewers and compare their classifications before the main sample. Link the observation to the exact field in the CRM and to the health score note that supports it.

Sample across the full observation window, including the days when the queue was busiest and the days when coverage changed. Collect the exception reason and the action taken before the close so the analysis can separate detection from resolution. Include reopened, unresolved, and exception cases in the sample frame. Record the date of the sample and the version of the queue export. Do not let a summary written later replace the evidence that was available at the first decision.

Use a random or systematic start rule for the first case, then take consecutive cases within each stratum. Use the same collection form for every case and keep the original record linked to the extracted fields. Use a random or systematic start rule and then take consecutive cases within each stratum. Use a random start and a fixed interval when the frame is ordered, and record both. If the case moved between owners, include both owner periods in the review note and name the handoff time.

Record the eligible count, the sample size, and every exclusion so another reviewer can reproduce the sample. A second reviewer should re-extract a small subsample and record disagreements before the full analysis. Stratify by risk, channel, and owner so the sample is not dominated by the easiest queue. Record the sampling frame as an export with a date, filter, and row count. Keep the review note with the case so the next sampler does not have to reconstruct the decision from memory.

Keep case-level detail available for review, because a summary chart without records cannot support a decision. Use case-level review to explain an outlier before treating it as a process signal. Use a simple before-and-after comparison only when the definitions and observation window are the same. When a rate changes, check whether the case mix or the evidence standard changed at the same time. When a value is missing, record the missing state and the reason instead of leaving an empty field in the extract.

Separate observation from interpretation; a high completion rate on a small easy sample is not evidence of a strong control. Keep descriptive findings separate from causal claims. Write the finding in plain language and separate what was observed from what the team believes caused it. Keep descriptive findings separate from causal claims and label exploratory results. At Customer Success Managers, the health score review should end with a dated note and a named owner for the next check.

Changes in definitions, staffing, or the health dashboard during the window can create a trend that is not a process effect. The cited sources describe recordkeeping, privacy, security, or control practice; they do not supply a local benchmark. If the reviewer also performed the work, note the independence limitation and use a second reviewer for a subsample. A missing record is evidence about the record, not proof that the work was never done. If the evidence is sensitive, record the approved reference and access path rather than copying the content.

The method also cannot observe work that never entered the queue, which may be the most important missing population. A clean summary can hide missing evidence, especially when the operator who did the work is also the reviewer. If the queue changed during the window, explain how the change affects the denominator and the interpretation. Record the access problems that prevented a case from being reviewed. The note should distinguish what was observed, what was inferred, and what still needs an owner decision.

Choose one next action: fix the source, simplify the escalation record, change the queue, adjust access, or pause the work until the owner decides. Keep the review note with the health score evidence so the next owner can see why the process changed. If the action requires a policy or access change, route it to the owner and record the decision. Keep the prior procedure available so the next reviewer can compare the old and new states. Record this decision in the study log with the case key, the review date, and the person who approved the classification.

If the action affects unclear owner, route it to customer sponsor and record the approval before implementation. If the result is inconclusive, say so and choose a narrower question rather than claiming a trend. Report the action result even if the change did not work; a failed test is still evidence. Link the action to the case evidence that prompted it, not only to the meeting notes. Keep the evidence reference beside the coded value so a reviewer can test the same case without rebuilding the queue.