Auditing access changes around renewal risk

Purpose and scope

A grounded customer success note for customer success leaders: renewal risk with explicit evidence, decision rights, and a verified close.

What the protocol tests

A grounded customer success note for customer success leaders: renewal risk with explicit evidence, decision rights, and a verified close.

Report denominators and unresolved records.

Keep the customer decision with the owner.

Limits and reporting discipline

Methodology

The scope is one defined population of renewal risk cases observed from first request through verified close in the support desk. If the population is too small to sample, say so and report the complete set rather than implying a larger study. The definitions should be short enough to use at the queue and precise enough to defend in a review with account executive. Record the exclusions in advance so a difficult case cannot be removed after the result is known. Use the same note in the case file and the study extract; do not create a second version in a personal document.

The protocol does not estimate an industry benchmark; it produces a local description that customer success leaders can review. State how duplicate requests are identified and whether they are merged, linked, or counted separately. Write the primary measure and one supporting measure; a measure without a denominator or observation window is not ready. The protocol should be short enough for an operator to follow and specific enough for a reviewer to repeat. When a case needs an exception, record the reason, the owner, and the date before continuing the review.

A population that changes size during the day should be reported with the start, end, and net change, not only the closing total. If duplicate requests are merged, keep the duplicate count and the merge rule in the dataset. The start event should be the first recorded request, not the first time the team opened the case. If the queue was archived during the window, report the loss and exclude the affected period rather than assuming the cases were resolved. Keep the measurement note short enough to read with the case and specific enough to support the reported reopen rate.

The unit is one renewal risk case, from the opening request to the verified close, and linked events stay with that case. A denominator that changes during the window should be reported separately for each period. A correction should be classified by the original case and by the correction event, with both dates retained. Separate cases opened before the window from cases opened and closed inside it, and report both counts. If two reviewers disagree, preserve both classifications until the definition or the evidence is resolved.

Define how a correction after close is linked to the original case and whether it creates a new case. Test the data dictionary on a small set of cases and revise ambiguous fields before the main sample. Use the same status vocabulary in the queue, the evidence extract, and the report so a case does not change category. State how duplicate, cancelled, and reopened cases enter the numerator and denominator. Link the observation to the exact field in the renewal forecast and to the QBR packet that supports it.

A case that has no destination change should be defined as cancelled or rejected, not counted as complete. Record the source system, the record identifier, the extraction time, and the reviewer for every extracted value. Use the same status vocabulary in the queue, the evidence extract, and the report so a case does not change category. If two reviewers disagree, preserve both classifications until the definition is clarified. Do not let a summary written later replace the evidence that was available at the first decision.

Mark missing, unavailable, and confirmed negative values separately; collapsing them hides the difference between no evidence and evidence of absence. If a field is derived, record the formula and the source fields used to calculate it. Store sensitive evidence in the approved repository and keep only the reference in the analysis file. Keep the original request, the approval, and the destination value linked to the same key. If the case moved between owners, include both owner periods in the review note and name the handoff time.

Each case needs the source, the observation time, the field supported, the owner, and the reviewer. Version the dictionary and report which version was used for each observation period. Run a five-case pilot before the full sample and revise any field that two reviewers interpret differently. Pilot the dictionary with two reviewers and compare their classifications before the main sample. Keep the review note with the case so the next sampler does not have to reconstruct the decision from memory.

Each case needs the source, the observation time, the field supported, the owner, and the reviewer. Version the dictionary and report which version was used for each observation period. Use one row per case and one column per required field; do not bury evidence in a long free-text note. Pilot the dictionary with two reviewers and compare their classifications before the main sample. When a value is missing, record the missing state and the reason instead of leaving an empty field in the extract.

Include adverse cases: a missing renewal risk memo, a conflicting source, an unavailable account executive, and a tight deadline. A second reviewer should re-extract a small subsample and record disagreements before the full analysis. Use a random or systematic start rule and then take consecutive cases within each stratum. Record the sampling frame as an export with a date, filter, and row count. At Customer Success Managers, the renewal risk review should end with a dated note and a named owner for the next check.

Stop expanding the sample only when the review question is answered, not when the result starts to look favorable. Use the same collection form for every case and keep the original record linked to the extracted fields. Use a random or systematic start rule and then take consecutive cases within each stratum. Keep a table of planned, eligible, selected, and reviewed counts for each stratum. If the evidence is sensitive, record the approved reference and access path rather than copying the content.

Use consecutive renewal risk cases where possible, then stratify by owner, channel, and risk class. If a source cannot be accessed, mark it unavailable and record the access problem rather than asking the operator to reconstruct it from memory. Record the eligible count and every exclusion before the sample is drawn. Do not let the reviewer choose the sample after seeing the results. The note should distinguish what was observed, what was inferred, and what still needs an owner decision.

Compare health score freshness across owners only after checking the case mix, the channel, and the evidence standard. A small sample should be reported with the number of cases that drive the result. Record the sample date, the denominator, the numerator, and the exclusion count for every reported rate. End with the decision the evidence supports and the decision it does not support. Record this decision in the study log with the case key, the review date, and the person who approved the classification.

For renewal risk, report the age distribution and the share of cases waiting on a named decision rather than a single average. Report denominators with every rate and separate missing evidence from negative evidence. Keep the case-level table available for review and summarize only the measures that answer the question. Report the number of cases that drive a small-sample result and the direction of the effect. Keep the evidence reference beside the coded value so a reviewer can test the same case without rebuilding the queue.

Report what remains unknown and what evidence would be needed to answer the next question. Changes in staffing, access, or the support desk during the window can create a difference that is not a process effect. Report the limits of the support desk data, including fields that were unavailable or changed during the window. Record the access problems that prevented a case from being reviewed. If the source changes after the observation, preserve the earlier value and add a dated note instead of replacing it.

Small samples are sensitive to one case; report the instability instead of presenting a percentage as a stable rate. Report the unknown population and the evidence that would be needed to describe it. Do not treat a missing record as proof that the work did not happen; it is evidence about the record. Record the access problems that prevented a case from being reviewed. At Customer Success Managers, the reviewer should be able to explain this step to customer sponsor from the case record alone.

Record the decision and the next check so the study does not end as an archive. Report what moved, what stayed the same, and what the study still cannot explain. Link the action to the specific renewal risk cases that motivated it. Use a short action log that names the change, the owner, the due date, and the measure that will show whether it worked. Use the same note in the case file and the study extract; do not create a second version in a personal document.

Repeat the protocol after the change with the same definitions, window, and exclusions. Keep the review note with the renewal risk evidence so the next owner can see why the process changed. If the action requires a policy or access change, route it to the owner and record the decision. Record the owner, due date, and exact change that will be tested at the next review. When a case needs an exception, record the reason, the owner, and the date before continuing the review.