151 lines
6.7 KiB
Markdown
151 lines
6.7 KiB
Markdown
# Change response and recovery analysis
|
|
|
|
Use this reference only when a recent intervention or suspected change point matters:
|
|
budget/bid scaling, rollback, targeting or word-pack changes, creative replacement,
|
|
Ad Group restructuring, a sudden deterioration, a new “black horse,” or a request to
|
|
rescue a previously successful entity.
|
|
|
|
## 1. Reconstruct the timeline before assigning cause
|
|
|
|
Create a compact event timeline from the report and intervention log. Split relevant
|
|
history into:
|
|
|
|
1. **Pre-change control:** the last sufficiently stable configuration and its mature
|
|
KPI cohorts.
|
|
2. **Transition:** delivery dates whose outcomes are immature or mix old and new
|
|
configuration effects.
|
|
3. **Post-change:** only cohorts mature enough for each KPI.
|
|
|
|
Compare like-for-like phases at the lowest trustworthy entity. Show exact dates,
|
|
spend, sample, KPI results, and configuration when known. Rolling 7D/14D windows remain
|
|
useful context, but must not blur a sharp phase break.
|
|
|
|
Timing is not causality. Use language such as `与变更时间一致` until competing
|
|
explanations are checked. If deterioration began before restructuring, say so: the
|
|
restructure may have amplified or failed to fix the problem, but is not the sole
|
|
demonstrated cause.
|
|
|
|
## 2. Separate expansion shock from organic deterioration
|
|
|
|
Call an expansion-related deterioration plausible only when a confirmed budget, bid,
|
|
reach, allocation, or traffic-mix change precedes the mature decline. Observed spend
|
|
alone is not budget history.
|
|
|
|
- If budget/delivery increased and downstream quality fell after the change, test an
|
|
expansion or traffic-mix hypothesis.
|
|
- If spend stayed broadly stable while acquisition remained efficient but retention,
|
|
revenue, or another downstream KPI fell, classify it first as downstream quality
|
|
deterioration. Investigate audience, inventory, placement, cohort concentration,
|
|
tracking, and mix shifts; do not label it a budget shock.
|
|
- If acquisition and downstream quality both worsen, keep creative fatigue,
|
|
targeting saturation, competition, and structural changes in the hypothesis set.
|
|
|
|
Never invent a causal mechanism from aggregate metrics. State the breakdown needed to
|
|
confirm it, such as placement, device, audience, search term, keyword/match type, geo,
|
|
frequency, or auction diagnostics.
|
|
|
|
## 3. Decide rollback, containment, or pause
|
|
|
|
Prefer a controlled rollback/recovery test when all are true:
|
|
|
|
- a dated intervention plausibly precedes the mature deterioration;
|
|
- a prior configuration is known and operationally restorable;
|
|
- that prior phase has credible evidence, not merely one good day;
|
|
- the rollback does not destroy a currently verified child winner or introduce a
|
|
broader structural risk.
|
|
|
|
Rollback is not recovery. If the prior configuration also missed current KPI, label
|
|
the rollback `止损/风险收敛`, not `恢复到有效状态`. Compare it against both the damaged
|
|
phase and the current KPI contract.
|
|
|
|
During the recovery test, hold other controllable variables fixed. Use the operator's
|
|
configured maturity/sample rule. If none exists, a small run of multiple mature
|
|
cohorts may serve as a provisional checkpoint, followed by a complete recent-window
|
|
validation; clearly label this as a default evidence rule, not a universal truth.
|
|
|
|
Define exits in advance:
|
|
|
|
- **Recover:** multiple mature cohorts return to target without a guardrail breach;
|
|
- **Continue cautiously:** directional improvement exists but sample is insufficient;
|
|
- **Escalate/retest:** results remain poor or mixed and a clean hypothesis is testable;
|
|
- **Pause/reduce:** persistent mature failure continues with meaningful exposure and
|
|
no unresolved data issue.
|
|
|
|
Avoid daily edits during an observation window unless a hard loss, compliance, or
|
|
tracking guardrail is breached.
|
|
|
|
## 4. Validate a new or apparent “black horse”
|
|
|
|
Early averages can be dominated by one cohort. Separate:
|
|
|
|
- launch/early cohorts;
|
|
- the standout cohort or event;
|
|
- subsequent mature cohorts;
|
|
- immature cohorts, which must not be treated as zeros.
|
|
|
|
Show how much of the retained outcomes, revenue, or other success measure comes from
|
|
the best day/segment. One strong cohort or one to two good days supports `promising`,
|
|
not `durable winner`. Require the configured minimum sample and multi-cohort
|
|
consistency before scale.
|
|
|
|
When an early winner weakens but has not accumulated persistent failure, prefer
|
|
hold/protect at a stable test level over immediate scale or pause. Specify the next
|
|
mature checkpoint and the failure condition.
|
|
|
|
## 5. Rescue a restructured Campaign or Ad Group
|
|
|
|
Do not decide `replace everything` or `abandon the Campaign` from its aggregate. Build
|
|
a comparison across:
|
|
|
|
- the prior clean control structure;
|
|
- copied/reused groups;
|
|
- new word packs, audiences, creatives, or match types;
|
|
- current child winners worth protecting.
|
|
|
|
Measure whether weakness is broad or concentrated and whether the decline started
|
|
before or after each structure change. Historical success is context, but reassess it
|
|
against the current KPI contract; a phase that met acquisition KPI but failed
|
|
downstream quality is not an all-KPI control.
|
|
|
|
A clean rescue design should:
|
|
|
|
1. preserve one interpretable control;
|
|
2. isolate one material hypothesis per test group;
|
|
3. reduce duplicate/overlapping audiences, keywords, or creatives where evidence
|
|
supports overlap risk;
|
|
4. protect verified child winners from being reset or drowned by the test;
|
|
5. use mature, predeclared success and stop conditions.
|
|
|
|
Do not recommend swapping a whole word pack without keyword/search-term and match-type
|
|
evidence when those are the real levers. Request deeper dimensions only when they can
|
|
change the decision.
|
|
|
|
## 6. Report the change decision
|
|
|
|
For an explicitly requested named-Ad follow-up, the optional presentation template is:
|
|
|
|
```text
|
|
<Ad名称>近7日专项判断
|
|
近7日消耗 | 安装 / 成熟结果 | 主要KPI | 下游KPI | 达标天数
|
|
趋势：<量级、KPI和变更前后走势>
|
|
结论：<保持、回退、低量观察、制作新变体、暂停或扩量判断>
|
|
```
|
|
|
|
Populate it only from the current source and confirmed context. This is a follow-up
|
|
template, not a standard daily-brief section. Never preserve an example Ad name or
|
|
example metrics as default content.
|
|
|
|
For each change-related recommendation include:
|
|
|
|
- entity path and stable ID;
|
|
- declared or inferred change and timestamp;
|
|
- pre-change, transition, and post-change mature evidence;
|
|
- observed fact versus causal hypothesis;
|
|
- action state: rollback/recovery, hold/protect, clean-control retest,
|
|
pause/reduce, or scale candidate;
|
|
- variables to hold fixed;
|
|
- pacing impact, including the spend gap created by rollback/pause;
|
|
- next mature review point and explicit recover/escalate/exit condition.
|
|
|
|
Never claim the intervention was executed. This skill remains analysis-only.
|