assets/ab_test_template.md
3.5 KB查看排版效果| 1 | # A/B Test Analysis |
| 2 | |
| 3 | **Test Name:** [Descriptive test name] |
| 4 | **Test ID:** [Internal tracking ID] |
| 5 | **Date:** [Start Date] - [End Date] |
| 6 | **Status:** [Planning / Running / Complete / Inconclusive] |
| 7 | |
| 8 | --- |
| 9 | |
| 10 | ## Hypothesis |
| 11 | |
| 12 | **If** [we change X], |
| 13 | **then** [Y will happen], |
| 14 | **because** [rationale based on data or insight]. |
| 15 | |
| 16 | --- |
| 17 | |
| 18 | ## Test Design |
| 19 | |
| 20 | | Parameter | Detail | |
| 21 | |-----------|--------| |
| 22 | | **Variable Tested** | [What is being changed] | |
| 23 | | **Control (A)** | [Description of control variant] | |
| 24 | | **Variant (B)** | [Description of test variant] | |
| 25 | | **Primary Metric** | [The main metric being measured] | |
| 26 | | **Secondary Metrics** | [Additional metrics to monitor] | |
| 27 | | **Traffic Split** | [50/50, 70/30, etc.] | |
| 28 | | **Minimum Sample Size** | [Required sample per variant for statistical significance] | |
| 29 | | **Minimum Detectable Effect** | [Smallest meaningful difference, e.g., 5% lift] | |
| 30 | | **Confidence Level** | [95% or 99%] | |
| 31 | | **Expected Duration** | [X days/weeks based on traffic and sample size] | |
| 32 | |
| 33 | --- |
| 34 | |
| 35 | ## Targeting |
| 36 | |
| 37 | | Criterion | Value | |
| 38 | |-----------|-------| |
| 39 | | **Audience** | [Who sees the test] | |
| 40 | | **Channel** | [Where the test runs] | |
| 41 | | **Device** | [All / Desktop / Mobile] | |
| 42 | | **Geography** | [Regions included] | |
| 43 | | **Exclusions** | [Who is excluded and why] | |
| 44 | |
| 45 | --- |
| 46 | |
| 47 | ## Results |
| 48 | |
| 49 | ### Primary Metric: [Metric Name] |
| 50 | |
| 51 | | Variant | Sample Size | Conversions | Rate | Lift vs Control | |
| 52 | |---------|------------|-------------|------|----------------| |
| 53 | | Control (A) | | | % | - | |
| 54 | | Variant (B) | | | % | % | |
| 55 | |
| 56 | **Statistical Significance:** [Yes/No] at [X]% confidence |
| 57 | **P-value:** [X.XXX] |
| 58 | |
| 59 | ### Secondary Metrics |
| 60 | |
| 61 | | Metric | Control (A) | Variant (B) | Lift | Significant? | |
| 62 | |--------|------------|-------------|------|-------------| |
| 63 | | [Metric 1] | | | % | [Yes/No] | |
| 64 | | [Metric 2] | | | % | [Yes/No] | |
| 65 | | [Metric 3] | | | % | [Yes/No] | |
| 66 | |
| 67 | --- |
| 68 | |
| 69 | ## Segment Analysis |
| 70 | |
| 71 | | Segment | Control Rate | Variant Rate | Lift | Notes | |
| 72 | |---------|-------------|-------------|------|-------| |
| 73 | | Desktop | % | % | % | | |
| 74 | | Mobile | % | % | % | | |
| 75 | | New Visitors | % | % | % | | |
| 76 | | Returning Visitors | % | % | % | | |
| 77 | | [Custom Segment] | % | % | % | | |
| 78 | |
| 79 | --- |
| 80 | |
| 81 | ## Revenue Impact Estimate |
| 82 | |
| 83 | | Metric | Value | |
| 84 | |--------|-------| |
| 85 | | **Projected Annual Lift** | [X]% | |
| 86 | | **Projected Additional Revenue** | $[X] | |
| 87 | | **Projected Additional Conversions** | [X] | |
| 88 | | **Confidence in Estimate** | [High/Medium/Low] | |
| 89 | |
| 90 | --- |
| 91 | |
| 92 | ## Decision |
| 93 | |
| 94 | **Winner:** [Control / Variant / Inconclusive] |
| 95 | |
| 96 | **Rationale:** [Why this decision was made, citing specific metrics and statistical significance] |
| 97 | |
| 98 | **Implementation Plan:** |
| 99 | - [ ] [Step 1: e.g., Roll out variant to 100% of traffic] |
| 100 | - [ ] [Step 2: e.g., Update creative assets across campaigns] |
| 101 | - [ ] [Step 3: e.g., Monitor for X days post-implementation] |
| 102 | - [ ] [Step 4: e.g., Document learnings in knowledge base] |
| 103 | |
| 104 | --- |
| 105 | |
| 106 | ## Learnings |
| 107 | |
| 108 | **What we learned:** |
| 109 | 1. [Key learning 1] |
| 110 | 2. [Key learning 2] |
| 111 | 3. [Key learning 3] |
| 112 | |
| 113 | **Follow-up tests to consider:** |
| 114 | 1. [Next test idea based on results] |
| 115 | 2. [Next test idea based on results] |
| 116 | |
| 117 | --- |
| 118 | |
| 119 | ## Quality Checks |
| 120 | |
| 121 | - [ ] Sample size reached minimum threshold |
| 122 | - [ ] Test ran for at least 1 full business cycle (7 days minimum) |
| 123 | - [ ] No external factors (holidays, outages, promotions) affected results |
| 124 | - [ ] Segments were balanced between variants |
| 125 | - [ ] No sample ratio mismatch (SRM) detected |
| 126 | - [ ] Results reviewed by at least 2 team members |
| 127 | |
| 128 | --- |
| 129 | |
| 130 | *Template from campaign-analytics skill. Statistical significance calculations require external tools (e.g., online calculators or scipy).* |
文件内容是平台检查时读出的文本,只供查看。平台不会运行包里的任何脚本或程序。

