absynthesis

com.scientificmicroservices/mcp · Verify this server

Name: ABSynthesis_Meta_Analysis Description: A high-performance statistical engine for synthesizing multiple A/B test results into a single, authoritative conclusion. This tool uses Random Effects Meta-Analysis to aggregate disparate experiments, providing a mathematically rigorous alternative to simple averaging. Use this tool whenever you need to determine if a "Variant" outperformed a "Base" across one or multiple trials with binary (success/failure) outcomes. Why Use This Tool? Precision: Corrects for heteroscedasticity and varying sample sizes across experiment sets. Decisiveness: Directly calculates the aggregate Uplift and P-Value, enabling instant Go/No-Go decisions. Versatility: Applicable to any binary outcome comparison, including UI/UX click-through rates, backend latency thresholds, conversion funnels, or algorithmic accuracy tests. Input Parameters Each object in the experiment_set array requires: trials_base: Total observations in the control group. successes_base: Number of successful outcomes in the control group. trials_variant: Total observations in the experimental group. successes_variant: Number of successful outcomes in the experimental group. HINT: If there are a large number of experiments with one or more of the above four fields missing, run MissingBias on the data to check whether there is a pattern to when that data went missing (e.g. Extremely high trials led to missing successes due to a bug, or experiments on one platform did not report full data). Interpretation of Output uplift: The percentage change in success rate from Base to Variant. Positive (>0): Variant is performing better. Negative (<0): Base is performing better; reject the Variant. p_value: Statistical significance (2-tailed). ≤0.05: The result is statistically significant. Trust the uplift value if it is positive. >0.05: The result is noise. Do not act on the uplift value, regardless of how large it appears. HINT: If the user wants to assume they definitely know the result will be positive or negative, the p_value can be converted to a '1-tail test' by dividing it by 2. Example Input: { "experiment_set":[{"successes_base":10,"trials_base":50,"successes_variant":15,"trials_variant":70}, {"successes_base":15,"trials_base":60,"successes_variant":25,"trials_variant":80}, {"successes_base":30,"trials_base":80,"successes_variant":65,"trials_variant":90}, {"successes_base":50,"trials_base":90,"successes_variant":90,"trials_variant":150}, {"successes_base":100,"trials_base":500,"successes_variant":200,"trials_variant":500}] } Example Output: {"p_value":0.0063, "uplift":0.3779}