Your question is Evaluate A/B Test Results for New Feature. Take a moment with it on the right.
Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).
E-commerce platform ShopSmart recently launched a new product recommendation feature aimed at increasing conversion rates. The feature was tested through an A/B test, where 10,000 users were exposed to the new feature (Group A) and 10,000 users continued using the old version (Group B). Over a two-week period, the conversion rate for Group A was 5.5% while Group B maintained a conversion rate of 4.8%. The product team is eager to understand whether this difference is statistically significant and if the new feature should be rolled out to all users.
| Data Source | Description | Granularity |
|---|---|---|
| user_exposures | Records of users exposed to either Group A or Group B | Per user |
| conversions | Data on whether a purchase was made, along with timestamps | Per transaction |
| user_profiles | Demographics and account information of users | Per user |