Validate Real Value of New Feature | Dataford Interview Questions

Company Context

NotionFlow is a B2B collaboration SaaS platform used by 6 million monthly active users across startups, mid-market companies, and enterprise teams. The company monetizes through seat-based subscriptions and has recently launched an AI meeting summary feature inside its workspace product to improve team productivity and reduce manual note-taking.

Problem

The feature has early signs of traction: 22% of eligible users tried it at least once in the first month, and internal stakeholders are calling it a success based on activation and positive anecdotal feedback. However, weekly repeat usage is only 7%, and support tickets show mixed reactions: some users say it saves time, while others say summaries are generic or not trustworthy. Leadership wants to know whether the feature is creating real user value or just generating novelty-driven engagement.

You are the product manager responsible for evaluating the feature and recommending whether to invest further, reposition it for a narrower segment, or de-prioritize it.

Deliverables

Define what “real user value” means for this feature and how you would distinguish it from short-term curiosity.
Identify the most important user segments and jobs-to-be-done to evaluate first.
Propose a measurement plan using qualitative and quantitative signals, including leading and lagging indicators.
Recommend a decision framework for whether to scale, iterate, narrow the target audience, or stop investing.
Explain the key trade-offs and risks in your evaluation approach.

Constraints

You have 6 weeks to produce a recommendation for the executive team.
Engineering can support only lightweight instrumentation changes during this period.
There is no clean randomized experiment yet; the feature was launched broadly to paid workspaces.
Sales is already using the feature in enterprise pitches, so a full rollback has commercial implications.

Company Context

Problem

You are the product manager responsible for evaluating the feature and recommending whether to invest further, reposition it for a narrower segment, or de-prioritize it.

Deliverables

Define what “real user value” means for this feature and how you would distinguish it from short-term curiosity.

Identify the most important user segments and jobs-to-be-done to evaluate first.

Propose a measurement plan using qualitative and quantitative signals, including leading and lagging indicators.

Recommend a decision framework for whether to scale, iterate, narrow the target audience, or stop investing.

Explain the key trade-offs and risks in your evaluation approach.

Constraints

You have 6 weeks to produce a recommendation for the executive team.

Engineering can support only lightweight instrumentation changes during this period.

There is no clean randomized experiment yet; the feature was launched broadly to paid workspaces.

Sales is already using the feature in enterprise pitches, so a full rollback has commercial implications.