Team Experiments Q1 2025 goals

[jurajmajerik]

Changes

Team Experiments Q1 goals. Please have a look and feel free to suggest any changes.

Checklist

• Words are spelled using American English
• Titles are in sentence case
• Feature names are in sentence case too

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
posthog	✅ Ready (Inspect)	Visit Preview	Dec 19, 2024 5:26pm

[raquelmsmith]

I feel like it usually makes more sense - and is easier for people to get excited about - if they're responsible for a single objective (or sometimes two).

So instead of having 5 objectives and everyone responsible for all 5, is there a way to divvy these up and find a theme within them? eg do some of the "new features" also fall into the "Data and statistics" section, and there's one person who's going to take the bulk of that work on?

[jurajmajerik]

easier for people to get excited about - if they're responsible for a single objective

I agree, and I’ve been thinking about that too, but I’ve been struggling to come up with such a structure.

For example, Daniel has been significantly contributing across all five areas, and I suspect he’ll continue doing so next quarter. So what’s the point of assigning Anders to "Data & Statistics" when we’ll all likely end up working on it anyway?

Or, Anders will be working on the new Data Collection calculator, which suits him well since it’s a statistical problem. But it’s also a UI/UX problem, and he’ll implement this end to end.

I guess I’m just struggling with how to break these up so that a single person is responsible for each big objective. We could have one person responsible for an objective with others assigned to subtasks, but what’s the significance of that? Would the objective owner somehow oversee the subtasks? I feel like that’s not really how we work here.

Anyway, I’m open to more feedback on how to restructure this 🙏

[danielbachhuber]

Looks like a good start! I left some questions to help facilitate further conversation

[danielbachhuber]

What features do our competitors have that we don't, and which of those features are the most important to close the feature gap/

[jurajmajerik]

The proposed new features are ones that:

• Our major competitors already have
• Our large/priority customers have requested

I’ve used my judgment to prioritize them, taking into account how much we can realistically build in a single quarter.

[danielbachhuber]

How frequently is this a problem for our customers? What % of customers would this impact?

[jurajmajerik]

I assume you're asking about Winsorization here. We don’t have data for it, but I suspect many of our users won’t even notice there’s a data quality issue caused by outliers. We absolutely need to solve this, as it impacts correctness. Even if it affects only 10% of experiments, that means 10% of experiments are producing unreliable results. Correctness is a top priority for experiments, and we should proactively fix this, even if customers aren’t explicitly asking for it or aware of the issue.

[danielbachhuber]

How might you quantify "ensure a great experience for our pilot users"? Also, what does success look like w/r/t adoption numbers?

[jurajmajerik]

Feel free to clarify this further and think about what you'd like it to be. In past quarterly plannings, I've seen objectives like "Get positive feedback from 5 pilot customers" specified.

[jurajmajerik]

I took another pass at this. Every objective except the first one has a clear owner. Collaboration is still encouraged of course! @danielbachhuber, I figure you'll finish some of the Data & statistics work this quarter. We can update it later based on how it goes.