Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: q1-25 surveys objectives #10303

Merged
merged 8 commits into from
Jan 13, 2025
Merged

chore: q1-25 surveys objectives #10303

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
582c141
chore: q1-25 surveys objectives
marandaneto Jan 9, 2025
8cc9392
fix
marandaneto Jan 9, 2025
f09cf35
fix
marandaneto Jan 9, 2025
aa1bbf5
Update contents/teams/surveys/objectives.mdx
marandaneto Jan 9, 2025
1187d6c
Update contents/teams/surveys/objectives.mdx
marandaneto Jan 9, 2025
ebe8276
Update contents/teams/surveys/objectives.mdx
marandaneto Jan 9, 2025
18d1daa
Merge branch 'master' into chore/q1-25-surveys
marandaneto Jan 10, 2025
25e44ce
fix
marandaneto Jan 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 9 additions & 26 deletions contents/teams/surveys/index.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ template: team
---

## Values

- Fast, iterative and high output rather not slow and thoughtful - achieving this
- Feedback-driven not spec-driven - we do a decent job at this
- Missionary (we have a clear problem definition and are aligned on how impactful a solution would be) not mercenary - glimpses of this
Expand All @@ -15,6 +16,7 @@ template: team
## Personas

### Company Persona

- Primary
- Size:
- 20-75 employees
Expand All @@ -32,6 +34,7 @@ template: team
- Shopify stores/no-code companies

### User Persona

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we are missing Founders from user persona here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not touch this section, so in case we'd like to amend things here, we can do it over another PR after getting context on who wrote this, based on which resources (even more important), etc, right now I have no clue :(

- Primary
- Role
- Product-minded front-end engineer
Expand All @@ -56,38 +59,18 @@ template: team
## Jobs to be done
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be under the objectives section now


### Surveys

- Primary
- Gather customer feedback on your website.
- Gather customer feedback on your website or Mobile app.
- Turn customer feedback into actionable insights that drive your product development.
- Secondary
- Meet your customers where they are, email or mobile surveys.
marandaneto marked this conversation as resolved.
Show resolved Hide resolved
- Survey automation to take action based on survey responses.

## Feature ownership
You can find out more about the features we own [here](/handbook/engineering/feature-ownership)

## Long term vision
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think this is about feature flags and not surveys


Imagine Bob is a product manager, and Alice is an engineer, both of whom love using PostHog.

During their weekly growth review, Posthog shows them that one of their workflows is performing 50% worse than other SaaS companies with a similar flow. They decide to build a new feature together, but they're unsure of the impact, so Bob & Alice decide to gate the feature via a feature flag.
## Slack channel

Alice builds the feature and runs the PostHog CLI, automatically converting his feature branch to a feature-flagged version. During creation, he selects the team template they normally use, called "Autorollout based on conversion metric", using the conversion metric that Posthog suggests. The feature progressively rolls out to internal users, then to beta users, then to remaining users. If their conversion metric falls by more than 20% the feature automatically rolls back and alerts their team. Alice requests a feature flag review from Bob.
[#team-surveys](https://posthog.slack.com/messages/team-surveys)

Bob checks the Posthog UI and because it's such an important feature - adds a safety condition for Sentry errors increasing by 30% and a few counter metrics. This should result in an automatic rollback as well. Bob starts the experiment.

Thankfully, nothing goes wrong when the feature is rolled out. The team is disappointed that the feature doesn't seem to move any of the core company metrics, however. This doesn't fit into either of Alice's or Bob's model, so they dig deeper why this was the case.

Before they even start, PostHog automatically does some impact analysis on their core metrics, and generates some insights into what properties are highly correlated with conversion & which aren't.

As it turns out, people in USA and India love their new feature and show a 40% increase in conversion. Other countries, especially the UK, seem to dislike it so much that it negatively affects conversion. In the end, these forces balance out, leading to similar total conversion rates.

They suspect it might have something to do with their positioning in other countries, so they run a marketing experiment using PostHog, where PostHog automatically generates recommended copy text to try out. It generates 5 variants, and they test these in all countries.

As it turns out, copy wasn't the issue, and there's no significant change here. They watch a few recordings from the experiment to confirm there's nothing off here.

Since it's not a positioning issue, Bob & Alice decide that it makes sense to introduce some personalisation, and let people opt-in to the new feature, and have it on by default for USA and India. They can customise this right from the feature flag, and set this up such that any users who opt-in on their UI automatically get the flag.

PostHog keeps analysing metrics for this flag over time, and notifies Bob and Alice when their customers behaviour change. For example, if the conversion for users in UK has taken a turn for the better, or if enterprise customers have taken a turn for the worse.
## Feature ownership

Our long term vision is to make all of this possible.
You can find out more about the features we own [here](/handbook/engineering/feature-ownership)
48 changes: 33 additions & 15 deletions contents/teams/surveys/objectives.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,19 +1,37 @@
### Objective: Surveys
# Goal 1: Talk to customers

Last quarter we polished up our new surveys UI.
This quarter, we want to address some issues that came up, and then focus on building out some more ambitious features on top of Surveys.
We'll only succeed if we are close to our customers, let's make our relationship with them stronger.

Broadly, we will:
1. Address outstanding experiments UI/UX issues
2. Build no-code experiments
3. Enterprise-3000 for surveys
* Global CSS
* Approvals
* Previews
* Progress bars for Surveys
* Back and Forward buttons for Multiple question surveys.
* Backend events for survey activations.
* Email surveys, which link out to full-screen surveys/ embed survey questions in them.
* Work on full-screen survey forms.
- Do 5 customer interviews
- Find the top 5 reference customers and build a relationship with them
- Do a case study with a top reference customer

# Goal 2: Improve quality

There are a couple of reported bugs that were not addressed quickly, we'd like to speed up this process and fix them.

- Reduce [SLA](https://en.wikipedia.org/wiki/Service-level_agreement) to fix bugs from months to weeks
- Reduce [paper cut](https://en.wikipedia.org/wiki/Lingchi) issues by 50%
- Currently around ~30 [open bugs](https://github.com/PostHog/posthog/issues?q=is%3Aopen+is%3Aissue+label%3Afeature%2Fsurveys+label%3Abug)

# Goal 3: Finish up [WIP](https://en.wikipedia.org/wiki/Work_in_process) features

We've started and nearly finished a couple of features our customers requested, but they are not [GA](https://en.wikipedia.org/wiki/Software_release_life_cycle#General_availability_(GA)) yet, the goal is to finish them up, gather feedback and put them in front of customers.

- [Adaptive Sampling](https://github.com/PostHog/posthog/issues/25722)
- Currently under a feature flag
- [Partial Responses](https://github.com/PostHog/posthog/issues/19300)
- There are Draft PRs
- [Survey Actions](https://github.com/PostHog/posthog/issues/27378)
- Currently under a feature flag
- [AI Summaries](https://github.com/PostHog/posthog/issues/27379)
- Currently under early access features

# Goal 4: Mobile Surveys
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pending PostHog/posthog#27309 here, not sure if we should work on that yet.


We want to bring the same functionality to Mobile that we have for Web Surveys, [GH Issue](https://github.com/PostHog/posthog/issues/25085).

- React Native
- iOS
- Android
- Flutter
Loading