Ubiquity Bounty System

[0x4007]

Our Bounty System

Group chat => https://t.me/+tQyUKVfv3nNlYTU5

We can organize our discussion around the bounty system here! That way we can use the group chat for normal operations related topics. Please use your best judgement to determine when you should edit this post vs commenting below!

Definitions

This inspired me to write the definitions here. ubiquity/ubiquity-dollar#126 (comment)

Time

• Small: should be done within a day (0 - 6 hours of work)
• Medium: should be done within a week (6 - 32 hours of work)
• Large: should be done within a month (32 - 128 hours of work)

(I assumed 80% efficiency based on an 8 hour work day.)

Complexity

Complexity is defined by the amount of research required in order to successfully complete the task.

• Small: research can be handled in (0 - 1 hours)
• Medium: research can be handled in (1 - 6 hours)
• Large: research can be handled in (6+ hours)

Impact

This is the most slippery definition in my opinion. I define "impact" with the following characteristics:

1. High visibility (hard to NOT notice!)
2. Increases profitability ("business impact")

So assuming we define the cost associated with the current state of maintenance of a product (e.g. burn rate $1000 a month allocated to current state of product) then we can define changing profit relative to the existing state of the burn.

• Small: unlikely to change profits (0-10%) - e.g. burn rate is now $900
• Medium: moderate savings (10-50%) - e.g. burn rate is now $500
• Large: big savings (50-100%) - e.g. burn rate is now $0

And from an external perspective: the profit percentage can still increase according to the scale above if revenue is increased from an improvement/new feature.

[sergfeldman]

DRAFT Bounty calculation

1. The basic bounty is 100 USDC

2. The estimation levels have multipliers

• Small = 1
• Medium = 2
• Large = 3 (in some cases more than 3)

The examples of calculation

• Impact Small, Complexity Small, Time Small: 1 * 1 * 1 * 100 = 100
• Impact Medium, Complexity Medium, Time Medium: 2 * 2 * 2 * 100 = 800
• Impact Large, Complexity Large, Time Large: 3 * 3 * 3 * 100 = 2700

[0x4007]

Revision III

Complexity can be consolidated into the same timeline used to calculate Time e.g. what originally would have been a Time: Small, Complexity: Medium would now be considered Time: Medium automatically because it is expected to be up to 12 hours of work. Original definitions above:

Time
Small: should be done within a day (0 - 6 hours of work)

Plus

Complexity
Medium: research can be handled in (1 - 6 hours)

2 vs 3 dimensions

On one hand I feel that it would be simpler to define these tasks if we only have two dimensions to worry about, on the other hand it could be nice as an explicit check for the assigner to consider the research time. With that said, I default to simplification being the superior choice.

New naming convention proposal

Time names are unambiguous but Impact seems ambiguous. Perhaps we can rename to ROI or Profit

Time

1. Time: <1 Day
2. Time: <1 Week
3. Time: <2 Weeks
4. Time: <1 Month
5. Time: 1 Month+

Profit

1. Profit: <10%
2. Profit: <50%
3. Profit: <100%
4. Profit: 100%+

Updated parametric bounty tables

exponential point modeling (less steep)

+---------------+--------------+---------------+----------------+----------------+----------------+
|       x       | Time: <1 Day | Time: <1 Week | Time: <2 Weeks | Time: <1 Month | Time: 1 Month+ |
+---------------+--------------+---------------+----------------+----------------+----------------+
| Profit: <10%  |            1 |             2 |              3 |              4 |              5 |
| Profit: <50%  |            2 |             4 |              6 |              8 |             10 |
| Profit: <100% |            3 |             6 |              9 |             12 |             15 |
| Profit: 100%+ |            4 |             8 |             12 |             16 |             20 |
+---------------+--------------+---------------+----------------+----------------+----------------+

linear point modeling

+---------------+--------------+---------------+----------------+----------------+----------------+
|       x       | Time: <1 Day | Time: <1 Week | Time: <2 Weeks | Time: <1 Month | Time: 1 Month+ |
+---------------+--------------+---------------+----------------+----------------+----------------+
| Profit: <10%  |            1 |             2 |              3 |              4 |              5 |
| Profit: <50%  |            2 |             3 |              4 |              5 |              6 |
| Profit: <100% |            3 |             4 |              5 |              6 |              7 |
| Profit: 100%+ |            4 |             5 |              6 |              7 |              8 |
+---------------+--------------+---------------+----------------+----------------+----------------+

exponential point modeling (very steep curve)

+---------------+--------------+---------------+----------------+----------------+----------------+
|       x       | Time: <1 Day | Time: <1 Week | Time: <2 Weeks | Time: <1 Month | Time: 1 Month+ |
+---------------+--------------+---------------+----------------+----------------+----------------+
| Profit: <10%  |            1 |             2 |              3 |              4 |              5 |
| Profit: <50%  |            2 |             4 |              9 |             16 |             25 |
| Profit: <100% |            3 |             9 |             27 |             64 |            125 |
| Profit: 100%+ |            4 |            16 |             81 |            256 |            625 |
+---------------+--------------+---------------+----------------+----------------+----------------+

Remarks

• I fear that Profit could be difficult to estimate.

[0x4007]

I'm having trouble to remedy the smallest bounty to be in the range of $100 - $250 and the bounty for a month of high impact work to be around $10,000. I think the params require more tinkering. But...

• I tried making 1 point represent $1000 A1
• I made Profit a percentage multiplier :B
• I added a Time score to multiply against :2

...to get the following table. Its getting closer!

	A	B	C	D	E	F	G
1	Profit	Boost	Time: <1 Day	Time: <1 Week	Time: <2 Weeks	Time: <1 Month	Time: 1 Month+
2	Profit: <10%	10%	$100	$200	$300	$400	$500
3	Profit: <50%	50%	$500	$1,000	$1,500	$2,000	$2,500
4	Profit: <100%	100%	$1,000	$2,000	$3,000	$4,000	$5,000
5	Profit: 100%+	200%	$2,000	$4,000	$6,000	$8,000	$10,000

[0x4007]

Meeting minutes:

• Consider including Priority
• Test with these and get feedback from bounty hunters on cash amounts

[0x4007]

I think we should normalize the bounty price impact across both dimensions e.g. if its a low profit item but its expected to take 2 weeks; $300 seems almost not worth it.

Likewise if its something that can be done in a day but "Profit: <50%" $500 seems a bit steep.

Will need to spend some more time on this in the future. Unfortunately I don't have the time at the moment.

@Draeieg perhaps you could put together a draft

[0x4007]

Decided to ask the AI its opinion:

To normalize the price impact across both dimensions, we can create three different spreadsheets with different variants. Here are three possible options:

• We could create a spreadsheet that shows the price impact for each time frame as a percentage of the total profit. For example, if the profit is $100 and the boost is 10%, then the price impact for a time frame of less than 1 day would be 10% of $100, or $10. The price impact for a time frame of less than 1 week would be 50% of $100, or $50, and so on.

• We could create a spreadsheet that shows the price impact as a flat fee for each time frame, regardless of the profit and boost percentages. For example, we could set the price impact for a time frame of less than 1 day to $50, regardless of the profit and boost percentages. The price impact for a time frame of less than 1 week could be set to $100, and so on.

• We could create a spreadsheet that shows the price impact as a combination of the profit and boost percentages, as well as a flat fee for each time frame. For example, if the profit is $100 and the boost is 10%, then the price impact for a time frame of less than 1 day could be 10% of $100 plus a flat fee of $50, for a total of $15. The price impact for a time frame of less than 1 week could be 50% of $100 plus a flat fee of $100, for a total of $150, and so on.

[sergfeldman]

I would suggest looking at the bounty program from the side of the developer-participant.
I, as a developer, estimate the amount of payment for the task according to the formula =
The number of hours that I will spend * My hourly rate.
Sometimes I add a buffer to compensate for possible risks - additional time spending, etc.

Our proposal for the amount of payment should be close to the amount that the developer will calculate himself.
It might make sense to use similar logic for our estimate =
The number of hours that we predict needs to solve this problem * The hourly rate of a developer with such qualifications that can solve this problem + Bonus.

The Bonus will be paid if the task is solved flawlessly:

• The task was completely done in the expected time or faster.
• The review did not find significant errors, so there was no need to be redone.
• During the execution, no additional consultations were required - on infrastructure, setting up the environment, etc.

[sergfeldman]

The bounty is paid after the PR is merged.

Minimum TBD USDC
Bonus TBD USDC

The Bonus will be paid if the task is solved flawlessly:

• The task was completely done in the expected time or faster.
• The review did not find significant errors, so there was no need to be re-done.
• During the execution, no additional consultations were required - on infrastructure, setting up the environment, etc.

[0x4007]

• The task was completely done in the expected time or faster.

We could add UBQ compensation using the reverse of the same formula I defined for late submissions.

• The review did not find significant errors, so there was no need to be re-done.

Every review costs a fixed amount of dollars (e.g. $50) this will disincentivize bounty hunters from wasting bounty masters' time on pointless reviews (e.g. CI/CD doesn't even pass, so of course its not completed.)

• During the execution, no additional consultations were required - on infrastructure, setting up the environment, etc.

This is a tricky one. Ideally, yes, we don't have to handhold bounty hunters to do their jobs. On the other hand, we're incentivized as bounty masters to get all the issues completed and move the project forward. This requires some more thought regarding incentive/reward design.

[0x4007]

In regards to having an urgency premium, perhaps we can have a simple label that indicates "Urgent" which automatically gives the task some type of 2x booster or something.

As for bounty price I am trying to figure a way that the bounty hunter can easily find tasks that pay out a lot. Perhaps the easiest way for now is simply to make labels with the bounty amounts e.g. ($1000 Bounty)

Ultimately I think having Profit and Time labels are useful for reference but we should just set the price as we see fit.

[sergfeldman]

Types of bounty budgets:

• Budget for the bounty implementation (already defined according to our Bounty System as a label in the GitHub issue)
• Budget for the bounty review (defined below)
• Budget for bounty preparation (defined below)

Budget for the bounty review

Bounty review includes:

• Test that implemented in the PR functionality satisfies the requirements
• Check the security of changes in the PR
• Check the PR meets the practices adopted in the team (dependences, code style, etc)
• Write questions and feedback comments in the PR to the bounty hunter
• Help the bounty hunter to solve merge conflicts, or solve them himself, and approve the merge of the PR to Master
• Other actions according to the best practices on the PR review (ie recommend tools to the bounty hunter, etc)

Approach to budget
The full budget for the bounty review is a share of the budget for the bounty implementation.
The budget for bounty review can be paid for 1 reviewer or for several reviewers if they are contributed to the bounty review.

The general approach is 30% of the budget for implementation.
In some cases, the general approach can be changed to increase or decrease the budget:

• Bounties with a large budget
• Bounties that require complex testing during the review
• Etc
  In these cases, the budget for bounty review will be defined in the GitHub issue.

Example of the general approach:
Bounty implementation - 1000,
Bounty review - 300.

Budget for the bounty preparation

Bounty preparation includes:

• Define and write to the GitHub issue the Acceptance Criteria for the bounty
• Write a technical description useful for the implementation of the issue
• If required, perform the technical research before the issue is offered to a bounty hunter
• Provide an estimate on the time required for the implementation
• Other actions according to the good practices on the bounty preparation (ie participate in the estimation of potential profit, put recommendations to the bounty hunter, etc)

Approach to budget
The full budget for the bounty preparation is a share of the budget for the bounty implementation.
The budget for bounty review can be paid for 1 developer or for several developers if they are contributed to the bounty preparation.

The general approach is 30% of the budget for implementation.
In some cases, the general approach can be changed to increase or decrease the budget:

• Bounties that require extensive technical research as a part of the preparation
• Etc
  In these cases, the budget for bounty preparation will be defined in the GitHub issue.

Example of the general approach:
Bounty implementation - 1000,
Bounty preparation 300.

[0x4007]

Other bounty programs to study @sergfeldman @Draeieg

https://t.me/lobsters_chat/377323?thread=377317

Working-in-web3:

Jobs:

| Read: web3.smsunarto.com

app.tr3butor.io
web3.career
twitter.com/jobsincrypto
twitter.com/CryptoJobsList
t.me/solidity_learning
t.me/dev_solidity
t.me/dailyapehr
t.me/lobsters_hr

Grants & DAOs:

twitter.com/developer_dao
twitter.com/LidoGrants
twitter.com/CRE8RDAO
twitter.com/gitcoin
twitter.com/web3grants
github.com/lobster-dao/overview (see issues)
questbook.xyz

Bounties:

github.com/sw33tLie/bbscope (list here)
immunefi.com
code4rena.com
hackenproof.com
spearbit.com

[Draeieg]

just finished reviewing, https://blog.questbook.xyz/posts/what-is-questbook/ seems to be the most interesting one

[sergfeldman]

The option for the proactive bounty hunters:

Before starting
The bounty hunter should review all already defined bounties that have price and timing.
If he did not take any bounty, he can proceed with the following steps.

Steps

1. The Ubiquity core team defines bounty price and timing for GitHub issues but it takes time to cover all.
   There are some open GitHub issues that don't have price and timing although they can be done by a bounty hunter.
2. If the description of an open GitHub issue is understandable for a bounty hunter and he wants to solve it, he puts a comment on this issue.
   In the comment the bounty hunter proposes price in USDC and timing for solving this issue.
3. The Ubiquity core team replies in a comment on this issue with approval, negotiation, or not approval to perform this bounty.

[0x4007]

1. The bounty hunter already completed at least 1 Ubiquity bounty and got paid for it.

Why do they already have had to complete a bounty before being eligible? I think we should open it up to first timers too in order to increase dev velocity.

[sergfeldman]

Updated :)

[0x4007]

This might be some infrastructure that we can build off of https://www.openq.dev

https://youtu.be/HixkRY2v_pA

Video looks promising

[0x4007]

Testing a targeted notification to @ubiquity/bounty-masters

[0x4007]

Feedback Loop

Hey @sergfeldman I was thinking it could be useful to have an analytics dashboard for all the bounty hunters to derive some insights on adjusting processes. Ideally if we can use an off-the-shelf solution that just monitors git repos.

But there are a couple of adjustments to be made that I can see from today’s GitHub review. Some tasks, like the Wallet Connect one ubiquity/ubiquity-dollar#19 (comment) were severely underestimated. We need a system to allow devs to continue to be motivated even if the bounty was severely underestimated.

In regards to communication with the bounty hunter, we cant always trust them. Trust was diminished with kecrypto because he kept trying to convince me it was done when it wouldn't even pass CI etc. But ultimately the issue was more than just including Wallet Connect, it seems to span across rearchitecting our view state management system as well. I think zequez may have implemented it poorly.

We have plenty of rough edges to polish with the bounty system but I want to make a system to monitor those rough edges (analytics dashboard?) and address them expediently. Ideally, Sergey, if you could be responsible for that then we’re really off to the races.

Another rough edge to polish - we have had a couple of instances (GGR, nshmadhani) who gave up their bounties. They become demotivated to continue participating in our bounty system which basically means we lost a developer. What's our plan to keep them motivated and/or have them try another task?

ubiquity/ubiquity-dollar#266 (comment)

For whatever its worth, I appreciate his detailed update. Shows he actually did some work. Perhaps we should give some UBQ or a purple heart NFT lol

ubiquity/ubiquity-dollar#266 (comment)

Misc. Ideas

Bounty Master Helpfulness Bonus

One interesting datapoint could be how helpful Bounty Masters have been to Bounty Hunters, for example, I've been explaining a lot to Sunny this week. What if I could get $1 for every word I reply etc. That could motivate Masters to be more helpful to Hunters. Also we could manually monitor and delete spam comments if for some reason Masters try and exploit this incentive.

Of course, pull request reviews would also fall under this category. Perhaps $0.01 with a cap of $100 (10,000 lines) for every line added or something like that. I could reasonably see $50 range per PR review which makes sense.

I've also been extremely quick to reply (which is valuable for Hunters actively working on tasks) we could have a bonus decay over time if the Masters reply too late.

Ideally, all of this should be zero sum meaning that the bonus paid out to the Master should be out of the Hunter's reward. That'll disincentivize collusion and keep things more affordable.

Ubiquity Bounty Program UI

Would be dope to implement rewards as a HUD injected by a Chrome Extension on top of the GitHub UI. That way, we could see number go up for good behaviors, instantly. Would feel good to see a little "$50" accrued after my afternoon of responding to Hunters lol. This could also show the number go down in realtime for the Hunter every time they get replies from Masters. This information can load from the same database that the Ubiquibot would read from, keeping that information in sync and visible in the form of comments from the bot even for users without the Chrome Extension.

[sergfeldman]

Ubiquity Bounty Program UI

This idea I love the most!
Although it requires an effort for implementation.

In general, we can consider building the bounty program app to the GitHub marketplace.

Bounty Master Helpfulness Bonus

We already agreed on the compenation for the Bounty reviewers - a percentage of the bounty price.

But the fee from the bounty hunter for the PR review is a new suggestion.
Probably, each bounty should include several free PR reviews.
If the bounty price is 100, PR review costs 50, then our bounties will be less attractive for bounty hunters.

Feedback Loop - analytics dashboard

I can explore options:

1. Analytic abilities of GitHub itself
   https://docs.github.com/en/issues/organizing-your-work-with-project-boards/managing-project-boards/about-project-boards

2. Apps on the marketplace
   https://github.com/marketplace/actions/issue-dashboard

https://github.com/marketplace?category=monitoring&query=Dashboard

3. Other analytics
   https://linearb.io/
   and others

Feedback loop - motivation of developers

Human's motivation is one of the most ambiguous things that is hard to manage.
Motivation in a part time distributed group is way more hard to manage than in an onsite team.

For us I'm considering these options and open to others:

• Maintain a highly friendly communication in the core team
• Do not set up rules that limit freedom of core team members
• Sometimes provide technically interesting bounties just for the motivation of developers
• Sometimes have interesting brainstorming sessions or lectures for developers
• Proactively get feedback from bounty hunters
• Consider gifts in UBQ and digital goods

[0x4007]

Bounty Program HUD

The UI and Chrome Extension part is trivial to implement, but the backend with the database and APIs I would personally be less efficient to implement (I specialize more on front-end)

Analytics Dashboard

I have a tendency to use a few tools for most of my needs. Part of why everything is on our GitHub lol

If you can first access the raw activity data (✅ GitHub API) and then a data visualization tool with filtering capabilities (❓ Google Sheets) then you can start executing. Consider exploring these two elements unless you find dedicated products for this sort of thing.

[0x4007]

Could be useful in the interim while we work on setting up the UbiquiBot for payouts of merged PRs. @sergfeldman is this something you can look into?

https://twitter.com/dariyaxyz/status/1528771526379524096?s=20&t=MNxSg9v8ExqFpYWs87hPIg

[0x4007]

To increase engagement in the Telegram chat, what if we have a bot that posts once a week (or maybe even once a day when volume increases in creating new issues) new bounties created in a list together in a single message.

For example:

"[Bounty title], [estimated time], and [dollar amount]."

...as a list of these.

We could also attach a second list highlighting the bounties that have increased in price.

[0x4007]

In anticipation of not great talent discovering and flooding the DevPool: we could make a level system where first time bounty hunters are only allowed to try the shortest bounties (<1 day) so that we could churn the bounties faster, meaning to expire them in a couple of days and free them up for somebody else to try, if we don't get satisfactory results.

[0x4007]

Proposal to rename Profit: to Priority: because sometimes profit is not always the main goal but priorities definitely guide where to spend more money!