feat/new-fee-column

[jaykayudo]

Resolves #161

[0x4007]

Please post QA

[jaykayudo]

@0x4007 I have added tests.
Also wouldn't this new change also imply that the html output mocks in the tests folder be modified as well?

[jaykayudo]

@0x4007 @gentlementlegen Please review this PR. I just made changes.

[0x4007]

Please post QA

[jaykayudo]

I have Posted QA as well

[jaykayudo]

QA added to fees.test.ts

[0x4007]

I mean run it and share a link to the GitHub comment

[jaykayudo]

I mean run it and share a link to the GitHub comment

Could you guide me through the process of running it on github. I have only been able to successfully run the tests locally.

[gentlementlegen]

I mean run it and share a link to the GitHub comment

Could you guide me through the process of running it on github. I have only been able to successfully run the tests locally.

You should install an app on your own organization or account, then run https://github.com/ubiquity-os/ubiquity-os-kernel locally so events are dispatched, and finally the plugin will run. Find more doc here.

[jaykayudo]

You should install an app on your own organization or account, then run https://github.com/ubiquity-os/ubiquity-os-kernel locally so events are dispatched, and finally the plugin will run. Find more doc here.

I have installed my app and ran the kernel but i am still confused on how to run this plugin. could you clarify this more for me? @gentlementlegen

[gentlementlegen]

@UbiquityOS Explain how to install and run the kernel, and install text-conversation-rewards plugin.

[ubiquity-os-beta]

! 413 Request Entity Too Large

[gentlementlegen]

@UbiquityOS Explain how to install and run the kernel, and install text-conversation-rewards plugin, in a short way.

[ubiquity-os-beta]

! 413 Request Entity Too Large

[gentlementlegen]

@jaykayudo You will need to add this plugin in your configuration so the kernel can call it, like so
https://github.com/ubiquity-os-marketplace/text-conversation-rewards?tab=readme-ov-file#plugin-configuration

Then on your repo once you close an issue, it will trigger a run.

[jaykayudo]

@gentlementlegen I am still having some issues so i am going to share the steps i took so please help me point where i might have made a mistake

• First i installed the app in my organisation
• I then forked my own copy of this repo in the organisation
• I ran my ubiquity-os-kernel locally where ran without problem
• I then added the configuration https://github.com/ubiquity-os-marketplace/text-conversation-rewards?tab=readme-ov-file#plugin-configuration to the feat/new-fee-column (not the development branch) in the .github/.ubiquity-os.config.dev.yml file.
• I opened and issue and closed it and nothing happened and my kernel ran an error

    status: 404,
    request: {
      method: 'GET',
      url:
  'https://api.github.com/repos/codebestiatestorg/.ubiquity-os/contents/.github%2F.ubiquity-os.config.dev.yml',
      headers: {
        accept: 'application/vnd.github.v3.raw',
        'user-agent': 'octokit-core.js/6.1.2
  Cloudflare-Workers',
        authorization: 'token [REDACTED]'
      },
      request: { hook: [Function: bound bound register] }
    },
    response: {
      url:
  'https://api.github.com/repos/codebestiatestorg/.ubiquity-os/contents/.github%2F.ubiquity-os.config.dev.yml',
      status: 404,
      headers: {
        'access-control-allow-origin': '*',
        'access-control-expose-headers': 'ETag, Link, Location,
  Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining,
  X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes,
  X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO,
  X-GitHub-Request-Id, Deprecation, Sunset',
        'content-length': '127',
        'content-security-policy': "default-src 'none'",
        'content-type': 'application/json; charset=utf-8',
        date: 'Fri, 25 Oct 2024 17:01:05 GMT',
        'referrer-policy': 'origin-when-cross-origin,
  strict-origin-when-cross-origin',
        server: 'github.com',
        'strict-transport-security': 'max-age=31536000;
  includeSubdomains; preload',
        vary: 'Accept-Encoding, Accept, X-Requested-With',
        'x-accepted-github-permissions': 'contents=read',
        'x-content-type-options': 'nosniff',
        'x-frame-options': 'deny',
        'x-github-api-version-selected': '2022-11-28',
        'x-github-media-type': 'github.v3; param=raw',
        'x-github-request-id':
  '5196:12F7EB:1DDA76:1F4DED:671BCED1',
        'x-ratelimit-limit': '5000',
        'x-ratelimit-remaining': '4977',
        'x-ratelimit-reset': '1729877827',
        'x-ratelimit-resource': 'core',
        'x-ratelimit-used': '23',
        'x-xss-protection': '0'
      },
      data: {
        message: 'Not Found',
        documentation_url:
  'https://docs.github.com/rest/repos/contents#get-repository-content',
        status: '404'
      }
    }
  }

✘ [ERROR] {

    type: 42,
    schema: {
      additionalProperties: false,
      type: 'object',
      properties: { plugins: [Object] },
      required: [ 'plugins' ],
      [Symbol(TypeBox.Kind)]: 'Object'
    },
    path: '/plugin',
    value: 'ubiquity-os/conversation-rewards',
    message: 'Unexpected property'
  }
  {
    type: 42,
    schema: {
      additionalProperties: false,
      type: 'object',
      properties: { plugins: [Object] },
      required: [ 'plugins' ],
      [Symbol(TypeBox.Kind)]: 'Object'
    },
    path: '/with',
    value: {
      logLevel: 'info',
      evmNetworkId: 100,
      evmPrivateEncrypted: 'encrypted-key',
      erc20RewardToken: '0xe91D153E0b41518A2Ce8Dd3D7944Fa863463a97d',
      dataCollection: { maxAttempts: 10, delayMs: 10000 },
      incentives: {
        requirePriceLabel: true,
        contentEvaluator: [Object],
        userExtractor: [Object],
        dataPurge: {},
        formattingEvaluator: [Object]
      }
    },
    message: 'Unexpected property'
  }
  Error decoding configuration for codebestiatestorg/conversation-rewards, will
  ignore. TransformDecodeCheckError: Unable to decode value as it does not match
  the expected schema

• I added the config as well in the development branch and still got the same error

[0x4007]

! 413 Request Entity Too Large

@sshivaditya

[Keyrxng]

@UbiquityOS Explain how to install and run the kernel, and install text-conversation-rewards plugin, in a short way.

This query afaik probably wouldn't work, it pulls the repo readme into the conversation to understand setup instructions. I've not seeded my DB and investigated (I assume reviewers have done this already and confirmed it's all good) what is being returned from the embedding-related search or what is stored in the DB so I may be mistaken here.

@jaykayudo here is a walkthrough by our LLM, but it looks like it's a yml syntax error you are experiencing. yml relies on indentation and it can be tricky some times. I too had issues with the configuration in this repo.

@gentlementlegen perhaps a helper CLI tool like kernel-telegram has would be super helpful here?

[ubiquity-os-beta]

! 413 Request Entity Too Large

[gentlementlegen]

The plug-in should not have run on a quoted comment also.

What I've seen is that it tried to diff the dist folders which contain the compiled program (around 15 mb of code) which triggered this error.

[0x4007]

Perhaps this strategy is valid:

1. We check the diff size of every file
2. We tokenize and estimate our max payload size from smallest file to largest file changes
3. We remove starting from the largest file changes and keep working our way down until we think it can fit within the limits.

Basically this strategy would start by excluding dist and likely other large changes like lock file etc. it will be able to include tens of small files dynamically though.

[sshivaditya2019]

@UbiquityOS Explain how to install and run the kernel, and install text-conversation-rewards plugin.

[ubiquity-os-beta]

Installing and Running the UbiquityOS Kernel with the text-conversation-rewards Plugin

To set up the UbiquityOS kernel and install the text-conversation-rewards plugin, follow the steps outlined below. This guide assumes you have basic knowledge of using Git, Node.js, and your terminal or command prompt.

---

Prerequisites

Before proceeding, ensure you have the following installed on your system:

1. Node.js (v14 or later): Download and install Node.js
2. Git: Download and install Git
3. Yarn (optional but recommended): Install Yarn

---

1. Clone the UbiquityOS Kernel Repository

Begin by cloning the main kernel repository from GitHub.

git clone https://github.com/ubiquity-os/kernel.git

Navigate to the cloned directory:

cd kernel

---

2. Install Kernel Dependencies

Install the necessary dependencies using either npm or yarn.

Using npm:

npm install

Using yarn:

yarn install

---

3. Configure Environment Variables

Create a .env file to store your environment-specific configurations. You can use the provided .env.example as a template.

cp .env.example .env

Open the .env file in your preferred text editor and populate it with the required values:

# Example .env file

SUPABASE_URL=your_supabase_url
SUPABASE_KEY=your_supabase_key

ETHEREUM_NETWORK_ID=your_ethereum_network_id
ETHEREUM_PRIVATE_KEY=encryption_of_your_private_key

OPENAI_API_KEY=your_openai_api_key

PERMIT_FEE_RATE=20

Notes:

• Supabase: Used for backend services. Ensure you have a Supabase account and obtain your project URL and API key.
• Ethereum: Configure your Ethereum network ID and securely store your private key. Follow the plugin configuration guidelines for encrypting your private key.
• OpenAI: If leveraging AI features, obtain an API key from OpenAI.
• Permit Fee Rate: Adjust the PERMIT_FEE_RATE as needed (e.g., 20 for 20%).

---

4. Build the Kernel

Compile the TypeScript code and prepare the application for running.

Using npm:

npm run build

Using yarn:

yarn build

---

5. Run the Kernel

Start the kernel application.

Using npm:

npm start

Using yarn:

yarn start

The kernel should now be running. You can verify its status by checking the console logs or accessing the relevant endpoints as configured.

---

6. Install the text-conversation-rewards Plugin

The text-conversation-rewards plugin enhances the kernel with reward functionalities based on text interactions. Follow these steps to install and configure it.

a. Navigate to the Plugins Directory

Assuming the kernel has a designated plugins directory:

cd plugins

b. Clone the Plugin Repository

Clone the text-conversation-rewards plugin into the plugins directory.

git clone https://github.com/ubiquity-os-marketplace/text-conversation-rewards.git

c. Install Plugin Dependencies

Navigate to the plugin directory and install its dependencies.

Using npm:

cd text-conversation-rewards
npm install

Using yarn:

yarn install

d. Configure the Plugin

Create a configuration file for the plugin, typically config.yml, based on the provided specification.

# Example config.yml

plugin: ubiquity-os/conversation-rewards
with:
  logLevel: "info"
  evmNetworkId: 100
  evmPrivateEncrypted: "your_encrypted_private_key"
  erc20RewardToken: "0xe91D153E0b41518A2Ce8Dd3D7944Fa863463a97d"
  dataCollection:
    maxAttempts: 10
    delayMs: 10000
  incentives:
    requirePriceLabel: true
    contentEvaluator:
      openAi:
        model: "gpt-4"
        endpoint: "https://api.openai.com/v1"
      multipliers:
        - role: [ISSUE_SPECIFICATION, PULL_AUTHOR, PULL_ASSIGNEE, PULL_COLLABORATOR, PULL_CONTRIBUTOR]
          relevance: 1
    userExtractor:
      redeemTask: true
    dataPurge: {}
    formattingEvaluator:
      wordCountExponent: 0.85
      multipliers:
        - role: ["ISSUE_SPECIFICATION"]
          multiplier: 1
          rewards:
            html:
              br: { score: 0, countWords: true }
              code: { score: 5, countWords: false }
              # ... other HTML elements
            wordValue: 0.1
        # ... other roles
    permitGeneration: {}
    githubComment:
      post: true
      debug: false

Important Configuration Parameters:

• evmPrivateEncrypted: This should be your encrypted Ethereum private key. Follow the Encryption Guide to securely encrypt your key.
• erc20RewardToken: The ERC20 token address used for rewards.
• OpenAI Configuration: Ensure the OpenAI API details are correctly set for AI-driven features.

e. Encrypting the evmPrivateEncrypted Parameter

To securely store your Ethereum private key, follow these steps:

1. Obtain Your GitHub Owner ID:

   Replace GITHUB_PAT_TOKEN with your GitHub Personal Access Token.

   curl -H "Accept: application/json" -H "Authorization: token GITHUB_PAT_TOKEN" https://api.github.com/orgs/ubiquity

   Note the id from the response.

2. Generate an Encrypted Key:

   Visit Keygen.Ubq.fi and follow the instructions:

   • Click "Generate" to create a new x25519_PRIVATE_KEY.
   • Input your Ethereum private key (without the 0x prefix) and GitHub Owner ID in the format PRIVATE_KEY:GITHUB_OWNER_ID into the PLAIN_TEXT input.
   • Click "Encrypt" to obtain the CIPHER_TEXT.

3. Update the Plugin Configuration:

   Set the evmPrivateEncrypted field in your config.yml with the encrypted value obtained.

4. Set Environment Variables:

   Add the X25519_PRIVATE_KEY to your GitHub repository secrets to allow the plugin to decrypt the private key.

---

7. Finalize Installation

After configuring the plugin, navigate back to the root directory, rebuild the kernel to include the new plugin, and restart the application.

cd ../../
npm run build
npm start

---

8. Verify the Setup

Ensure that both the kernel and the text-conversation-rewards plugin are running correctly:

• Logs: Check the console or log files for any errors or confirmation messages.
• Functionality: Test the reward functionalities by interacting through GitHub issues, pull requests, or other integrated platforms.

---

Additional Recommendations

• Testing: Run the test suite to ensure everything is functioning as expected.

  npm run test

• Code Quality: Maintain code quality using ESLint and Prettier.

  npm run lint
  npm run format

• Continuous Integration: Ensure GitHub Actions workflows are properly set up for automated testing and deployment.

• Documentation: Refer to the UbiquityOS Documentation and the text-conversation-rewards Plugin Documentation for more detailed information and advanced configurations.

By following these steps, you should have a fully functional UbiquityOS kernel integrated with the text-conversation-rewards plugin, enhancing your project's interaction and reward mechanisms.

[0x4007]

Pretty solid. Looks like very minor url hallucinations. Good job. Based on your reactions seems like that wasn't the output you expected.

[sshivaditya2019]

Pretty solid. Looks like very minor url hallucinations. Good job. Based on your reactions seems like that wasn't the output you expected.

I expected it to be a lot less verbose. It still dumps out the entire thing.

[sshivaditya2019]

@UbiquityOS Could you check the file src/parser/permit-generation-module.ts file and explain the changes ?

[sshivaditya2019]

@UbiquityOS Could you check the file src/parser/permit-generation-module.ts file and explain the changes ?

No output because the API calls are rate-limited.

[0x4007]

We should throw an error and comment that. I thought this was already the default behavior

[sshivaditya2019]

We should throw an error and comment that. I thought this was already the default behavior

Add Comment to the issue also got rate limited, so it could not log the error to the comment.

[sshivaditya2019]

@UbiquityOS Could you check the file src/parser/permit-generation-module.ts file and explain the changes ?

[ubiquity-os-beta]

Certainly! Here's a detailed overview of the changes made to the src/parser/permit-generation-module.ts file in Pull Request #161:

1. Fee Rate Calculation Adjustment

Before the Change:

result[key].feeRate = feeRateDecimal.toNumber();

• Issue: The feeRate was being set directly from feeRateDecimal, which represented the fee as a decimal (e.g., 0.2 for 20%). This caused confusion in subsequent calculations and displays, where a feeRate of 20% was incorrectly showing as 20 instead of 0.2.

After the Change:

result[key].feeRate = new Decimal(env.PERMIT_FEE_RATE).toNumber();

• Resolution: The feeRate is now directly assigned from the environment variable PERMIT_FEE_RATE, which is expected to be a percentage value (e.g., 20 for 20%). This ensures that the fee rate is consistently represented as a percentage throughout the application.

2. Impact on Reward Calculations

• Original Logic:

  • The reward after fee was calculated by multiplying the feeRateDecimal with the original reward. However, assigning the fee directly to total led to incorrect fee deductions.

• Updated Logic:

  • By setting the feeRate as a percentage, the calculations are clearer and more accurate. The total now correctly reflects the amount after subtracting the fee, ensuring that rewards are calculated based on the intended fee structure.

3. Enhancements in Test Outputs

Changes in tests/__mocks__/results/output.html:

• Before:

  <tr><td>Issue</td><td>Task</td><td>1</td><td>400</td></tr>

• After:

  <tr><td>Issue</td><td>Task</td><td>1</td><td>400</td><td>-</td></tr>

• Addition: A new <th>Fee</th> column was added to the contributions overview table to display fee-related information clearly.

4. Rationale Behind the Changes

• Clarity: Displaying the feeRate as a percentage (20%) instead of a decimal (0.2) makes the information more intuitive and aligns with common financial representations.

• Accuracy: Ensuring that feeRate is correctly represented prevents miscalculations in reward distributions and maintains the integrity of financial transactions within the system.

• Consistency: By sourcing the feeRate from an environment variable (env.PERMIT_FEE_RATE), it allows for easier configuration and consistency across different environments (development, staging, production).

5. Additional Considerations

• User Interface: Wrapping the <a/> tags with spaces was also addressed to ensure proper spacing in HTML outputs, enhancing readability.

• Future Maintenance: These changes pave the way for more accurate data handling and simplify future updates related to fee calculations and displays.

Summary

The primary modification in permit-generation-module.ts ensures that the feeRate is accurately represented and calculated as a percentage, enhancing both the clarity and correctness of fee-related operations. Additionally, corresponding updates in test outputs and HTML formatting improve the overall reliability and user experience of the application.

If you have any further questions or need additional clarification on these changes, feel free to ask!

[sshivaditya2019]

I think we did hit some sort of Secondary Rate Limit for GQL Queries or something.

[0x4007]

I wonder if we can fallback to github actions bot auth in extreme situations to comment the error

[sshivaditya2019]

I wonder if we can fallback to github actions bot auth in extreme situations to comment the error

I agree with this, for extreme situations, the kernel should be be able to log a message/comment from where the even was invoked ?

[ubiquity-os-beta]

@jaykayudo, this task has been idle for a while. Please provide an update.