Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate smoya/multi-parser-js into asyncapi/parser-js #963

Closed
smoya opened this issue Mar 14, 2024 · 32 comments
Closed

Integrate smoya/multi-parser-js into asyncapi/parser-js #963

smoya opened this issue Mar 14, 2024 · 32 comments
Assignees
Labels
bounty AsyncAPI Bounty

Comments

@smoya
Copy link
Member

smoya commented Mar 14, 2024

Context

@smoya/multi-parser is a library that allows users to parse an AsyncAPI document and get an output model based on a desired Parser-API version. Additionally, it allows to convert models from one Parser-API version to another.

This tool is required by some AsyncAPI tools that use the Parser-JS and have to send the parsed document model to another library, which could be compatible with a different Parser-API version than the one used in that particular version of the Parser-JS.
You have several examples of usage here:

Current issues

1. 👤 Ownership

@smoya/multi-parser is a library hosted in a GH repository under my particular ownership (@smoya). Not a big issue per se, but I don't wanna be the only maintainer of such a tool. I thought on donating this to AsyncAPI org (TSC would need to vote first) but the reality is that won't be needed, see next point:

2. 🔃 Release is not in sync with Parser-JS

Every new Parser-JS release is not triggering an update on the @smoya/multi-parser. I agree I could configure a bot like Renovate (Dependabot doesn't allow NPM package aliases, and this package makes use of them) to try update those.
But still we have two issue with that situation:

  1. Whenever a new major Parser-API version gets released, the need of small additions into the @smoya/multi-parser code will be needed. See https://github.com/smoya/multi-parser-js/blob/main/src/parse.ts#L45-L53. Could we automate it somehow by changing the code? Perhaps yes, but not ideal and probably not trustable in the long term. At this moment, this includes manual work and be always careful when a new version goes out.
  2. For a certain period of time after a new release of Parser-JS, @smoya/multi-parser will be outdated and users of the library who also use Parser-JS as dependency could have runtime errors due to not using the same version of the models (different Parser-JS versions!). Of course, solved when Renovate updates in case it's not a major Parser-API version change.

The solution I propose

To integrate @smoya/multi-parser code into the Parser-JS repository, and to release two packages on each Parser-JS release:

  • @asyncapi/parser, the current one that is already happening.
  • @asyncapi/multi-parser, this a new one.

In that way, everything gets in sync, and also, by adding tests, we could ensure new versions of the Parser-JS are working with the @asyncapi/multi-parser package. The same when we adopt a new version of the Parser-API in the Parser-JS, support in @asyncapi/multi-parser will be added in the same PR.

That means, we should have a monorepo model in this repository.
Due to the fact @asyncapi/multi-parser has as dependency the Parser-JS but not only once, but once for each Parser-API version (atm 3 aliased Parser-JS deps pointing to different version of the Parser-JS), plus the fact the library will be used only by few particular libraries and most users won't need it ever, @asyncapi/multi-parser should be released completely as a separate package.

Examples of monorepo within AsyncAPI can be found here:

Some discussions in Slack about that topic:

BTW, since @smoya/multi-parser is under Apache 2.0 license, copying code right directly into the Parser-JS is completely legal, and since Parser-JS uses the same license, no need to specify any atribution of change on the license. So no need to even first do the donation to AsyncAPI GH org :) Additionally, I'm giving my written approval to do whatever is needed with that code so it can be integrated into the Parser-JS. Once that's done, and all dependants update to @asyncapi/multi-parser, I will completely archive @smoya/multi-parser in favor of @asyncapi/multi-parser.

cc @jonaslagoni @magicmatatjahu

@smoya smoya added the help wanted Extra attention is needed label Mar 14, 2024
@jonaslagoni
Copy link
Member

Makes sense to me 👍

@asyncapi-bot asyncapi-bot added the bounty AsyncAPI Bounty label Mar 18, 2024
@aeworxet
Copy link
Contributor

Bounty Issue's service comment

Text labels: bounty/2024-Q2, bounty/advanced, bounty/coding
First assignment to third-party contributors: 2024-03-22 00:00:00 UTC+12:00
End Of Life: 2024-08-31 23:59:59 UTC-12:00

@asyncapi/bounty_team

@Vishal2002
Copy link

@smoya I think you should ask the Asyncapi org to add this issue under GSOC 2024 idealist. It would be a good contribution to have in the parser-js.

@smoya
Copy link
Member Author

smoya commented Mar 20, 2024

@smoya I think you should ask the Asyncapi org to add this issue under GSOC 2024 idealist. It would be a good contribution to have in the parser-js.

This is part now of the AsyncAPI Bounty Program 2024 Q2. Additionally, the GSOC 2024 ideas submission period finished last month, and we got accepted.

@derberg
Copy link
Member

derberg commented Mar 20, 2024

@smoya if that works, I guess we can start thinking about pulling other projects inside parser-js for better maintainance and cooperation - like avro, protobuf, openapi and raml data parsers?

and then we can also work on something that @jonaslagoni brought up in past, that DX of using parser with schema parsers is hard, so we could enable publishing of parser-js on steroids, with schema parsers included by default (while still publishing parser alone of course, lightweight for web)

@smoya
Copy link
Member Author

smoya commented Mar 20, 2024

@smoya if that works, I guess we can start thinking about pulling other projects inside parser-js for better maintainance and cooperation - like avro, protobuf, openapi and raml data parsers?

It makes sense to me, yeah.

and then we can also work on something that @jonaslagoni brought up in past, that DX of using parser with schema parsers is hard, so we could enable publishing of parser-js on steroids, with schema parsers included by default (while still publishing parser alone of course, lightweight for web)

+1. I wasn't aware of this suggestion but i really like it. I've found myself involved in the issue of having to manually load the schema parsers all the time so I believe it will be so helpful for lot of users who just want everything to work at once.

@derberg
Copy link
Member

derberg commented Mar 20, 2024

+1. I wasn't aware of this suggestion but i really like it. I've found myself involved in the issue of having to manually load the schema parsers all the time so I believe it will be so helpful for lot of users who just want everything to work at once.

yeah, I see the same with semantic-versioning that at some point of time, decided to include in main package a number of default, more used plugins - so managing dependencies is now much much better

@Vishal2002
Copy link

I want to give a try to this issue and want to work on it.

@smoya
Copy link
Member Author

smoya commented Mar 20, 2024

Hi @Vishal2002, thanks for showing interest. Let's wait at least a day since more contributors could jump in as well.
As per the Bounty rules:

Assignment of the Bounty Issue on GitHub to users that fall under the second and third categories is performed not earlier than three calendar days after the addition of the GitHub label bounty according to GitHub's timestamp.
Source: https://github.com/asyncapi/community/pull/897/files#diff-25ecb20a61754c225d6511ca08d7e7c9a14b9ca5a93e89bd42331e51c9ebf26dR39

In your case, you fall under the third category.

@Toyin5
Copy link

Toyin5 commented Mar 20, 2024

@smoya I want to work on this issue

@smoya
Copy link
Member Author

smoya commented Mar 20, 2024

@smoya I want to work on this issue
Cool! 🚀

Same answer as the one I provided to @Vishal2002 in #963 (comment)

@derberg
Copy link
Member

derberg commented Mar 21, 2024

@smoya in case you don't get experienced candidates for this one, maybe you can wait for @ayushnau (maybe he will be interested) as he is going to work now on basically the same (with small differences) asyncapi/generator#1044 - so monorepo in generator. Once he finish it, he can then do the same here maybe. Just an idea, I did not even check with @aeworxet if it is possible but I guess yes 🤔 you do not have to decide today or tomorrow, you can decide in a month

@smoya
Copy link
Member Author

smoya commented Mar 21, 2024

@smoya in case you don't get experienced candidates for this one, maybe you can wait for @ayushnau (maybe he will be interested) as he is going to work now on basically the same (with small differences) asyncapi/generator#1044 - so monorepo in generator. Once he finish it, he can then do the same here maybe. Just an idea, I did not even check with @aeworxet if it is possible but I guess yes 🤔 you do not have to decide today or tomorrow, you can decide in a month

Yeah, good advise. I'm still evaluating each candidate individually in the meantime.

@smoya
Copy link
Member Author

smoya commented Mar 21, 2024

Whats your opinion on this @ayushnau ? If you are willing to work on this, do you believe you could become available at any time soon? When approx?

Thanks! 🙏

@ayushnau
Copy link
Contributor

@smoya I'm excited about the opportunity to collaborate on this with you, I would love to work on this. Looking forward to it!

@ayushnau
Copy link
Contributor

I will be available starting from next Monday. to pay full attention to this.

@smoya
Copy link
Member Author

smoya commented Apr 3, 2024

@ayushnau are you gonna work on this then? If so, when are you finally planning to do it? Thanks!

@ayushnau
Copy link
Contributor

ayushnau commented Apr 4, 2024

i am on it once it got assigned to me.

@smoya
Copy link
Member Author

smoya commented Apr 4, 2024

i am on it once it got assigned to me.

As spoken in Slack, you are working in asyncapi/generator#1044 in parallel. You mentioned you need a couple of weeks to finish such a task. Let's wait 2 weeks until I assign this issue to you.
Please @ayushnau ping me max on the 18 of April so I can assign this task to you.

@ayushnau
Copy link
Contributor

ayushnau commented Apr 4, 2024

sure will do that.

@ayushnau
Copy link
Contributor

@smoya I would love to work on this please assign this issue to me.

@smoya
Copy link
Member Author

smoya commented Apr 18, 2024

@asyncapi/bounty_team issue got assigned today to @ayushnau

@smoya smoya removed the help wanted Extra attention is needed label Apr 18, 2024
@aeworxet
Copy link
Contributor

Bounty Issue's Timeline

Complexity Level Assignment date (by GitHub) Start date (by BP rules) End date (by BP rules) Draft PR submission Final PR submission Final PR merge
Advanced 2024-04-18 2024-04-22 2024-06-14 2024-05-10 2024-05-31 2024-06-14
Please note that the dates given represent deadlines, not specific dates, so if the goal is reached sooner, it's better.
Keep in mind the responsibility for violations of the Timeline.

@smoya
Copy link
Member Author

smoya commented Jun 5, 2024

Due to the fact Ci has been broken since long ago (we were not aware), and considering it took me some time (days) figuring out the reason, I believe @ayushnau should request an extension of 1 week for this Bounty Program delivery.

cc @aeworxet

@aeworxet
Copy link
Contributor

aeworxet commented Jun 6, 2024

Upon request of the AsyncAPI Maintainer, who is responsible for the resolution of the Bounty Issue from the AsyncAPI's side (@smoya), all remaining target dates of the Bounty Issue's Timeline are extended by one calendar week.

Bounty Issue's Timeline Extended

Complexity Level Assignment date (by GitHub) Start date (by BP rules) End date (by BP rules) Draft PR submission Final PR submission Final PR merge
Advanced 2024-04-18 2024-04-22 2024-06-21 2024-05-10 2024-05-31 2024-06-21
Please note that the dates given represent deadlines, not specific dates, so if the goal is reached sooner, it's better.
Keep in mind the responsibility for violations of the Timeline.

@ayushnau
Copy link
Contributor

@aeworxet @smoya

I am writing to inform you of a recent change in our release process for the GitHub action. The release process, which was previously managed using semantic release, has now been transitioned to changeset. This unexpected shift has caused some delays, as I was not fully prepared for the new process. Consequently, I kindly request an extension of two weeks to accommodate this transition and ensure a smooth and accurate release.

Thank you for your understanding and consideration.

@smoya
Copy link
Member Author

smoya commented Jun 21, 2024

@aeworxet @smoya

I am writing to inform you of a recent change in our release process for the GitHub action. The release process, which was previously managed using semantic release, has now been transitioned to changeset. This unexpected shift has caused some delays, as I was not fully prepared for the new process. Consequently, I kindly request an extension of two weeks to accommodate this transition and ensure a smooth and accurate release.

Thank you for your understanding and consideration.

I agree the uncertainty nature of this issue, the knowns VS unknowns, and the several areas this issue affects (code, CI, CD) makes a difficult and, somehow, unpredictable one.
Even though we had a model to follow (Studio case), we (including me as maintainer) did assumptions wrongly; later found we were missing some changes such as the replacement of semantic-release by changesets (a change that it is not so obvious) and how that change affects our release pipeline.

Being honest, I had some doubts few days ago if this task should be given an extension or not, but atm I truly believe it should get it.

@aeworxet has the last words.

@aeworxet
Copy link
Contributor

Upon request of the Bounty Program Participant (@ayushnau), all remaining target dates of the Bounty Issue's Timeline are extended by two calendar weeks.

Bounty Issue's Timeline Extended

Complexity Level Assignment date (by GitHub) Start date (by BP rules) End date (by BP rules) Draft PR submission Final PR submission Final PR merge
Advanced 2024-04-18 2024-04-22 2024-07-05 2024-05-10 2024-06-21 2024-07-05
Please note that the dates given represent deadlines, not specific dates, so if the goal is reached sooner, it's better.
Keep in mind the responsibility for violations of the Timeline.

@smoya
Copy link
Member Author

smoya commented Jul 23, 2024

@ayushnau After merging #1036 and related PRs, the release of both packages happened. I tested the @asyncapi/parser (basic stuff) and seems to work, however, when running a simple test for the @asyncapi/multi-parser, the following errors appears:

(node:55793) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.
(Use `node --trace-warnings ...` to show where the warning was created)
/Users/smoya/dev/asyncapi-parser-example/node_modules/@asyncapi/parser/esm/index.js:1
import { DiagnosticSeverity } from '@stoplight/types';
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at wrapSafe (internal/modules/cjs/loader.js:1001:16)
    at Module._compile (internal/modules/cjs/loader.js:1049:27)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
    at Module.load (internal/modules/cjs/loader.js:950:32)
    at Function.Module._load (internal/modules/cjs/loader.js:790:12)
    at Module.require (internal/modules/cjs/loader.js:974:19)
    at require (internal/modules/cjs/helpers.js:93:18)
    at Object.<anonymous> (/Users/smoya/dev/asyncapi-parser-example/node_modules/@asyncapi/multi-parser/cjs/parse.js:6:17)
    at Module._compile (internal/modules/cjs/loader.js:1085:14)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)

@asyncapi asyncapi deleted a comment from asyncapi-bot Jul 23, 2024
@ayushnau
Copy link
Contributor

@smoya working on it. will update you once i fix it.

@smoya
Copy link
Member Author

smoya commented Aug 2, 2024

This issue is now technically solved. Thanks @ayushnau for your work!

Cc @aeworxet

@aeworxet
Copy link
Contributor

aeworxet commented Aug 2, 2024

After a set of delays that were beyond the control of the Bounty Program Participant
#1036 (comment)
#1036 (comment)
#1036 (comment)
#1036 (comment)
#1036 (comment)
#1036 (review)
#1036 (comment)
#963 (comment)

Bounty Issue Completed 🎉

@ayushnau, please go to the AsyncAPI page on Open Collective and submit an invoice for USD 400.00 with the expense title Bounty parser-js#963, tag bounty, and full URL of this Bounty Issue in the description.

@smoya smoya closed this as completed Oct 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bounty AsyncAPI Bounty
Projects
Status: Completed
Development

No branches or pull requests

8 participants