Where do our Packages go?

[confused-Techie]

As we work our way through the all of the Orgs Repos, one question I feel we should try to get some shared confirmation on is where do our packages go?

As in, how do we import them into the programs that need them?

Right now we have a huge mix, packages being loaded locally from files (these will have to remain the same as it means they are bundled items), but then we have packages pointing to Atoms github, using legacy APIs, or non-public codeload APIs, we then have some referring to atom-community repos, do we want to fork these? Keep them as is? Then we even have packages being loaded via NPM, where they where published by one of the above.

I think we should collectively decide on a universal method, except in the instance of an edge case where something different should be required.

---

The options I see.

Ignoring the conversation of forking everything from atom-community/atom-ide-community, we could either decide on a proper API to load GitHub repos as dependencies, this has the advantage that it'll always work, and easily allows us to define tags. We could use CodeLoad APIs, but as those are not meant for public consumption, it's possible they may break.

Additionally we could publish to NPM. This would take a lot of work at first, but is the de-facto standard on this topic, so likely has some merit.

Then if we do, we likely would have to publish under a namespace since Atom has likely taken all of the non-namespaced names we would be attempting to us.

Obviously lets figure out what our best method would be, but personally my vote is for loading via Legacy GitHub Repo APIs. Meaning we know they will continue to work, and avoids the extra time it would take to manage and publish everything to NPM.

But what do you guys think?

[kaosine]

we could either decide on a proper API to load GitHub repos as dependencies, this has the advantage that it'll always work, and easily allows us to define tags. We could use CodeLoad APIs, but as those are not meant for public consumption, it's possible they may break.

Additionally we could publish to NPM. This would take a lot of work at first, but is the de-facto standard on this topic, so likely has some merit.

Then if we do, we likely would have to publish under a namespace since Atom has likely taken all of the non-namespaced names we would be attempting to us.

I recommend some version of these options. It's not far off from what I keep saying we should do. Could do things like @pulsar/essential, @pulsar/languages, and @pulsar/themes. Where anything for basic language and autocomplete go in the second (along with probably some useful snippets from each). I feel for testing this would probably be easiest and would do file detection to make sure we recognize basic necessary languages. While theoretically, the latter two would be bundled during build and wouldn't affect whether it builds or not...

[confused-Techie]

Heck, there's gotta be a way to set it up as the npm registry in GH even. Since I swear maurico tried it in AC but probably didn't have the right perms and everything for it to process right... 🤔

Thats very true, make our life as easy as possible and just publish here. I would be totally down with that personally.

Do you think that holds an advantage over referencing the GitHub repo, like we are doing a lot?

Because while this involves the downside of having to configure everything to publish, it does reduce the concern of getting versioning working just right and such

[kaosine]

I mean....just having it either in NPM's own registry or ours in gh's registry would make it easy to just run an update for the @latest or @next like we've done for the stuff we've needed on the website...

[confused-Techie]

Yeah fantastic point. Everything else can't update as well. Really just wanting to find a good justification for the extra effort. But awesome then, that really seems like the way to go for me

[kaosine]

I mean, they'd still be in GitHub, we just need to decouple the pulsar version imo from the version required by the addons. Since I still think we need to reset to v0.x right now seeing as we're unstable and definitely not as far along as we probably should be for a v1 or even a v2....(and it's weird to be at v1.63 for a rebirth imo)

[mauricioszabo]

One needs a personal access token to access the GitHub repo. So a person that don't want to configure this can't download the package

[DeeDeeG]

personally my vote is for loading via Legacy GitHub Repo APIs. Meaning we know they will continue to work, and avoids the extra time it would take to manage and publish everything to NPM.

I basically agree with this, and that could even be a short summary of my post below...

---

My thoughts (starting here with what I'm familiar with):

• I like GitHub "tarball URLs" (explained below) for being easy.
• I don't mind the thought of the npmjs.com registry as a second option which is perhaps a bit sleeker and more official-looking for users/package consumers? But going out of the way to move to npmjs.com seems like a workflow drag, so I would be perfectly happy to keep using GitHub "tarball URLs" for now or for the foreseeable future.
• I'm not super keen on the GitHub Registry because of hoops it makes you jump through (particularly needing Personal Access Tokens/PATs configured on each machine that installs the packages, and .npmrc configuration files in each respective repo that depends on the packages).

Overview of these options:

• The npm registry is a de-facto standard indeed. If we nab the @pulsar org on there, or @pulsar-edit if that's not available, then we can publish packages as like @pulsar/component or whatever and have it "just work"
  • Downsides: Need accounts and publishing workflows to deal with npmjs.com (package registry site). A bit of a hassle. Have to figure out and manage who has permissions and not forget our logins for one more site. Takes time to publish.
• Getting stuff from GitHub. We can do this as "git URLs" like https://github.com/pulsar-edit/pulsar-assets, but I find these slow and quirky in use. We can do this as "tarball URLs" such as "https://codeload.github.com/pulsar-edit/pulsar-assets/legacy.tar.gz/7e0d12ca1a7e47b26f497757bf07fe9a03f6e10b", which in my experience works much better.
  • Downside: "tarball URLs" can be long, look a bit clunky, actual version info is likely to be horizontally overflowed off-screen at the end of the URL in some terminals and text editors?
  • Downside: "git URLs" actually involve doing a git clone in a temporary dir and running a local npm install (fetches all devDependencies, too!), running all the local installation scripts for the package, then copying the result of that into node_modules, as far as I understand. Not ideal. Takes super long on some packages. Generally inefficient, especially on Windows where operations on many small files are quite slow.
  • Downside: GitHub could theoretically move or disable these tarball URLs as a pseudo-unofficial API, but so many projects depend on this, I don't expect them to change these URLs. I also hope they would at least make an announcement and give us time to migrate to some other solution.
• Using GitHub Packages to publish and host our packages...
  • This requires end-users to have a GitHub Personal Access Token (PAT) (and by extension a GitHub account) configured in their local machine's ~/.npmrc in order to even install these packages, and requires some .npmrc configuration locally in each project to make fetching from GitHub Packages instead of npmjs.com work. Supposedly
  • Upside: No accounts or permission wrangling needed for interfacing with npmjs.com. It's all just GitHub.com.
    • Downside: Not sure what the permission wrangling looks like for GitHub Packages???

[kaosine]

The npm registry is a de-facto standard indeed. If we nab the @Pulsar org on there, or @pulsar-edit if that's not available, then we can publish packages as like @pulsar/component or whatever and have it "just work"

• Downsides: Need accounts and publishing workflows to deal with npmjs.com (package registry site). A bit of a hassle. Have to figure out and manage who has permissions and not forget our logins for one more site. Takes time to publish.

Despite the downsides, I feel like this is the best option if we can't get the npm through GH working. Especially since I don't have much hope for GH's packages with how needlessly complicated GitHub makes things. We could make an org on npm that people can join if that's an issue you're seeing there. Which might be our best solution imo. Heck I wouldn't be surprised if for the updates on the package side we couldn't establish a workflow action to update the tags somehow and periodically have the main check for these and bump versions accordingly while generating new builds.

[kaosine]

In fact, and I'm not sure how deep it goes, but we can enable github linking from our profiles too and that may solve some issues?

[mauricioszabo]

Why don't we just link then as git dependencies, like we do with GitHub package? It's something suported by npm and we don't need to do anything else.

The only issue I have is that we'll need to remember to publish tags to each package all the time, otherwise we'll always clone the latest version (so, if we want to debug something that used to work in an older version, we will also clone the "latest" version of every package, not the version that was originally used to create the binary at the time)

[kaosine]

This is why I'd recommend making images for our development. Not only that, but maybe we could see if there's a way to prompt the user for which version they really want? Because the ideal theory would be, we'd automatically generate builds based off the latest, next and preview tags. This would handle my issue with the development cycle. I still really want to have those separated versions.

Which with the images/containers, I really wish we had someway to support this as where debuggers and compilers are run(and even where language- or ide- pull from for their autocomplete). Even allowing people to create, destroy, select from various places. I had forked a repo for this, but may have to restart. Since I would like to make one's for documentation, and ones for core, packaging and backend respectively to make it easier for people to spin up these, do the setup tasks, and run what they need to.

[mauricioszabo]

I actually have no idea what you meant...