Add some new scripts to package.json, to make npm run command easier

[pmario]

add some new scripts to package.json, to make npm run command easier

• relies on: Make pluginlibrary edition consistent with all other editions #8265

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
tiddlywiki5	✅ Ready (Inspect)	Visit Preview	Jun 27, 2024 6:28pm

[Jermolene]

Thanks @pmario I am very much in favour of improving the core package.json scripts.

[Jermolene]

Thanks @pmario these are powerful extensions, but do make the package.json file harder to read overall. I suppose that means that we'll need very good documentation here

[pmario]

The structure changed completely.

• We do have the basic commands:
  • start, test
• A wiki command, that is used with +plugins/tiddlywiki/filesystem +plugins/tiddlywiki/tiddlyweb plugin prefixes
• A cs command for "client server"
• server and server-external-js, because there "naming pattern" is completely different

pluginlibrary is missing, because we need to handle the PR: Make pluginlibrary edition consistent with all other editions #8265 first

multiwikiserver is missing since it does not have a tiddlywiki.info file.

share has no tiddlers/ directory so it's missing too - I don't know what to do with it

---

If I clone a new repo, one of the first things I usually do is run npm run to see what's going on.
The command creates this output. Which should be somewhat self explaining.

Since wiki:info is the first thing, we do get some help info, without running any commands. So for most users that will be enough info to get going. If they run npm run wiki they get an error -> which is OK

Everything needed is there now, to be selected with the mouse and "copy / pasted" into the command line.

A similar output is created for npm run cs, which can be used for editions that have a separated *-server edition.

@Jermolene -- what do you think?

[pmario]

Feedback from npm run cs

[Jermolene]

• A cs command for "client server"

I think that is too obscure. Can we use "server" instead?

• server and server-external-js, because there "naming pattern" is completely different

If we cater for external-js then perhaps we should also cater for lazy loading?

multiwikiserver is missing since it does not have a tiddlywiki.info file.

I think you've got a blank folder there because at some point in the past you've checked out the multi-wiki-support branch, and then weirdly when you check out a different branch, git seems not to remove the directory.

share has no tiddlers/ directory so it's missing too - I don't know what to do with it

That edition is just a wrapper around the "share" plugin, and doesn't need any tiddlers.

[pmario]

Just an info: Problems with missing tiddlywiki.info files has been solved with Fix tiddlywiki --editions command #8535

[pmario]

• The cs command has been renamed to edition
• The library command starts the local library server at port: 8888,
  • so it can be tested with a second server at port: 8080

npm run now shows this info

Possible commands are

• npm start ... starts tw5.com-server --listen
• npm test ... runs all tests

---

• npm run wiki --edition=<wiki name> ... this one starts wikis without a server edition
• npm run edition --edition=<edition name> ... this one starts wikis, that have a *-server edition
• npm run library ... builds all plugins and starts the library server on port:8888
• npm run server-external-js ... starts the corresponding edition
• npm run help ... runs node ./tiddlywiki.js --help ... to get all CLI info

[pmario]

Updated screenshot in last post. Moved help, lint and lint:fix up in the hierarchy

[pmario]

I wrote:

• server and server-external-js, because there "naming pattern" is completely different

Jeremy wrote:
If we cater for external-js then perhaps we should also cater for lazy loading?

I do have no idea how lazy loading works and how to set it up. @linonetwo can you make suggestions here?

@linonetwo ... In general, what do you think?

[linonetwo]

Sure, I use lazy-loading everyday. Simply add a rootTiddler will enable lazy loading

https://github.com/tiddly-gittly/TidGi-Desktop/blob/fe3a29eff1fb7e70307c2f3d29483681d535403d/src/services/wiki/wikiWorker/startNodeJSWiki.ts#L125

Try add

"server:lazy": "node ./tiddlywiki.js ./editions/server root-tiddler=$:/core/save/lazy-all --listen",

or this, so there are existing tiddlers:

"server:lazy": "node ./tiddlywiki.js ./editions/tw5.com-server --listen root-tiddler=$:/core/save/lazy-all",

[Jermolene]

@Jermolene ... I think this doc should be in the dev-edtion. right?

I think it makes sense to document them along with the other command line details in tw5.com.

[Jermolene]

In terms of which Node.js versions that we target, perhaps it would make sense to adopt the policy to target the oldest LTS version of Node.js, which would currently be Node.js 18. That's the version used by our CI tests, as it happens.

We could still have an npm script that uses --watch-path, though, since the scripts are so loosely coupled with the core. It would perhaps give people a reason to upgrade.

[pmario]

I think we should target the oldest active LTS version which would be Node.js 20.x.

The maintenance time for 18.x ends in April 2025.. If we start to switch version at that date we have no time left to deal with any problems in the next version.

Schedule see: https://nodejs.org/en/about/previous-releases#release-schedule

[pmario]

@Jermolene @saqimtiaz ... I did add npm run watch which starts the tw5.com-server edition.

The problem I see is, that the client is not notified, that there is a new server running. But the command seems to work

[pmario]

@Jermolene -- IMO this one can be merged for further testing. -- It only will affect developers -- So users should have no problems

[pmario]

Converted to draft -- It does not work for me anymore.

[pmario]

Sorry had a typo in the command line :/ Everything is OK

[Jermolene]

Hi @pmario there's a lot going on here, and the net effect is to make our package.json file much longer and more complex. It's not clear why some commands use colon as a separator, some use period, and some use dashes.

We should add some documentation for these npm commands, and transclude it into the readme shown on GitHub.

[pmario]

... It's not clear why some commands use colon as a separator, some use period, and some use dashes.

• The dot as a separator is a typo.

• The colons are used for :info text only

• The run commands have no separator

---

@Jermolene

server-external-js is an edition, which could be used with npm run edition --edition=external-js but it would be necessary to rename the directory external-js-server -- Should I rename it?

[linonetwo]

add some documentation for these npm commands, and transclude it

If do so, then there is no need to put them in fields like wiki:info. Devs will read the readme instead.

[pmario]

Hi @pmario there's a lot going on here, and the net effect is to make our package.json file much longer and more complex.

@Jermolene -- It seems your concern is, that this PR makes the package.json way to complex. That's because the edition and -server directory naming is not consistent. Some editions have it's own -server configuration for testing and some editions do not.

Having an eg: tw5.com and a tw5.com-server configuration is a huge advantage, since the -server test configuration can be significantly different to the published tw5.com directory, which contains the single file settings.

If every edition would have it's own -server configuration it would probably shorten the possible new lines in package.json by 30% or 40%

But that would mean, that the editions directory would contain a lot more subdirectories. The change would be backwards compatible, since the -server directories would be new.

The only directory which would need a name change is the server-external-js because it would need to be external-js-server and external-js

I did not suggest these changes since they would introduce many more edition subdirectories. So it would probably increase the complexity there. But imo it would be much more consistent, which is a good thing.

[pmario]

@Jermolene We only need a decision.

[pmario]

close this one. "merge master" did mess it up :/

[github-actions]

@pmario Signatures to the CLA must target the 'tiddlywiki-com' branch.

[Jermolene]

close this one. "merge master" did mess it up :/

It happens to me frequently too, maddening

[pmario]

@Jermolene -- I just found out that the changes I did create with this PR would have only worked with Windows :(

I'll create a new discussion with a proposal soon.

[Jermolene]

@Jermolene -- I just found out that the changes I did create with this PR would have only worked with Windows :(

Ouch. I think we hit the same block with some earlier experiments a few years ago. It suggests that we should prefer our tooling to be within TW itself. As you noted in another ticket, our --build command is similar to npm run. I've no objection to making the --build command more capable.

For example, at the moment we define commands as arrays of strings, but we could intersperse more complex JSON structures that for example permit any argument to be wikified. For example:

"build": {
	"encrypted": [
		"--password",
		{
			"type": "function",
			"text": "[<version>match[X]then[password]else[drowssap]]"
		},
		"--render", "$:/core/save/all",
		{
			"type": "wikified",
			"text": "myfile-<<version>>.html"
		},
		"text/plain",
		"--clearpassword"],

We could have interactive nodes that prompt the user for a string at run time.

[linonetwo]

That will be a lot of work. If everyone is using VSCode, we can provide vscode launch script, I saw it will also prompt for input, and have nice UI.

I prefer npm instead of build script, because I can have GUI out of box in vscode, so only need one click, for a fast write and test cycle.

[Jermolene]

That will be a lot of work

I think it's actually relatively simple. It doesn't touch any other subsystems. The basic functionality already exists and there wouldn't be any backwards compatibility concerns.

[Jermolene]

With respect to VSCode integration, I've no objection to introducing a VSCode-specific configuration file in the repo.

[Jermolene]

For example, at the moment we define commands as arrays of strings, but we could intersperse more complex JSON structures that for example permit any argument to be wikified. For example:

I've started an initial implementation in this branch: https://github.com/TiddlyWiki/TiddlyWiki5/tree/dynamic-build-commands

There is support for command tokens being wikified or being treated as a filter. I intend to add a way to prompt the user for a string. We might even explore extending this JSON language to permit setting variables that can be referenced in later tokens.

No doubt the documentation will be the hard part.