Skip to content

How to Hack on Calva

Jump to bottom Edit New page
Peter Strömberg edited this page Nov 3, 2021 · 50 revisions

Calva is a TypeScript project. Some things are written in ClojureScript. The ClojureScript code is compiled by shadow-cljs into a :node-library, which is required from the TypeScript code.

The Setup:

Start by saying hello in #calva at the Clojurians Slack. Then:

  1. Clone your fork of https://github.com/BetterThanTomorrow/calva
  2. Checkout the dev branch. This is where all Calva development happens.
  3. Open the project root directory in VS Code. (You are using VS Code and Calva, right?)
  4. Open an integrated Terminal and npm install

The dev process looks like so:

  1. From the VS Code Terminal menu, choose: Run Build Task… (ctrl+shift+b).
    1. Wait for it to
      1. Build the :node-library, :calva-lib, and run its tests
      2. Build the TypeScript code
      3. Start watching.
  2. (Optional, if you want to play with the cljs part of Calva) Jack in Calva to Calva itself.
    1. Run jack-in and select to start the :calva-lib build
    2. Wait until you see "Build completed" in the terminal for the shadow-cljs build (see below and related issue)
    3. After the build is completed, choose to connect to the :calva-lib cljs REPL.
  3. Start the extension in debug mode (the Extension Host): F5.
  4. Hack away. The watch tasks will rebuild the extension really quickly.
    • You'll find the output from the build task in the vscode Terminal view, and for the cljs part in its Jack-in terminal.
  5. Restart the extension host when you have new TS code to test (cljs code will hot-reload, just as you are used to).

About waiting for the build to complete:

Currently when jacking into Calva, when you start the :calva-lib build then also select the same build to connect to before the terminal shows that the shadow build completed, then open the extension host, things do not work, like evaluation. If you wait for the build to show completed in the terminal before selecting :calva-lib to connect to in the prompt, then open the extension host, things work as expected.

Tips

Some (one, at the moment) tips to make Calva hacking as smooth as possible.

Save time using the Copy Jack-in command

When working with the TypeScript code, you are back in the land where developers haven't yet discovered Interactive Programming. It is a cycle of recompiling, restarting, retesting over and over again. With VS Code and Calva recompile and restart are quick, but restarting can be prolonged if you need to also start the REPL of the project you use for retesting.

You can save quite a lot of time using the Copy Jack-in command line command together with Calva Connect to a Running REPL in the session where you run the development version of Calva. Then you don't need to wait for the REPL to start when you have restarted the debug extension host.

Note: You can still run the copied jack-in command line in the VS Code integrated terminal. VS Code will keep any terminal sessions during such reloads.

Debugging

Depending on what parts of Calva you are working with, the debug options look differently. Generally:

  • TypeScript: Use the debugger and the debug console. So, when you have started the extension in debug mode as per above, you go back to the Calva code base and place debug breakpoints which you then trigger in the debugged instance. Note that you have three types of breakpoints at your disposal, and you can also mix them somewhat. Logpoints are a bit like Clojure's tap>.
  • ClojureScript: Here you have full Interactive Programming powers. Evaluate a function to change its behaviour, use shadow-cljs hot-reload, tap> values out to the shadow-cljs inspector, inspect values flowing through the functions using inline defs (see this video for an example of this technique), or whatever is your favorite trick.

Code that does not integrate with VS Code can also be explorered from unit-tests:

  • TypesScript - src/extension-test/unit - Use the Mocha TestExplorer to get a super nice UI for this.)
  • ClojureScript - src/cljs-lib/test/calva - These tests run automatically every time you save a file. You can of course also run them using any Interactive Programming tricks you fancy.

See below for more on testing.

NB: A great Contribute to Calva opportunity.

If you are looking for ways to contribute to Calva, here is an area where you can make a huge difference: All too much of the TypeScript code is unnecessarily integrated with VS Code (imports vscode or operate on input that relies on vscode). Extracting such code out to modules that can be unit tested, and setting up some first unit tests, is a super opportunity to help the Calva project.

Some of the code can and should be moved over to the ClojureScript library. The major deciding factor here is wether the code uses any TokenCursor facilities, either directly or by proxy. We haven't found a good way to work with the TokenCursor from ClojureScript yet. (That's not to say there isn't a way, of course.)

Testing

Calva has four bodies of automatic testing, all of which are run as part of the Calva CI pipeline:

  1. Unit tests for the ClojureScript library, calva-lb
  2. Unit tests for the TypeScript code.
  3. Integration tests for the Calva extension.
  4. Unit tests for the tmLanguage Syntax grammar.

calva-lib

While developing, the calva-lib tests are run automatically as part of the watcher process. You only need to be concerned with the grammar tests if you are hacking on the grammar. We haven't written about how to run those, it involves using Atom, contact us and we'll show you.

TypsScript unit tests

You can run the TS unit tests in (at least) two ways:

  1. From the command line: npm run unit-test.
  2. Using the Mocha Test Explorer Extension.

It is recommended you use both methods.

The coverage is slim so far. Please consider helping the Calva project by adding test suites and adding to the existing ones.

Integration tests

These are just a stub right now. Please help in adding some meaningful tests.

Running the integration tests is done from the command line npm run integration-test. However, you need to not have VS Code running when you do it.

Syntax grammar

Calva has two sets of grammars.

  1. One is the tokeniation used for structural editing, navigation, rainbow parens, and various features in Calva. That is tested via the TS unit tests mentioned above.
  2. The other grammar is the tmLanguage file that VS Code uses for basic syntax highlighting.
Updating the tmLanguge grammar

The tmLanguage Clojure grammar originated in Atom and still needs Atom for the automated testing:

  1. Open the src/calva-fmt/atom-language-clojure directory in Atom.
  2. Run Package Specs
  3. Update the grammar cson (I do this in VS Code, because I love VS Code).
  4. In Atom, Run Package Specs

You probably need to update the specs too, depending on the change. Consider writing a test that catches if your change to the grammar would be removed.

For actually making the grammar work in VS Code, you need to convert the CSON grammar file to JSON:

  1. From the root folder of the Calva repo run npm run update-grammar. This updates the clojure.tmLanguage.json file used by VS Code.
  2. Restart the debug session to test the grammar in VS Code.

Please don't hesitate to holler at @pez on the Clojurians slack.

Editing Documentation

The Calva source for the Calva documentation, located at calva.io, is in the docs/site directory. It's built using MkDocs + MkDocs Material and is hosted by GitHub Pages.

If you're making changes to the docs that are not associated with code changes, make the changes on the master branch. If you're making doc changes associated with code changes, make them on the branch that contains the code changes.

  1. Install MkDocs + Material. From the Calva directory, run:

    pip install -r requirements.txt

    (Or your preferred method from https://squidfunk.github.io/mkdocs-material/getting-started/)

    If you get an error with the pip install method above, you may have an older version of python. If you install python 3.7 and run the following command, it should work:

    python3.7 -m pip install -r requirements.txt

  2. Edit mkdocs.yml and comment out the plugins: section:

    #plugins:
#  - social:
#      cards: !ENV [MKDOCS_SOCIAL_CARDS, false]
#      cards_color:
#        fill: "#db9550"
#        text: "#fafafa"

    Remember not to commit this change! It is temporary, while the Material for MkDocs maintainer is providing some solution. (Sorry, long story. 😄)

  3. Run the MkDocs server:

    mkdocs serve

  4. Visit http://localhost:8000

  5. Edit docs. It's all Markdown. When you save, the changes will be reloaded in the browser.

Note: The MkDocs Markdown is a bit pickier than GitHub's, so make sure to use four indents, put an empty line before bulleted lists, and so on. The VS Code extension markdownlint helps with following the appropriate style. And make sure to check the results at your local site.

Before Sending Pull Requests

Feel invited to send us a PR just as a way to get feedback. You don't need to wait until your code works or the change is completed. Work in progress is totally PR-able.

Make sure you are directing the pull request at the dev branch, unless you have reasons to target some other branch. Also, ”some other branch” is never master, which happens to be the default.

Also:

  1. Please strongly consider filing an issue for the thing you are fixing, whether it is a bug, a change to a feature, or a new feature. (Unless, of course, such an issue already exists.)
  2. If your PR is more than just fixing a typo, have your work in a feature branch. Since most PRs will be of a work-in-progress nature, you are welcome to follow our convention of naming our feature branches wip/<short-feature-title>.
  3. Update CHANGELOG.md under [Unreleased]. Link your entry to the issue you are fixing.
  4. In PR commit message, mention any issues you are addressing or fixing (maybe, as per 1.). Use the fixes/closes syntax, if applicable. (If you are not fixing the entire issue, say something like ”Addressing: #42”).
    • This will make people looking at that issue see that there is work being done, where it is carried out, and what is being changed. Also using the ”Fixes” syntax will make GitHub automatically close the issue, linking the commit that did it, when the commit is merged onto master. GitHub is awesome with this, let's leverage!
  5. Keep the changes focused on the thing you are adding/fixing. This will make it smoother for us to consider whether the changes should be merged or not. If you see room for refactoring and code style changes and such (there is plenty of this), you are welcome to suggest these changes in separate PR.

Please also see:

After Sending Pull Requests

When you have issued your pull request, it is best to ping us about it to catch our attention. The #calva channel of the Clojurians Slack is a good place for this ping, mention @pez, @cfhse and @ksthen even.

We will try to get back to you quickly with our assessment of whether this PR fits the Calva path. Depending on the nature of the change it might need some discussion.

We use Circle-CI to automatically build any updates to a PR. It will run the automated tests and give you feedback on the PR page on Github. It will also build a VSIX package for you. Every time you have pushed updates to the PR, please download it and smoke test it some. Our release process relies on that the VSIX packages are sanity checked along the development process. See Testing VSIX Packages.

To Get Your PR Merged

  1. The Calva team will have to agree it fits the path Calva is travelling along.
  2. All tests must pass. Please add some basic unit tests for the thing you are fixing.
    • Since much of the code is not factored for testing, this can be a bit tricky, so we do not always demand it.
  3. Any changes to README.md are done (or, if you prefer we write it, communicated to us).
  4. The VSIX package is tested by you.

Again, see the Pull Request Template for more.

See also: Testing VSIX Packages

Just One More Thing

Happy Calva Hacking! ❤️

Add a custom footer
Add a custom sidebar
Clone this wiki locally