Upgrade to LTS 22.26 GHC 9.6.5 #5142

ChrisPenner · 2024-06-27T01:19:27Z

Overview

Upgrades to LTS 22.26; GHC 9.6.5

Recommended you update your local GHCup to: HLS 2.9.0.0 and Stack 2.15.5

Does someone who knows nix want to update the flake as well? It seems ghc 965 isn't being found, I'm sure it's an easy fix for someone who knows how it works 🙃

Implementation notes

Updates the LTS
Removes redundant applicative updates
Replaces use of NanoID lib with secure crypto instead, just cuz the NanoID lib has bitrotted
Move from x509-* to crypton-x509-* libs since HVR quit Haskell
Bump fuzzyfind
Removes a few custom pinnings now that they've been added to the snapshot

Interesting/controversial decisions

Not really

Test coverage

Existing tests

Loose ends

Should I fix the JWT lib deprecations now?

unison-share-projects-api/src/Unison/Share/API/Hash.hs

aryairani · 2024-06-27T02:46:55Z

Does someone who knows nix want to update the flake as well? It seems ghc 965 isn't being found, I'm sure it's an easy fix for someone who knows how it works 🙃

cc @sellout

aryairani · 2024-06-27T02:48:40Z

.github/workflows/bundle-ucm.yaml

+        with:
+          stack-version: 2.15.5


we can also change the default on unisonweb/actions/stack/install if we like that better.

Yeah, I figured it was a bit of a chicken-and-egg thing. TBH I kind of like specifying here because it's easy to see what it's using and easy to bump again in the future 😄

I'm glad you added it as an arg for that step.

Yeah, I like specifying it in this project too; I just wish we didn't have to specify it 4 times in this project, but maybe that's fine. Replace All.

You can read JSON in GH workflows, and also in Nix (I don’t know if there are other places we need versions).

So, we could write a dependency-versions.json file or something that the flake can read into its versions attrSet, and that workflows can read as well. I’ve been wanting to extract it from the flake anyway.

aryairani · 2024-06-27T02:52:37Z

Nice!

On testing, could you smoke test that the new file watch code works locally

sellout · 2024-06-27T13:08:48Z

Does someone who knows nix want to update the flake as well? It seems ghc 965 isn't being found, I'm sure it's an easy fix for someone who knows how it works 🙃

I would normally agree with you, but haskell.nix (which is still used a lot by our flake) doesn’t support much in the way of GHC. But I’ll give it a shot.

ChrisPenner · 2024-06-27T19:40:47Z

contrib/cabal.project

Contributed by neduard for keeping the cabal build up to date. I haven't tested this myself.

ChrisPenner · 2024-06-27T19:41:11Z

flake.nix

@@ -27,10 +27,10 @@
    ]
    (system: let
      versions = {
-        ghc = "928";
+        ghc = "965";


This change doesn't actually work.

ChrisPenner · 2024-06-27T19:41:38Z

lib/unison-sqlite/src/Unison/Sqlite/Connection.hs

@@ -172,7 +172,7 @@ execute conn@(Connection _ _ conn0) sql@(Sql s params) = do
 --
 -- This function does not support parameters, and is mostly useful for executing DDL and migrations.
 executeStatements :: Connection -> Text -> IO ()
-executeStatements conn@(Connection _ _ (Sqlite.Connection database)) sql = do
+executeStatements conn@(Connection _ _ (Sqlite.Connection database _tempNameCounter)) sql = do


This type just changed.

aryairani · 2024-06-28T16:58:57Z

@ChrisPenner Ok excellent, thanks. In that case, I'm just holding to sort out the Nix questions, which we'll try to do later this afternoon. I'll start a Discord thread to answer the question of which versions we should be pinning.

aryairani · 2024-07-06T18:00:01Z

@sellout "No space left on device" for nix-dev-cache.yaml on Ubuntu 😬
According to https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories, the disk size is 14GB.
One can pay for bigger runners (debatable), but not on the free/oss plan.

sellout · 2024-07-08T04:45:41Z

"No space left on device" for nix-dev-cache.yaml on Ubuntu 😬

A re-run succeeded. I grabbed the log archive before and after. I’m guessing it needed to build something the first time that it didn’t the second¹, and that pulled in enough transitive deps to push us over the limit.

Since this job builds the devShells as well as the Cabal packages, it should contain all the dependencies (tooling & Haskell libs) needed by Unison no matter which Unison packages need to be rebuilt in a given PR, but we could just be very close to the limit. We could probably break this into multiple jobs, so one makes sure all the dependencies are built, and a second only ever builds Unison packages.

On the plus side, even if this job fails, it will at least push what it’s finished to the cache, so a re-run is more likely to succeed (as it just did) … but we definitely don’t want to be manually re-running the job on every PR. And pinning certain packages (GHC, HLS, etc.) in Cachix can make sure they don’t get GCed.

I checked – it didn’t try to build any GHCs 😅 ↩

aryairani · 2024-07-08T14:49:09Z

So we'll plan to merge this after #5041; I'm going to clear the approval so I don't merge it by accident.

waiting for #5041

ChrisPenner · 2024-07-08T17:35:55Z

@aryairani I checked out the conflicts with #5041 and I'm not too concerned, nothing major; that is, assuming Ormolu has already been re-run on this branch right?

sellout · 2024-07-08T17:46:19Z

assuming Ormolu has already been re-run on this branch right?

It hasn’t – it appears that in general CI only checks Ormolu against files that were modified in the PR (which is good), and at least some things had slipped through in the period that CI wasn’t running Ormolu¹. But I do think it’s good practice to run Ormolu all at once when we upgrade it.

It’s just unfortunate that it’s difficult to separate upgrading Ormolu from the rest of this PR. I can run Ormolu as another commit on this PR, or we can do a separate PR immediately after this. I also figured since we were waiting on #5041, that running Ormolu after we merge that would be prudent.

And since I couldn’t figure out where I linked this before, here is the effect of running Ormolu on top of this PR: sellout@694b065

I’ve noticed things like reordered imports that I didn’t touch when I format some files locally, etc. ↩

aryairani · 2024-07-08T20:27:13Z

@ChrisPenner You can be in charge of merging this PR when you are ready.

There is also an ormolu Github action set up you can run when ready, which hopefully does the right thing; cc @sellout

ChrisPenner · 2024-07-08T22:25:53Z

Ah, thanks Greg, yeah we can wait then. I'm getting pretty close with project roots now anyways :)

neduard · 2024-07-12T22:09:15Z

Hey @ChrisPenner , I was trying out this branch and I needed some changes from trunk so I had a go at merging it in & fixing the conflicts: neduard#3 - figured you'll probably encounter the same so it might be worth sharing in case it saves you a bit of time. As before feel free to either cherrypick or simply use as a reference.

Something worth pointing out is that I had to revert #5192 since numerals requires text (>=0.11 && <1.3) but LTS 22.26 has text 2.0.2 ( cc @aryairani ), but other than that the changes were minimal 🚀

aryairani · 2024-07-13T03:26:32Z

Hey @ChrisPenner , I was trying out this branch and I needed some changes from trunk so I had a go at merging it in & fixing the conflicts: neduard#3 - figured you'll probably encounter the same so it might be worth sharing in case it saves you a bit of time. As before feel free to either cherrypick or simply use as a reference.

Something worth pointing out is that I had to revert #5192 since numerals requires text (>=0.11 && <1.3) but LTS 22.26 has text 2.0.2 ( cc @aryairani ), but other than that the changes were minimal 🚀

@neduard I used allow-newer-deps: [numerals] for our stack build, because https://github.com/roelvandijk/numerals wanted containers (<6), so it probably accepted text-2.0.2 due to that flag, before the error was even on my radar. The author of numerals invites a PR with updated bounds, but I didn't have the time/knowledge to do it; but maybe you'd want to take a stab at it since you're more familiar with cabal than I am.

Otherwise, is there a similar directive to cabal to ignore the bounds on numerals (only)? I don't really want to revert #5192.

neduard · 2024-07-13T09:31:32Z

That's a much better shout @aryairani , thanks! I was so focused on getting it to build that I didn't think of other ways of solving it. I've raised the PR upstream and in the meantime worked around it by adding the following to contrib/cabal.project

allow-newer:
  numerals:text,
  numerals:containers

aryairani · 2024-07-15T11:59:29Z

That's a much better shout @aryairani , thanks! I was so focused on getting it to build that I didn't think of other ways of solving it. I've raised the PR upstream and in the meantime worked around it by adding the following to contrib/cabal.project
allow-newer:
  numerals:text,
  numerals:containers

Okay great — neduard#3 is merged though, is that still where to look?

sellout · 2024-07-15T15:51:23Z

worked around it by adding the following to contrib/cabal.project

Thanks, @neduard. We really need to get a Cabal GitHub check into CI (even if it’s not required). It should generally be trivial to keep it in sync, but easy to overlook without a reminder.

aryairani · 2024-07-15T17:28:16Z

worked around it by adding the following to contrib/cabal.project

Thanks, @neduard. We really need to get a Cabal GitHub check into CI (even if it’s not required). It should generally be trivial to keep it in sync, but easy to overlook without a reminder.

Yeah, the Cabal Github check is missing a) because we didn't have anyone to maintain the .cabal file (no longer true) and b) because it would have doubled the resources needed to run CI (not fully true anymore, but not a non-issue either). I guess we could run cabal build on linux only; wouldn't have run tests. We should have some sort of build cache.

sellout · 2024-07-15T17:41:31Z

Yeah, the Cabal Github check is missing a) because we didn't have anyone to maintain the .cabal file (no longer true) and b) because it would have doubled the resources needed to run CI (not fully true anymore, but not a non-issue either). I guess we could run cabal build on linux only; wouldn't have run tests. We should have some sort of build cache.

I opened #5225 so we could avoid conflating this discussion (my apologies for starting it here), and so we have somewhere to point when it comes up again.

neduard · 2024-07-15T18:49:29Z

@ChrisPenner @aryairani #5224 ready for merge into this one (or directly into trunk if you so wish) 😊 - please have a look at the description as there's a couple of things I'm unsure regarding updating a few transcripts and the jit tests potentially being cached but still requiring a couple of small updates - hoping you'll know more!

Reviewing the last merge into unisonweb#5142, I noticed some duplicated and out- of-date information. This brings things up-to-date and slightly reduces the duplication. - bumped Ormolu used by ci.yaml from 0.5.2.0 to 0.7.2.0, to match flake.nix - removed Markdown that claimed ci.yaml was using Ormolu 0.5.0.1 - moved description from `base-codebase` in Markdown to comment on `runtime_tests_codebase` in ci.yaml (and updated it to refer to builtin-tests/interpreter-tests.md instead of builtin-tests/base.md) - removed `unison_src_test_results` as it’s no longer managed as a single variable - moved other comments from Markdown to ci.yaml - added Markdown recommending to look in ci.yaml for specifics - rearranges the order of vars in ci.yaml to match the order they were presented in Markdown One thing I wasn’t sure how to map over: Markdown claims Racket 8.7 is used in CI, but ci.yaml doesn’t mention any Racket version.

With unisonweb#5142, Ormolu was upgraded from 0.5.2.0 to 0.7.2.0. This formats the codebase to avoid spurious formatting comingled in other commits. Almost all of the changes are simply wrapping single constraints in parens, like ```diff -hashBranch :: forall m. Monad m => Branch m -> m BranchHash +hashBranch :: forall m. (Monad m) => Branch m -> m BranchHash ``` There is also some reordering of language pragmas and imports, indentation correction (some of which gets precedence wrong), and switching some Haddock from `-- ^` to `-- |` .

ChrisPenner added 8 commits June 26, 2024 17:19

Mostly Upgrade LTS to ghc 9.6.5

2d1bf19

Temporarily ignore deprecations from jose

61b2666

Swap from x509 to crypton-x509 since old packages are abandoned

4a9b358

Fix redundant liftA2 imports

61e0522

Fix servant request stuff

745660d

Use random token rather than NanoID

a7f234a

Update opt-parse help

f3d4455

Upgrade watch expression stuff

7538111

ChrisPenner commented Jun 27, 2024

View reviewed changes

unison-share-projects-api/src/Unison/Share/API/Hash.hs Show resolved Hide resolved

ChrisPenner added 3 commits June 26, 2024 18:30

Clean up stack.yaml pins

b85bea8

Bump stack version in CI

87e15d7

Attempt to bump nix flake

06e730d

ChrisPenner force-pushed the cp/ghc-upgrade branch from 28d9375 to 06e730d Compare June 27, 2024 01:39

aryairani reviewed Jun 27, 2024

View reviewed changes

neduard mentioned this pull request Jun 27, 2024

[DO NOT MERGE] GHC 964 cabal compilation #5096

Closed

neduard added 3 commits June 27, 2024 17:03

lsp 2.3.0.0

c97d93e

Use GHC.IsList

a35d706

cleanup cabal

95a0eb9

neduard mentioned this pull request Jun 27, 2024

Update lsp version and add cabal fixes #5148

Merged

ChrisPenner added 2 commits June 27, 2024 11:53

Merge neduard's lsp fixes

c3fe983

update transcripts

5dcc096

ChrisPenner force-pushed the cp/ghc-upgrade branch from b844f1d to 5dcc096 Compare June 27, 2024 18:53

ChrisPenner added 2 commits June 27, 2024 12:16

Remove Primes from module names

e94f870

Resolve conflicts with trunk

563e148

ChrisPenner commented Jun 27, 2024

View reviewed changes

Merge pull request #5155 from sellout/cp/ghc-upgrade

ecf5fe1

sellout mentioned this pull request Jul 8, 2024

Add IOG cache to substituters #5131

Closed

sellout mentioned this pull request Jul 9, 2024

Pin Haskell tool versions for VS Code #5196

Closed

sellout mentioned this pull request Jul 15, 2024

test Cabal in CI #5225

Open

ChrisPenner mentioned this pull request Jul 15, 2024

Merge newest trunk in "Upgrade to LTS 22.26 GHC 9.6.5" #5224

Merged

aryairani merged commit 6ed06f3 into trunk Jul 15, 2024
35 checks passed

aryairani deleted the cp/ghc-upgrade branch July 15, 2024 20:48

sellout mentioned this pull request Jul 15, 2024

Avoid duplicating YAML info in Markdown #5227

Merged

sellout mentioned this pull request Jul 19, 2024

Reformat all Haskell code #5239

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade to LTS 22.26 GHC 9.6.5 #5142

Upgrade to LTS 22.26 GHC 9.6.5 #5142

ChrisPenner commented Jun 27, 2024 •

edited

Loading

aryairani commented Jun 27, 2024

aryairani Jun 27, 2024 •

edited

Loading

ChrisPenner Jun 27, 2024

aryairani Jun 28, 2024

sellout Jun 28, 2024

aryairani commented Jun 27, 2024

sellout commented Jun 27, 2024

ChrisPenner Jun 27, 2024 •

edited

Loading

ChrisPenner Jun 27, 2024

ChrisPenner Jun 27, 2024

aryairani commented Jun 28, 2024

aryairani commented Jul 6, 2024

sellout commented Jul 8, 2024

aryairani commented Jul 8, 2024

ChrisPenner commented Jul 8, 2024

sellout commented Jul 8, 2024

aryairani commented Jul 8, 2024

ChrisPenner commented Jul 8, 2024

neduard commented Jul 12, 2024 •

edited

Loading

aryairani commented Jul 13, 2024 •

edited

Loading

neduard commented Jul 13, 2024

aryairani commented Jul 15, 2024

sellout commented Jul 15, 2024

aryairani commented Jul 15, 2024

sellout commented Jul 15, 2024

neduard commented Jul 15, 2024 •

edited

Loading

Upgrade to LTS 22.26 GHC 9.6.5 #5142

Upgrade to LTS 22.26 GHC 9.6.5 #5142

Conversation

ChrisPenner commented Jun 27, 2024 • edited Loading

Overview

Implementation notes

Interesting/controversial decisions

Test coverage

Loose ends

aryairani commented Jun 27, 2024

aryairani Jun 27, 2024 • edited Loading

Choose a reason for hiding this comment

ChrisPenner Jun 27, 2024

Choose a reason for hiding this comment

aryairani Jun 28, 2024

Choose a reason for hiding this comment

sellout Jun 28, 2024

Choose a reason for hiding this comment

aryairani commented Jun 27, 2024

sellout commented Jun 27, 2024

ChrisPenner Jun 27, 2024 • edited Loading

Choose a reason for hiding this comment

ChrisPenner Jun 27, 2024

Choose a reason for hiding this comment

ChrisPenner Jun 27, 2024

Choose a reason for hiding this comment

aryairani commented Jun 28, 2024

aryairani commented Jul 6, 2024

sellout commented Jul 8, 2024

Footnotes

aryairani commented Jul 8, 2024

ChrisPenner commented Jul 8, 2024

sellout commented Jul 8, 2024

Footnotes

aryairani commented Jul 8, 2024

ChrisPenner commented Jul 8, 2024

neduard commented Jul 12, 2024 • edited Loading

aryairani commented Jul 13, 2024 • edited Loading

neduard commented Jul 13, 2024

aryairani commented Jul 15, 2024

sellout commented Jul 15, 2024

aryairani commented Jul 15, 2024

sellout commented Jul 15, 2024

neduard commented Jul 15, 2024 • edited Loading

ChrisPenner commented Jun 27, 2024 •

edited

Loading

aryairani Jun 27, 2024 •

edited

Loading

ChrisPenner Jun 27, 2024 •

edited

Loading

neduard commented Jul 12, 2024 •

edited

Loading

aryairani commented Jul 13, 2024 •

edited

Loading

neduard commented Jul 15, 2024 •

edited

Loading