Show constraint sources in dependency solver errors #10524

9999years · 2024-11-04T19:51:27Z

Before:

[__0] rejecting: memory-0.18.0 (constraint from user target requires ==0.17.0)

After:

[__0] rejecting: memory-0.18.0
  (constraint from cabal.project requires ==0.17.0)

First shot at #10519. Seems unavoidably large.

Builds off of #9578.

Future work: Propagate line numbers into these constraints!

Implementation strategy:

Propagate provenance (ProjectConfigPath/ConstraintSource) into project config types (Distribution.Client.ProjectConfig, D.C.ProjectConfig.Legacy)
Extract and attach that information in D.C.ProjectConfig.findProjectPackages
Attach that information to ProjectPackageLocation (returned from findProjectPackages). Note: Might be worth unifying that with PackageLocation, would make the new type PackageLocationProvenance.
Use that information to write better ConstraintSources into the solver; this is mostly implemented.

Overall it seems like all the information exists, but it takes a lot of work to propagate it into the right parts of the system.

We have a lot of `showType` functions that are effectively a `Pretty` instance but less composable. Let's make them proper `Pretty` instances. Split off of haskell#10524

cabal-testsuite/PackageTests/NewSdist/MultiTarget/all-test-sute.out

cabal-testsuite/PackageTests/ConstraintSource/SourceRepositoryPackage/cabal.out

ulysses4ever · 2024-11-09T18:56:40Z

@jasagredo just a heads-up: on the last Cabal meeting @9999years asked to not comment on their PRs while a PR is in the draft mode. I remember it because I’m guilty in it more than many!

9999years · 2024-11-18T20:37:38Z

Needs a release note.

9999years · 2024-11-19T20:55:10Z

Added a release note and tests, addressed comments. Should be ready to go!

geekosaur

It's a pity 3.14.1.0 is breathing down our necks; it would be nice to see this go in, seems like it'd be a big UX improvement.

cabal-install/cabal-install.cabal

cabal-install-solver/src/Distribution/Solver/Types/WithConstraintSource.hs

cabal-install/src/Distribution/Client/CmdBuild.hs

grayjay

I started reviewing this PR, but I had a question. It looks like a lot of the changes relate to adding ConstraintSources to targets in v2 commands. Why do the build targets need ConstraintSources? I had thought that targets for most commands only specified the subset of the project to build and didn't affect solving. The only exception I can think of is v2-install, which can solve for packages outside of the project (e.g., "cabal install text-2.0"), but then I would expect all targets to be able to use the same ConstraintSource (ConstraintSourceUserTarget), which I think is the current behavior on master.

cabal-install-solver/src/Distribution/Solver/Types/ConstraintSource.hs

cabal-install-solver/src/Distribution/Solver/Types/WithConstraintSource.hs

cabal-install-solver/src/Distribution/Solver/Types/SourcePackage.hs

grayjay · 2024-11-25T21:43:57Z

cabal-install/src/Distribution/Client/CmdBench.hs

+      =<< readTargetSelectors
+        (localPackages baseCtx)
+        (Just BenchKind)
+        (map (\target -> WithConstraintSource{constraintInner = target, constraintSource = ConstraintSourceCommandlineFlag}) targetStrings)


Why does this line use ConstraintSourceCommandlineFlag as the ConstraintSource? If I understand correctly, targetStrings is a list of targets provided to the bench command, so ConstraintSourceUserTarget seems like a better match. This comment also applies to many of the other commands.

I'll change it if you want. I think it's really unclear which is supposed to be which, and I have no clue what Cabal means when it tells me a constraint comes from a 'user target'. What is ConstraintSourceCommandlineFlag for, if not command line arguments?

grayjay · 2024-11-26T04:48:23Z

cabal-install/tests/UnitTests/Distribution/Solver/Modular/DSL.hs

+                ( WithConstraintSource
+                    { constraintInner =
+                        NamedPackage (C.mkPackageName p) []
+                    , constraintSource = ConstraintSourceUnknown


Should this be ConstraintSourceUserTarget?

I think that the targets parameter should be replaced with one that contains constraint source information. Right now, the constraint source is unknown because that information is missing. We don't know what this function is getting called with or where the targets come from.

grayjay · 2024-11-26T04:59:53Z

cabal-testsuite/PackageTests/ConstraintSource/SourceRepositoryPackage/cabal.project

+-- This is `my-lib` 0.9.
+source-repository-package
+  type: git
+  location: https://github.com/9999years/cabal-testsuite-my-lib.git
+  tag: 9a0af0aa81325c71e744def11db06265840ffb5f


Could this test use an existing package in the Haskell organization as in the postCheckoutCommand test?

cabal/cabal-testsuite/PackageTests/postCheckoutCommand/cabal.positive.project

Lines 3 to 8 in 02da008

source-repository-package

type: git

-- A Sample repo to test post-checkout-command

location: https://github.com/haskell/bytestring

post-checkout-command: true

tag: 0.10.9.0

I wanted a smaller repository so the tests would run faster; my repository is 136k vs bytestring is about 15m.

I also need a package that depends on a conflicting version of a library in the repo.

Grep for location: in this repository and you'll see we're already cloning tons of non-Haskell org repos in the tests.

grayjay · 2024-11-26T07:38:08Z

cabal-testsuite/PackageTests/ConstraintSource/SourceRepositoryPackage/cabal.out

+Could not resolve dependencies:
+[__0] next goal: my-lib (user goal)
+[__0] rejecting: my-lib; 2.0, 1.0
+      (constraint from cabal.project requires ==0.9)


Similarly to the message in use-local-version-of-package.out, I think that this message would be clearer if it explained that the constraint came from the version of a package listed in cabal.project. It could also help to mention that it is a source repository package.

cabal-install/src/Distribution/Client/Setup.hs

grayjay · 2024-11-26T23:20:30Z

cabal-testsuite/PackageTests/NewSdist/MultiTarget/all-test-suite.test.hs

Did you mean to also change the content of this test?

No, @jasagredo asked me to rename it #10524 (comment)

We have a lot of `showType` functions that are effectively a `Pretty` instance but less composable. Let's make them proper `Pretty` instances. Split off of haskell#10524

9999years · 2024-12-20T22:50:43Z

@grayjay I fixed a semantic conflict with #10546, mind re-reviewing?

grayjay · 2024-12-28T07:21:22Z

@9999years I still have a question about the overall change from my last review: #10524 (review)

Before: [__0] rejecting: memory-0.18.0 (constraint from user target requires ==0.17.0) After: [__0] rejecting: memory-0.18.0 (constraint from cabal.project requires ==0.17.0)

@grayjay

From @grayjay: > The dependency solver currently doesn't describe preferences in error > messages, so I think that it would take a lot more work to make this > useful.

...provided that nothing else changes.

9999years · 2025-01-04T00:07:13Z

Why do the build targets need ConstraintSources?

@grayjay I'm looking at this now and the issue is that build targets (as Strings or TargetSelectors) are turned into instances of PackageSpecifier UnresolvedSourcePackage basically immediately, and those are used all over and can include things like source-repository-packages, so we definitely want constraint sources for those... we can mark them as unknown constraint sources but that doesn't lead to great UX, so it seems like it's easiest to just add ConstraintSourceCommandlineFlag or ConstraintSourceImplicitTarget to the target arguments as early as possible.

grayjay · 2025-01-10T05:44:51Z

@9999years I realized that I forgot to clarify the type of target that I was talking about in the comment above and at the Cabal meeting. It seems like Cabal uses "target" in multiple ways due to the change from v1 to v2 commands:

solver targets: The dependency solver takes a set of package names as an argument and solves for those packages and their dependencies.
v1 command build targets: v1 build targets had a relatively simple syntax. Those targets were passed to the dependency solver with very little modification.
v2 command build targets: v2 build targets have a syntax that is much more complex and expressive than v1 targets. For example, they can specify all packages, a component type, or a file. If I understand correctly, for most commands, the build targets only specify the subset of the project to build and are not passed to the dependency solver. However, cabal still needs to pass targets to the solver. I think that those targets are derived from the structure of the project, such as local packages or source repository packages listed in cabal.project. They usually shouldn't change between build commands.

I would expect the solver targets to be associated with constraints, because cabal needs to constrain the versions to match the versions of local packages, when present. I wouldn't expect the v2 targets to be used for any constraints, since they don't affect solving (except for v2 install).

I can see why PackageSpecifiers might need to be associated with ConstraintSources in some cases, but I was wondering why this PR adds ConstraintSources to v2 build targets. Here are two specific examples:

This change seems to add ConstraintSources to targets in the build command: https://github.com/haskell/cabal/pull/10524/files#diff-1de9e22a937e0b39047b8f4cb7695e4b70a68245ef9a8bc3282114d8856b1225R195-R203
This PR contains a lot of changes involving the TargetSelector type, which represents v2 build targets. This change appears to add ConstraintSources to the parsing of TargetSelectors: https://github.com/haskell/cabal/pull/10524/files#diff-ff2f819843c952b293028168b4e8f610ca4f5ea554445d813e6bcb4a17e190f8R258-R268

EDIT: I think that the change from v1 to v2 commands also explains why the messages about constraint sources in the solver currently aren't very clear. The things that they describe, such as "user targets", became more internal with v2 commands.

9999years · 2025-02-10T19:48:50Z

@grayjay I'm not sure how to explain this. I've done my best to explain the changes to TargetSelector in #10524 (comment). I think it will make more sense if you attempt to make the change you desire yourself (removing the constraint annotations from TargetSelectors) and the compiler will show you the places where those annotations are needed. I found myself left with a choice:

At the last moment possible, you can add an annotation to the selectors, e.g. ConstraintSourceUnknown (bad UX, low confidence that it's actually the correct thing to apply to unannotated strings)
Alternatively, you can add the annotations as early as possible, e.g. immediately after the command receives the arguments you can annotate them with ConstraintSourceCommandlineFlag. Then, you propagate those constraints to the points where they're actually used.

I've investigated making the changes here more minimal but I found myself increasingly convinced this only reduces the quality of the patch and leaves more work for someone to do in the future.

ulysses4ever · 2025-02-10T20:23:28Z

I'm thinking maybe we should bite the bullet and merge it, and be ready to revert if something goes horribly wrong for the users after the next release. I just really don't like the impasse I'm observing :-) But then again, I have minimal expertise.

9999years · 2025-02-12T00:17:50Z

Yeah, I wish it was easier to make this change in a less invasive fashion :/

grayjay · 2025-02-13T11:23:40Z

I'm not worried about the correctness of the code, but rather the technical debt. I'm concerned that this PR is very large (+1779, -1009, ignoring tests) and adds a lot of code that appears unnecessary and untested (adding constraint source information to the build targets of v2 commands, other than install).

I spent more time reading the code, and my best guess as to why this change touched so many lines is the change to PackageSpecifier. That type is used by the code that finds all packages in a project as well as the code that reads v2 build targets. Moving the ConstraintSource out of TargetSelector and/or related types and only associating a ConstraintSource with packages where needed could help reduce the amount of code.

9999years force-pushed the dependency-provenance branch 2 times, most recently from 4811a3c to 0c8471d Compare November 4, 2024 22:29

9999years mentioned this pull request Nov 4, 2024

Convert display functions into Pretty instances #10526

Merged

2 tasks

9999years force-pushed the dependency-provenance branch 9 times, most recently from 9ebbba8 to 522bec9 Compare November 9, 2024 00:25

jasagredo reviewed Nov 9, 2024

View reviewed changes

cabal-testsuite/PackageTests/NewSdist/MultiTarget/all-test-sute.out Outdated Show resolved Hide resolved

cabal-testsuite/PackageTests/ConstraintSource/SourceRepositoryPackage/cabal.out Show resolved Hide resolved

9999years force-pushed the dependency-provenance branch from 522bec9 to e52d016 Compare November 18, 2024 19:59

9999years changed the title ~~[WIP] Show constraint sources in dependency solver errors~~ Show constraint sources in dependency solver errors Nov 18, 2024

9999years marked this pull request as ready for review November 18, 2024 19:59

9999years force-pushed the dependency-provenance branch 2 times, most recently from c10bd7d to 9854c9d Compare November 18, 2024 20:37

9999years force-pushed the dependency-provenance branch 2 times, most recently from 9492b41 to b27b413 Compare November 19, 2024 00:45

9999years requested review from jasagredo, philderbeast and ulysses4ever November 19, 2024 20:33

geekosaur approved these changes Nov 19, 2024

View reviewed changes

cabal-install/cabal-install.cabal Show resolved Hide resolved

ulysses4ever requested a review from grayjay November 20, 2024 02:42

jasagredo reviewed Nov 20, 2024

View reviewed changes

cabal-install-solver/src/Distribution/Solver/Types/WithConstraintSource.hs Show resolved Hide resolved

cabal-install/src/Distribution/Client/CmdBuild.hs Outdated Show resolved Hide resolved

9999years mentioned this pull request Nov 25, 2024

PR template: Make it better! #10594

Open

grayjay reviewed Nov 26, 2024

View reviewed changes

grayjay added the cabal-install: solver label Dec 5, 2024

9999years force-pushed the dependency-provenance branch 2 times, most recently from e180d68 to 2844e3e Compare December 6, 2024 23:03

9999years mentioned this pull request Dec 9, 2024

"configuration changed" is way too vague #10617

Open

9999years force-pushed the dependency-provenance branch from 2844e3e to 524ba6d Compare December 20, 2024 19:17

9999years added 5 commits January 3, 2025 10:22

Add ConstraintSource to local packages

11ed225

Before: [__0] rejecting: memory-0.18.0 (constraint from user target requires ==0.17.0) After: [__0] rejecting: memory-0.18.0 (constraint from cabal.project requires ==0.17.0)

Remove ConstraintSources from preferences

a0b9b2a

From @grayjay: > The dependency solver currently doesn't describe preferences in error > messages, so I think that it would take a lot more work to make this > useful.

Prevent rebuilds when the project file changes

1be674d

...provided that nothing else changes.

Fix DedupUsingConfigFromComplex test

e6f0f78

Fix HLint nits

b84c092

9999years force-pushed the dependency-provenance branch from 611d65c to b84c092 Compare January 3, 2025 18:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Show constraint sources in dependency solver errors #10524

Show constraint sources in dependency solver errors #10524

9999years commented Nov 4, 2024 •

edited

Loading

ulysses4ever commented Nov 9, 2024

9999years commented Nov 18, 2024

9999years commented Nov 19, 2024

geekosaur left a comment

grayjay left a comment

grayjay Nov 25, 2024

9999years Dec 6, 2024

grayjay Nov 26, 2024

9999years Dec 6, 2024

grayjay Nov 26, 2024

9999years Dec 6, 2024 •

edited

Loading

9999years Dec 6, 2024

grayjay Nov 26, 2024

9999years Dec 6, 2024

grayjay Nov 26, 2024

9999years Dec 6, 2024

9999years commented Dec 20, 2024

grayjay commented Dec 28, 2024

9999years commented Jan 4, 2025

grayjay commented Jan 10, 2025 •

edited

Loading

9999years commented Feb 10, 2025 •

edited

Loading

ulysses4ever commented Feb 10, 2025

9999years commented Feb 12, 2025

grayjay commented Feb 13, 2025

	source-repository-package
	type: git
	-- A Sample repo to test post-checkout-command
	location: https://github.com/haskell/bytestring
	post-checkout-command: true
	tag: 0.10.9.0

Show constraint sources in dependency solver errors #10524

Are you sure you want to change the base?

Show constraint sources in dependency solver errors #10524

Conversation

9999years commented Nov 4, 2024 • edited Loading

ulysses4ever commented Nov 9, 2024

9999years commented Nov 18, 2024

9999years commented Nov 19, 2024

geekosaur left a comment

Choose a reason for hiding this comment

grayjay left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

9999years Dec 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

9999years commented Dec 20, 2024

grayjay commented Dec 28, 2024

9999years commented Jan 4, 2025

grayjay commented Jan 10, 2025 • edited Loading

9999years commented Feb 10, 2025 • edited Loading

ulysses4ever commented Feb 10, 2025

9999years commented Feb 12, 2025

grayjay commented Feb 13, 2025

9999years commented Nov 4, 2024 •

edited

Loading

9999years Dec 6, 2024 •

edited

Loading

grayjay commented Jan 10, 2025 •

edited

Loading

9999years commented Feb 10, 2025 •

edited

Loading