Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lazy trees #6530

Draft
wants to merge 390 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 247 commits
Commits
Show all changes
390 commits
Select commit Hold shift + click to select a range
37d859b
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Aug 19, 2022
a115c4f
GitHub fetcher: Restore the lastModified field
edolstra Aug 22, 2022
91aea15
Fix macOS build, where le32toh is not available
edolstra Aug 22, 2022
13c0db4
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Aug 25, 2022
def7b25
readLittleEndian(): Fix 64-bit integer truncation
edolstra Aug 26, 2022
034340a
Fix potential duplicate activity IDs in forked child processes
edolstra Aug 26, 2022
f95c425
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Aug 29, 2022
30be644
Make EvalState::inputAccessors keyed by the accessor number
edolstra Aug 29, 2022
7da3a30
Remove no_pos_tag
edolstra Aug 29, 2022
301f388
Remove FIXME
edolstra Aug 29, 2022
440214f
ZipInputAccessor: Fix invalid read
edolstra Aug 30, 2022
89f1021
Improve display of Git inputs
edolstra Aug 31, 2022
120bec5
GitInputScheme: Do not record 'ref' for dirty trees
edolstra Aug 31, 2022
8a43eaa
GitInputScheme: Add some progress indication
edolstra Aug 31, 2022
7c1bdff
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Sep 2, 2022
c0dd35a
ZipInputAccessor: Improve error messages
edolstra Sep 2, 2022
2d5cfca
Fix accessing 'toString path'
edolstra Sep 2, 2022
3667cf5
Whitespace
edolstra Sep 12, 2022
b293b33
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Sep 12, 2022
48a5879
Decode virtual paths in user-thrown errors
edolstra Sep 12, 2022
1b8065f
posToXML(): Fix displaying paths
edolstra Sep 12, 2022
85c1959
Remove some FIXMEs
edolstra Sep 12, 2022
432a3a1
Move isUri() and resolveUri() out of filetransfer.cc
edolstra Sep 12, 2022
2a1c63c
Support flake references in the old CLI
edolstra Sep 13, 2022
28b62dd
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Sep 28, 2022
a291e37
Improve error messages from call-flake.nix
edolstra Sep 28, 2022
c3c0682
Don't show "from call site" when we don't know the call site
edolstra Sep 28, 2022
cbade16
Handle unlocked overriden inputs
edolstra Sep 29, 2022
241dd54
warnOnce(): Fix boost exception when the message contains a format ch…
edolstra Sep 30, 2022
5c8d67d
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Oct 10, 2022
0286edb
Format GitHub paths as URLs
edolstra Oct 10, 2022
1483c56
Patch libzip to return timestamps in the Unix epoch
edolstra Oct 10, 2022
7317196
Input::getAccessor(): Get the fingerprint from the final accessor
edolstra Oct 10, 2022
5115909
Fix handling of relative paths
edolstra Oct 10, 2022
12dd8d4
Fix 'nix-instantiate --find-file' and add a test
edolstra Oct 12, 2022
e2353b9
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Oct 26, 2022
0402dd0
Interpret absolute paths relative to the root FS rather than the curr…
edolstra Oct 26, 2022
1683872
Use __nix_virtual__ instead of __virtual__
edolstra Oct 26, 2022
e424a8b
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Oct 27, 2022
a653e98
Encode virtual paths as /nix/store/virtual000<N>
edolstra Oct 27, 2022
f02da62
builtins.trace: Decode virtual paths
edolstra Oct 28, 2022
b275aa4
Don't use warnOnce() for the toString warning
edolstra Oct 28, 2022
8342317
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Nov 2, 2022
4072024
GitArchiveInputScheme: Bring back the narHash attribute
edolstra Nov 3, 2022
64a69b4
Fix dirOf on the root of a flake
edolstra Nov 16, 2022
561440b
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Nov 16, 2022
b27cd88
Partially revert "Format GitHub paths as URLs"
edolstra Nov 25, 2022
515b908
Fix decoding virtual paths that are at the root of the tree
edolstra Nov 25, 2022
39a783f
Require flakes for the -I flake:... feature
edolstra Dec 1, 2022
116acc1
Fix readDir for accessors whose readDirectory doesn't return types
edolstra Dec 2, 2022
fcdca3d
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Dec 5, 2022
730f6bf
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Dec 7, 2022
57397a3
Formatting
edolstra Dec 7, 2022
d162222
Remove test for .path in nix flake metadata
edolstra Dec 7, 2022
6d104bb
Don't allow appending a non-absolute path to the root of a source tree
edolstra Dec 8, 2022
f620184
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Dec 9, 2022
d950e3a
Improve error message when flake.nix exists but is not under Git control
edolstra Dec 9, 2022
12c554a
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Dec 12, 2022
a322306
For backward compatibility, allow appending non-root paths to the roo…
edolstra Dec 14, 2022
d8620d7
Append a slash in ./${"foo"}
edolstra Dec 14, 2022
8becbb0
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Dec 14, 2022
3bcaa7d
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Dec 15, 2022
210cd8c
Merge toDerivations() into toDerivedPaths()
edolstra Dec 15, 2022
81a4516
InstallableFlake::toDerivedPaths(): Support paths and store paths
edolstra Dec 15, 2022
880a72b
nix build --json: Only show non-zero startTime / stopTime
edolstra Dec 16, 2022
15d2e0e
Merge remote-tracking branch 'origin/master' into tmp
edolstra Dec 20, 2022
b48e641
Add builtins.filterPath
edolstra Dec 19, 2022
5e3cd3e
Fix FilteringInputAccessor::pathExists()
edolstra Dec 21, 2022
29dff7e
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Jan 11, 2023
20a0a74
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Jan 12, 2023
fa5af1e
Fix support for relative non-flake inputs (path:./bla)
edolstra Jan 12, 2023
9286b1a
Split GitInputScheme::getAccessor() more
edolstra Jan 20, 2023
9512afa
Typo
edolstra Jan 20, 2023
3621d07
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Jan 26, 2023
3522978
Fix tests
edolstra Jan 26, 2023
f4f0f8a
Remove toString deprecation warning
edolstra Jan 26, 2023
37b4a9e
Shut up a gcc warning
edolstra Jan 26, 2023
31bb875
Use libgit2 to provide direct access to Git repositories
edolstra Jan 25, 2023
aaf8b1b
Use C++20
edolstra Jan 31, 2023
b14830b
TarArchive: Remove a duplicate constant and increase the buffer size
edolstra Jan 31, 2023
5c29abc
GitArchiveInputScheme: Revert to downloading tarballs
edolstra Jan 31, 2023
4142982
Remove ZipInputAccessor
edolstra Jan 31, 2023
ca26ce9
Check tarball cache validity
edolstra Feb 3, 2023
219510b
GitArchiveInputScheme: Verify the locked tree hash
edolstra Feb 3, 2023
7b1cda9
GitArchiveInputScheme: Restore the lastModified attribute
edolstra Feb 3, 2023
00b746d
Warn if the computed tree hash differs from the one reported by GitHub
edolstra Feb 3, 2023
bb421ac
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Feb 8, 2023
12f1413
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Feb 17, 2023
02d5c54
Fix clang build
edolstra Feb 17, 2023
26ff9c1
Revert "Shut up a gcc warning"
edolstra Feb 17, 2023
f4009fd
Don't use std::span just yet
edolstra Feb 17, 2023
ece20d5
GitInputAccessor: Support symlinks
edolstra Feb 17, 2023
5d1e5a0
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Feb 21, 2023
c8f7ca2
Fix addErrorContext
edolstra Feb 21, 2023
273df09
Remove unused file
edolstra Feb 21, 2023
21f0a98
Remove unnecessary -lgit2
edolstra Feb 21, 2023
03618bb
Fix GitHub test
edolstra Feb 21, 2023
56a845d
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Feb 22, 2023
98a90cc
Use libgit2 to get the revCount
edolstra Feb 24, 2023
430bfcf
Use libgit2 to get lastModified
edolstra Feb 24, 2023
13e8d70
Use libgit2 for getting isShallow
edolstra Feb 24, 2023
23806db
Use libgit2 to resolve references to commit hashes
edolstra Feb 24, 2023
ab2b0fb
Add a helper class to avoid leaking libgit2 objects
edolstra Feb 24, 2023
8c8f242
Handle revisions used as refs for compatibility
edolstra Feb 24, 2023
579ecd3
Remove a test for the non-presence of git
edolstra Feb 24, 2023
1b0b3b1
Add git_oid -> Hash conversion
edolstra Feb 24, 2023
834d878
Use libgit2 to get workdir info
edolstra Mar 1, 2023
1017b11
Remove dead code
edolstra Mar 1, 2023
86ca2c5
Use libgit2 to create the local cache repos
edolstra Mar 1, 2023
e952d36
Refactor
edolstra Mar 1, 2023
d6cdb07
Rename
edolstra Mar 1, 2023
334348f
Remove use of git cat-file
edolstra Mar 1, 2023
733861d
Support allRefs in git URLs
edolstra Mar 2, 2023
3a1a33d
Use libgit2 for fetching repos
edolstra Mar 2, 2023
b19534a
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Mar 6, 2023
854a311
Fix appending an empty string to a path
edolstra Mar 6, 2023
b8218b5
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Mar 7, 2023
f21e1cf
Fix outPath in flakes
edolstra Mar 8, 2023
aa823c2
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Mar 8, 2023
d7d49fd
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Mar 9, 2023
bb72d1b
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Mar 9, 2023
6192fd0
Fix some clang errors/warnings
edolstra Mar 9, 2023
4f1b0d7
Fix clang compilation
edolstra Mar 10, 2023
cdb946e
Fix test regression
edolstra Mar 13, 2023
7052e19
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Mar 15, 2023
2f1a90b
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Mar 16, 2023
0671807
Fix makefile
edolstra Mar 16, 2023
b2a205e
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Mar 16, 2023
ccd2ad2
Remove git caching
edolstra Mar 17, 2023
2a4462e
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Mar 17, 2023
f432967
Use libgit2 for getting the HEAD ref of local repos
edolstra Mar 20, 2023
f23b969
Check git_remote_fetch() return value
edolstra Mar 20, 2023
2ffa909
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Mar 24, 2023
4b9215c
Handle patches that have a timestamp after the filename
edolstra Mar 28, 2023
9a15ec8
Indentation
edolstra Mar 30, 2023
870e750
Fix root/overriden flakes with dir attribute
edolstra Mar 29, 2023
64b9e50
Add CanonPath::makeRelative()
edolstra Mar 30, 2023
2154084
Allow patches to be applied to flake inputs
edolstra Mar 28, 2023
a0b0cac
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Mar 30, 2023
a1501cf
EvalState::rootPath(): Take a CanonPath
edolstra Apr 3, 2023
8ee59d7
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Apr 3, 2023
791e222
Fix tests
edolstra Apr 6, 2023
5b2be3a
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Apr 25, 2023
0dfc214
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Apr 26, 2023
91aefbd
Factor out FilteringInputAccessor
edolstra Apr 26, 2023
d102273
Fix clang compilation
edolstra May 1, 2023
39079dd
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra May 9, 2023
2b9ded5
Fix parsing of relative flake inputs without 'path:'
edolstra May 10, 2023
a9d4780
Revert to using git for fetching remote git repos
edolstra May 11, 2023
2dc2f58
Log git fetch errors
edolstra May 11, 2023
ac73702
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra May 12, 2023
3c4d678
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Jun 2, 2023
e59caad
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Jun 19, 2023
2f18e78
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Jun 26, 2023
f355c34
Fix handling of relative path flakes (and add some tests)
edolstra Jun 26, 2023
56272db
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Aug 3, 2023
3494c29
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Sep 9, 2023
4e16d5e
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Sep 20, 2023
6b70509
Remove dead code
edolstra Sep 21, 2023
eaa785e
Improve error message
edolstra Sep 22, 2023
ea5c2e3
Tarball fetcher: Use the tarball cache
edolstra Oct 11, 2023
b219d76
Tarball cache: Add cache entries for all URLs in the redirect chain
edolstra Oct 11, 2023
0e48afb
Remove the "facts" cache
edolstra Oct 11, 2023
0b72b31
Remove debug statement
edolstra Oct 12, 2023
6513f69
Fix lastModified handling
edolstra Oct 13, 2023
94c028f
Fix file:// URLs
edolstra Oct 13, 2023
e350f84
Remove bad.tar.xz check, since libarchive doesn't care
edolstra Oct 13, 2023
fa8618f
Eliminate old downloadTarball()
edolstra Oct 13, 2023
144987e
Revert cache version
edolstra Oct 13, 2023
6a95ff9
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Oct 13, 2023
b4df104
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Oct 17, 2023
601de7f
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Oct 23, 2023
c677ea7
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Oct 23, 2023
aa85359
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Nov 3, 2023
ddca787
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Nov 20, 2023
ec272de
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Nov 22, 2023
b39148e
Post-merge cleanup
edolstra Nov 22, 2023
3099b51
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Nov 28, 2023
0aa13b6
Cleanup
edolstra Nov 30, 2023
be1543e
Merge remote-tracking branch 'upstream/master' into lazy-trees
Ericson2314 Dec 1, 2023
653c100
Merge remote-tracking branch 'upstream/master' into lazy-trees
Ericson2314 Dec 1, 2023
575902b
Merge remote-tracking branch 'upstream/master' into lazy-trees
Ericson2314 Dec 7, 2023
d38f4d9
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Dec 13, 2023
dab2be3
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Dec 13, 2023
2055e28
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Dec 13, 2023
c0b1df7
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Feb 15, 2024
748567b
Doxygen
edolstra Feb 15, 2024
5558d65
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Feb 19, 2024
d022bce
FileInputScheme: Display path properly
edolstra Feb 19, 2024
7eb3ba0
PosixSourceAccessor: Support roots that are not directories
edolstra Feb 19, 2024
d9cfae2
Cleanup
edolstra Feb 21, 2024
43de535
Remove unnecessary pointer
edolstra Feb 21, 2024
0d02a63
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Feb 21, 2024
d2292a1
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Feb 21, 2024
edcd6ce
Merge remote-tracking branch 'origin/master' into lazy-trees
9999years Mar 1, 2024
793a838
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Mar 4, 2024
1c56bdd
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Mar 7, 2024
49f9099
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Apr 2, 2024
ce23764
Remove duplicate comment
edolstra Apr 2, 2024
3369a81
EvalState::findFile(): Resolve symlinks
edolstra Apr 3, 2024
3f70f20
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Apr 19, 2024
1abf058
Merge remote-tracking branch 'upstream/master' into lazy-trees
roberth Apr 21, 2024
147593d
tests/libexpr: Re-enable C api tests that build
roberth Apr 21, 2024
c6b4f01
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Apr 22, 2024
cf02c6a
Fix test
edolstra Apr 22, 2024
4a9b7c8
Fix pre-commit check
edolstra Apr 22, 2024
0efcbbf
Run the flake-regressions test suite
edolstra Apr 24, 2024
c4cd4cd
Hack
edolstra Apr 24, 2024
93cf285
PathInputScheme: Improve path display
edolstra Apr 25, 2024
2fec506
BasicDerivation: Add applyRewrites() method
edolstra Apr 25, 2024
5ddd8e4
Introduce a new string context for representing paths
edolstra Apr 25, 2024
1302bfa
Re-enable some tests
edolstra Apr 25, 2024
9c10fcf
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Apr 30, 2024
71575a2
Debug
edolstra Apr 30, 2024
333d35a
callPathFilter(): Pass a lazy store path
edolstra Apr 30, 2024
0950094
Fix fileset compatibility
edolstra Apr 30, 2024
2101916
Test baseNameOf behaviour on the root of a flake
edolstra Apr 30, 2024
fc4d948
Cleanup
edolstra Apr 30, 2024
db6c219
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Apr 30, 2024
a65c8a9
Debug
edolstra Apr 30, 2024
c5ae41d
Copy roots to the store as /nix/store/<hash1>-<hash2>-source
edolstra Apr 30, 2024
ea54be0
filterPath: Ensure /nix/store/<hash>-<hash>-source
edolstra Apr 30, 2024
a435a40
Disable some GitHub narHash related tests for now
edolstra Apr 30, 2024
7332321
Run more flake regression tests
edolstra Apr 30, 2024
299d0b7
TarballInputScheme: Fix fetchToStore() caching
edolstra May 3, 2024
3a96ba0
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra May 6, 2024
25635e5
EvalState: Rename decodePaths() -> prettyPrintPaths()
edolstra May 6, 2024
b5be3b0
Pretty-print virtual paths in more contexts
edolstra May 6, 2024
2409e6b
Add tests for tree laziness
edolstra May 6, 2024
44d20c3
Warn against use of `toString ./path` as a derivation attribute
edolstra May 6, 2024
4133514
Remove unused
edolstra May 6, 2024
2cb003c
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra May 6, 2024
8fc36e2
Fix concatenating to an attrset
edolstra May 6, 2024
bcb1be4
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra May 6, 2024
f0cf5d8
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra May 7, 2024
a123f87
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra May 8, 2024
26f1431
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra May 10, 2024
b7255f8
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra May 13, 2024
584fece
flake-regressions.sh: Make the sort order deterministic
edolstra May 14, 2024
4d39908
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra May 14, 2024
e255be6
builtins.toPath: Rewrite virtual paths
edolstra May 14, 2024
a33aad3
Use fetchToStore() instead of computeStorePath()
edolstra May 14, 2024
35bbd85
Warn about double copies
edolstra May 14, 2024
b6be10d
Skip a broken test
edolstra May 15, 2024
57e1692
Backwards compatibility hack to handle `/. + path`
edolstra May 15, 2024
57b2cc2
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Jul 2, 2024
1b20945
Formatting
edolstra Jul 2, 2024
53fabdc
Busywork
edolstra Jul 2, 2024
34e4a61
Merge branch 'master' into lazy-trees
Ericson2314 Jul 5, 2024
0d3a157
Merge remote-tracking branch 'origin/master' into lazy-trees
edolstra Jul 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/manual/generate-manpage.nix
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found an issue

the nix edit man page says

Get the filename and line number used by nix edit:
    | # nix eval --raw nixpkgs#hello.meta.position
    | /nix/store/fvafw0gvwayzdan642wrv84pzm5bgpmy-source/pkgs/applications/misc/hello/default.nix:15

but that returns a virtual path instead of the path used by nix edit

$ nix eval ".#hello.meta.position"
"/nix/store/virtual0000000000000000000000004-source/pkgs/by-name/he/hello/package.nix:34"

nix edit ".#hello" works properly and is an editable path

https://github.com/NixOS/nixpkgs/blob/e80f82aa1edc853339ae889568cfa430808b0dfe/pkgs/stdenv/generic/make-derivation.nix#L137-L142
builtins.unsafeGetAttrPos is used to create meta.position

Copy link
Member

@Artturin Artturin Nov 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another issue

In this filter https://github.com/serokell/nixfmt/blob/7e9e06eefed52d6d698b828ee83b83ea5c163f3b/flake.nix#L66-L68

Only the top level files and directories are passed in the path arg

applying

diff --git a/flake.nix b/flake.nix
index 8e4dca6..77f6065 100644
--- a/flake.nix
+++ b/flake.nix
@@ -65,7 +65,7 @@
           name = "nixfmt-src";
           filter = path: type:
             let relPath = lib.removePrefix (toString ./. + "/") (toString path);
-            in lib.any (re: builtins.match re relPath != null) regexes;
+            in lib.trace (toString path) lib.any (re: builtins.match re relPath != null) regexes;
         };

       in {

and doing nix build ".#packages.x86_64-linux.nixfmt.src"

gives the following his branch

trace: /.github
trace: /.gitignore
trace: /.reuse
trace: /.stylish-haskell.yaml
trace: /CHANGELOG.md
trace: /CONTRIBUTING.md
trace: /LICENSE
trace: /LICENSES
trace: /MAINTENANCE.md
trace: /README.md
trace: /default.nix
trace: /flake.lock
trace: /flake.nix
trace: /js
trace: /main
trace: /nixfmt.cabal
trace: /shell.nix
trace: /src
trace: /test

and the following on 2.17

trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/.github
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/.gitignore
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/.reuse
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/.stylish-haskell.yaml
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/CHANGELOG.md
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/CONTRIBUTING.md
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/LICENSE
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/LICENSES
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/MAINTENANCE.md
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/README.md
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/default.nix
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/flake.lock
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/flake.nix
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/js
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/js/404.html
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/js/JSInterface.hs
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/js/index.html
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/main
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/main/Main.hs
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/main/System
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/main/System/IO
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/main/System/IO/Atomic.hs
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/main/System/IO/Utf8.hs
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/nixfmt.cabal
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/shell.nix
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/src
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/src/Nixfmt
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/src/Nixfmt/Lexer.hs
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/src/Nixfmt/Parser
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/src/Nixfmt/Parser/Float.hs
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/src/Nixfmt/Parser.hs
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/src/Nixfmt/Predoc.hs
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/src/Nixfmt/Pretty.hs
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/src/Nixfmt/Types.hs
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/src/Nixfmt/Util.hs
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/src/Nixfmt.hs
trace: /nix/store/w5yw6i1b5bvywh8snd99pdkgpknlylsp-source/test

Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{ toplevel }:

with builtins;
with import ./utils.nix;
with import <nix/utils.nix>;

let

Expand Down
2 changes: 1 addition & 1 deletion doc/manual/local.mk
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ dummy-env = env -i \
NIX_STATE_DIR=/dummy \
NIX_CONFIG='cores = 0'

nix-eval = $(dummy-env) $(bindir)/nix eval --experimental-features nix-command -I nix/corepkgs=corepkgs --store dummy:// --impure --raw
nix-eval = $(dummy-env) $(bindir)/nix eval --experimental-features nix-command -I nix=doc/manual --store dummy:// --impure --raw

$(d)/%.1: $(d)/src/command-ref/%.md
@printf "Title: %s\n\n" "$$(basename $@ .1)" > $^.tmp
Expand Down
1 change: 1 addition & 0 deletions flake.nix
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,7 @@
bzip2 xz brotli editline
openssl sqlite
libarchive
libgit2
boost
lowdown-nix
gtest
Expand Down
18 changes: 11 additions & 7 deletions src/libcmd/common-eval-args.cc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found another issue when using fetchGit with an empty repository:

❯ cd $(mktemp -d)
❯ git init
Initialized empty Git repository in /tmp/tmp.oOrUtuE7kX/.git/
❯ nix eval --impure --expr 'fetchGit ./.'
warning: Git tree '/tmp/tmp.oOrUtuE7kX' is dirty
{ lastModified = 0; lastModifiedDate = "19700101000000"; narHash = "sha256-pQpattmS9VmO3ZIQUFn66az8GSmB4IvYhTTCFn6SUmo="; outPath = "/nix/store/0ccnxa25whszw7mgbgyzdm4nqc0zwnm8-source"; rev = "0000000000000000000000000000000000000000"; revCount = 0; shortRev = "0000000"; submodules = false; }
❯ nix run github:edolstra/nix/lazy-trees -- eval --impure --expr 'fetchGit ./.'
error: bad optional access
❯ git commit --allow-empty --allow-empty-message --message ""
[main (root-commit) 3abe881] 
❯ nix eval --impure --expr 'fetchGit ./.'
{ lastModified = 1700607798; lastModifiedDate = "20231121230318"; narHash = "sha256-pQpattmS9VmO3ZIQUFn66az8GSmB4IvYhTTCFn6SUmo="; outPath = "/nix/store/0ccnxa25whszw7mgbgyzdm4nqc0zwnm8-source"; rev = "3abe881a3f62393f05062c4638d1ecc7496d25f4"; revCount = 1; shortRev = "3abe881"; submodules = false; }
❯ nix run github:edolstra/nix/lazy-trees -- eval --impure --expr 'fetchGit ./.'
error (ignored): error: end of string reached
error:
       … while calling the 'fetchGit' builtin

         at «string»:1:1:

            1| fetchGit ./.
             | ^

       … while fetching the input 'git+file:///tmp/tmp.oOrUtuE7kX'

       error: access to path '/tmp/tmp.oOrUtuE7kX' is forbidden because it is not under Git control; maybe you should 'git add' it to the repository '/tmp/tmp.oOrUtuE7kX'?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another very general issue: There's 4 different results you can get when printing lazy paths from this PR:

  • Directly:

    $ nix run github:edolstra/nix/lazy-trees -- eval --impure \
      --expr '(builtins.fetchTree "path:${toString ./.}").outPath'
    /home/tweagysil/test/flake-filesets/
    
  • With toString:

    $ nix run github:edolstra/nix/lazy-trees -- eval --impure \
      --expr 'toString (builtins.fetchTree "path:${toString ./.}").outPath'
    "/nix/store/virtual0000000000000000000000004-source"
    
  • With string interpolation:

    $ nix run github:edolstra/nix/lazy-trees -- eval --impure \
      --expr '"${(builtins.fetchTree "path:${toString ./.}").outPath}"'
    "/nix/store/j62mpn1vvrmc8bgrsp5ypggd2qcb07iq-source"
    
  • Using --json:

    $ nix run github:edolstra/nix/lazy-trees -- eval --impure --json \
      --expr '(builtins.fetchTree "path:${toString ./.}").outPath'
    "/"
    

Paths were already hard to understand, but this makes it even more confusing.

Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
#include "flake/flakeref.hh"
#include "store-api.hh"
#include "command.hh"
#include "fs-input-accessor.hh"
#include "tarball.hh"

namespace nix {

Expand Down Expand Up @@ -149,27 +151,29 @@ Bindings * MixEvalArgs::getAutoArgs(EvalState & state)
for (auto & i : autoArgs) {
auto v = state.allocValue();
if (i.second[0] == 'E')
state.mkThunk_(*v, state.parseExprFromString(i.second.substr(1), absPath(".")));
state.mkThunk_(*v, state.parseExprFromString(i.second.substr(1), state.rootPath(absPath("."))));
else
v->mkString(((std::string_view) i.second).substr(1));
res.insert(state.symbols.create(i.first), v);
}
return res.finish();
}

Path lookupFileArg(EvalState & state, std::string_view s)
SourcePath lookupFileArg(EvalState & state, std::string_view s)
{
if (EvalSettings::isPseudoUrl(s)) {
auto storePath = fetchers::downloadTarball(
state.store, EvalSettings::resolvePseudoUrl(s), "source", false).first.storePath;
return state.store->toRealPath(storePath);
state.store, EvalSettings::resolvePseudoUrl(s), "source", false).first;
auto accessor = makeStorePathAccessor(state.store, storePath);
state.registerAccessor(accessor);
return accessor->root();
}

else if (hasPrefix(s, "flake:")) {
settings.requireExperimentalFeature(Xp::Flakes);
auto flakeRef = parseFlakeRef(std::string(s.substr(6)), {}, true, false);
auto storePath = flakeRef.resolve(state.store).fetchTree(state.store).first.storePath;
return state.store->toRealPath(storePath);
auto [accessor, _] = flakeRef.resolve(state.store).lazyFetch(state.store);
return accessor->root();
}

else if (s.size() > 2 && s.at(0) == '<' && s.at(s.size() - 1) == '>') {
Expand All @@ -178,7 +182,7 @@ Path lookupFileArg(EvalState & state, std::string_view s)
}

else
return absPath(std::string(s));
return state.rootPath(absPath(std::string(s)));
}

}
3 changes: 2 additions & 1 deletion src/libcmd/common-eval-args.hh
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ namespace nix {
class Store;
class EvalState;
class Bindings;
struct SourcePath;

struct MixEvalArgs : virtual Args
{
Expand All @@ -24,6 +25,6 @@ private:
std::map<std::string, std::string> autoArgs;
};

Path lookupFileArg(EvalState & state, std::string_view s);
SourcePath lookupFileArg(EvalState & state, std::string_view s);

}
7 changes: 5 additions & 2 deletions src/libcmd/editor-for.cc
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,11 @@

namespace nix {

Strings editorFor(const Path & file, uint32_t line)
Strings editorFor(const SourcePath & file, uint32_t line)
{
auto path = file.getPhysicalPath();
if (!path)
throw Error("cannot open '%s' in an editor because it has no physical path", file);
auto editor = getEnv("EDITOR").value_or("cat");
auto args = tokenizeString<Strings>(editor);
if (line > 0 && (
Expand All @@ -13,7 +16,7 @@ Strings editorFor(const Path & file, uint32_t line)
editor.find("vim") != std::string::npos ||
editor.find("kak") != std::string::npos))
args.push_back(fmt("+%d", line));
args.push_back(file);
args.push_back(path->abs());
return args;
}

Expand Down
3 changes: 2 additions & 1 deletion src/libcmd/editor-for.hh
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
#pragma once

#include "types.hh"
#include "input-accessor.hh"

namespace nix {

/* Helper function to generate args that invoke $EDITOR on
filename:lineno. */
Strings editorFor(const Path & file, uint32_t line);
Strings editorFor(const SourcePath & file, uint32_t line);

}
3 changes: 1 addition & 2 deletions src/libcmd/installable-flake.cc
Original file line number Diff line number Diff line change
Expand Up @@ -96,8 +96,7 @@ DerivedPathsWithInfo InstallableFlake::toDerivedPaths()
auto v = attr->forceValue();

if (v.type() == nPath) {
PathSet context;
auto storePath = state->copyPathToStore(context, Path(v.path));
auto storePath = v.path().fetchToStore(state->store);
return {{
.path = DerivedPath::Opaque {
.path = std::move(storePath),
Expand Down
13 changes: 7 additions & 6 deletions src/libcmd/installables.cc
Original file line number Diff line number Diff line change
Expand Up @@ -211,9 +211,10 @@ void SourceExprCommand::completeInstallable(std::string_view prefix)

evalSettings.pureEval = false;
auto state = getEvalState();
Expr *e = state->parseExprFromFile(
resolveExprPath(state->checkSourcePath(lookupFileArg(*state, *file)))
);
auto e =
state->parseExprFromFile(
resolveExprPath(
lookupFileArg(*state, *file)));

Value root;
state->eval(e, root);
Expand Down Expand Up @@ -397,10 +398,10 @@ ref<eval_cache::EvalCache> openEvalCache(
EvalState & state,
std::shared_ptr<flake::LockedFlake> lockedFlake)
{
auto fingerprint = lockedFlake->getFingerprint();
auto fingerprint = lockedFlake->getFingerprint(state.store);
return make_ref<nix::eval_cache::EvalCache>(
evalSettings.useEvalCache && evalSettings.pureEval
? std::optional { std::cref(fingerprint) }
? fingerprint
: std::nullopt,
state,
[&state, lockedFlake]()
Expand Down Expand Up @@ -444,7 +445,7 @@ std::vector<std::shared_ptr<Installable>> SourceExprCommand::parseInstallables(
else if (file)
state->evalFile(lookupFileArg(*state, *file), *vFile);
else {
auto e = state->parseExprFromString(*expr, absPath("."));
auto e = state->parseExprFromString(*expr, state->rootPath(absPath(".")));
state->eval(e, *vFile);
}

Expand Down
8 changes: 4 additions & 4 deletions src/libcmd/repl.cc
Original file line number Diff line number Diff line change
Expand Up @@ -590,14 +590,14 @@ bool NixRepl::processLine(std::string line)
Value v;
evalString(arg, v);

const auto [path, line] = [&] () -> std::pair<Path, uint32_t> {
const auto [path, line] = [&] () -> std::pair<SourcePath, uint32_t> {
if (v.type() == nPath || v.type() == nString) {
PathSet context;
auto path = state->coerceToPath(noPos, v, context, "while evaluating the filename to edit");
return {path, 0};
} else if (v.isLambda()) {
auto pos = state->positions[v.lambda.fun->pos];
if (auto path = std::get_if<Path>(&pos.origin))
if (auto path = std::get_if<SourcePath>(&pos.origin))
return {*path, pos.line};
else
throw Error("'%s' cannot be shown in an editor", pos);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would happen when the position of the lambda can't be determined, right? I think this error message doesn't reflect that very well.

Expand Down Expand Up @@ -872,7 +872,7 @@ void NixRepl::addVarToScope(const Symbol name, Value & v)

Expr * NixRepl::parseString(std::string s)
{
Expr * e = state->parseExprFromString(std::move(s), curDir, staticEnv);
Expr * e = state->parseExprFromString(std::move(s), state->rootPath(curDir), staticEnv);
return e;
}

Expand Down Expand Up @@ -930,7 +930,7 @@ std::ostream & NixRepl::printValue(std::ostream & str, Value & v, unsigned int m
break;

case nPath:
str << ANSI_GREEN << v.path << ANSI_NORMAL; // !!! escaping?
str << ANSI_GREEN << v.path().to_string() << ANSI_NORMAL; // !!! escaping?
break;

case nNull:
Expand Down
26 changes: 15 additions & 11 deletions src/libexpr/attr-path.cc
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ std::pair<Value *, PosIdx> findAlongAttrPath(EvalState & state, const std::strin
}


std::pair<std::string, uint32_t> findPackageFilename(EvalState & state, Value & v, std::string what)
std::pair<SourcePath, uint32_t> findPackageFilename(EvalState & state, Value & v, std::string what)
{
Value * v2;
try {
Expand All @@ -118,21 +118,25 @@ std::pair<std::string, uint32_t> findPackageFilename(EvalState & state, Value &

// FIXME: is it possible to extract the Pos object instead of doing this
// toString + parsing?
auto pos = state.forceString(*v2, noPos, "while evaluating the 'meta.position' attribute of a derivation");
PathSet context;
auto path = state.coerceToPath(noPos, *v2, context, "while evaluating the 'meta.position' attribute of a derivation");

auto colon = pos.rfind(':');
if (colon == std::string::npos)
throw ParseError("cannot parse meta.position attribute '%s'", pos);
auto fn = path.path.abs();

auto fail = [fn]() {
throw ParseError("cannot parse 'meta.position' attribute '%s'", fn);
};

std::string filename(pos, 0, colon);
unsigned int lineno;
try {
lineno = std::stoi(std::string(pos, colon + 1, std::string::npos));
auto colon = fn.rfind(':');
if (colon == std::string::npos) fail();
std::string filename(fn, 0, colon);
auto lineno = std::stoi(std::string(fn, colon + 1, std::string::npos));
return {SourcePath{path.accessor, CanonPath(fn.substr(0, colon))}, lineno};
} catch (std::invalid_argument & e) {
throw ParseError("cannot parse line number '%s'", pos);
fail();
abort();
}

return { std::move(filename), lineno };
}


Expand Down
2 changes: 1 addition & 1 deletion src/libexpr/attr-path.hh
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ std::pair<Value *, PosIdx> findAlongAttrPath(
Value & vIn);

/* Heuristic to find the filename and lineno or a nix value. */
std::pair<std::string, uint32_t> findPackageFilename(EvalState & state, Value & v, std::string what);
std::pair<SourcePath, uint32_t> findPackageFilename(EvalState & state, Value & v, std::string what);

std::vector<Symbol> parseAttrPath(EvalState & state, std::string_view s);

Expand Down
10 changes: 6 additions & 4 deletions src/libexpr/eval-cache.cc
Original file line number Diff line number Diff line change
Expand Up @@ -442,8 +442,10 @@ Value & AttrCursor::forceValue()
if (v.type() == nString)
cachedValue = {root->db->setString(getKey(), v.string.s, v.string.context),
string_t{v.string.s, {}}};
else if (v.type() == nPath)
cachedValue = {root->db->setString(getKey(), v.path), string_t{v.path, {}}};
else if (v.type() == nPath) {
auto path = v.path().path;
cachedValue = {root->db->setString(getKey(), path.abs()), string_t{path.abs(), {}}};
}
else if (v.type() == nBool)
cachedValue = {root->db->setBool(getKey(), v.boolean), v.boolean};
else if (v.type() == nInt)
Expand Down Expand Up @@ -580,7 +582,7 @@ std::string AttrCursor::getString()
if (v.type() != nString && v.type() != nPath)
root->state.error("'%s' is not a string but %s", getAttrPathStr()).debugThrow<TypeError>();

return v.type() == nString ? v.string.s : v.path;
return v.type() == nString ? v.string.s : v.path().to_string();
}

string_t AttrCursor::getStringWithContext()
Expand Down Expand Up @@ -622,7 +624,7 @@ string_t AttrCursor::getStringWithContext()
if (v.type() == nString)
return {v.string.s, v.getContext(*root->state.store)};
else if (v.type() == nPath)
return {v.path, {}};
return {v.path().to_string(), {}};
else
root->state.error("'%s' is not a string but %s", getAttrPathStr()).debugThrow<TypeError>();
}
Expand Down
Loading