[Feature Request] Disable caching for local/file system feeds

[leastprivilege]

For our builds we are using a local nuget feed to get dependencies from.

This local feed gets updated for every build - the version number of the packages do not always change.

Since nuget prefers the local cache (.nuget) over the file system source - we need to clear that cache manually all the time to make this work.

It would be nice if caching could be disabled on a per feed basis (especially for local / file system based feeds).

This is for SDK style / package reference based projects.

Build script: https://github.com/IdentityServer/IdentityServer4/blob/master/build.ps1

[rrelyea]

It is possible to use fallback folders to get this behavior today.
Configure a fallback folder via nuget.config.

NuGet will search the global package folder, then the fallback folders.
If either of those locations has the appropriate package, no package sources will be searched.
If the package was found in the fallback folder, it won't be copied to the global package folder.

[rrelyea]

To configure FallbackPackageFolders via nuget.config:

<configuration>
  <fallbackPackageFolders>
    <add key="XYZ Offline Packages" value="C:\somePath\someFolder\"/>
  </fallbackPackageFolders>
</configuration>

We don't have this covered in documentation very well. We added the feature in NuGet 3.5, I believe...and I've opened a doc issue with some details: https://github.com/NuGet/docs.microsoft.com-nuget/issues/1487

[leastprivilege]

I tried this

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <fallbackPackageFolders>
    <add key="IdentityServer Build Packages" value="./nuget"/>
  </fallbackPackageFolders>
</configuration>

But now the behavior has changed

I am referencing version 2.5.0-* in my directory.targets

and my fallback folder has 2.5.0-preview.2.50, but nuget.org has 2.5.0-preview.2.6

with the previous approach 2.5.0-preview.2.50 was referenced during the build - now 2.5.0-preview.2.6 is used.

I guess because nuget.org is used before the fallback folder. Can I make the nuget client look into the fallback folder first before the package sources?

[rrelyea]

If you are using * in a version (2., 1.0.0-, etc...) -- a full restore will always check all package sources.
We need to collect all possible versions to find the latest matching one.

[leastprivilege]

But it seems the fallback folder is not checked if a matching version is found in the package sources.

But the fallback folder has the more recent version in this case.

I still think that what we really want is to be able to disable caching per package source.

[304NotModified]

This local feed gets updated for every build - the version number of the packages do not always change.

Isn't this the issue? Why not adding the build number to the package?

[MrAntix]

Please consider re-opening this

We use a local folder for development of nuget packages, this involves creating a package and building the consuming app referencing that package.

If there are bugs or additional features we want to put in the same release, we have to clear the nuget caches, sometimes having to restart windows due to locks.

Then we must restore all the packages again including those which are not local development but from nuget.org

To help us out, we would like to be able to say "don't cache this package source" when it is from a local path.

nuget.config

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
     <add key="local" value="./nuget" disableCache="True" />
  </packageSources>
</configuration>

Using a new version number is not what we want to do until the package is to be released.

This would be opt in so no backwards compatibility issues.

Many thanks

[dobri-002]

Disabling cache for specific feeds would be very useful. We are fighting every day with exactly the same issues that @MrAntix described.

[CEbbinghaus]

+1 for the ability to disable the cache for certain feeds. Not only for not copying the package to the cache upon resolution but also for not resolving packages from the local cache. This would allow Project A to depend on a local copy of package X while Project B can use the remote and cache it without impacting Project A.

Please consider adding this feature

[gioce90]

Oh I'm happy to see I'm not alone facing this problem! This should be a must-have feature, because - as all of you - I'm facing this same problem and fallback folder nor a build number suit's well when we are talking about LOCAL DEVELOPMENT.

[gioce90]

@rrelyea can you re-open this feature request? There are really good reasons to have this feature.
I want to explain my point (@everyone: please thumb-up or comment if the situation is somehow relative to you).

I think that we all can agree that the best thing is to have a robust CI/CD that:

• creates official release version of a package when we close the PR to the main branch (or some somehow similar approach)
• creates a pre-release version of a package when we push commits on a feature branch

(It's not always exactly like this, but in general terms we can say that it's the norm)

But when we talk about local development, things get a little confusing. I think the current state of NuGet forces us to do a lot of pointless pushes to the remote repository, just creating an incredible number of useless pre-release builds when we could just test those features directly from another project of ours on the local machine.

That's my point. If you want to understand better why, follow my example:

• I have the package called MyPackage with version 8.0.1.
• Now I need to implement a new feature. I create a branch and every single push to the remote repository will pack and publish a new pre-release version with this format: x.y.z-myFeatureBranchName-date.build. Where:
  • x.y.z is a SemVer versioning
  • myFeatureBranchName is self-explanatory
  • date is the date when the pipeline runs
  • build is a progressive number, first time is 1, second time is 2, etc.
  • EXAMPLE: 8.0.2-myNewFeature-20241111.1
• Before to produce an official version of MyPackage, I need to tests these changes from ANOTHER project, a bigger one called MyApp. I don't want to open and close a PR to the main branch of MyPackage project before everything is tested and works also in this other MyApp project. I want to create only the official release version at the end, when everything is fine in both projects.
• So, I open my MyApp's .csproj and now I change the version used of MyPackage from 8.0.1 to 8.0.2-myNewFeature-20241111.1 (look the progressive build number).
• I test everything and, damn, I notice that I have to change a really small detail in the package project.
• Now I've done this little change on the package project, I have to commit, push, wait the pipelines to run (runs that my company pays and require an amount of time considerable), go to the MyApp project, change the version of MyPackage to 8.0.2-myNewFeature-20241111.2
  • Note I have to be careful to change the "build" part every single time... when the focus is compromised due to pipeline execution lengths, it's perfectly easy to make a mistake here).
• If I see other problems, repeat the previous step until I get a satisfactory result.

... do you see how much time I wasted?

POSSIBLE SOLUTION:
It would be MUCH faster if NuGet provided a simple and straightforward way to use a Local Feed without caching issues.
In this way, in my MyApp project I would simply set the version of MyPackage to "8.0.2-Local"... without changing the version every time I build the package on the local machine!
And only when I am really ready to push these commits to the remote package repository, on the MyApp project I will switch that "-Local" suffix with the pre-release or official release version of the package.

Please note that:

• Of course this is only to speed up the local development and testing of a package project by another, always local, project. This is not meant to invalidate or skip the pre-release CI/CD. But now the developer is free to push commits only when he wants.
• I now that this is actually achievable by removing the PackageReference tag and introducing a Project Reference. But replacing the PackageReference tag with the Reference tag in its place seems wrong to me (and it feels to interrupt that seamless versioning, IMHO). Also, in this way I lose control of other things, like SourceLink, .nuspec etc...
• fallback package folder doesn't resolve the problem if a previous "MyPackage.8.0.2-Local" local version is already in the global package folder. Because the point here is to not change that 8.0.2 every single time (in local, of course).

MY CURRENT SOLUTION:

• I created a Directory.Builds.props where I have MSBuild targets that checks if the build is in Local or in a Remote pipeline and, if is in Local, creates a local pre-release version (with "-Local" suffix) in a Local Feed.
  • The "-Local" suffix is useful to understand that is a pre-release local version, that is personal of the developer and of the local machine, and it would never be published on the remote feed. Furthermore, this package has DevelopmentDependency flag to true.
• I recently resolved a issue with the cache: packages in the Local Feed should not be cached in the global-packages folder because my continuous builds in local requires to delete the old cached package. I achieve this always with MSBuild in the Directory.Builds.props file, where I delete ONLY the previous version of the specific package (and I avoid to clean all the cache)... so everything works but I recommend NuGet to accept this feature request and made this out-of-the-box.

This is my opinion, yours is welcome.

[rrelyea]

@rrelyea can you re-open this feature request?

Sorry, I'm no longer working on NuGet

[gioce90]

I opened a new feature request here: #13918 adding more context.