Merge pull request #1256: Merge milestones/M153 to releases/shipped

- Includes Git 2.22
- Moves placeholder database into SQLite for better performance.
- Fixes when libgit2 repos are acquired and disposed for better performance of checkout (where detected) and other operations that rely heavily on the read-object hook.
- Adds additional logging for folder and file placeholder creation.
- Other bug fixes for reliability

.vsconfig

@@ -0,0 +1,13 @@
+{
+  "version": "1.0",
+  "components": [
+    "Microsoft.Component.MSBuild",
+    "Microsoft.VisualStudio.Workload.NativeDesktop"
+    "Microsoft.VisualStudio.Workload.ManagedDesktop",
+    "Microsoft.VisualStudio.Workload.NetCoreTools",
+    "Microsoft.Net.Core.Component.SDK.2.1",
+    "Microsoft.VisualStudio.Component.VC.v141.x86.x64",
+    "Microsoft.Net.Component.4.6.1.TargetingPack",
+    "Microsoft.Net.Component.4.6.1.SDK",
+  ]
+}

AuthoringTests.md

@@ -2,39 +2,103 @@
 
 ## Functional Tests
 
-### 1. Running the functional tests
+#### Runnable functional test projects
 
-Our functional tests are in the `GVFS.FunctionalTests` project.  They are built on NUnit 3, which is available as a set of NuGet packages.
+- `GVFS.FunctionalTests`
+- `GVFS.FunctionalTests.Windows`
 
-To run the functional tests:
-1. Open GVFS.sln in Visual Studio
-2. Build all, which will download the NUnit framework and runner
-3. You have three options for how to run the tests, all of which are equivalent.
-   1. Run the `GVFS.FunctionalTests` project.  Even better, set it as the default project and hit F5.
-   2. Use the command line runner.  After building, execute `Scripts\RunFunctionalTests.bat`
-   3. If you want to use Visual Studio's Test Explorer, you need to install the NUnit 3 Test Adapter in VS | Tools | Extensions and Updates.
+`GVFS.FunctionalTests` is a .NET Core project and contains all cross-platform functional tests.  `GVFS.FunctionalTests.Windows`, contains functional tests that require Windows. Additionally, `GVFS.FunctionalTests.Windows` includes all the `GVFS.FunctionalTests` allowing it to run both cross-platform and Windows-specific functional tests.
 
-Running the `GVFS.FunctionalTests` project is probably the most convenient for developers.  `RunFunctionalTests.bat` will be used on the build machines.
+#### Other functional test projects
+
+*GVFS.NativeTests*
+
+`GVFS.NativeTests` contains tests written in C++ that use the Windows API directly.  These tests are called from the managed tests (see above) using PInvoke.
+
+*GVFS.FunctionalTests.LockHolder*
+
+The `LockHolder` is a small program that allows the functional tests to request and release the `GVFSLock`.  `LockHolder` is useful for simulating different timing/race conditions.
+
+## Running the Functional Tests
+
+The functional tests are built on NUnit 3, which is available as a set of NuGet packages.
+
+### Windows
+
+1. Build VFS for Git:
+
+    **Option 1:** Open GVFS.sln in Visual Studio and build everything.
+
+    **Option 2:** Run `Scripts\BuildGVFSForWindows.bat` from the command line
+
+2. Run the VFS4G installer that was built in step 2.  This will ensure that ProjFS is properly installed/enabled on your machine, and that VFS4G will be able to find the correct version of the pre/post-command hooks. The installer will be placed in `BuildOutput\GVFS.Installer.Windows\bin\x64\<Debug or Release>`
+3. Run the tests **with elevation**.  Elevation is required because the functional tests create and delete a test service.
+
+   **Option 1:** Run the `GVFS.FunctionalTests.Windows` project from inside Visual Studio launched as Administrator.
+
+   **Option 2:** Run `Scripts\RunFunctionalTests.bat` from CMD launched as Administrator.
+
+#### Selecting Which Tests are Run
+
+By default, the functional tests run a subset of tests as a quick smoke test for developers.  There are three mutually exclusive arguments that can be passed to the functional tests to change this behavior:
+
+- `--full-suite`: Run all configurations of all functional tests
+- `--extra-only`: Run only those tests marked as "ExtraCoverage" (i.e. the tests that are not run by default)
+- `--windows-only`: Run only the tests marked as being Windows specific
+
+**NOTE** `Scripts\RunFunctionalTests.bat` already uses some of these arguments.  If you run the tests using `RunFunctionalTests.bat` consider locally modifying the script rather than passing these flags as arguments to the script.
+
+### Mac
+
+1. Build VFS for Git: `Scripts/Mac/BuildGVFSForMac.sh`
+2. Run the tests: `Scripts/Mac/RunFunctionalTests.sh `
+
+If you need the VS for Mac debugger attached for a functional test run:
+
+1. Make sure you've built your latest changes
+2. Run `./ProjFS.Mac/Scripts/LoadPrjFSKext.sh`
+3. Open GVFS.sln in VS for Mac
+4. Run->Run With->Custom Configuration...
+5. Select "Start external program" and specify the published functional test binary (e.g. `/Users/<USERNAME>/Repos/VFSForGit/Publish/GVFS.FunctionalTests`)
+6. Specify any desired arguments (e.g. [a specific test](#Running-Specific-Tests) )
+7. Run Action -> "Debug - .Net Core Debugger"
+8. Click "Debug"
+
+### Customizing the Functional Test Settings
 
 The functional tests take a set of parameters that indicate what paths and URLs to work with.  If you want to customize those settings, they
 can be found in [`GVFS.FunctionalTests\Settings.cs`](/GVFS/GVFS.FunctionalTests/Settings.cs).
 
-### 2. Running Full Suite of Tests vs. Smoke Tests
 
-By default, the VFS for Git functional tests run a subset of tests as a quick smoke test for developers. To run all tests, pass in the `--full-suite` flag.
+## Running Specific Tests
+
+Specific tests can be run by adding the `--test=<comma separated list of tests>` command line argument to the functional test project/scripts.  
+
+Note that the test name must include the class and namespace and that `Debug` or `Release` must be specified when running the functional test scripts.
+
+*Example*
+
+Windows (Script):
+
+`Scripts\RunFunctionalTests.bat Debug --test=GVFS.FunctionalTests.Tests.EnlistmentPerFixture.GitFilesTests.CreateFileTest`
+
+Windows (Visual Studio):
+
+1. Set `GVFS.FunctionalTests.Windows` as StartUp project
+2. Project Properties->Debug->Start options->Command line arguments (all on a single line): `--test=GVFS.FunctionalTests.Tests.EnlistmentPerFixture.GitFilesTests.CreateFileTest`
 
-### 3. Running specific tests
+Mac:
 
-Specific tests can be run by specifying `--test=<comma separated list of tests>` as the command line arguments to the functional test project.
+`Scripts/Mac/RunFunctionalTests.sh Debug --test=GVFS.FunctionalTests.Tests.EnlistmentPerFixture.GitFilesTests.CreateFileTest`
 
-### 4. How to write a functional test
+## How to Write a Functional Test
 
 Each piece of functionality that we add to VFS for Git should have corresponding functional tests that clone a repo, mount the filesystem, and use existing tools and filesystem
 APIs to interact with the virtual repo.
 
 Since these are functional tests that can potentially modify the state of files on disk, you need to be careful to make sure each test can run in a clean 
 environment.  There are three base classes that you can derive from when writing your tests.  It's also important to put your new class into the same namespace
-as the base class, because NUnit treats namespaces like test suites, and we have logic that keys off of that for deciding when to create enlistments.
+as the base class, because NUnit treats namespaces like test suites, and we have logic that keys off that for deciding when to create enlistments.
 
 1. `TestsWithLongRunningEnlistment`
 
@@ -49,11 +113,11 @@ as the base class, because NUnit treats namespaces like test suites, and we have
 
 3. `TestsWithEnlistmentPerTestCase`
 
-   Derive from this class if you need a brand new enlistment per test case.  This is the most reliable, but also most expensive option.
+   Derive from this class if you need a new enlistment created for each test case.  This is the most reliable, but also most expensive option.
 
-### 5. Updating the remote test branch
+## Updating the Remote Test Branch
 
-By default, `GVFS.FunctionalTests` clones `master`, checks out the branch "FunctionalTests/YYYYMMDD" (with the day the FunctionalTests branch was created), 
-and then removes all remote tracking information. This is done to guarantee that remote changes to tip cannot break functional tests. If you need to update 
+By default, the functional tests clone `master`, check out the branch "FunctionalTests/YYYYMMDD" (with the day the FunctionalTests branch was created), 
+and then remove all remote tracking information. This is done to guarantee that remote changes to tip cannot break functional tests. If you need to update 
 the functional tests to use a new FunctionalTests branch, you'll need to create a new "FunctionalTests/YYYYMMDD" branch and update the `Commitish` setting in `Settings.cs` to have this new branch name.  
 Once you have verified your scenarios locally you can push the new FunctionalTests branch and then your changes.

GVFS.sln

@@ -103,6 +103,7 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "GVFS.Installer.Mac", "GVFS\
 		{FAC6EFC5-A890-4CB2-8C80-6358E358C637} = {FAC6EFC5-A890-4CB2-8C80-6358E358C637}
 		{374BF1E5-0B2D-4D4A-BD5E-4212299DEF09} = {374BF1E5-0B2D-4D4A-BD5E-4212299DEF09}
 		{35CA4DFB-1320-4055-B8F6-F12E0F252FF0} = {35CA4DFB-1320-4055-B8F6-F12E0F252FF0}
+		{AECEC217-2499-403D-B0BB-2962B9BE5970} = {AECEC217-2499-403D-B0BB-2962B9BE5970}
 	EndProjectSection
 EndProject
 Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "GVFS.SignFiles", "GVFS\GVFS.SignFiles\GVFS.SignFiles.csproj", "{2F63B22B-EE26-4266-BF17-28A9146483A1}"
@@ -120,6 +121,7 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "GVFS.SignFiles", "GVFS\GVFS
 		{4CE404E7-D3FC-471C-993C-64615861EA63} = {4CE404E7-D3FC-471C-993C-64615861EA63}
 		{24D161E9-D1F0-4299-BBD3-5D940BEDD535} = {24D161E9-D1F0-4299-BBD3-5D940BEDD535}
 		{93B403FD-DAFB-46C5-9636-B122792A548A} = {93B403FD-DAFB-46C5-9636-B122792A548A}
+		{AECEC217-2499-403D-B0BB-2962B9BE5970} = {AECEC217-2499-403D-B0BB-2962B9BE5970}
 	EndProjectSection
 EndProject
 Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "GVFS.VirtualFileSystemHook.Windows", "GVFS\GVFS.VirtualFileSystemHook\GVFS.VirtualFileSystemHook.Windows.vcxproj", "{2D23AB54-541F-4ABC-8DCA-08C199E97ABB}"
@@ -143,7 +145,6 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "GVFS.Mac", "GVFS\GVFS\GVFS.
 EndProject
 Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "GVFS.Windows", "GVFS\GVFS\GVFS.Windows.csproj", "{32220664-594C-4425-B9A0-88E0BE2F3D2A}"
 	ProjectSection(ProjectDependencies) = postProject
-		{AECEC217-2499-403D-B0BB-2962B9BE5970} = {AECEC217-2499-403D-B0BB-2962B9BE5970}
 		{2D23AB54-541F-4ABC-8DCA-08C199E97ABB} = {2D23AB54-541F-4ABC-8DCA-08C199E97ABB}
 		{798DE293-6EDA-4DC4-9395-BE7A71C563E3} = {798DE293-6EDA-4DC4-9395-BE7A71C563E3}
 		{5A6656D5-81C7-472C-9DC8-32D071CB2258} = {5A6656D5-81C7-472C-9DC8-32D071CB2258}
@@ -191,7 +192,7 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "GVFS.Hooks.Mac", "GVFS\GVFS
 		{A4984251-840E-4622-AD0C-66DFCE2B2574} = {A4984251-840E-4622-AD0C-66DFCE2B2574}
 	EndProjectSection
 EndProject
-Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "GVFS.Upgrader", "GVFS\GVFS.Upgrader\GVFS.Upgrader.csproj", "{AECEC217-2499-403D-B0BB-2962B9BE5970}"
+Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "GVFS.Upgrader", "GVFS\GVFS.Upgrader\GVFS.Upgrader.csproj", "{AECEC217-2499-403D-B0BB-2962B9BE5970}"
 EndProject
 Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "GVFS.PostIndexChangedHook.Windows", "GVFS\GVFS.PostIndexChangedHook\GVFS.PostIndexChangedHook.Windows.vcxproj", "{24D161E9-D1F0-4299-BBD3-5D940BEDD535}"
 	ProjectSection(ProjectDependencies) = postProject
@@ -263,9 +264,11 @@ Global
 		{03769A07-F216-456B-886B-E07CAF6C5E81}.Debug.Mac|x64.ActiveCfg = Debug|x64
 		{03769A07-F216-456B-886B-E07CAF6C5E81}.Debug.Mac|x64.Build.0 = Debug|x64
 		{03769A07-F216-456B-886B-E07CAF6C5E81}.Debug.Windows|x64.ActiveCfg = Debug|x64
+		{03769A07-F216-456B-886B-E07CAF6C5E81}.Debug.Windows|x64.Build.0 = Debug|x64
 		{03769A07-F216-456B-886B-E07CAF6C5E81}.Release.Mac|x64.ActiveCfg = Release|x64
 		{03769A07-F216-456B-886B-E07CAF6C5E81}.Release.Mac|x64.Build.0 = Release|x64
 		{03769A07-F216-456B-886B-E07CAF6C5E81}.Release.Windows|x64.ActiveCfg = Release|x64
+		{03769A07-F216-456B-886B-E07CAF6C5E81}.Release.Windows|x64.Build.0 = Release|x64
 		{5A6656D5-81C7-472C-9DC8-32D071CB2258}.Debug.Mac|x64.ActiveCfg = Debug|x64
 		{5A6656D5-81C7-472C-9DC8-32D071CB2258}.Debug.Windows|x64.ActiveCfg = Debug|x64
 		{5A6656D5-81C7-472C-9DC8-32D071CB2258}.Debug.Windows|x64.Build.0 = Debug|x64
@@ -413,7 +416,7 @@ Global
 		{FA273F69-5762-43D8-AEA1-B4F08090D624}.Debug.Windows|x64.ActiveCfg = Debug|x64
 		{FA273F69-5762-43D8-AEA1-B4F08090D624}.Debug.Windows|x64.Build.0 = Debug|x64
 		{FA273F69-5762-43D8-AEA1-B4F08090D624}.Release.Mac|x64.ActiveCfg = Release|x64
-		{FA273F69-5762-43D8-AEA1-B4F08090D624}.Release.Mac|x64.Build.0 = Debug|x64
+		{FA273F69-5762-43D8-AEA1-B4F08090D624}.Release.Mac|x64.Build.0 = Release|x64
 		{FA273F69-5762-43D8-AEA1-B4F08090D624}.Release.Windows|x64.ActiveCfg = Release|x64
 		{FA273F69-5762-43D8-AEA1-B4F08090D624}.Release.Windows|x64.Build.0 = Release|x64
 		{4CC2A90D-D240-4382-B4BF-5E175515E492}.Debug.Mac|x64.ActiveCfg = Debug|x64
@@ -430,6 +433,8 @@ Global
 		{AECEC217-2499-403D-B0BB-2962B9BE5970}.Release.Mac|x64.ActiveCfg = Release|x64
 		{AECEC217-2499-403D-B0BB-2962B9BE5970}.Release.Windows|x64.ActiveCfg = Release|x64
 		{AECEC217-2499-403D-B0BB-2962B9BE5970}.Release.Windows|x64.Build.0 = Release|x64
+		{AECEC217-2499-403D-B0BB-2962B9BE5970}.Debug.Mac|x64.Build.0 = Debug|x64
+		{AECEC217-2499-403D-B0BB-2962B9BE5970}.Release.Mac|x64.Build.0 = Release|x64
 		{24D161E9-D1F0-4299-BBD3-5D940BEDD535}.Debug.Mac|x64.ActiveCfg = Debug|x64
 		{24D161E9-D1F0-4299-BBD3-5D940BEDD535}.Debug.Windows|x64.ActiveCfg = Debug|x64
 		{24D161E9-D1F0-4299-BBD3-5D940BEDD535}.Debug.Windows|x64.Build.0 = Debug|x64

GVFS/FastFetch/CheckoutStage.cs

@@ -252,7 +252,7 @@ private void HandleAllFileDeleteOperations()
 
         private void HandleAllFileAddOperations()
         {
-            using (FastFetchLibGit2Repo repo = new FastFetchLibGit2Repo(this.tracer, this.enlistment.WorkingDirectoryRoot))
+            using (FastFetchLibGit2Repo repo = new FastFetchLibGit2Repo(this.tracer, this.enlistment.WorkingDirectoryBackingRoot))
             {
                 string availableBlob;
                 while (this.AvailableBlobShas.TryTake(out availableBlob, Timeout.Infinite))

GVFS/FastFetch/FastFetch.csproj

@@ -47,7 +47,7 @@
 
   <ItemGroup>
     <PackageReference Include="CommandLineParser" Version="2.1.1-beta" />
-    <PackageReference Include="LibGit2Sharp.NativeBinaries" Version="1.0.165" />
+    <PackageReference Include="LibGit2Sharp.NativeBinaries" Version="2.0.278" />
     <PackageReference Include="StyleCop.Analyzers" Version="1.0.2">
       <PrivateAssets>all</PrivateAssets>
     </PackageReference>

GVFS/FastFetch/GitEnlistment.cs

@@ -8,6 +8,7 @@ public class GitEnlistment : Enlistment
     {
         private GitEnlistment(string repoRoot, string gitBinPath)
             : base(
+                  repoRoot,
                   repoRoot,
                   repoRoot,
                   null,

GVFS/GVFS.Build/GVFS.props

@@ -3,7 +3,7 @@
 
   <PropertyGroup Label="Parameters">
     <GVFSVersion>0.2.173.2</GVFSVersion>
-    <GitPackageVersion>2.20190312.1</GitPackageVersion>
+    <GitPackageVersion>2.20190610.5</GitPackageVersion>
   </PropertyGroup>
 
   <PropertyGroup Label="DefaultSettings">