Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use linked resource instead of filesystem #2658

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
/*******************************************************************************
* Copyright (c) 2023 Red Hat, Inc. and others.
* All rights reserved. This program and the accompanying materials
* are made available under the terms of the Eclipse Public License 2.0
* which accompanies this distribution, and is available at
* https://www.eclipse.org/legal/epl-2.0/
*
* SPDX-License-Identifier: EPL-2.0
*******************************************************************************/
package org.eclipse.jdt.ls.core.internal.managers;

import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;
import java.nio.file.StandardOpenOption;
import java.nio.file.attribute.BasicFileAttributeView;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.stream.Stream;

import org.eclipse.core.internal.preferences.EclipsePreferences;
import org.eclipse.core.resources.IFile;
import org.eclipse.core.resources.IFolder;
import org.eclipse.core.resources.IProject;
import org.eclipse.core.resources.IProjectDescription;
import org.eclipse.core.resources.IResource;
import org.eclipse.core.resources.ResourcesPlugin;
import org.eclipse.core.runtime.CoreException;
import org.eclipse.core.runtime.IPath;
import org.eclipse.core.runtime.NullProgressMonitor;
import org.eclipse.core.runtime.Platform;
import org.eclipse.core.runtime.Status;
import org.eclipse.jdt.core.IJavaProject;

class LinkResourceUtil {

private static final IPath METADATA_FOLDER_PATH = ResourcesPlugin.getWorkspace().getRoot().getLocation().append(".projects");

private static boolean isNewer(Path file, Instant instant) throws CoreException {
try {
Instant creationInstant = Files.getFileAttributeView(file, BasicFileAttributeView.class).readAttributes().creationTime().toInstant();
// match accuracy
ChronoUnit smallestSupportedUnit = Stream.of(ChronoUnit.NANOS, ChronoUnit.MILLIS, ChronoUnit.SECONDS) // IMPORTANT: keeps units in Stream from finer to coarser
.filter(creationInstant::isSupported).filter(instant::isSupported) //
.findFirst().orElse(null);
if (Platform.OS_MACOSX.equals(Platform.getOS())) {
// macOS filesystem has second-only accuracy (despite Instant.isSupported returns)
smallestSupportedUnit = ChronoUnit.SECONDS;
}
if (smallestSupportedUnit != null) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check on MacOS today, looks like we still have the problem here.

smallestSupportedUnit is assigned to ChronoUnit.NANOS, though the accuracy of creationInstant is second, makes this method returns false finally.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK thanks, so should we just hardcode the rule for macOS to trim to the second?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess so. I couldn't think of any better solution for this case.

(I was thinking about find out all the already existing metadata files before import started, but that may have too much perf penalty and may be not worth it.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking about find out all the already existing metadata files before import started, but that may have too much perf penalty and may be not worth it.

The difficulty here is that we don't know at that stage what folders the importers are going to consider as projects, so we're not sure which files matter or not.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess you can just those files start from rootPaths, because all the metadata files that exist before import happens will not be linked, so we don't need to care about what folders will be imported by which importers.

I'm not optimistic about the performance though...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean recursively crawling the whole subtrees under rootPAths for pre-existing .classpath, .project and so on?
Indeed, it could impact heavily performance for users using the wrong root (eg their home or filesystem root), but for a regular development project, it should be fast enough (I don't expect it to take more than 1 seconds for the biggest projects). But if we're ready to loose 1 second, we can just put a 1 second wait directly after Instant.now , and the check will then be reliable on all OSs ;)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the new patch, I've hardcoded that Instant should be truncated and compared by the second on macOS.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But if we're ready to loose 1 second, we can just put a 1 second wait directly after Instant.now , and the check will then be reliable on all OSs

That's a good insight! I think there are some key metrics we need to track when roll out this change. Then we can make the decision based on the data, like:

  • how many users happen to meet the two equally instants on second during import.
  • how many users fail to link the files.

For example, if the data shows that very few users have two equally instant on second, then we can just simply trim to the second.

creationInstant.truncatedTo(smallestSupportedUnit);
instant.truncatedTo(smallestSupportedUnit);
} else {
throw new CoreException(Status.error("No supported time unit!"));
}
return creationInstant.equals(instant) || creationInstant.isAfter(instant);
} catch (IOException ex) {
throw new CoreException(Status.error(ex.getMessage(), ex));
}
}

/**
* Get the redirected path of the input path. The path will be redirected to
* the workspace's metadata folder ({@link LinkResourceUtil#METADATA_FOLDER_PATH}).
* @param projectName name of the project.
* @param path path needs to be redirected.
* @return the redirected path.
*/
private static IPath getMetaDataFilePath(String projectName, IPath path) {
if (path.segmentCount() == 1) {
return METADATA_FOLDER_PATH.append(projectName).append(path);
}

String lastSegment = path.lastSegment();
if (IProjectDescription.DESCRIPTION_FILE_NAME.equals(lastSegment)) {
return METADATA_FOLDER_PATH.append(projectName).append(lastSegment);
}

return null;
}

private static void linkFolderIfNewer(IFolder settingsFolder, Instant instant) throws CoreException {
if (settingsFolder.isLinked()) {
return;
}
if (settingsFolder.exists() && !isNewer(settingsFolder.getLocation().toPath(), instant)) {
return;
}
if (!settingsFolder.exists()) {
// not existing yet, create link
File diskFolder = getMetaDataFilePath(settingsFolder.getProject().getName(), settingsFolder.getProjectRelativePath()).toFile();
diskFolder.mkdirs();
settingsFolder.createLink(diskFolder.toURI(), IResource.NONE, new NullProgressMonitor());
} else if (isNewer(settingsFolder.getLocation().toPath(), instant)) {
// already existing but not existing before import: move then link
File sourceFolder = settingsFolder.getLocation().toFile();
File targetFolder = getMetaDataFilePath(settingsFolder.getProject().getName(), settingsFolder.getProjectRelativePath()).toFile();
File parentTargetFolder = targetFolder.getParentFile();
if (!parentTargetFolder.isDirectory()) {
parentTargetFolder.mkdirs();
}
sourceFolder.renameTo(targetFolder);
jdneo marked this conversation as resolved.
Show resolved Hide resolved
settingsFolder.createLink(targetFolder.toURI(), IResource.REPLACE, new NullProgressMonitor());
}
}

private static void linkFileIfNewer(IFile metadataFile, Instant instant, String ifEmpty) throws CoreException {
if (metadataFile.isLinked()) {
return;
}
if (metadataFile.exists() && !isNewer(metadataFile.getLocation().toPath(), instant)) {
return;
}
File targetFile = getMetaDataFilePath(metadataFile.getProject().getName(), metadataFile.getProjectRelativePath()).toFile();
targetFile.getParentFile().mkdirs();
try {
if (metadataFile.exists()) {
Files.move(metadataFile.getLocation().toFile().toPath(), targetFile.toPath(), StandardCopyOption.REPLACE_EXISTING);
} else {
Files.writeString(targetFile.toPath(), ifEmpty != null ? ifEmpty : "", StandardOpenOption.CREATE);
}
} catch (IOException ex) {
throw new CoreException(Status.error(ex.getMessage(), ex));
}
metadataFile.createLink(targetFile.toURI(), IResource.REPLACE, new NullProgressMonitor());
}

public static void linkMetadataResourcesIfNewer(IProject project, Instant instant) throws CoreException {
linkFileIfNewer(project.getFile(IProjectDescription.DESCRIPTION_FILE_NAME), instant, null);
jdneo marked this conversation as resolved.
Show resolved Hide resolved
linkFolderIfNewer(project.getFolder(EclipsePreferences.DEFAULT_PREFERENCES_DIRNAME), instant);
linkFileIfNewer(project.getFile(IJavaProject.CLASSPATH_FILE_NAME), instant, null);
linkFileIfNewer(project.getFile(".factorypath"), instant, "<factorypath/>");
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -17,16 +17,19 @@

import java.io.File;
import java.net.URI;
import java.time.Instant;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import java.util.TreeMap;
import java.util.Map.Entry;
import java.util.stream.Collectors;
import java.util.stream.Stream;

Expand Down Expand Up @@ -145,6 +148,8 @@ private void updateEncoding(IProgressMonitor monitor) throws CoreException {
protected void importProjects(Collection<IPath> rootPaths, IProgressMonitor monitor) throws CoreException, OperationCanceledException {
SubMonitor subMonitor = SubMonitor.convert(monitor, rootPaths.size() * 100);
MultiStatus importStatusCollection = new MultiStatus(IConstants.PLUGIN_ID, -1, "Failed to import projects", null);
IProject[] alreadyExistingProjects = ProjectsManager.getWorkspaceRoot().getProjects();
Instant importSessionStart = Instant.now();
for (IPath rootPath : rootPaths) {
File rootFolder = rootPath.toFile();
try {
Expand All @@ -163,6 +168,18 @@ protected void importProjects(Collection<IPath> rootPaths, IProgressMonitor moni
JavaLanguageServerPlugin.logException("Failed to import projects", e);
}
}
if (!generatesMetadataFilesAtProjectRoot()) {
Set<IProject> newProjects = new HashSet<>(Arrays.asList(getWorkspaceRoot().getProjects()));
newProjects.removeAll(Arrays.asList(alreadyExistingProjects));
for (IProject project : newProjects) {
try {
LinkResourceUtil.linkMetadataResourcesIfNewer(project, importSessionStart);
} catch (CoreException e) {
importStatusCollection.add(e.getStatus());
JavaLanguageServerPlugin.logException("Failed to import projects", e);
}
}
}
if (!importStatusCollection.isOK()) {
throw new CoreException(importStatusCollection);
}
Expand Down Expand Up @@ -349,10 +366,29 @@ public static IProject createJavaProject(IProject project, IProgressMonitor moni
return createJavaProject(project, null, "src", "bin", monitor);
}

/*
* ⚠ These value is duplicated in ProjectsManager as both bundles must remain independent,
* but the same dir should be used for .project or .settings/.classpath.
* So when updating one, think about updating the other.
*/
public static final String GENERATES_METADATA_FILES_AT_PROJECT_ROOT = "java.import.generatesMetadataFilesAtProjectRoot";

/**
* Check whether the metadata files needs to be generated at project root.
*/
public static boolean generatesMetadataFilesAtProjectRoot() {
String property = System.getProperty(GENERATES_METADATA_FILES_AT_PROJECT_ROOT);
if (property == null) {
return true;
}
return Boolean.parseBoolean(property);
}

public static IProject createJavaProject(IProject project, IPath projectLocation, String src, String bin, IProgressMonitor monitor) throws CoreException, OperationCanceledException {
if (project.exists()) {
return project;
}
Instant creationRequestInstant = Instant.now();
JavaLanguageServerPlugin.logInfo("Creating the Java project " + project.getName());
//Create project
IProjectDescription description = ResourcesPlugin.getWorkspace().newProjectDescription(project.getName());
Expand Down Expand Up @@ -400,6 +436,10 @@ public static IProject createJavaProject(IProject project, IPath projectLocation
//Add JVM to project class path
javaProject.setRawClasspath(classpaths.toArray(new IClasspathEntry[0]), monitor);

if (!generatesMetadataFilesAtProjectRoot()) {
LinkResourceUtil.linkMetadataResourcesIfNewer(project, creationRequestInstant);
}

JavaLanguageServerPlugin.logInfo("Finished creating the Java project " + project.getName());
return project;
}
Expand Down Expand Up @@ -434,6 +474,7 @@ public IStatus runInWorkspace(IProgressMonitor monitor) {
updateEncoding(monitor);
project.deleteMarkers(BUILD_FILE_MARKER_TYPE, false, IResource.DEPTH_ONE);
long elapsed = System.currentTimeMillis() - start;
replaceLinkedMetadataWithLocal(project);
JavaLanguageServerPlugin.logInfo("Updated " + projectName + " in " + elapsed + " ms");
} catch (Exception e) {
String msg = "Error updating " + projectName;
Expand Down Expand Up @@ -641,6 +682,13 @@ public void reportProjectsStatus() {
JavaLanguageServerPlugin.sendStatus(ServiceStatus.ProjectStatus, "OK");
}
}

private void replaceLinkedMetadataWithLocal(IProject p) throws CoreException {
if (new File(p.getLocation().toFile(), IJavaProject.CLASSPATH_FILE_NAME).exists() &&
p.getFile(IJavaProject.CLASSPATH_FILE_NAME).isLinked()) {
p.getFile(IJavaProject.CLASSPATH_FILE_NAME).delete(false, false, null);
}
}

class UpdateProjectsWorkspaceJob extends WorkspaceJob {

Expand Down
7 changes: 0 additions & 7 deletions org.eclipse.jdt.ls.filesystem/.classpath

This file was deleted.

45 changes: 0 additions & 45 deletions org.eclipse.jdt.ls.filesystem/.project

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

16 changes: 0 additions & 16 deletions org.eclipse.jdt.ls.filesystem/META-INF/MANIFEST.MF

This file was deleted.

6 changes: 0 additions & 6 deletions org.eclipse.jdt.ls.filesystem/build.properties

This file was deleted.

14 changes: 0 additions & 14 deletions org.eclipse.jdt.ls.filesystem/plugin.properties

This file was deleted.

10 changes: 0 additions & 10 deletions org.eclipse.jdt.ls.filesystem/plugin.xml

This file was deleted.

13 changes: 0 additions & 13 deletions org.eclipse.jdt.ls.filesystem/pom.xml

This file was deleted.

Loading