index: data: support imports

Stepping stone to simplifying `dvc fetch/pull` by using index.

Fetch handles regular imports through index already, but not repo
imports because their processing is much more involved (e.g. chained
imports) in the current arch.

With `FileStorage` support introduced into `DataIndex` and `datafs`
supporting imports overall, `dvcfs` can now handle repo imports (even
chained ones). This will soon allow us to handle repo imports the same
way we handle regular ones, improve performance and get rid of a lot of
messy code (e.g. DependencyRepo).

Related iterative/scmrepo#207
Related iterative/dvc-data#315
Related https://github.com/iterative/studio/issues/5261

dvc/dependency/repo.py

@@ -36,11 +36,18 @@ class RepoDependency(Dependency):
     }
 
     def __init__(self, def_repo: Dict[str, str], stage: "Stage", *args, **kwargs):
+        from dvc.fs import DVCFileSystem
+
         self.def_repo = def_repo
         self._objs: Dict[str, "HashFile"] = {}
         self._meta: Dict[str, "Meta"] = {}
         super().__init__(stage, *args, **kwargs)
 
+        self.fs = DVCFileSystem(
+            self.def_repo[self.PARAM_URL],
+            rev=self.def_repo.get(self.PARAM_REV_LOCK),
+        )
+
     def _parse_path(self, fs, fs_path):  # noqa: ARG002
         return None
 

dvc/repo/index.py

@@ -161,7 +161,7 @@ def _load_storage_from_out(storage_map, key, out):
     except NoRemoteError:
         pass
 
-    if out.stage.is_import and not out.stage.is_repo_import:
+    if out.stage.is_import:
         dep = out.stage.deps[0]
         storage_map.add_data(FileStorage(key, dep.fs, dep.fs_path))