Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

利用jindofs-sdk包初始化FileSystem对象实例失败 #108

Open
baisui1981 opened this issue Jul 27, 2023 · 3 comments
Open

利用jindofs-sdk包初始化FileSystem对象实例失败 #108

baisui1981 opened this issue Jul 27, 2023 · 3 comments

Comments

@baisui1981
Copy link

按照 https://github.com/aliyun/alibabacloud-jindodata/blob/master/docs/user/3.x/hadoop/jindosdk_on_hadoop.md 文档所示，

  1. 下载jindofs-sdk，并且加入classpath中
  2. 在阿里云OSS中创建了基于OSS HDFS bucket，
  3. 本地依赖hadoop：3.2.1 版本相关的依赖包代码

希望能够初始化org.apache.hadoop.fs.FileSystem 实例对象。

在执行过程中报告以下错误，不知道是哪儿出错了，本机是MacOS机器

Caused by: java.lang.RuntimeException: JbootNative loading failed: java.lang.UnsatisfiedLinkError: /private/tmp/libjboot-3397ed5b1f5602cb_20211216_104547.dylib: dlopen(/private/tmp/libjboot-3397ed5b1f5602cb_20211216_104547.dylib, 1): no suitable image found.  Did find:
	/private/tmp/libjboot-3397ed5b1f5602cb_20211216_104547.dylib: cannot load 'libjboot-3397ed5b1f5602cb_20211216_104547.dylib' (load command 0x80000034 is unknown)
	/private/tmp/libjboot-3397ed5b1f5602cb_20211216_104547.dylib: cannot load 'libjboot-3397ed5b1f5602cb_20211216_104547.dylib' (load command 0x80000034 is unknown)
	at java.lang.ClassLoader$NativeLibrary.load(Native Method)
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
	at java.lang.Runtime.load0(Runtime.java:809)
	at java.lang.System.load(System.java:1086)
	at com.alibaba.jboot.NativeCodeLoader.loadLibrary(NativeCodeLoader.java:125)
	at com.alibaba.jboot.NativeCodeLoader.loadLibrary(NativeCodeLoader.java:82)
	at com.alibaba.jboot.NativeCodeLoader.load(NativeCodeLoader.java:44)
	at com.alibaba.jboot.JbootNative.<clinit>(JbootNative.java:19)
	at com.alibaba.jboot.JbootCache.<init>(JbootCache.java:11)
	at com.aliyun.emr.fs.internal.ossnative.OssFileletSystem.<init>(OssFileletSystem.java:36)
	at com.aliyun.emr.fs.internal.ossnative.OssNativeStore$Context.<init>(OssNativeStore.java:99)
	at com.aliyun.emr.fs.oss.JindoOssFileSystem.initNativeStoreContext(JindoOssFileSystem.java:140)
	at com.aliyun.emr.fs.oss.JindoOssFileSystem.initialize(JindoOssFileSystem.java:135)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:172)
	at com.qlangtech.tis.hdfs.impl.HdfsFileSystemFactory$HdfsUtils.getFileSystem(HdfsFileSystemFactory.java:320)
	at com.qlangtech.tis.hdfs.impl.HdfsFileSystemFactory$1.create(HdfsFileSystemFactory.java:173)
	at com.qlangtech.tis.hdfs.impl.HdfsFileSystemFactory$1.visit(HdfsFileSystemFactory.java:149)
	at com.qlangtech.tis.hdfs.impl.HdfsFileSystemFactory$1.visit(HdfsFileSystemFactory.java:145)
	at com.qlangtech.tis.config.authtoken.impl.OffUserToken.accept(OffUserToken.java:37)
	at com.qlangtech.tis.hdfs.impl.HdfsFileSystemFactory.createFS(HdfsFileSystemFactory.java:145)
	at com.qlangtech.tis.hdfs.impl.HdfsFileSystemFactory.getFileSystem(HdfsFileSystemFactory.java:113)
	at com.qlangtech.tis.hdfs.impl.TestAliayunJindoFSFactory.testListChild(TestAliayunJindoFSFactory.java:77)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

	at com.alibaba.jboot.JbootNative.checkLoaded(JbootNative.java:25)
	at com.alibaba.jboot.JbootNative.instance(JbootNative.java:30)
	at com.alibaba.jboot.JbootNative.initOnceNativeClass(JbootNative.java:74)
	at com.alibaba.jboot.JbootCache.<init>(JbootCache.java:11)
	at com.aliyun.emr.fs.internal.ossnative.OssFileletSystem.<init>(OssFileletSystem.java:36)
	at com.aliyun.emr.fs.internal.ossnative.OssNativeStore$Context.<init>(OssNativeStore.java:99)
	at com.aliyun.emr.fs.oss.JindoOssFileSystem.initNativeStoreContext(JindoOssFileSystem.java:140)
	at com.aliyun.emr.fs.oss.JindoOssFileSystem.initialize(JindoOssFileSystem.java:135)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:172)
	at com.qlangtech.tis.hdfs.impl.HdfsFileSystemFactory$HdfsUtils.getFileSystem(HdfsFileSystemFactory.java:320)
	... 32 more
@yanbinyang
Copy link
Collaborator

@baisui1981 你是不是 mac m1/m2 的机型？是的话，可能得用jindosdk-4.6.4-linux-el7-aarch64.tar.gz

@baisui1981
Copy link
Author

baisui1981 commented Jul 27, 2023 •
edited
Loading

@baisui1981 你是不是 mac m1/m2 的机型？是的话，可能得用jindosdk-4.6.4-linux-el7-aarch64.tar.gz

谢谢如此快速的回复！我现在用的 jindofs-sdk包是否只能在centOS服务器上使用？我现在就按照你说的，试试

@yanbinyang
Copy link
Collaborator

yanbinyang commented Jul 27, 2023 •
edited
Loading

@baisui1981 你是不是 mac m1/m2 的机型？是的话，可能得用jindosdk-4.6.4-linux-el7-aarch64.tar.gz

谢谢如此快速的回复！我现在用的 jindofs-sdk包是否只能在centOS服务器上使用？我现在就按照你说的，试试

不是，我们支持多平台使用的，4.x所有版本，基本上都支持macos x86、大部分linux发行版。
如果是macos arm版本，由于使用的人不多，目前只有4.6.4版本做了发布，以供demo使用。
下载链接参见jindodata_download.md。
安装说明参见install_dependeny_jindosdk.md

另外，你看的文档是我们的老版本3.x文档。目前最新4.x版本的文档参见jindosdk_ide_hadoop.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@yanbinyang @baisui1981