Ozone ofs in presto #7259
Replies: 9 comments 28 replies
-
|
Yes, ofs is a Hadoop compatible protocol and should be able to be used by Presto. Our Ozone cluster is currently being used by Presto to query Hive tables on Ozone (instead of HDFS). |
Beta Was this translation helpful? Give feedback.
-
|
Hi @ivandika3 Thanks for the quick response. I tried the OFS protocol by adding the jar
|
Beta Was this translation helpful? Give feedback.
-
|
@lukmanulhakkeem Does the client runs on the docker environment as well? Maybe you can take a look at @adoroszlai patch #5016 for docker-compose setup and script for Trino, Iceberg, and Ozone. I'm personally not familiar with the details for Trino/Presto on Ozone, just saying that it's possible to run Trino/Presto on top of Ozone. Edit: Seems #5016 runs it on top of S3G, instead of ofs. |
Beta Was this translation helpful? Give feedback.
-
|
No, the client is running in my local. As you mentioned the above PR is using S3G. Is there any idea how this internal IP is getting to the client. I am not aware of the internal communication inside the ozone. I guess it will work If I managed to supply the public IP instead of internal IP to the client. But not sure how it can be done. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, maybe because the datanode IPs was not exposed to localhost. Can try upload the key using ozone cli using I'll try to come up with PoC when I have time. |
Beta Was this translation helpful? Give feedback.
-
|
Sure @ivandika3 i will try it out. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, usually I will follow https://cwiki.apache.org/confluence/display/OZONE/Run+Ozone+cluster+from+IDE to setup local Ozone cluster from Intellij. Can try this order:
Afterwards, you can do some sanity checking first by uploading 1 or 2 objects. |
Beta Was this translation helpful? Give feedback.
-
|
This is not only the issue from presto. I followed the below documentation to try locally using hadoop I can create volume and schema using |
Beta Was this translation helpful? Give feedback.
-
|
@ivandika3 @adoroszlai - So presto should be there in the same machine where ozone om is installed right? I am assuming this would be the same regardless of my setup like docker or in a physical cluster. Please correct me If I am wrong |
Beta Was this translation helpful? Give feedback.
-
|
@lukmanulhakkeem Here's the client requirements:
So not necessarily the same machine, you can whether you can connect by using ping or telnet on OM and DN. |
Beta Was this translation helpful? Give feedback.
-
No, docker will create an internal network that cannot be accessed from outside unless you use port forwarding. Even if you port forward the datanodes, the datanode network address returned by SCM and OM to client will be the docker internal network address, which cannot be used by client to connect from outside docker network. |
Beta Was this translation helpful? Give feedback.
-
|
Yes I understand that. But will that be the same if I am having ozone in a physical machine, for eg say ozone in Cloudera. |
Beta Was this translation helpful? Give feedback.
-
|
Not familiar with Ozone on Cloudera, but should be similar client setup. |
Beta Was this translation helpful? Give feedback.
-
|
Does the client directly interact with data nodes both during read and write? |
Beta Was this translation helpful? Give feedback.
-
|
Yes, client will directly connect to datanodes for read and write ops. |
Beta Was this translation helpful? Give feedback.
-
|
@ivandika3 - I tried presto and ozone in the same machine. This is working with hive catalog in presto. But I tried the remote connectivity to an ozone instance in cloudera. It throws the same error I posted in the previous thread. The client is trying to access the internal IP of the datanode which is in the cloudera VM. So this is not only with the docker setup. Is there a way I can specify datanode host at the client side using ozone-site.xml or hdfs-site.xml? |
Beta Was this translation helpful? Give feedback.
-
|
@ivandika3 I made the changes to use hostname always for the connection [ But Select is not working. I am getting below error when I am doing I checked |
Beta Was this translation helpful? Give feedback.
-
|
@ivandika3 @adoroszlai any idea on this? |
Beta Was this translation helpful? Give feedback.
-
@lukmanulhakkeem Glad to know it works.
Since Ozone uses gRPC to read (with Netty transport), perhaps you can change the |
Beta Was this translation helpful? Give feedback.
-
|
Thanks very much @ivandika3 . I will try this |
Beta Was this translation helpful? Give feedback.
-
|
@ivandika3 Now select is working..Thanks |
Beta Was this translation helpful? Give feedback.
-
|
@lukmanulhakkeem try using Docker's host network driver. This eliminates container-specific IP addresses and NAT. On the flip side, you'll need to configure different set of ports for each service with multiple instances (e.g. each datanode) to avoid port conflict. |
Beta Was this translation helpful? Give feedback.
-
|
@ivandika3 @adoroszlai - I cloned the repo and checkout to 
But I am getting the below error in presto when used the above jar 
But it was working fine with the below Jar from the maven repository. https://mvnrepository.com/artifact/org.apache.ozone/ozone-filesystem-hadoop3/1.4.0 Is there any change in the protobuf in ozone? |
Beta Was this translation helpful? Give feedback.
-
|
Where did you put the |
Beta Was this translation helpful? Give feedback.
-
|
@adoroszlai Maybe related to #6359? |
Beta Was this translation helpful? Give feedback.
-
|
I manually installed the jar to local maven repository and added that dependency in the |
Beta Was this translation helpful? Give feedback.
-
|
@lukmanulhakkeem Ozone comes with two different FS jars for Hadoop 3, where the only difference is that:
Protobuf itself is not present in either jar, it needs to be provided, just like Hadoop's other dependencies. Since it was working fine with non-shaded protobuf, I guess your environment comes with non-shaded protobuf. Please use |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @adoroszlai . Then I will check from the presto end. |
Beta Was this translation helpful? Give feedback.
-
Hi folks, I am able to connect to ozone using the iceberg catalog from presto using the s3 gateway. But is it possible to connect to ozone from presto using the ofs protocol? Did anyone try this?
Beta Was this translation helpful? Give feedback.
All reactions