Skip to content

Commit

Permalink
feature: use colima and latest packages at this point
Browse files Browse the repository at this point in the history
  • Loading branch information
fithisux committed Oct 4, 2023
1 parent f96b8ba commit 93b6630
Show file tree
Hide file tree
Showing 8 changed files with 27 additions and 8,209 deletions.
9 changes: 5 additions & 4 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,18 @@ FROM adoptopenjdk/openjdk8:latest

WORKDIR /opt

ENV HADOOP_VERSION=3.3.4
ENV HADOOP_VERSION=3.3.6
ENV METASTORE_VERSION=3.1.3
ENV MYSQL_CONNECTORJ=mysql-connector-j-8.1.0

ENV HADOOP_HOME=/opt/hadoop-${HADOOP_VERSION}
ENV HIVE_HOME=/opt/apache-hive-metastore-3.1.3-bin
ENV HADOOP_OPTIONAL_TOOLS="hadoop-aws"

RUN curl -L https://repo1.maven.org/maven2/org/apache/hive/hive-standalone-metastore/3.1.3/hive-standalone-metastore-3.1.3-bin.tar.gz | tar zxf -
RUN curl -L https://repo1.maven.org/maven2/org/apache/hive/hive-standalone-metastore/${METASTORE_VERSION}/hive-standalone-metastore-${METASTORE_VERSION}-bin.tar.gz | tar zxf -
RUN curl -L https://dlcdn.apache.org/hadoop/common/hadoop-${HADOOP_VERSION}/hadoop-${HADOOP_VERSION}.tar.gz | tar zxf -
RUN curl -L https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-j-8.0.32.tar.gz | tar zxf -
RUN cp mysql-connector-j-8.0.32/mysql-connector-j-8.0.32.jar ${HIVE_HOME}/lib/ && rm -rf mysql-connector-j-8.0.32
RUN curl -L https://dev.mysql.com/get/Downloads/Connector-J/${MYSQL_CONNECTORJ}.tar.gz | tar zxf -
RUN cp ${MYSQL_CONNECTORJ}/${MYSQL_CONNECTORJ}.jar ${HIVE_HOME}/lib/ && rm -rf ${MYSQL_CONNECTORJ}

RUN apt-get update && apt-get install -y netcat

Expand Down
16 changes: 16 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,20 @@ My experiments with minio, trino and delta lake. It is supporting material for m

[Manipulating Delta Lake tables on MinIO with Trino](https://fithis2001.medium.com/manipulating-delta-lake-tables-on-minio-with-trino-74b25f7ad479)

Start it with

```
export UID=${UID}
export GID=${GID}
docker-compose up
```

end it with


```
docker-compose down -v
```


Tested on Macos x64 Ventura 13.5.2 with Colima 0.5.5 .
Loading

0 comments on commit 93b6630

Please sign in to comment.