Skip to content

Commit

Permalink
Add Java SDK sample for Maven and Gradle users (#91)
Browse files Browse the repository at this point in the history
  • Loading branch information
sgrebnov authored Jul 18, 2024
1 parent 158f8e9 commit 7a25d02
Show file tree
Hide file tree
Showing 6 changed files with 291 additions and 0 deletions.
6 changes: 6 additions & 0 deletions client-sdk/spice-java-sdk-sample/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
/target/
/.gradle/
/build/
/gradle/
/gradlew
/gradlew.bat
180 changes: 180 additions & 0 deletions client-sdk/spice-java-sdk-sample/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
# Spice with Java sdk sample

Use [Spice.ai Java SDK](https://github.com/spiceai/spice-java) to query [Spice.ai](https://github.com/spiceai/spiceai) from Java.

## Clone this sample

```shell
git clone https://github.com/spiceai/samples.git
cd samples/client-sdk/spice-java-sdk-sample
```

## Start spice runtime

[Install Spice.ai OSS](https://docs.spiceai.org/getting-started) and start Spice runtime:

```shell
spice run
```

```shell
Spice.ai runtime starting...
2024-07-16T19:16:34.192387Z INFO spiced: Metrics listening on 127.0.0.1:9000
2024-07-16T19:16:34.195177Z INFO runtime::opentelemetry: Spice Runtime OpenTelemetry listening on 127.0.0.1:50052
2024-07-16T19:16:34.197072Z INFO runtime: Initialized results cache; max size: 128.00 MiB, item ttl: 1s
2024-07-16T19:16:34.197759Z INFO runtime::http: Spice Runtime HTTP listening on 127.0.0.1:3000
2024-07-16T19:16:34.197770Z INFO runtime::flight: Spice Runtime Flight listening on 127.0.0.1:50051
2024-07-16T19:16:34.885084Z INFO runtime: Dataset taxi_trips registered (s3://spiceai-demo-datasets/taxi_trips/2024/), acceleration (arrow, 10s refresh), results cache enabled.
2024-07-16T19:16:34.886257Z INFO runtime::accelerated_table::refresh_task: Loading data for dataset taxi_trips
2024-07-16T19:16:40.494038Z INFO runtime::accelerated_table::refresh_task: Loaded 2,964,624 rows (421.71 MiB) for dataset taxi_trips in 5s 607ms.
```
## Maven Users
### Prerequisites
1. A Java Development Kit (JDK), version 17 or higher - for example [OracleJDK](https://www.oracle.com/java/technologies/downloads/). Installed version can be verified with `java -version`.
1. [Maven](https://maven.apache.org/install.html). Verify the Maven installation with: `mvn -version`
### Build sample
```shell
mvn clean compile
```
```shell
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------< ai.spice.example:taxi-trips-app >-------------------
[INFO] Building taxi-trips-app 1.0-SNAPSHOT
[INFO] from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- clean:3.2.0:clean (default-clean) @ taxi-trips-app ---
[INFO] Deleting /Users/sg/spice/samples/client-sdk/spice-java-sdk-sample/target
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ taxi-trips-app ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /Users/sg/spice/samples/client-sdk/spice-java-sdk-sample/src/main/resources
[INFO]
[INFO] --- compiler:3.13.0:compile (default-compile) @ taxi-trips-app ---
[INFO] Recompiling the module because of changed source code.
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent!
[INFO] Compiling 1 source file with javac [debug target 1.8] to target/classes
[WARNING] bootstrap class path is not set in conjunction with -source 8
not setting the bootstrap class path may lead to class files that cannot run on JDK 8
--release 8 is recommended instead of -source 8 -target 1.8 because it sets the bootstrap class path automatically
[WARNING] source value 8 is obsolete and will be removed in a future release
[WARNING] target value 8 is obsolete and will be removed in a future release
[WARNING] To suppress warnings about obsolete options, use -Xlint:-options.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.712 s
[INFO] Finished at: 2024-07-16T12:43:33-07:00
[INFO] ------------------------------------------------------------------------

```
### Run sample
> **IMPORTANT:** The Spice SDK uses the Apache Arrow Flight library, which requires the `--add-opens=java.base/java.nio=ALL-UNNAMED` parameter to operate.
>
> Read [Apache Arrow Java Compatibility](https://arrow.apache.org/docs/java/install.html#java-compatibility) for more details.
```shell
_JAVA_OPTIONS="--add-opens=java.base/java.nio=ALL-UNNAMED" mvn exec:java -Dexec.mainClass="ai.spice.example.App"
```
```shell
Picked up _JAVA_OPTIONS: --add-opens=java.base/java.nio=ALL-UNNAMED
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------< ai.spice.example:taxi-trips-app >-------------------
[INFO] Building taxi-trips-app 1.0-SNAPSHOT
[INFO] from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- exec:3.3.0:java (default-cli) @ taxi-trips-app ---
[ai.spice.example.App.main()] INFO org.apache.arrow.memory.BaseAllocator - Debug mode disabled. Enable with the VM option -Darrow.memory.debug.allocator=true.
[ai.spice.example.App.main()] INFO org.apache.arrow.memory.DefaultAllocationManagerOption - allocation manager type not specified, using netty as the default type
[ai.spice.example.App.main()] INFO org.apache.arrow.memory.CheckAllocator - Using DefaultAllocationManager at memory-netty/16.1.0/arrow-memory-netty-16.1.0.jar!/org/apache/arrow/memory/netty/DefaultAllocationManagerFactory.class
VendorID tpep_pickup_datetime fare_amount
2 2024-01-14T08:32:55 70.0
1 2024-01-14T08:13:28 70.0
2 2024-01-14T08:31:56 6.5
1 2024-01-14T08:15:17 16.3
1 2024-01-14T08:49:57 10.7
2 2024-01-14T08:28:37 14.2
2 2024-01-14T08:42:59 12.0
2 2024-01-14T08:49:26 7.2
1 2024-01-14T08:10:31 7.2
1 2024-01-14T08:17:35 5.1
```
## Gradle Users
### Prerequisites
1. A Java Development Kit (JDK), version 17 or higher - for example [OracleJDK](https://www.oracle.com/java/technologies/downloads/). Installed version can be verified with `java -version`.
1. [Gradle Build Tools version 7.0 or later](https://gradle.org/install/). Installed version can be verified with `gradle -v`.
### Init gradle wrapper
```shell
gradle wrapper
```
```shell
Welcome to Gradle 8.9!

Here are the highlights of this release:
- Enhanced Error and Warning Messages
- IDE Integration Improvements
- Daemon JVM Information

For more details see https://docs.gradle.org/8.9/release-notes.html

Starting a Gradle Daemon (subsequent builds will be faster)

BUILD SUCCESSFUL in 2s
1 actionable task: 1 executed
```
### Build sample
```shell
./gradlew clean build
```
```shell
Downloading https://services.gradle.org/distributions/gradle-8.9-bin.zip
............10%.............20%.............30%.............40%.............50%.............60%.............70%.............80%.............90%.............100%
Starting a Gradle Daemon, 1 incompatible Daemon could not be reused, use --status for details

BUILD SUCCESSFUL in 40s
6 actionable tasks: 5 executed, 1 up-to-date
```
### Run sample
`./gradlew run`
```shell
> Task :run
[main] INFO org.apache.arrow.memory.BaseAllocator - Debug mode disabled. Enable with the VM option -Darrow.memory.debug.allocator=true.
[main] INFO org.apache.arrow.memory.DefaultAllocationManagerOption - allocation manager type not specified, using netty as the default type
[main] INFO org.apache.arrow.memory.CheckAllocator - Using DefaultAllocationManager at memory-netty/16.1.0/c608dab8b8e59d4dc1609a645340f83fa4a145ed/arrow-memory-netty-16.1.0.jar!/org/apache/arrow/memory/netty/DefaultAllocationManagerFactory.class
VendorID tpep_pickup_datetime fare_amount
2 2024-01-02T14:54:44 56.9
1 2024-01-02T14:58:35 25.5
1 2024-01-02T14:21:32 12.1
1 2024-01-02T14:36:26 10.0
2 2024-01-02T14:25:25 38.0
2 2024-01-02T14:41:57 44.3
2 2024-01-02T14:47:52 66.0
2 2024-01-02T14:01:17 21.9
2 2024-01-02T14:27:29 44.3
2 2024-01-02T14:54:39 8.6
```
25 changes: 25 additions & 0 deletions client-sdk/spice-java-sdk-sample/build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
plugins {
id 'application'
}

group = 'ai.spice.example'
version = '1.0-SNAPSHOT'

repositories {
mavenCentral()
}

dependencies {
implementation 'ai.spice:spiceai:0.1.0'
}

application {
mainClass = 'ai.spice.example.App'
}

// The Spice SDK uses Apache Arrow Flight library which requires `--add-opens=java.base/java.nio=ALL-UNNAMED` parameter to operate.
// Read https://arrow.apache.org/docs/java/install.html#java-compatibility for more details.

tasks.withType(JavaExec) {
jvmArgs '--add-opens=java.base/java.nio=ALL-UNNAMED'
}
18 changes: 18 additions & 0 deletions client-sdk/spice-java-sdk-sample/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>ai.spice.example</groupId>
<artifactId>taxi-trips-app</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>taxi-trips-app</name>
<url>http://maven.apache.org</url>
<dependencies>
<dependency>
<groupId>ai.spice</groupId>
<artifactId>spiceai</artifactId>
<version>0.1.0</version>
<scope>compile</scope>
</dependency>
</dependencies>
</project>
14 changes: 14 additions & 0 deletions client-sdk/spice-java-sdk-sample/spicepod.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
version: v1beta1
kind: Spicepod
name: gospice-sdk-sample

datasets:
- from: s3://spiceai-demo-datasets/taxi_trips/2024/
name: taxi_trips
description: taxi trips in s3
params:
file_format: parquet
acceleration:
enabled: true
refresh_check_interval: 10s
refresh_mode: full
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
/*
Copyright 2024 The Spice.ai OSS Authors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/

package ai.spice.example;

import org.apache.arrow.flight.FlightStream;
import org.apache.arrow.vector.VectorSchemaRoot;

import ai.spice.SpiceClient;

// _JAVA_OPTIONS="--add-opens=java.base/java.nio=ALL-UNNAMED" mvn exec:java -Dexec.mainClass="ai.spice.example.App"
public class App {
public static void main(String[] args) {
try (SpiceClient client = SpiceClient.builder()
.build()) {

FlightStream stream = client
.query("SELECT \"VendorID\", \"tpep_pickup_datetime\", \"fare_amount\" FROM taxi_trips LIMIT 10");

while (stream.next()) {
try (VectorSchemaRoot batches = stream.getRoot()) {
System.out.println(batches.contentToTSVString());
}
}
} catch (Exception e) {
System.err.println("An unexpected error occurred: " + e.getMessage());
}
}
}

0 comments on commit 7a25d02

Please sign in to comment.