-
Notifications
You must be signed in to change notification settings - Fork 2
/
Copy pathgetting-started-android.html
executable file
·508 lines (457 loc) · 20.3 KB
/
getting-started-android.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
<!DOCTYPE html>
<html lang="en">
<head>
<base href=".">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Android</title>
<link rel="stylesheet" href="assets/css/dark-frontend.css" type="text/css" title="dark">
<link rel="alternate stylesheet" href="assets/css/light-frontend.css" type="text/css" title="light">
<link rel="stylesheet" href="assets/css/bootstrap-toc.min.css" type="text/css">
<link rel="stylesheet" href="assets/css/jquery.mCustomScrollbar.min.css">
<link rel="stylesheet" href="assets/js/search/enable_search.css" type="text/css">
<link rel="stylesheet" href="assets/css/extra_frontend.css" type="text/css">
<link rel="stylesheet" href="assets/css/prism-tomorrow.css" type="text/css" title="dark">
<link rel="alternate stylesheet" href="assets/css/prism.css" type="text/css" title="light">
<script src="assets/js/mustache.min.js"></script>
<script src="assets/js/jquery.js"></script>
<script src="assets/js/bootstrap.js"></script>
<script src="assets/js/scrollspy.js"></script>
<script src="assets/js/typeahead.jquery.min.js"></script>
<script src="assets/js/search.js"></script>
<script src="assets/js/compare-versions.js"></script>
<script src="assets/js/jquery.mCustomScrollbar.concat.min.js"></script>
<script src="assets/js/bootstrap-toc.min.js"></script>
<script src="assets/js/jquery.touchSwipe.min.js"></script>
<script src="assets/js/anchor.min.js"></script>
<script src="assets/js/tag_filtering.js"></script>
<script src="assets/js/language_switching.js"></script>
<script src="assets/js/styleswitcher.js"></script>
<script src="assets/js/lines_around_headings.js"></script>
<script src="assets/js/prism-core.js"></script>
<script src="assets/js/prism-autoloader.js"></script>
<script src="assets/js/prism_autoloader_path_override.js"></script>
<script src="assets/js/trie.js"></script>
<link rel="icon" type="image/png" href="assets/images/nnstreamer_logo.png">
</head>
<body class="no-script
">
<script>
$('body').removeClass('no-script');
</script>
<nav class="navbar navbar-fixed-top navbar-default" id="topnav">
<div class="container-fluid">
<div class="navbar-right">
<a id="toc-toggle">
<span class="glyphicon glyphicon-menu-right"></span>
<span class="glyphicon glyphicon-menu-left"></span>
</a>
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar-wrapper" aria-expanded="false">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<span title="light mode switch" class="glyphicon glyphicon-sunglasses pull-right" id="lightmode-icon"></span>
<form class="navbar-form pull-right" id="navbar-search-form">
<div class="form-group has-feedback">
<input type="text" class="form-control input-sm" name="search" id="sidenav-lookup-field" placeholder="search" disabled>
<span class="glyphicon glyphicon-search form-control-feedback" id="search-mgn-glass"></span>
</div>
</form>
</div>
<div class="navbar-header">
<a id="sidenav-toggle">
<span class="glyphicon glyphicon-menu-right"></span>
<span class="glyphicon glyphicon-menu-left"></span>
</a>
<a id="home-link" href="index.html" class="hotdoc-navbar-brand">
<img src="assets/images/nnstreamer_logo.png" alt="Home">
</a>
</div>
<div class="navbar-collapse collapse" id="navbar-wrapper">
<ul class="nav navbar-nav" id="menu">
<li class="dropdown">
<a class="dropdown-toggle" role="button" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
API References<span class="caret"></span>
</a>
<ul class="dropdown-menu" id="modules-menu">
<li>
<a href="doc-index.html">NNStreamer doc</a>
</li>
<li>
<a href="gst/nnstreamer/README.html">NNStreamer Elements</a>
</li>
<li>
<a href="nnstreamer-example/index.html">NNStreamer Examples</a>
</li>
<li>
<a href="API-reference.html">API reference</a>
</li>
</ul>
</li>
<li>
<a href="doc-index.html">Documents</a>
</li>
<li>
<a href="gst/nnstreamer/README.html">Elements</a>
</li>
<li>
<a href="tutorials.html">Tutorials</a>
</li>
<li>
<a href="API-reference.html">API reference</a>
</li>
</ul>
<div class="hidden-xs hidden-sm navbar-text navbar-center">
</div>
</div>
</div>
</nav>
<main>
<div data-extension="core" data-hotdoc-in-toplevel="True" data-hotdoc-project="NNStreamer" data-hotdoc-ref="getting-started-android.html" class="page_container" id="page-wrapper">
<script src="assets/js/utils.js"></script>
<div class="panel panel-collapse oc-collapsed" id="sidenav" data-hotdoc-role="navigation">
<script src="assets/js/full-width.js"></script>
<div id="sitenav-wrapper">
<iframe src="hotdoc-sitemap.html" id="sitenav-frame"></iframe>
</div>
</div>
<div id="body">
<div id="main">
<div id="page-description" data-hotdoc-role="main">
<h1 id="nnstreamer-api-library-for-android">NNStreamer API Library for Android</h1>
<h2 id="note-the-api-is-separated-into-a-new-repository">Note: The API is separated into a <a href="https://github.com/nnstreamer/api">new repository</a>
</h2>
<h2 id="prerequisite">Prerequisite</h2>
<p>We assume that you already have experienced Android application developments with Android Studio.</p>
<ul>
<li>Host PC:
<ul>
<li>OS: Ubuntu 22.04 x86_64 LTS</li>
<li>Android Studio: Ubuntu version</li>
<li>Android SDK: Min version 28 (Pie)</li>
<li>Android NDK: Use default ndk-bundle in Android Studio <strong>( < 26.0 )</strong>
</li>
<li>GStreamer: gstreamer-1.0-android-universal-1.24.0</li>
</ul>
</li>
</ul>
<h2 id="build-library">Build library</h2>
<h3 id="environment-variables">Environment variables</h3>
<p>First of all, you need to set-up the development environment as following:</p>
<pre><code class="language-bash">$ export ANDROID_DEV_ROOT=$HOME/Android # Set your own path (default location: $HOME/Android)
$ mkdir -p $ANDROID_DEV_ROOT/tools/sdk
$ mkdir -p $ANDROID_DEV_ROOT/tools/ndk
$ mkdir -p $ANDROID_DEV_ROOT/gstreamer-1.0
$ mkdir -p $ANDROID_DEV_ROOT/workspace
$
$ vi ~/.bashrc
# The environment variables to develop an Android application with NNStreamer
#
export JAVA_HOME=/opt/android-studio/jre # JRE path in Android Studio
export ANDROID_DEV_ROOT=$HOME/android # Set your own path (default location: "$HOME/Android".)
# $ANDROID_DEV_ROOT/tools/sdk/: Android SDK root directory (default location: $HOME/Android/Sdk)
# $ANDROID_DEV_ROOT/tools/ndk/: Android NDK root directory (default location: $HOME/Android/Sdk/ndk/<ndk-version>)
# $ANDROID_DEV_ROOT/gstreamer-1.0/: GStreamer binaries
# $ANDROID_DEV_ROOT/workspace/nnstreamer/: The git repository of NNStreamer
# $ANDROID_DEV_ROOT/workspace/api/: The git repository of ML API
export ANDROID_SDK=$ANDROID_DEV_ROOT/tools/sdk
export ANDROID_NDK=$ANDROID_DEV_ROOT/tools/ndk
export ANDROID_SDK_ROOT=$ANDROID_SDK
export ANDROID_NDK_ROOT=$ANDROID_NDK
export GSTREAMER_ROOT_ANDROID=$ANDROID_DEV_ROOT/gstreamer-1.0
export NNSTREAMER_ROOT=$ANDROID_DEV_ROOT/workspace/nnstreamer
export ML_API_ROOT=$ANDROID_DEV_ROOT/workspace/api
</code></pre>
<h3 id="install-required-packages">Install required packages</h3>
<p>Some required packages should be installed as below.</p>
<pre><code class="language-bash">$ sudo apt install subversion curl pkg-config gradle
</code></pre>
<h3 id="download-android-studio">Download Android Studio</h3>
<p>Download and install Android Studio to compile an Android source code.
You can see the installation guide <a href="https://developer.android.com/studio/install">here</a>.</p>
<p>For example,</p>
<pre><code class="language-bash">$ firefox https://developer.android.com/studio &
Then, download the **Android Studio** IDE into the /opt folder as follows.
$ cd /opt
$ sudo curl -O https://redirector.gvt1.com/edgedl/android/studio/ide-zips/2023.3.1.20/android-studio-2023.3.1.20-linux.tar.gz
$ sudo tar xvzf ./android-studio-2023.3.1.20-linux.tar.gz
</code></pre>
<p>Now, run the <strong>Android Studio</strong> IDE as follows.</p>
<pre><code class="language-bash">$ /opt/android-studio/bin/studio.sh
</code></pre>
<p>Finally, install SDK into the <code>$ANDROID_SDK</code> folder as follows.
The <code>yes</code> command automatically agrees to the license question for the Android SDK.</p>
<pre><code class="language-bash">$ cd $ANDROID_SDK/tools/bin
$ yes | ./sdkmanager --licenses
</code></pre>
<h4 id="proxy-setting">Proxy Setting</h4>
<p>If your network is maintained a proxy of the office, you need to set-up the proxy and SSL configuration.</p>
<ul>
<li>Proxy setting: File > Settings > Appearance & Behavior > System Settings > HTTP Proxy</li>
<li>SSL Certificate: File > Settings > Tools > Server Certificates > Register your certificate and check <code>Accept non-trusted certificates automatically</code>
</li>
</ul>
<h3 id="download-ndk">Download NDK</h3>
<p>Use the default NDK in Android Studio.
To install NDK in Android Studio, navigate to configure -> Appearance & Behavior -> System Settings -> Android SDK -> SDK Tools and then select NDK.</p>
<p>If you need to set a specific version, download and decompress it to compile normally a GStreamer-based plugin (e.g., NNStreamer).
You can download older version from <a href="https://developer.android.com/ndk/downloads/older_releases.html">here</a>.</p>
<h3 id="download-gstreamer-binaries">Download GStreamer binaries</h3>
<p>You can get the prebuilt GStreamer binaries from <a href="https://gstreamer.freedesktop.org/data/pkg/android/">here</a>.</p>
<p>For example,</p>
<pre><code class="language-bash">$ cd $ANDROID_DEV_ROOT/gstreamer-1.0
$ curl -O https://gstreamer.freedesktop.org/data/pkg/android/1.24.0/gstreamer-1.0-android-universal-1.24.0.tar.xz
$ tar xJf gstreamer-1.0-android-universal-1.24.0.tar.xz
</code></pre>
<h3 id="download-nnstreamer-source-code-and-ml-api-source-code">Download NNStreamer source code and ML API source code</h3>
<pre><code class="language-bash">$ cd $ANDROID_DEV_ROOT/workspace
$ git clone https://github.com/nnstreamer/nnstreamer.git
$ git clone https://github.com/nnstreamer/api.git
</code></pre>
<h3 id="build-android-api">Build Android API</h3>
<p>Run the build script in NNStreamer.</p>
<ul>
<li>Build options
<ol>
<li>Target ABI: Default arm64-v8a, specify the ABI (armeabi-v7a, arm64-v8a) to be built for with <code>--target_abi={TARGET-ABI}</code>.</li>
<li>Type: Default all.
<ul>
<li>
<code>--build_type=all</code> to include all features.</li>
<li>
<code>--build_type=single</code> to enable SingleShot API only.</li>
<li>
<code>--build_type=lite</code> to get the minimized library with GStreamer core elements.</li>
</ul>
</li>
<li>Including sub-plugins: Default TensorFlow-Lite and NNFW enabled.
To enable each neural network frameworks, you should download and set-up proper environment.
<ul>
<li>
<code>--enable_tflite=yes</code> to build with TensorFlow-Lite.</li>
<li>
<code>--enable_tflite_qnn_delegate</code> to build with tflite QNN delegate.</li>
<li>
<code>--enable_snpe=yes</code> to build with SNPE (Qualcomm Snapdragon Neural Processing Engine).</li>
<li>
<code>--enable_nnfw=yes</code> to build with NNFW (Samsung on-device neural network inference framework).</li>
<li>
<code>--enable_snap=yes</code> to build with SNAP (Samsung Neural Acceleration Platform).</li>
<li>
<code>--enable_qnn=yes</code> to build with QNN (Qualcomm® AI Engine Direct).</li>
</ul>
</li>
<li>Enabling tracing: Default no.
<ul>
<li>
<code>--enable_tracing=yes</code> to build with tracing and Gst-Shark.</li>
</ul>
</li>
<li>Run test: Default no. <code>--run_test=yes</code> to run the instrumentation test.</li>
<li>Other options
<ul>
<li>
<code>--nnstreamer_dir=<path></code> path to NNStreamer root directory. Default <code>NNSTREAMER_ROOT</code> is used if this is not set.</li>
<li>
<code>--ml_api_dir=<path></code> path to ML API root directory. <code>ML_API_ROOT</code> is used if this is not set.</li>
<li>
<code>--result_dir=<path></code> path to build result. Default path is <code>ML_API_ROOT/android_lib</code>.</li>
<li>
<code>--gstreamer_dir=<path></code> path to GStreamer binaries. Default path is <code>GSTREAMER_ROOT_ANDROID</code>.</li>
<li>
<code>--android_sdk_dir=<path></code> path to Android SDK. Default path is <code>ANDROID_SDK_ROOT</code>.</li>
<li>
<code>--android_ndk_dir=<path></code> path to Android NDK. Default path is <code>ANDROID_NDK_ROOT</code>.</li>
</ul>
</li>
</ol>
</li>
</ul>
<pre><code class="language-bash">$ cd $ML_API_ROOT
$ bash ./java/build-nnstreamer-android.sh
</code></pre>
<p>After building the Android API, you can find the library(.aar) in <code>$ML_API_ROOT/android_lib</code>.</p>
<ul>
<li>Build result
<ol>
<li>nnstreamer-[BUILD_DATE].aar: NNStreamer library for Android</li>
<li>nnstreamer-native-[BUILD_DATE].zip: shared objects and header files for native developer</li>
</ol>
</li>
</ul>
<h3 id="run-the-unittest-optional">Run the unit-test (Optional)</h3>
<p>To run the unit-test, you will need an Android Emulator running or a physical device connected and in usb debugging mode.
Make sure to select the appropriate target ABI for the emulator.
Before running the unit-test, you should download the test model and copy it into your target device manually.</p>
<p>Make directory and copy test model and label files into the internal storage of your own Android target device.</p>
<p>You can download these files from <a href="https://github.com/nnsuite/testcases/tree/master/DeepLearningModels/">nnsuite testcases repository</a>.</p>
<pre><code># You must put the below model and label files in the internal storage of your Android target device.
## For TensorFlow Lite
# Copy {nnsuite testcases repository}/tensorflow-lite/Mobilenet_v1_1.0_224_quant/* into
{INTERNAL_STORAGE}/nnstreamer/test/mobilenet_v1_1.0_224_quant.tflite
{INTERNAL_STORAGE}/nnstreamer/test/labels.txt
{INTERNAL_STORAGE}/nnstreamer/test/orange.png
{INTERNAL_STORAGE}/nnstreamer/test/orange.raw
# Copy {nnsuite testcases repository}/tensorflow-lite/add_tflite/add.tflite into
{INTERNAL_STORAGE}/nnstreamer/test/add.tflite
## For SNPE
# Copy {nnsuite testcases repository}/snpe/inception_v3/* into
{INTERNAL_STORAGE}/nnstreamer/snpe_data/inception_v3_quantized.dlc
{INTERNAL_STORAGE}/nnstreamer/snpe_data/imagenet_slim_labels.txt
{INTERNAL_STORAGE}/nnstreamer/snpe_data/plastic_cup.jpg
{INTERNAL_STORAGE}/nnstreamer/snpe_data/plastic_cup.raw
</code></pre>
<p>To check the testcases, run the build script with an option <code>--run_test=yes</code>.
You can find the result in <code>$ML_API_ROOT/android_lib</code>.</p>
<pre><code class="language-bash">$ cd $ML_API_ROOT
$ bash ./java/android/build-nnstreamer-android-lib.sh --run_test=yes
</code></pre>
<h3 id="using-model-file-with-scoped-storage">Using Model File with Scoped Storage</h3>
<p>Android keeps trying to protect app and user data on external storage. As a result, "scoped storage" is introduced in Android 10 and enhanced in Android 11.
It makes an application has <a href="https://developer.android.com/training/data-storage#scoped-storage">access only to the app-specific directory on external storage</a>.</p>
<p>With scoped storage, consider either options below:</p>
<ol>
<li>
<p>(<strong>Recommended</strong>) Provide your model files with <a href="https://developer.android.com/guide/topics/resources/providing-resources#OriginalFiles"><code>assets</code></a>.</p>
<ul>
<li>Place your model files in <code>assets/models/</code>.</li>
<li>Copy it into app-specific directory using <a href="https://developer.android.com/reference/android/content/res/AssetManager">AssetManager</a>.</li>
<li>Use the File object with NNStreamer Java API.</li>
</ul>
<p>Code example:</p>
<pre><code class="language-java">/**
* Copy files in `assets/models` into app-specific directory.
*
* @param context The application context
*/
void copyModelFromAssetsToAppDir(Context context) {
AssetManager am = context.getResources().getAssets();
String[] files = null;
// Get names of files in `assets/models` directory.
try {
files = am.list("models");
} catch (Exception e) {
Log.e("TAG", "Failed to get asset file list");
e.printStackTrace();
return;
}
// Copy files into app-specific directory.
for (String filename : files) {
try {
InputStream in = am.open("models/" + filename);
String outDir = context.getFilesDir().getAbsolutePath();
// Use `getExternalFilesDir` if you want external directory.
File outFile = new File(outDir, filename);
OutputStream out = new FileOutputStream(outFile);
byte[] buffer = new byte[1024];
int read;
while ((read = in.read(buffer)) != -1) {
out.write(buffer, 0, read);
}
in.close();
out.flush();
out.close();
} catch (IOException e) {
Log.e("TAG", "Failed to copy file: " + filename);
e.printStackTrace();
return;
}
}
}
</code></pre>
</li>
<li>
<p>Provide your model files with absolute path.</p>
<ul>
<li>Place your model files in the device's external storage.</li>
<li>(If your app targets API level 30 (Android 11) or later) Set <code>android:preserveLegacyExternalStorage="true"</code> in your <code>AndroidManifest.xml</code> to use the deprecated method <code>getExternalStorageDirectory</code>.</li>
<li>Use method <code>getExternalStorageDirectory</code> to get the File object.</li>
<li>Use the File object with NNStreamer Java API.</li>
<li>
<strong>Note:</strong> Use this option only for test purpose. This assumes that the model files should be in the right hardcoded path in the target device.</li>
</ul>
<p>Code example:</p>
<pre><code class="language-java">/**
* Return a model file in external storage.
*
* @return The File object
*/
File getModelFile() {
String root = Environment.getExternalStorageDirectory().getAbsolutePath();
File model = new File(root + "/path/to/modelfile/sample.tflite");
if (!model.exists()) {
return null;
}
return model;
}
</code></pre>
</li>
</ol>
<h3 id="using-tensorflow-lite-nnapi-delegate">Using TensorFlow Lite NNAPI Delegate</h3>
<p>If the TensorFlow Lite model file is provided from external storage, TensorFlow Lite NNAPI delegate fails to use available backend like GPU, DSP, or NPU.</p>
<p>To use NNAPI delegate properly, you should provide the model file from <strong>internal</strong> storage. You may copy the model files into <strong>internal</strong> app-specific directory (<code>getFilesDir</code>) or cache directory (<code>getCacheDir</code>).</p>
<p>Here are sample Java methods that copy given files into internal app-specific directory:</p>
<pre><code class="language-java">/**
* Copy a model file into app-specific internal storage and return it.
*
* @param context The application context
* @param model The File object of a model file
*
* @return The copied File object
*/
File modelFromFilesDir(Context context, File model) {
File appSpecificFile = new File(context.getFilesDir(), model.getName());
appSpecificFile.mkdirs();
// Copy the model file in external storage to app specific internal storage.
try {
Files.copy(model.toPath(), appSpecificFile.toPath(), StandardCopyOption.REPLACE_EXISTING);
} catch (Exception e) {
e.printStackTrace();
return null;
}
return appSpecificFile;
}
/**
* Copy a model file into app-specific cache directory and return it.
*
* @param context The application context
* @param model The File object of a model file
*
* @return The copied File object
*/
File modelFromCacheDir(Context context, File model) {
File cacheFile = new File(context.getCacheDir(), model.getName());
cacheFile.getParentFile().mkdirs();
// Copy the model file in external storage to app-specific cache directory.
try {
Files.copy(model.toPath(), cacheFile.toPath(), StandardCopyOption.REPLACE_EXISTING);
} catch (Exception e) {
e.printStackTrace();
return null;
}
return cacheFile;
}
</code></pre>
</div>
</div>
<div id="search_results">
<p>The results of the search are</p>
</div>
<div id="footer">
</div>
</div>
<div id="toc-column">
<div class="edit-button">
</div>
<div id="toc-wrapper">
<nav id="toc"></nav>
</div>
</div>
</div>
</main>
<script src="assets/js/navbar_offset_scroller.js"></script>
</body>
</html>