-
Notifications
You must be signed in to change notification settings - Fork 0
/
hrman2wingpu.html
executable file
·250 lines (140 loc) · 13.9 KB
/
hrman2wingpu.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
<!DOCTYPE html>
<html>
<head>
<meta name="description" content="HRMAn - Host Response to Microbe Analysis" />
<title>HRMAn - Host Response to Microbe Analysis</title>
<link rel="shortcut icon" href="favicon.ico">
<!--<script src="ga.js"></script>-->
<!-- Global site tag (gtag.js) - Google Analytics -->
<script async src="https://www.googletagmanager.com/gtag/js?id=UA-111014695-1"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'UA-111014695-1');
</script>
</head>
<body>
<!--<textarea theme="spacelab">-->
<xmp theme="cerulean" style="display:none;">
###[__Home__](index.html) : [__News__](news.html) : [__HRMAn2.0__](hrman2.html) : [__Sample data__](sample_data.html) : [__Downloads__](downloads.html) : [__Tutorials__](tutorials.html) : [__About us__](About_us.html)
###HRMAn 2.0 setup
####SETUP ON WINDOWS (+GPU ACCELERATION)
__1.__ Download and install [KNIME Analytics platform](https://www.knime.com/download-installer/2/64bit), leaving the install path as default. KNIME will ask you how much memory to allocate during this process. Make sure to add as much as possible.
__2.__ Go to your KNIME installation directory (typically __C>Program Files>KNIME__) and open the "*knime.ini*" file with a text editor.
Change the value in line 18 from *-Xmx2048m* to the maximum amount of RAM memory you can allow KNIME to use (e.g. *-Xmx16g for 16 GB*).
*This might already be changed from the installation setting in __step 1__.*
The more the better.
OPTIONAL: Add "-Dknime.dl.installationtesttimeout=60000" to the last line of the .ini file. Some user have reported an error when using HRMAn which is caused by long loading times of the deep learning environment. Adding this line should resolve this issue.
Save the changes in the file and close the Explorer Window.
__3.__ Create a folder for your workspace somewhere convenient and which you will remember (e.g. __Desktop>KNIME_workspace__). This folder will be used to as your local KNIME workspace that holds all your future analysis pipelines, including HRMAn.
__4.__ Create a folder for your KNIME temporary files somewhere convenient and which you will remember (e.g. __Desktop>KNIME_temp__). __DO NOT ADD SPACES ANYWHERE IN YOUR FILEPATH NAMES__. This folder will be used by KNIME to write temporary files during the analysis. Sometime the files can build up, so it’s a good idea to occasionally clear the contents of this folder (while KNIME is **NOT** running).
__5.__ Start KNIME Analytics platform. Upon the first start KNIME will as which folder to use for your local workspace. Navigate to the folder you have created in __step 3__, tick "*Use this as the default and do not ask again*" and confirm by clicking "*LAUNCH*".
__6.__ Within KNIME Analytics platform, use the top menu bar and navigate to __File>Preferences>KNIME__.
In the menu change the folder with your temporary files from the default to the folder you have created in __step 4__ by selecting "*Browse*" and navigating to the folder within Explorer.
__7.__ *Optional*: Open the KNIME Preferences (see __step 6__) and go to __KNIME>KNIME GUI__. Here you can set the Console View Log Level to __INFO__, which will display more information when HRMAn is executing (e.g. progress of some python snippets). If you prefer a cleaner console, leave the level at __WARN__, otherwise decrease to __ERROR__ (not recommended).
__8.__ Next, open the KNIME Preferences (see __step 6__) and go to __Install/Update>Available Software sites__. In the menu make sure that all items are ticked (including "*KNIME Community Extensions (Trusted)*").
__9.__ To run HRMAn, Anaconda needs to be installed to manage the python environments. Download Anaconda from [__here__](https://repo.anaconda.com/archive/Anaconda3-2020.11-Windows-x86_64.exe) and install leaving everything at default!!
__10.__ Setup the python environments: First open an Anaconda prompt window (Use the windows search and type "*Anaconda Prompt*" into the search).
Then one after another enter the following commands into the prompt window and confirm each one by pressing enter:
conda create -n HRMAn python=3.6
When conda asks you to proceed, type y and hit enter:
proceed ([y]/n)?
This creates a new python environment for HRMAn and KNIME which will hold all required packages. Next enter
conda activate HRMAn
and confirm by pressing enter.
This will activate the environment and you should see in your terminal that __(base)__ has changed to __(HRMAn)__.
Next enter:
pip install numpy==1.18.1 pandas==0.23.4 scipy==1.4.1 matplotlib==3.1.3 scikit-image==0.16.2 cellpose==0.6 openpyxl==3.0.3 tensorflow-gpu==1.14.0 keras==2.2.4 nvidia-ml-py3
and again press enter. This will install all necessary python packages into the HRMAn environment. This step might take a while depending on your computer and your internet connection.
Additionally, pytorch needs to be changed for the GPU version. First enter:
pip uninstall torch
and confirm by hitting enter.
Once done, enter:
pip install torch===1.7.1 torchvision===0.8.2 torchaudio===0.7.2 -f https://download.pytorch.org/whl/torch_stable.html
and again confirm by pressing enter.
This should have changed the pytorch package to the GPU version. Once completed you can close the prompt window.
__11.__ Next CUDA and cuDNN need to be set up. Please make sure you have the latest Nvidia graphics driver installed before starting with this. CUDA version 10.2 (required) can be downloaded from [here](https://developer.download.nvidia.com/compute/cuda/10.2/Prod/network_installers/cuda_10.2.89_win10_network.exe). Additionally, cuDNN version 8.0.4 for CUDA 10.2 can be downloaded from [here](https://developer.nvidia.com/compute/machine-learning/cudnn/secure/8.0.4/10.2_20200923/cudnn-10.2-windows10-x64-v8.0.4.30.zip) (*To download cuDNN you might have to create an Nvidia Developer account*).
Next click on the downloaded CUDA 10.2 installer .exe and follow the on screen prompts.
Chose "*Custom (Advanced)*" installation.
Leave the install path and directory as default but deselect all components from the installation menu, except CUDA itself.
Once completed, close the installer window and open a new *Windows command prompt*. In the command prompt type:
nvcc --version
and hit enter. The result should show the CUDA 10.2 version, this step ensures that CUDA 10.2 was installed properly.
Next, go to your downloads folder and unzip the cuDNN folder. Open the folder and copy the files from the cuDNN folder into the corresponding folder in your CUDA 10.2 installation directory (Typically __C>Program Files>NVIDIA GPU Computing Toolkit>CUDA>v10.2__). Simply copy the files from *bin* to *bin*, *include* to *include* and *lib* to *lib*. All done.
__12.__ Download HRMAn and the training workflow from here:
[HRMAn2.0](https://github.com/HRMAn-Org/HRMAn/raw/master/HRMAn2.0.knwf)
[CNN training for HRMAn2.0](https://github.com/HRMAn-Org/HRMAn/raw/master/Annotation%2BTraining_for_HRMAn2.0.knwf)
[Template for bulk upload](https://github.com/HRMAn-Org/HRMAn/raw/master/HRMAn_2.0_bulk_input_template.xlsx)
__13.__ Double click on the "*HRMAn2.0.knwf*" file in your Downloads folder which will open the workflow in KNIME. Upon the first opening of the workflow, KNIME will notify you that some software extensions are missing and ask to automatically search for extensions. Confirm this with "*Yes*", follow the installation and re-start KNIME when asked to.
KNOWN ERROR: With newer versions of KNIME, it occasionally happens that the software packages cannot be obtained automatically as described above. This also affects __step 14__. In this case follow the instructions in __step 14__ to install new KNIME extensions. Overall make sure that the following extensions are installed:
+ Dropbox Nodes for KNIME
+ KNIME Deep Learning - Keras Integration
+ KNIME Deep Learning - TensorFlow 2 Integration
+ KNIME Deep Learning - TensorFlow Integration
+ KNIME Excel Support
+ KNIME Image Processing
+ KNIME Image Processing - Deep Learning Extension
+ KNIME Image Processing - ImageJ Integration (Beta)
+ KNIME Image Processing - Python Extensions
+ KNIME JavaScript Views
+ KNIME Javasnippet
+ KNIME Parallel Chunk Loop Nodes
+ KNIME Python Integration
+ KNIME Python Scripting extension
+ Vernalis KNIME Nodes
__14.__ Once KNIME has re-started, use the top menu bar and go to __Help>Install New Software…__
In the menu that opens, first untick the “*Hide items that are already installed*” tickbox on the bottom right.
Then enter the following address into the “*Work with*” address bar: [*https://download.nodepit.com/dropbox/4.2*](https://download.nodepit.com/dropbox/4.2.). Next tick the tickbox for Dropbox nodes from the list.
Next navigate back to the address bar and select *--All available software Sites--* from the list and type “*python*” into the filter make sure that you tick the boxes next to __KNIME Python integration__, __KNIME Image processing – Python Extensions__, __KNIME Python Scripting extension__ (If they were already automatically installed they will appear greyed out and not selectable…all good then!).
Then search for "*KERAS*" in the filter and tick the box next to __KNIME Deep Learning – Keras Integration__ (If it was already automatically installed it will appear greyed out and not selectable…all good then!).
Next, search for "*Image*" and select __KNIME Image Processing – Deep Learning Extension__.
Once you have made the selections above click on "*Next*", follow the installation and let KNIME re-start as prompted.
__15.__ When KNIME has re-started, open the KNIME Preferences (see __step 6__) and go to __KNIME>Python__. Make sure that "*Python 3*" and "*Conda*" are ticked in the top area of the menu. Next go to the "*Python 3 (Default)*" part of the menu and select HRMAn as Name of the Python 3 Conda environment (this was created in __step 10__).
*The occasional warning under "Python 2" can be ignored (this is not used by HRMAn).*
Next, within the Preferences go to __KNIME>Python Deep Learning__. In the menu tick "*Use special Deep Learning configuration as defined below*". Make sure that "*Conda*" is ticked.
Then click on the "*New environment…*" button in the Keras part of the menu and select "*Create new GPU environment*" in the menu that opens.
This new environment should, once creation has completed, appear in the menu under "*Keras*". *The occasional warning under TensorFlow 2 can be ignored (this is not used by HRMAn).*
__16.__ Once again double click on the "*HRMAn2.0.knwf*" file in your Downloads folder which will open the workflow in KNIME. This time no error messages should appear. On top of the workflow view, there will be a yellow ribbon warning that the workflow has not been saved yet.
Click on "*Save as*" within the yellow ribbon and save it as __HRMAn2.0__ into your local workspace (__Local>HRMAN2.0__).
Repeat this step with the "*CNN-training-for-HRMAN2.0.knwf*" file in your Download folder.
####Congrats! You’re now fully set up to run HRMAn on your Windows computer with GPU acceleration for all deep learning components!
For more information on CUDA/Tensorflow/Keras/KNIME compatibility you can check these ressources:
[Tensorflow tested build configurations](https://www.tensorflow.org/install/source#tested_build_configurations)
[KNIME Deep Learning Integration Installation Guide](https://docs.knime.com/2019-06/deep_learning_installation_guide/index.html)
</xmp>
<br><br>
<h5 style="color: #317eac; line-height: 28px; font-size: 18pt; font-family: 'Telex',sans-serif; text-align: center"> Copyright © 2017 - <script type="text/javascript">var year = new Date(); document.write(year.getFullYear());</script> <a href="https://frickellab.com/" style="color: #2fa4e7">HRMAn team</a>,<br><a href="https://www.crick.ac.uk/" style="color: #2fa4e7">The Francis Crick Institute,<br>London</a></h5>
<script src="v/0.2/strapdown.js"></script>
</body>
</html>