-
Notifications
You must be signed in to change notification settings - Fork 0
/
Mimir-powercap
253 lines (208 loc) · 8.75 KB
/
Mimir-powercap
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
################################################################
#Get an account at the ICL machine
##1)Fill form to get an account at the KNM machine from ICL: https://admin.icl.utk.edu/acctmgr/apply.html
#Once you have an account
##2)To connect:
################################################################
#Requirements to install Mimir with Powercap capability
##1) Install or load mpich
#Build from source
wget http://www.mpich.org/static/downloads/3.2/mpich-3.2.tar.gz
tar xvf mpich-3.2.tar.gz
cd mpich-3.2
./configure --prefix=<dir-location>
make -j32
make install
#Load module on KNM machine (Methanol)
module load mpich/3.3beta2
#Check the mpi version
mpichversion
##2) Install PAPI with powercap capability:
#PAPI installed through module does not have the powercap component
cd $HOME
mkdir Downloads
cd Downloads
git clone https://bitbucket.org/icl/papi.git
cd papi/src
./configure --prefix=$HOME/papi --with-components="powercap"
make && make install
make test
#Check powercap component:
cd $HOME/papi/src/components/powercap/tests
./powercap_basic #(Check if the test passed)
./powercap_limit #(Check if the test passed)
##3) Add powercap calibration to PAPI
#Create a powercap script for calibration
##This is machine dependet, this value is NOT the power limit from user.
cp powercap_limit.c powercap_calibrate.c
vim powercap_calibrate.c
#Change line 133: values[limit_map[i]] = (215 * 1e6); ##plus 10 Watts. To set up the maximum TDP of the machine.
#Modify makefile to integrate the new calibration script.
#(TO DO: Autocreate the makefile with cmake without having to add this lines)
#Add these lines:
TESTS = powercap_basic powercap_limit powercap_calibrate
powercap_calibrate.o: powercap_calibrate.c
$(CC) $(CFLAGS) $(OPTFLAGS) $(INCLUDE) -c powercap_calibrate.c -o powercap_calibrate.o
powercap_calibrate: powercap_calibrate.o $(UTILOBJS) $(PAPILIB)
$(CC) $(INCLUDE) -o powercap_calibrate powercap_calibrate.o $(UTILOBJS) $(PAPILIB) $(LDFLAGS)
#Run the makefile:
make
./powercap_calibrate #To test if the set up the new max power to 215 is correct.
##4) Install Mimir-dev
cd $HOME
git clone https://github.com/TauferLab/Mimir-dev.git
cd Mimir-dev
git checkout power-capping
vim ~/.bashrc
## Added these lines to the ~/.bashrc:
## User specific aliases and functions
export PATH=/nfs/apps/mpich/3.3beta2/bin:$PATH
export LD_LIBRARY_PATH=$HOME/papi/lib/:$LD_LIBRARY_PATH
##Then
source ~/.bashrc
autoreconf -i
./configure --prefix=$HOME/sw/mimir --with-papi=$HOME/sw/papi/ #papi where the bin, lib and include are
make clean
make
make install
################################################################
#Generation the data (Joe Teague)
##To learn about the generation of data look at the next slides. (A set of steps is being prepared for easy running and generation of data)
##Slides: https://docs.google.com/presentation/d/1aA7sXwjXjlmnb7W8DKmZ9u4WXH03NdCa1e-8JmVuO8k/edit?usp=sharing
################################################################
#Set up environment to create and collect results
## (TO DO: Check ldd executables to see if I have the same links!Libraries and versions!)
## (TO DO: CHECK DATA IF IT'S FROM LOCAL OR FROM A GITHUB)
## 1)Add right paths to ~/.bashrc
### User specific aliases and functions
export PATH=/nfs/apps/mpich/3.3beta2/bin:$PATH
export LD_LIBRARY_PATH=$HOME/papi/lib/:$LD_LIBRARY_PATH
export PATH=$HOME/papi/bin:$PATH
export PATH=$HOME/local/x86/bin:$PATH
export C_INCLUDE_PATH=$HOME/papi/include:$C_INCLUDE_PATH
export C_INCLUDE_PATH=$HOME/Downloads/papi/src/testlib:$C_INCLUDE_PATH
export LD_RUN_PATH=$HOME/papi/lib/:$LD_RUN_PATH
export LD_LIBRARY_PATH=$HOME/Downloads/papi/src:$LD_LIBRARY_PATH
#In terminal:
source ~/.bashrc
## 2) Create folder for later to save the running tests
###(TO DO: EXPLORE IF YOU CAN CREATE A GITHUB REPOSITORY WHERE JUST TO ADD WITHOUT SAVING ALL FILES IN HOME DIR)
## For setting up the folder you can do PLAN A or PLAN B
### PLAN A: CREATE LOCAL FOLDER
cd $HOME
mkdir results
cd results
mkdir methanol/wc-reducebykey/frequency#/dataset #dataset= 4B72 or the specific you are will use
cd $HOME/methanol/wc-reducebykey/frequency#/dataset
### PLAN B: SAVE ON REPOSITORY
cd $HOME
git clone https://github.com/polaya07/results_Mimirpowercap.git
cd results_Mimirpowercap/methanol/wc-reducebykey/
#create folder to save results (path must be congruent in freq_test.sh script)
mkdir frequency# #Replace '#' with the run_num
cd frequency#
mkdir dataset #e.g. 4B72
## 3) Clone the scripts to run the tests:
cd $HOME
git clone https://github.com/TauferLab/Mimir-scripts.git
cd Mimir-scripts
git checkout KNM-dev
cd Mimir-scripts/linux_powercap
### 3.1)Check this files: multi_freq_test.sh, config.normal.h and freq_test.sh
** multi_freq_test.sh
#### In this file there are 6 parameters, next are the explanation each of them in case you want to modify:
$1: dataset (4B72)
$2: powercapping % (1.000 given by percentage)
$3: buffersize_num(MB) (64 MB)
$4: duration(s) (280 s)
$5: benchmark (reducebykey)
$6: run_num (5) //you must create the folder where to save the results
#### Check if this path is at the end of the script
$HOME/Downloads/papi/src/components/powercap/tests/powercap_calibrate
** config.normal.h
####Check the directory of mimir:
INSTALLDIR="$HOME/mimir"
** freq_test.sh
#Check if the next line is after machine=`hostname -s` is defined
$HOME/Downloads/papi/src/components/powercap/tests/powercap_calibrate
###############################################################
#Run the frequency tests
## (TO DO: ADD DEPENDENCIES AND CALLS INSIDE FUNCTIONS ON THE READ ME FILE!)
##For example:
#multi_freq_test.sh
#->freq_test.sh
#->->run_hw_stats.sh
#->->-> $HOME/Mimir-dev/tool/profile_hw_state
#->->test_$5.sh #### $5 is replace with the type of benchmark the user is running
#->->->test_$5
#->->->-> run.job.sh
## (TO DO: Organize the linux-powercap folder and create a folder with all the tests)
## Join:
cd linux_powercap
./join_multi_freq_test.sh
## Wordcount:
cd linux_powercap
./multi_freq_test.sh
#What do you expect to see if is successfully running:
#filename=/home/polaya/results/methanol/wc-reducebykey/frequency5/4B72/mimir-0.558-power-buff64m-unified-reducebykey-words-4B72-normal-1-68_2019-5-29-12-36-51_trace.txt
#gather values 1559147811698264600
#gather values 1559147811831580580
#gather values 1559147811967673712
#gather values 1559147812103759546
#gather values 1559147812240003925
#gather values 1559147812376030204
#...
#end profiling
#freq_test.sh: done
#Trying all powercap events
#Found powercap component at cid 2
#
#CURRENT LIMITS
#EVENT: powercap:::POWER_LIMIT_A_UW:ZONE0 LIMIT: 4095.25 Watts
#EVENT: powercap:::POWER_LIMIT_B_UW:ZONE0 LIMIT: 320.00 Watts
#
#SETTING LIMITS 10 WATTS BELOW CURRENT LIMITS
#
#READING LIMITS TO MAKE SURE THEY ARE SET
#
#NEW LIMITS
#EVENT: powercap:::POWER_LIMIT_A_UW:ZONE0 LIMIT: 4085.25 Watts
#EVENT: powercap:::POWER_LIMIT_B_UW:ZONE0 LIMIT: 310.00 Watts
#
#RESET LIMITS BEFORE EXITING...
#READING RESET LIMITS TO MAKE SURE THEY ARE SET
#
#RESET LIMITS
#EVENT: powercap:::POWER_LIMIT_A_UW:ZONE0 LIMIT: 215.00 Watts
#EVENT: powercap:::POWER_LIMIT_B_UW:ZONE0 LIMIT: 215.00 Watts
#done
#PASSED
## Push results
cd $HOME/results_Mimirpowercap/methanol/wc-reducebykey/
git add --all
git commit -m "New results"
git push origin master
###############################################################
# Visualization of results
## (TO DO: FIND A WAY FOR PLOTTING WITHOUT CHANGING TO LOCAL)
# clone Mimir-scripts and results repos in your local machine
git clone https://github.com/TauferLab/Mimir-scripts.git
git clone https://github.com/polaya07/results_Mimirpowercap.git
# Access to analyze-powercap to parse and plot results
# Always pull the directory with results
cd Mimir-scripts
git checkout KNM-dev
cd analyze_powercap/visualization
#./parse.sh run_num name
./parse.sh 5 P_test #Change the specific folders where to run
#python hw_stats_plot.py 'name*' testcomplete.png
python hw_stats_plot.py 'P_test*' P_test.png #Plot
###############################################################
#BIG TO DOS:
#1) Check why are we not getting memory power consumption!!!
#2) Check datasets for wordcount (Where are the saved or are they automatically generated)?
#The data
#3) Get results from Join!! Check why it does not run after the gathering anf profiling
#4) Ask why are we calibrating at the end of the three powercapping instead of after each one?
#Inside freq_test.sh I'm calibrating too. In between each test!