forked from schaap/p2p-testframework
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathHOWTO
534 lines (416 loc) · 33.3 KB
/
HOWTO
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
This HOWTO will introduce you into the usage of the P2P testing framework. Topics covered are building tests, running tests, reviewing results and extending the framework. Throughout this HOWTO commands and files will be assumed to be run in the root of the P2P test framework, which is the directory that contains the ControlScripts directory. This is the working directory that is assumed throughout all documentation of the framework.
During the first parts of this HOWTO an example will be cosntructed that will instruct the framework to connect to some hosts using SSH, run the swift client to transfer a file, to plot some statistics on that and finally to present the results in HTML.
The last part of this HOWTO will demonstrate how to develop a new module for the framework by example of the development of the file:fakedata module.
=== BUILDING TESTS ===
To run a test you first have to build the scenario and campaign files. In the scenario files you define which hosts will run which clients to send which files. As an example we will build a test that sends a single file from host1 to host2, both accessable over ssh, using the swift client. We will use one scenario file to define the file object, one file to define the hosts and one to define the clients and put it all together. The separation into multiple files is just an example: you could just as well use one file or a different separation.
One thing we will not explicitly do here, but what you should do when writing your own scenarios (and what I did when writing this), is referring to the documentation. The base entry point is the README file, which documents the parameters to all the generic objects. Apart from that you should always open the file of every module you use: module specific documentation is placed at the beginning of the module file. Finding these files is simple: as an example module file:local is located in ControlScripts/modules/file/local . This is actually the very reason the module is called file:local. This is, by the way, also the way that is used throughout the framework to refer to specific files.
Once you got the hang of the basic syntax the REFERENCE becomes very useful. I found going through it every time I wrote a test was actually the fastest way not to miss anything. The REFERENCE, of course, only contains the documentation for the modules delivered with the framework.
= file: TestSpecs/files/my_file =
[file:local]
name=myfile
path=/home/me/someNiceFileToTransfer
rootHash=0123456789012345678901234567890123456789
This creates a single file object named 'myfile'. It points to the local file given by path=. Since we'll be transferring this file using swift it is useful to also give the rootHash. Of course the root hash here is bogus ;).
= file: TestSpecs/hosts/my_hosts =
[host:ssh]
name=my_seeder
hostname=myseederhost.foo.bar
[host:ssh]
name=my_leecher
hostname=myleecherhost.foo.bar
user=my_alter_ego
This creates two host objects named 'my_seeder' and 'my_leecher'. They instruct the framework to use SSH to connect to the hosts under the given hostnames (this can also be an IP). In case of the leecher a different username than the logged in user is to be used.
= file: TestSpecs/scenarios/my_scenario =
[client:swift]
name=seedingswift
location=git://github.com/gritzko/swift.git
source=git
builder=make
remoteClient=yes
listenPort=15000
wait=300
[client:swift]
name=leechingswift
location=/home/me/prebuilt_swift_dir
tracker=myseederhost.foo.bar:15000
[execution]
host=my_seeder
file=myfile
client=seedingswift
seeder=yes
[execution]
host=my_leecher
file=myfile
client=leechingswift
[processor:gnuplot]
script=TestSpecs/processors/simple_log_gnuplot
[processor:savehostname]
[viewer:htmlcollection]
This first creates two client objects named 'seedingswift' and 'leechingswift'. The seedingswift client is instructed to have its source pulled using git (source=git) from the given repository (location=). It is to be built remotely (remoteClient=yes) using make (builder=make). Two swift specific parameters for the seedingswift client are given to instruct the client to listen on port 15000 and to wait for 300 seconds before terminating. The leechingswift client uses a locally prebuilt binary swift located in /home/me/prebuilt_swift_dir/. This client will be uploaded to the leeching host and executed. It is instructed to use myseederhost.foo.bar:15000 as its tracker.
Then the file proceeds with declaring two executions. Executions are the combination of host, client and file. The first execution instructs the framework to run the seedingswift client on the my_seeder host to transfer myfile and it tells the framework that that host will be a seeder (seeder=yes). The latter is important to do correct: only seeding executions will have the actual files needed to seed uploaded, non-seeding executions will only upload the meta data. The second execution runs the leechingswift client on the my_leecher host to transfer myfile again.
The last lines instruct the framework to run two postprocessors: gnuplot and savehostname. The former runs gnuplot on the gathered data with a supplied script, the simple_log_gnuplot script in this case, and the latter just saves the hostname as given above in single files. The output of both of these will be used by the htmlcollection viewer which will be run in the end. That viewer will generate an HTML overview of what has been going on.
= file: TestSpecs/my_campaign =
[scenario]
name=scenario1
file=TestSpecs/files/my_file
file=TestSpecs/hosts/my_hosts
file=TestSpecs/scenarios/my_scenario
timelimit=60
[scenario]
name=scenario2
file=TestSpecs/hosts/my_hosts
file=TestSpecs/files/my_file
file=TestSpecs/scenarios/my_scenario
This is the campaign file. The campaign file is the complete description of the campaign, using indirections into the scenario files. It can't contain other objects than scenario objects and just instructs the framework which files to concatenate in order to create a full scenario file. It also gives the scenarios a name and optionally a time limit (in seconds). Note that the order of the file parameters is important: the files are simply concatenated in the order they are given and if an object is declared before it is used, the framework will simply complain. For example, if we would specify the my_file scenario file after the my_scenario scenario file, the framework will complain after parsing the first execution: it can't find file object myfile.
=== RUNNING TESTS ===
Now that your very interesting and elaborate test suite has been built, it is time to run it. The most easy way is:
./ControlScripts/run_campaign.py TestSpecs/my_campaign
This will run the scenarios in your campaign file. You can instruct the framework to check your campaign instead, without actually running everything:
./ControlScripts/run_campaign.py --check TestSpecs/my_campaign
Note that the syntax and sanity checks will be run during the actual run as well: a check run simply stops before any uploading and executing is done. When developing campaigns it is advisable to do a check run first, for example to establish whether your hosts are reachable without user interaction.
Several more options are available, mainly for debugging. Just run
./ControlScripts/run_campaign.py
without any other options to get a list of them.
And that's all there is to it. Just run it.
= Access to hosts =
One important note on access to hosts: this needs to be done without user interaction! This goes for everything in the framework, but accessing hosts is the most important example. Usually you will access some hosts over SSH. Make sure you can access those hosts without having to type anything! Create a key for your own identity and use ssh-agent to make sure you don't need to enter the passwords for your private keys. (You do have passwords on your private keys, right?)
A typical session for me goes like this:
ssh-agent bash
ssh-add
[Type password to private key]
./ControlScript/run_campaign.py TestSpecs/my_campaign
exit
The host:ssh module will check whether your hosts are reachable, but you can do so by hand yourself:
ssh yourhost "date"
This should connect to the host, print the date, and fall back to your local prompt. If anything happens in between, such as extra output or user interaction, the framework will not work. Of course, you should add those parameters you also give to the framework, such as a different username or extra parameters.
=== REVIEWING RESULTS ===
After your tests have run, or failed, you should always review some results. The results can by default be found in the Results/ directory. Say you have just ran the above campaign my_campaign, and it was 17:00:00 on the 24th of November 2011. The results will then be in Results/my_campaign-2011.11.24-17.00.00/. In this directory you will first find err.log. Always review this: it is extra output from the scenarios. This file is especially important when something failed (the output of the framework will direct you here, as well).
Apart from the err.log file there is the scenarios directory which holds one directory for each scenario. Inside each scenario's directory are all the logs and results of that scenario. Firstly there is the scenarioFile file, which is the concatenation of scenario files used to initialize the scenario. This is useful for debugging and also automatically documents the setup of your tests. Note that when line numbers are mentioned in error lines, they always refer to this file.
The executions directory contains one directory for each execution, numbered exec_0, exec_1, etc. Inside these you will find the logs and parsedLogs directories, which contains the raw logs from the clients and the interpreted logs after a parser has been run on them (for using other parsers than the default ones: consult the full documentation). You can of course use these logs to do your own extended analyses.
Next to the executions directory are the processed and views directories, which respectively contain post-processed data, such as graphs or formatted logs, and views, such as the HTML overview.
When everything from your my_campaign campaign went well, you should usually first check the actual output. The htmlcollection view was defined, which takes together all processed data and puts it into an HTML page. To view this, you could run:
firefox Results/my_campaign-2011.11.24-17.00.00/scenarios/scenario1/views/collection.html
=== EXTENDING THE FRAMEWORK ===
The framework is built with extensions in mind. There are several categories of extensions you can make:
- host modules
- file modules
- client modules
- parser modules
- processor modules
- viewer modules
- tc modules
- builder modules
- source modules
- workload modules
Those are quite some extension points. This HOWTO doesn't even use all of them and no effort will be made to discuss each extension in detail. For a particular extension's details, please refer to the README and other documentation.
The general process of creating a new module is this:
1) Read up on the API the module should implement;
2) Copy the skeleton file to your own module;
3) Read your new module (which is just the skeleton) and read up on any mentioned APIs you can use, as well as the global API;
4) Write your implementation in the skeleton that is currently your module, be sure to document and check all places where it says TODO;
5) Thoroughly test your implementation and adjust as needed.
That looks a lot like generic software development and in fact it is. But due to the use of the skeleton a lot of the hard labor is taken out of it. Every module is also based on a generic parent class that does most of the heavy lifting and administration. This leaves just the particular details for your module to be filled in. For example, if you would like to add a new client (probably the most commonly made extension) you need to define the layout on disk of your client (so it can be moved/uploaded), tell the framework how to run it and instruct how to retrieve the logs. And that's it. For a simple client with only one binary and logging on stderr this would take a total of 5 lines of code. Most effort would probably be in reading through all the comments in the skeleton implementation.
As an example of this process the development of file:fakedata is documented below.
= 1) Read up on the API the module should implement =
As a first step, let's run the doxygen tool to get our documentation.
cd Docs/
doxygen doxy
With the documentation in place, we'll take a look at the API of the file object.
firefox html/index.html
*click on Classes*
*click on core::file:file*
Read through the class' methods to get an idea of what functionality the file:fakedata class will have to offer.
Based on everything we can read here one could build an implementation. But let's make life easy on ourselves.
= 2) Copy the skeleton file to your own module =
cp ControlScripts/modules/file/_skeleton_.py ControlScripts/modules/file/fakedata.py
Check.
= 3) Read your new module and read up on any mentioned APIs and the global API =
There's a number of TODOs in the skeleton file and a lot of comments. Most comments give examples on possible implementations, as well. In fact the examples show you the complete implementation of an empty file.
With each method defined there are descriptions of how to implement that method. It's a good idea to also have a look at the types of the arguments passed to the methods we will be implementing. Going over all methods one could note we really only deal with host objects in this module. So let's look up the host class in our firefox: *click on Classes* and *click on core::host::host*. Read through the methods to know what services a host can provide. Interesting bits are:
- getTestDir
- sendCommand
- sendFile
- sendFiles
Also important is the static Campaign class. *Click on classes* and *click on core::campaign::Campaign* to read the documentation on that. Since we need a utility provided with the framework for file:fakedata, the testEnvDir property might be useful.
Now that we've read up on our supporting code and have an idea of what to do, we can get to implementing the module.
= 4) Write the implementation of your module =
When implementing a module there are two important points to take into account, next to building your complete implementation:
- Go over all places where it says 'TODO' (search for it)
- Document what you're doing and how your module works
The TODOs are there to make sure you touch all points you should, either because they need some administrative touches, or you should carefully consider whether to implement and/or extend the function. The documentation is obviously needed to make sure others can use your module. The most important documentation comes at the top of your class: there it should say what the module does, how to use it, and other important things about it.
A full contextual diff will be placed at the end of this file, so the exact implementation won't be discussed here, just a number of specific ways of getting there. As such, the documentation and administrative changes can be reviewed in the diff.
sendToSeedingHost is where the really interesting stuff happens. Note that the binary parameter tells us to use an already existing binary on the remote host (and which binary), so we should check whether it is set before trying to compile the binary remotely. Uploading and compiling the files involves some interaction with the rest of the framework and for that reason its development is detailed below. The following thoughts are relevant:
- The fakedata utility is in Utils/fakedata/ and consists of all .cpp and .h files there;
- On the remote host the source should have its own (temporary) directory;
- There are many compilers out there and we can't easily take all of them into account;
- Errors might occur and we should handle those.
The first thought touches on finding those files locally. In the Campaign object the testEnvDir property tells us where the testing environment is located. From there we can get to the fakedata utility: "{0}/Utils/fakedata/".format( Campaign.testEnvDir ) should be the directory holding the files.
The second thought has to do with making sure we don't overwrite other files and at the same time don't pollute the remote host with our stuff. A remote temporary directory would be ideal for that, as long as it is removed again when we stop. We'll create a remote temporary directory using the mktemp command. As for the base path we can give to mktemp: is there a good way to place this temporary directory? In fact you'll find there is: whenever a host is initialized a temporary directory is made available on it where temporary files for the testing framework can be placed. host.getTestDir() host.getPersistentTestDir() will tell you about this. The question is which to use. Will these files be needed after cleanup? Certainly not, only log files and similar output is needed after cleanup and this utility can be thrown away again. So host.getTestDir() is the right function to call.
Many compilers are available and we could write some very complex code trying to find out which compiler is supported, etc, etc. We might as well go for autoconf for that. Or we could just choose one. g++ is often available and for if it isn't: just let the user specify a manually compiled binary. This is a tradeoff between usability and complexity. In this case the complexity will grow far too much if we'd try and support many compilers. Those hosts lacking the g++ compiler are covered by the binary parameter.
The last thought, the occurence of errors, becomes more important with the choice for just one compiler. It's also a though that fall into two parts: finding problems and acting on them. Any problems that could occur in this case are during calls to remote commands. Luckily the host.sendCommand() function, which we'll use for running commands remotely, will return the output of the command. As such it is possible to just include some simple bash to always output, say, "OK" when everything went fine. Catch the output in a variable, check that the last line says "OK", and problems can be found. What to do, then, when a problem occurs? Raising an exception usually does it, they get logged and kill the current scenario.
Having thought about these things we can write most of the code for compiling the utility remotely. For running it one important question still arises: where to store the file? A convention that is used throughout the framework is that each module claims its own directory in a temporary directory's substructure: module_type/module_subtype/ . Or in this case: self.getFileDir(host). Wait, where did that come from? It's documented in the file parent object and will just return the complete path to the directory specific for your file module on the passed host. With this information it becomes a matter of filling in the blanks and just writing the code. See the diff for the results.
= 5) Test and adjust =
Testing your module should be done using your usual software testing techniques: cover all code, test corner cases, special parameters, etc. Make sure it works. It is usually easiest to write a small campaign in which to test a module you develop. I have a few around, for example, in which I can just plug a new client and have that client run on a few machines to send some files around. It is this one I also changed to use file:fakedata in order to test that.
When first developed the file:fakedata was not implemented correctly. Some typos, some small mistakes, the usual. The reason you test. More interestingly it turned out that no files were transmitted at all when using file:fakedata, even though the files were created as intended. An error in the design was found: torrents (which were used for its testing) name specific files, but file:fakedata chooses its own name. This led to the file being generated, but never being recognized by the seeder. The filename parameter was introduced because of this.
= Diff =
Below is the full contextual diff from file:_skeleton_ to file:fakedata. Note that the current version of file:fakedata is quite different, since it supports multiple files.
Note that this is actually the ported python version instead of the original bash version (for which this, now adapted, HOWTO was originally written).
*** ControlScripts/modules/file/_skeleton_.py 2012-03-09 13:27:03.546043891 +0100
--- ControlScripts/modules/file/fakedata.py 2012-03-09 13:27:03.572055429 +0100
***************
*** 1,59 ****
! # These imports are needed to access the parsing functions (which you're likely to use in parameter parsing),
! # the Campaign data object and the file parent class.
! from core.parsing import *
from core.campaign import Campaign
import core.file
! # NOTE: The last import above (import core.file) is different from usual. This is done to prevent trouble with python's
! # builtin file type. The import
! # from core.file import file
! # works perfectly, but hides the normal file type. The tradeoff is between a bit more typing (core.file.file instead of file)
! # and possible errors with regard to file and file (??).
!
! # You can define anything you like in the scope of your own module: the only thing that will be imported from it
! # is the actual object you're creating, which, incidentally, must be named equal to the module it is in. For example:
! # suppose you copy this file to modules/file/empty.py then the name of your class would be empty.
def parseError( msg ):
"""
A simple helper function to make parsing a lot of parameters a bit nicer.
"""
raise Exception( "Parse error for file object on line {0}: {1}".format( Campaign.currentLineNumber, msg ) )
! # TODO: Change the name of the class. See the remark above about the names of the module and the class. Example:
! #
! # class empty(core.file.file):
! class _skeleton_(core.file.file):
"""
! A skeleton implementation of a file subclass.
! Please use this file as a basis for your subclasses but DO NOT actually instantiate it.
! It will fail.
!
! Look at the TODO in this file to know where you come in.
"""
! # TODO: Update the description above. Example:
! #
! # """
! # An empty file object.
! #
! # To be used just to create an empty file.
! #
! # Extra parameters:
! # - filename The name of the file to be created.
! # """
def __init__(self, scenario):
"""
Initialization of a generic file object.
@param scenario The ScenarioRunner object this client object is part of.
"""
core.file.file.__init__(self, scenario)
- # TODO: Your initialization, if any (not likely). Oh, and remove the next line.
- raise Exception( "DO NOT instantiate the skeleton implementation" )
def parseSetting(self, key, value):
"""
Parse a single setting for this object.
--- 1,44 ----
! from core.parsing import isPositiveInt
from core.campaign import Campaign
import core.file
! import os
def parseError( msg ):
"""
A simple helper function to make parsing a lot of parameters a bit nicer.
"""
raise Exception( "Parse error for file object on line {0}: {1}".format( Campaign.currentLineNumber, msg ) )
! # The list of files needed for the fakedata utility
! fakedataGeneratorFiles = ['compat.h', 'fakedata.h', 'fakedata.cpp', 'genfakedata.cpp']
!
! class fakedata(core.file.file):
"""
! A file implementation for generated, fake data.
!
! This module uses Utils/fakedata to generate the data for the files.
! Extra parameters:
! - size A positive integer, divisible by 4096, that denotes the size of the generated file in bytes. Required.
! - binary The path of the remote binary to use. This might be needed when g++ does not work on one of the hosts
! this file is used on. Optional, defaults to "" which will have the binary compiled on the fly.
! - filename The name of the file that will be created. Optional, defaults to "fakedata".
"""
!
! size = None # The size of the file in bytes
! binary = None # Path to the remote binary to use
! filename = None # The filename the resulting file should have
def __init__(self, scenario):
"""
Initialization of a generic file object.
@param scenario The ScenarioRunner object this client object is part of.
"""
core.file.file.__init__(self, scenario)
def parseSetting(self, key, value):
"""
Parse a single setting for this object.
***************
*** 68,90 ****
generic settings parsed and to have any unknown settings raise an Exception.
@param key The name of the parameter, i.e. the key from the key=value pair.
@param value The value of the parameter, i.e. the value from the key=value pair.
"""
! # TODO: Parse your settings. Example:
! #
! # if key == 'filename':
! # if self.filename:
! # parseError( "Really? Two names? ... No." )
! # self.filename = value
! # else:
! # core.file.file.parseSetting(self, key, value)
! #
! # Do not forget that last case!
! #
! # The following implementation assumes you have no parameters specific to your file:
! core.file.file.parseSetting(self, key, value)
def checkSettings(self):
"""
Check the sanity of the settings in this object.
--- 53,80 ----
generic settings parsed and to have any unknown settings raise an Exception.
@param key The name of the parameter, i.e. the key from the key=value pair.
@param value The value of the parameter, i.e. the value from the key=value pair.
"""
! if key == 'size':
! if not isPositiveInt( value, True ):
! parseError( "The size must be a positive, non-zero integer" )
! if self.size:
! parseError( "Size already set: {0}".format(self.size) )
! self.size = int(value)
! elif key == 'binary':
! if self.binary:
! parseError( "The path to the fakedata binary has already been set: {0}".format( self.binary ) )
! self.binary = value
! elif key == 'filename' or key == 'fileName':
! if key == 'fileName':
! Campaign.logger.log( "Warning: the parameter fileName to file:fakedata has been deprecated. Use filename instead." )
! if self.filename:
! parseError( "The filename has already been set: {0}".format( self.filename ) )
! self.filename = value
! else:
! core.file.file.parseSetting(self, key, value)
def checkSettings(self):
"""
Check the sanity of the settings in this object.
***************
*** 92,105 ****
Any defaults may be set here as well.
An Exception is raised in the case of insanity.
"""
core.file.file.checkSettings(self)
! # TODO: Check your settings. Example:
! #
! # if not self.filename:
! # raise Exception( "A dummy file still needs a filename, dummy." )
def sendToHost(self, host):
"""
Send any required file to the host.
--- 82,102 ----
Any defaults may be set here as well.
An Exception is raised in the case of insanity.
"""
core.file.file.checkSettings(self)
!
! if not self.size:
! raise Exception( "The size parameter to file {0} is not optional".format( self.name ) )
! if not self.filename:
! self.filename = 'fakedata'
! if not self.binary:
! if not os.path.exists( os.path.join( Campaign.testEnvDir, 'Utils', 'fakedata' ) ):
! raise Exception( "The Utils/fakedata directory is required to build a fakedata file" )
! for f in fakedataGeneratorFiles:
! if not os.path.exists( os.path.join( Campaign.testEnvDir, 'Utils', 'fakedata', f ) ):
! raise Exception( "A file seems to be missing from Utils/fakedata: {0} is required to build the fakedata utility.".format( f ) )
def sendToHost(self, host):
"""
Send any required file to the host.
***************
*** 115,128 ****
change the name overriding that method is enough.
@param host The host to which to send the files.
"""
core.file.file.sendToHost(self, host)
- # TODO: Send any extra files here. These are the files that are required by all executions, whether they're seeding or leeching.
- # Seeding specific files are to be sent in sendToSeedingHost(...).
- #
- # Just having the default implementation send the meta file is usually enough.
def sendToSeedingHost(self, host):
"""
Send any files required for seeding hosts.
--- 112,121 ----
***************
*** 133,146 ****
The default implementation does nothing.
@param host The host to which to send the files.
"""
core.file.file.sendToSeedingHost(self, host)
! # TODO: Send the actual files to a seeding host. sendToHost(...) has already been called.
! # Note that self.getFileDir(...) is not guaranteed to exist yet. Example:
! #
! # host.sendCommand( 'mkdir -p "{0}/files/"; touch "{0}/files/{1}"'.format( self.getFileDir(host), self.filename ) )
def getFile(self, host):
"""
Returns the path to the files on the remote seeding host.
--- 126,160 ----
The default implementation does nothing.
@param host The host to which to send the files.
"""
core.file.file.sendToSeedingHost(self, host)
!
! res = host.sendCommand( 'mkdir -p "{0}/files"'.format( self.getFileDir(host) ) )
!
! binaryCommand = None
! if not self.binary:
! remoteBaseDir = '{0}/fakedata-source'.format( self.getFileDir(host) )
! host.sendCommand( 'mkdir -p "{0}"'.format( remoteBaseDir ) )
! for f in fakedataGeneratorFiles:
! host.sendFile( os.path.join( Campaign.testEnvDir, 'Utils', 'fakedata', f ), '{0}/{1}'.format( remoteBaseDir, f ), True )
! res = host.sendCommand( '( cd "{0}"; g++ *.cpp -o genfakedata && echo && echo "OK" )'.format( remoteBaseDir ) )
! if len(res) < 2:
! raise Exception( "Too short a response when trying to build genfakedata for file {0} in directory {1} on host {2}: {3}".format( self.name, remoteBaseDir, host.name, res ) )
! if res[-2:] != "OK":
! raise Exception( "Could not build genfakedata for file {0} in directory {1} on host {2}. Reponse: {3}".format( self.name, remoteBaseDir, host.name, res ) )
! binaryCommand = '{0}/genfakedata'.format( remoteBaseDir )
! else:
! res = host.sendCommand( '[ -e "{0}" -a -x "{0}" ] && echo "Y" || echo "N"' )
! if res != 'Y':
! raise Exception( "Binary {0} for file {1} does not exist on host {2}".format( self.binary, self.name, host.name ) )
! binaryCommand = self.binary
! res = host.sendCommand( '"{0}" "{1}/files/{2}" {3} && echo && echo "OK"'.format( binaryCommand, self.getFileDir(host), self.filename, self.size ) )
! if len(res) < 2:
! raise Exception( "Too short a response when trying to generate the fake data file {0} on host {1}: {2}".format( self.name, host.name, res ) )
! if res[-2:] != "OK":
! raise Exception( "Could not generate fake data file {0} on host {1}: {2}".format( self.name, host.name, res ) )
def getFile(self, host):
"""
Returns the path to the files on the remote seeding host.
***************
*** 155,174 ****
@param host The host on which to find the file(s).
@return The path to the (root of) the file(s) on the remote host, or None if they are not (yet) available.
"""
! # Note that this is the new name of getName(...), which made no sense in naming
! #
! # TODO: Send the path to the file uploaded to a seeding host. Example:
! #
! # "{0}/files/{1}".format( self.getFileDir(host), self.filename )
! #
! # This implementation assumes you don't really have files, which is unlikely but possible:
! return None
!
! # TODO: More methods exist, but they are pretty standard and you're unlikely to want to change them. Look at core.file for more details.
@staticmethod
def APIVersion():
- # TODO: Make sure this is correct. You don't want to run the risk of running against the wrong API version
return "2.0.0"
--- 169,178 ----
@param host The host on which to find the file(s).
@return The path to the (root of) the file(s) on the remote host, or None if they are not (yet) available.
"""
! return "{0}/files/{1}".format( self.getFileDir(host), self.filename )
@staticmethod
def APIVersion():
return "2.0.0"