diff --git a/docs/Conversation-on-Props.md b/docs/Conversation-on-Props.md new file mode 100644 index 0000000..ba6111a --- /dev/null +++ b/docs/Conversation-on-Props.md @@ -0,0 +1,120 @@ +# RunTypes JSON File + +(Mike Bottini here) + +From my commit message of the `runtypes_config.json` revision: + +--- + +Each RunType will now contain the following values: + +* `args`, containing an ordered list of arguments. This is for the +purpose of dialogue and presenting each parameter to the user. + +* `annotations`, containing key-value pairs of arguments and +descriptions for each one. This is for explaining each parameter +to the user. + +* `types`, containing key-value pairs of arguments and types. This +is for defining valid user input for each argument. + +* `translations`, containing key-value pairs of arguments and +.props values. Upon running in `main`, the program will take a Run +and its corresponding RunType and write values to its temporary +.props file. + +* `props_defaults`, containing values that will be written by +default to the .props file without any user input at all. +These are intrinsic to the RunType. + +For example, let's say that we have an RunType with the following: + + "SPAM" : { + args : [ + "foo" + "bar" + ], + + annotations : { + "foo" : "Lorem ipsum dolor sit amet, consectetur...", + "bar" : "Sed blandit metus quis nulla dapibus..." + }, + + "types" : { + "foo" : "string", + "bar" : "integer" + }, + + "translations" : { + "foo" : "controller.quux.spam.eggs.foo" + }, + + "props_defaults" : { + "controller.muffin.scone" : false, + "controller.waffle.pancake" : 50, + "controller.bacon.sausage" : "eggs" + } + } + +Upon a user creating a `SPAM` Run, the Dialogue will ask the user to +provide values for `foo` and `bar`, demanding a string and integer +as input, respectively. If needed, the user can obtain the annotations +(Lorem ipsum dolor sit amet...) to explain what he is supposed to input. +Let's say that he inputs `oatmeal` and `5`, respectively. + +Upon running the above run, `main` will write the following to +the temporary .props file: + + controller.quux.spam.eggs.foo=oatmeal + controller.muffin.scone=false + controller.waffle.pancake=50 + controller.bacon.sausage=eggs + +The remaining `bar` value will possibly be used in the actual JVM +invocation. + +# RunList JSON File + +The RunList is much simpler. Going with Benjamin Spriggs' request for more metadata, +I'm happy to provide something like the following schema: + + { + "Runs" : [ + { + "Tag" : "SPAM Run 1", + "RunType" : "SPAM", + "foo" : "oatmeal", + "bar" : 5 + } + ], + + "Meta" : { + "creation_date": "2012 23 1 1", + "name": "crepes", + "company": "Cream of Wheat, Inc" + } + } + +Note that `Runs` is an ordered list, as each run will be run sequentially in an +order that matters to the user. + +# Putting it Together In Main + +As stated above and in the meeting, `main` needs to pull from both the `RunList` +and the `RunTypes` JSON files. + +* It will take the tag name from the `RunList` to create a directory with the +results. + +* It will obtain the RunType of the current Run from the `RunList`. + +* It will write the props file with the props attributes defined in `RunTypes` +and the values defined in the `RunList`. + +* It will use a **hardcoded** function to create the JVM invocation, pulling +values from the `RunList`. Note that right now, we're just constructing +`run.sh` calls. Depending on how few arguments remain, this might not have +to be hardcoded! It might be possible to create a generic invocation with +any additional JVM Options, if necessary. + +* It then invokes the JVM with this command and saves the data. \ No newline at end of file diff --git a/docs/Conversation-on-Run-Templates.md b/docs/Conversation-on-Run-Templates.md new file mode 100644 index 0000000..c74c898 --- /dev/null +++ b/docs/Conversation-on-Run-Templates.md @@ -0,0 +1,37 @@ +## Run Types +``` +{ + "HBIR": { + "userargs": [ + "T1" + ], + "annotation": { + "someArg": "This is an example of an argument that gets put into a populated .props file." + }, + "types": { + "someArg": "string", + }, + "translation": { + "someArg": "com.spec.something" + } + } +} +``` + +TODO: Add more extensive notes. + +## Run Lists +Also talked about run lists. + +These are the things that SPECtate is going to use to construct a benchmarking run. How many injectors? How many backends? What values go into the `userargs` defined above? etc. +``` +[ +{ + "tag": "value", + "runtype": "HBIR", + "numa_nodes": 4, + "backends": 20 +}, +... +] +``` \ No newline at end of file diff --git a/docs/Home.md b/docs/Home.md new file mode 100644 index 0000000..5bbd60d --- /dev/null +++ b/docs/Home.md @@ -0,0 +1 @@ +Welcome to the SPECtate wiki! diff --git a/docs/INDEX.md b/docs/INDEX.md index e69de29..68e7b34 100644 --- a/docs/INDEX.md +++ b/docs/INDEX.md @@ -0,0 +1,10 @@ +# SPECtate + +Welcome to the SPECtate wiki! + +## Contents + +- [Conversation on Properties](docs/Conversation-on-Props.md) +- [Conversation on Run Templates](docs/Conversation-on-Run-Templates.md) +- [Conversation on SPECTate Configurations](docs/Tate-Config.md) +- [Documentation on working in virtual environments](docs/Working-in-Virtual-Environments.md) diff --git a/docs/Tate-Config.md b/docs/Tate-Config.md new file mode 100644 index 0000000..7135753 --- /dev/null +++ b/docs/Tate-Config.md @@ -0,0 +1,251 @@ +Based on the conversation today at the library, we've decided to go in the direction of having a single JSON file for everything. + +# Glossary + +## Run + +The information needed to perform a single invocation of SpecJBB. This currently looks like the following: + + { + "template_type" : "HBIR", + "args" : { + "arg1" : 5, + "arg2" : "foobar", + "arg3" : false + } + } + +TODO: There's a possibility of adding a third key-value pair to each Run, called `props_extra`. This would provide an opportunity to give fine-tuning capability for each Run. This is not necessary, and we're not even sure if we should put it in. But it's a possibility. + +## Template + + The information needed to + +* Determine which `args` are needed from the user. (`args`) +* Determine which type each `arg` is. (`types`) +* Determine the correspondence between `args` and `.props` properties. (`translations`) +* Provide default properties that will **always** be written to the `.props` files (`default_props`). +* Provide annotations for each `arg` for dialogue. + +An example of this is the following: + + { + "args" : [ + "Tag", + "Kit Version", + "JDK", + "RTSTART", + "JVM Options", + "NUMA Nodes", + "Data Collection", + "T1", + "T2", + "T3" + ], + + "annotations" : { + "Tag" : "Name of the run", + "Kit Version" : "Version of SpecJBB", + "JDK" : "Version of the JVM that will run SpecJBB", + "RTSTART" : "What percentage of total output will we start at", + "JVM Options" : "What additional arguments, if any, will be passed to the JVM", + "NUMA Nodes" : "How many NUMA nodes will SpecJBB use", + "Data Collection" : "What data collection process will monitor while running SpecJBB", + "T1" : "How many threads does Tier 1 have access to", + "T2" : "How many threads does Tier 2 have access to", + "T3" : "How many threads does Tier 3 have access to" + }, + + "types" : { + "Tag" : "string", + "Kit Version" : "string", + "JDK" : "string", + "RTSTART" : "integer", + "JVM Options" : "string", + "NUMA Nodes" : "integer", + "Data Collection" : "string", + "T1" : "integer", + "T2" : "integer", + "T3" : "integer" + }, + + "translations" : { + "RTSTART" : "specjbb.controller.rtcurve.start", + "T1" : "specjbb.forkjoin.workers.Tier1", + "T2" : "specjbb.forkjoin.workers.Tier2", + "T3" : "specjbb.forkjoin.workers.Tier3", + "NUMA Nodes" : "specjbb.group.count" + }, + + "default_props" : { + "specjbb.controller.type" : "HBIR", + "specjbb.time.server" : false, + "specjbb.comm.connect.client.pool.size" : 192, + "specjbb.comm.connect.selector.runner.count" : 4, + "specjbb.comm.connect.timeouts.connect" : 650000, + "specjbb.comm.connect.timeouts.read" : 650000, + "specjbb.comm.connect.timeouts.write" : 650000, + "specjbb.comm.connect.worker.pool.max" : 320, + "specjbb.customerDriver.threads" : 64, + "specjbb.customerDriver.threads.saturate" : 144, + "specjbb.customerDriver.threads.probe" : 96, + "specjbb.mapreducer.pool.size" : 27 + } + } + + +### Args + +**Run-specific** values that are specified by the user. In a Template, this is an **ordered list** to provide a common order for inputting values inside a dialogue. + +### Properties + +A property is a single line in the `.props` file that will be passed to SPECjbb. + +### Translations + +A translation maps an Arg to a Property. + +### Annotations + +A description maps an Arg to a human-readable description of what the argument actually means. + +### Types + +A type is the data type that an Arg must be. This is for validation. + +## RunList + +An **ordered list** of Runs. + +For example: + + [ + { + "template_type" : "HBIR", + "args" : { + "arg1" : 5, + "arg2" : "foobar", + "arg3" : false + } + }, + + { + "template_type" : "HBIR_RT", + "args" : { + "arg1" : "muffin" + "arg2" : 42, + "arg3" : "spam", + "arg4" : true + } + } + ] + +## TemplateData + +A key-value pair that maps **template types** to Templates. + +For example: + + { + "HBIR" : { + "args" : [], + "default_props" : {}, + "types" : {}, + "translations" : {} + }, + + "HBIR_RT" : { + "args" : [], + "default_props" : {}, + "types" : {}, + "translations" : {} + } + } + +## TateConfig + +A JSON file that contains an object with two values: a TemplateData object and a RunList object. + +For example: + + { + "TemplateData" : { + "HBIR" : { + "args" : [], + "default_props" : {}, + "types" : {}, + "translations" : {} + }, + + "HBIR_RT" : { + "args" : [], + "default_props" : {}, + "types" : {}, + "translations" : {} + } + }, + + "RunList" : [ + { + "template_type" : "HBIR", + "args" : { + "arg1" : 5, + "arg2" : "foobar", + "arg3" : false + } + }, + + { + "template_type" : "HBIR_RT", + "args" : { + "arg1" : "muffin" + "arg2" : 42, + "arg3" : "spam", + "arg4" : true + } + } + ] + } + +# Dialogue Treatment + +* This means that Dialogue programs only have to open one file. + +* Most code can remain unchanged; the only difference is that instead of writing to two separate JSON files, the RunList and Template Data objects are saved as values of an overarching TateConfig file. + +# Main Program Treatment + +This is more complicated. Let's list the interactions. + +* Dialogue creates or edits a TateConfig file and saves it. + +* Main program opens the TateConfig file. + +* Main program parses the RunList and selects a Run. + +--- + +## For each run... + +* Main program grabs the `template_type` attribute of the Run. + +* Main program goes to the `TemplateData` attribute of the TateConfig file and selects that specific Template. + +### For each value in `translations`... + +* Get the corresponding value for `arg` from the Run. + +* Write it to the `.props` file. + +### For each value in `props_default`... + +* Write it. No extra stuff needs to happen here. + +--- + +Now that we've written the `.props` file, we can do the JVM invocation. + +* Construct the JVM invocation(s). using the Run's `args` and `template_type`. + +* Call the JVM. \ No newline at end of file diff --git a/docs/Working-in-Virtual-Environments.md b/docs/Working-in-Virtual-Environments.md new file mode 100644 index 0000000..4664fde --- /dev/null +++ b/docs/Working-in-Virtual-Environments.md @@ -0,0 +1,23 @@ +# Working in Virtual Environments + +`SPECtate` uses [`pipenv`](https://github.com/pypa/pipenv) to manage python installs and dependencies. + +There's a fairly straightforward [wiki here, which is more or less the GitHub readme with some additional comments from the maintainers](https://docs.pipenv.org/). + +The whole idea boils down to: + +- You want to know what to install +- You want to know what versions to require +- You want builds and installs to be deterministic (no chance for setup failure) +- You don't want to manage all the details that require the above to work + +`pipenv` more or less works as a [`yarn`](https://yarnpkg.com/lang/en/docs/getting-started/) or a [`dep`](https://gist.github.com/subfuzion/12342599e26f5094e4e2d08e9d4ad50d) for python, providing lockfiles and install commands for easy setup. + +## Basic Workflow + +1. Clone the repo +1. `pipenv shell` - drops you into a shell where you're using `SPECtate`'s version of python and its dependencies +1. `pipenv check` - ensures you have everything you need (resolve conflicts, error on wrong versions of python, etc) + +Installing additional packages is as easy as `pipenv install `. +`pipenv --help` can give you more information and specifics, if you need to do something more involved. \ No newline at end of file