Skip to content

Commit

Permalink
[doc] Update prosoul-grimoirelab.md
Browse files Browse the repository at this point in the history 
This commit updates the documentation with the workflow
of using prosoul along with grimoirelab. A sample QM is
also added to the data folder.

Signed-off-by: Venu Vardhan Reddy Tekula <[email protected]>
  • Loading branch information
@vchrombie
vchrombie committed Aug 26, 2020
1 parent c457f54 commit c31f39e
Show file tree
Hide file tree
Showing 2 changed files with 205 additions and 20 deletions.
79 changes: 79 additions & 0 deletions django-prosoul/prosoul/data/developer_model_grimoirelab.json
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
{
"qualityModels": [
{
"goals": [
{
"attributes": [
{
"description": "",
"factoids": [],
"metrics": [
{
"calculation_type": "max",
"data_implementation": "Number of Commits",
"data_params": null,
"data_source_type": null,
"description": "",
"name": "Number of Commits",
"reverse_thresholds": false,
"thresholds": "1,5,10,20,50"
},
{
"calculation_type": "max",
"data_implementation": "Number of Created Issues",
"data_params": null,
"data_source_type": null,
"description": "",
"name": "Number of Issues",
"reverse_thresholds": false,
"thresholds": "1,5,10,20,50"
}
],
"name": "Vitality",
"subattributes": []
}
],
"description": "",
"name": "Product",
"subgoals": []
},
{
"attributes": [
{
"description": "",
"factoids": [],
"metrics": [
{
"calculation_type": "max",
"data_implementation": "Number of Issues Attended",
"data_params": null,
"data_source_type": null,
"description": "",
"name": "Number of Issues Attended",
"reverse_thresholds": false,
"thresholds": "1,5,10,20,50"
},
{
"calculation_type": "max",
"data_implementation": "Number of Closed Issues",
"data_params": null,
"data_source_type": null,
"description": "",
"name": "Number of Issues Closed",
"reverse_thresholds": false,
"thresholds": "1,5,10,20,50"
}
],
"name": "Attention",
"subattributes": []
}
],
"description": "",
"name": "Community",
"subgoals": []
}
],
"name": "Developer Quality Model (GrimoireLab)"
}
]
}
146 changes: 126 additions & 20 deletions doc/prosoul-grimoirelab.md
View file
Original file line number Diff line number Diff line change
@@ -1,39 +1,145 @@
# Using prosoul with GrimoireLab
# Using Prosoul with GrimoireLab

### Start Prosoul, Elasticsearch and Kibiter using docker

```
(acs@dellx) (master *% u=) ~/devel/prosoul/docker $ docker-compose up -d
Creating docker_prosoul_1
Creating docker_elasticsearch_1
Creating docker_kibiter_1
p0tt3r@wand ~/dev/prosoul/docker $ docker-compose up -d
Starting docker_elasticsearch_1 ... done
Starting docker_kibiter_1 ... done
Starting docker_prosoul_1 ... done
```

### Load GrimoireLab Metrics Data
The three services will be available at their respective ports.
- Elasticsearch at https://localhost:9200/
- Kibiter at http://localhost:5601/
- Prosoul at http://localhost:8000/

You need a GitHub API token in order to load GitHub data from GrimoireLab projects.
Let's perform an assessment for a few grimoirelab projects using the `gitqm` and `githubqm` backends and the Developer Quality Model (for GrimoireLab).

### Load GrimoireLab Data

With the `prosoul/tests/grimoirelab/init.sh` script the data for projects is collected,
and the quality model for doing the assessment and visualization is loaded.
You need to setup [SirMordred](https://github.com/chaoss/grimoirelab-sirmordred) project. You can use the [Getting-Started.md](https://github.com/chaoss/grimoirelab-sirmordred/blob/master/Getting-Started.md#getting-started-) guide and omit the `Getting the containers` section as the Elasticsearch and Kibiter are already available.

Now that you have the project configured in the PyCharm, we can execute micro-mordred/sirmordred. All you need to define the setup.cfg and projects.json with the required configurations and execute the backends.

Let's use micro-mordred for this purpose.

projects.json
```
acs@~/devel/prosoul/tests/grimoirelab $ ./init.sh <github-api-token>
{
"GrimoireLab": {
"gitqm": [
"https://github.com/chaoss/grimoirelab-elk" ,
"https://github.com/chaoss/grimoirelab-kidash" ,
"https://github.com/chaoss/grimoirelab-sirmordred" ,
"https://github.com/chaoss/grimoirelab-perceval" ,
"https://github.com/chaoss/grimoirelab-sortinghat"
] ,
"githubqm:issue": [
"https://github.com/chaoss/grimoirelab-elk" ,
"https://github.com/chaoss/grimoirelab-kidash" ,
"https://github.com/chaoss/grimoirelab-sirmordred" ,
"https://github.com/chaoss/grimoirelab-perceval" ,
"https://github.com/chaoss/grimoirelab-sortinghat"
]
}
}
```

Once the script execution has finished, you have in Elasticsearch the projects data needed for
doing the assessment and visualization of them using ProSoul.
setup.cfg
```
[gitqm]
raw_index = grimoirelab_gitqm_raw
enriched_index = grimoirelab_gitqm_enriched
category = commit
[githubqm:issue]
raw_index = grimoirelab_githubqm_issues_raw
enriched_index = grimoirelab_githubqm_issues_enriched
api-token = xxxx
sleep-for-rate = true
sleep-time = 300
category = issue
no-archive = true
```
You can find all the configurations here, link.

Now, execute the raw and enrich tasks of micro-mordred.
```
micro.py --raw --enrich --cfg ./setup.cfg --backends gitqm githubqm:issue
```

Setup alias for all the enriched indices.
```
POST /_aliases
{
"actions": [
{
"add": {
"indices": [
"grimoirelab_gitqm_enriched",
"grimoirelab_githubqm_issues_enriched"
],
"alias": "all_qm_data"
}
}
]
}
```

Now, you have all the required enriched data in `qm-enriched-data` index which is needed for the assessement and visualization using Prosoul.

### Import the QualityModel

* Import the [Developer Quality Model](https://github.com/Bitergia/prosoul/blob/master/django_prosoul/prosoul/data/developer_model.json) using prosoul web editor
Import the [Developer Quality Model](https://github.com/Bitergia/prosoul/blob/master/django_prosoul/prosoul/data/developer_model_grimoirelab.json) using prosoul web editor.

### Load the Metrics Data

### Create the Visualization and Assessment using ProSoul web interface
Login with the admin credentials and go to the admin panel, `Prosoul Site Administration` and click on `Metric datas`. You have to entry the metric items data by clicking on the `ADD METRIC DATA` button. You can add the metric data by filling out the web form using the `metric_name`.

Use the Visualize and Assess link in Prosoul web interface to generate them.
- Description: Number of Commits
- Implementation: Number of Commits

Similarly, you have to add all the metrics data which you use while making a Quality Model.

### Create the Assessment

The next step is to create the assessment using prosoul assessment web form.

- Quality Model: Developer Quality Model
- Elasticsearch URL: https://localhost:9200/
- Index with metrics data: all_qm_data

Click on `Create` and the assessment will be completed in a while.

After the assessment is completed, you can view the results using the prosoul web interface.

### Import the Dashboard

Setup alias for all the qm results indices.
```
POST /_aliases
{
"actions": [
{
"add": {
"indices": [
"all_qm_data_null_scores",
"all_qm_data_null_scores_by_quarters",
"all_qm_data_scores",
"all_qm_data_scores_by_quarters"
],
"alias": "all_qm_data_results"
}
}
]
}
```

Import the dashboard from the sigils repository.
```
p0tt3r@wand ~ $ kidash -g -e https://admin:admin@localhost:9200 --import ~/dev/sources/grimoirelab-sigils/json/qm-dashboard.json
```

In the web forms use:
After importing the dashboard, you can view the results in the dashboard and edit the visualizations as per your need.

* Elasticsearch URL: http://172.17.0.1:9200
* Kibana URL: http://172.17.0.1:5601
* Index with metrics data: grimoirelab
* Atribute Template: AttributeTemplateGrimoireLab

0 comments on commit c31f39e

Please sign in to comment.