Data cutting and network metrics #78

hechtlC · 2017-12-11T09:34:57Z

In this PR, we aim to introduce data cutting (see issue #38) and a bunch
of network metrics (see issue #73). See the respective issues for more
details.

Signed-off-by: Raphael Nömmer <[email protected]>

Signed-Off-By: Raphael Nömmer <[email protected]>

Add functionality to cut data sources to the same date ranges Add parameter in NetworkConf for that purpose Add cutting functionalities in the NetworkBuilder fixes #38 Signed-off-by: Christian Hechtl <[email protected]>

Signed-off-by: Christian Hechtl <[email protected]>

Signed-Off-By: Raphael Nömmer <[email protected]>

The cutting is replaced by the already existing splitting mechanism Signed-off-by: Christian Hechtl <[email protected]>

The timestamps are now extracted when the issue getter is called A warning message is printed when the data sources don't overlap Add project data getter in the NetworkBuilder for testing reasons Signed-off-by: Christian Hechtl <[email protected]>

Signed-off-by: Christian Hechtl <[email protected]>

Add checking of mail data in the cutting tests. Signed-off-by: Christian Hechtl <[email protected]>

Signed-off-by: Christian Hechtl <[email protected]>

Signed-off-by: Raphael Nömmer <[email protected]>

…on-r

Fix "author.email". Introduce "ref.name" column in tests (WIP and temporary fix). Add TODO item for fixing the test file "issues.list". Signed-off-by: Claus Hunsen <[email protected]> Signed-off-by: Christian Hechtl <[email protected]>

Signed-off-by: Raphael Nömmer <[email protected]>

To avoid a merge conflict in PR #78, we fix the indentation of a statement in the file 'util-read.R'. Signed-off-by: Claus Hunsen <[email protected]>

Merging the changes from se-passau/dev to avoid merge conflicts in #78. Signed-off-by: Claus Hunsen <[email protected]>

We now always have e-mail-address data available for authors, independent of the real data source containing those. If there is no data available, we add NAs. This is a follow-up for issue #69 and PR #71. Signed-off-by: Claus Hunsen <[email protected]>

clhunsen · 2017-12-11T15:28:16Z

I had no choice but merging the changes from se-passau/dev to resolve the merge conflicts effectively... Next time, we would like to aim for a rebase.

I still need to review the changes!

I also added a commit (7bfbe84) to enhance issue #69 and PR #71.

For easier internal use of data-source names (i.e., "commits", "issues", and "mails"), the data item containing the commit data is now called "commits" -- and not "commits.raw" anymore. All corresponding methods and method calls are renamed accordingly. This change will make it easier to handle data sources by their specific name, e.g., when performing a parameterizable subset 'proj.data[[data.source.name]]'. Note: The methods 'ProjectData$get.commits.raw()' and 'ProjectData$set.commits.raw()' are still there for compatibility reasons. They are now mere delegates to the new methods. Signed-off-by: Claus Hunsen <[email protected]>

ecklbarb · 2017-12-12T16:05:13Z

The attribute name does not exist in the function metrics.modularity() (line 104).

After the landing of commit a803425e6bdb54c1654fb9de1f9375499e3aa829, the general code for the data cutting mechanism is streamlined: 1) The 'ProjectData$data.timestamps' attribute is now transposed -- to map data sources per line to their respective timestamps in the columns. This way is more intuitive and better for later access. 2) All related methods are adapted accordingly. 3) The amount of inline documentation is increased significantly. Signed-off-by: Claus Hunsen <[email protected]> Reviewed-by: Thomas Bock <[email protected]>

This patch only applies some more readable code formatting to the networks-metrics module. Signed-off-by: Claus Hunsen <[email protected]>

@ecklbarb

In this patch, we fix two minor bugs in the network metrics 'node.degree' and 'modularity'. In the first, the unsorted result was not assigned correctly to the return value. In the latter, the single modularity value does not need a name (which was also an undefined variable). Props to @ecklbarb for reporting these two mistakes. Signed-off-by: Claus Hunsen <[email protected]>

Fix mistakes and add 'unify.date.ranges' documentation. Signed-off-by: Claus Hunsen <[email protected]>

Now, in the README file and the configuration module, we have 'ProjectConf' documentation first and 'NetworkConf' documentation second. Signed-off-by: Claus Hunsen <[email protected]>

This is just a code movement of the method 'get.pasta.items' for better structure in the 'ProjectData' class. Signed-off-by: Claus Hunsen <[email protected]>

In the metrics module, single-value returns are now named vectors. The name is the metrics name as used in the respective function definition. Additionally, the column names for the scale-freeness metric is changed to not include 'res.' at each column name's beginning. Signed-off-by: Claus Hunsen <[email protected]>

clhunsen

Basically, the PR looked good right away, but I improved it nevertheless by pushing some convenient commits myself. For me, this is ready to merge.

bockthom · 2017-12-13T12:09:59Z

README.md

@@ -124,19 +124,22 @@ Updates to the parameters can be done by calling `NetworkConf$update.variables(.
       - issue information: *`"issue.id"`*, *`"event.name"`*, `"issue.state"`, `"creation.date"`, `"closing.date"`, `"is.pull.request"`
  * **Note**: `"date"` is always included as this information is needed for several parts of the library, e.g., time-based splitting.
  * **Note**: For each type of network that can be built, only the applicable part of the given vector of names is respected.
-  * **Note**: For the edge attributes `"pasta"` and `"synchronicty"`, the network configuration's parameters `pasta` and `synchronicity` need to be set to `TRUE`, respectively (see below).
+  * **Note**: For the edge attributes `"pasta"` and `"synchronicty"`, the project configuration's parameters `pasta` and `synchronicity` need to be set to `TRUE`, respectively (see below).


@clhunsen Could you please fix the typo in "synchronicty" here?

bockthom

The PR looks good so far, beside the README.

There are several issues to fix in the README:

The files util-read.R and util-networks-metrics.R are missing in the section "File overview".
The "How to" section is not up to date any more as there should appear the NetworkBuilder, I guess.
In the section "Configuration Classes" the subsectioning is wrong: "Project Conf" should be on the same level as "Network Conf" and, therefore, start with ###.

@bockthom

- Add missing files and their respective descriptions. - Update how-to section's code snippet. - Fix some typos. - Fix the indentation of the 'ProjectConf' sections. - Add proper links to intra-document sections. - Add syntax highlighting for multi-line code snippets. Props to @bockthom for mentioning some of the points in PR #78. Signed-off-by: Claus Hunsen <[email protected]>

@bockthom

Fix typos in the roxygen documentation of some of the reading functions. Remove wrong documentation for the issue-reading function. Props to @bockthom for pointing this out. Signed-off-by: Claus Hunsen <[email protected]>

For better comprehensibility, the showcase file is renamed to 'showcase.R', as the name 'test.R' was misleading regarding the tests. Signed-off-by: Claus Hunsen <[email protected]>

Raphael-N and others added 30 commits August 23, 2017 20:00

Introduce file with network metrics

83bfc0e

Signed-off-by: Raphael Nömmer <[email protected]>

Add more network metrics, change issue data folder

76014b2

Signed-Off-By: Raphael Nömmer <[email protected]>

Add namespace requirements to smallworldness

ba08a24

Signed-Off-By: Raphael Nömmer <[email protected]>

Continue implementing metrics

6adabaf

Signed-Off-By: Raphael Nömmer <[email protected]>

Add more metrics for network analysis.

aeb456f

Signed-Off-By: Raphael Nömmer <[email protected]>

Merge remote-tracking branch 'origin/master' into issues

ff8fc10

Indroduce first plot, fix error when loading issue

c1ad7dc

Signed-Off-By: Raphael Nömmer <[email protected]>

Introduce handling of incomplete ranges

545850a

Add functionality to cut data sources to the same date ranges Add parameter in NetworkConf for that purpose Add cutting functionalities in the NetworkBuilder fixes #38 Signed-off-by: Christian Hechtl <[email protected]>

Adjust getter of NetworkConf to new type

5921070

Signed-off-by: Christian Hechtl <[email protected]>

Add plot functions for metrics and change encoding

7187e1f

Signed-Off-By: Raphael Nömmer <[email protected]>

Merge remote-tracking branch 'origin' into issues

1ab5289

Visual changes to hierarchy plot

3e616d8

Signed-Off-By: Raphael Nömmer <[email protected]>

Rebuild cutting mechanism for incomplete Ranges

90cdc09

The cutting is replaced by the already existing splitting mechanism Signed-off-by: Christian Hechtl <[email protected]>

Fix minor bugs in cutting mechanism

4be6f2f

The timestamps are now extracted when the issue getter is called A warning message is printed when the data sources don't overlap Add project data getter in the NetworkBuilder for testing reasons Signed-off-by: Christian Hechtl <[email protected]>

Add tests for the cutting mechanism on data and network side

9373d01

Signed-off-by: Christian Hechtl <[email protected]>

Add documentation to new getter

1ec406c

Add checking of mail data in the cutting tests. Signed-off-by: Christian Hechtl <[email protected]>

Minor changes to metrics behaviour

4c2bd70

Update pasta reading method and test

53a2460

Signed-off-by: Christian Hechtl <[email protected]>

Add comments to network metrics.

abfafb9

Signed-off-by: Raphael Nömmer <[email protected]>

Merge remote-tracking branch 'origin/master' into issues

2664408

Merge remote-tracking branch 'origin/issues'

3330ab5

Merge branch 'master' of https://github.com/hechtlC/codeface-extracti…

7c3e5dc

…on-r

Various minor changes to metrics

c0e0f2a

Signed-off-by: Raphael Nömmer <[email protected]>

Merge remote-tracking branch 'origin' into issues

01e09a9

Minor changes to metrics.R

894feeb

Signed-off-by: Raphael Nömmer <[email protected]>

Update issues.list test file

ea0a252

Signed-off-by: Raphael Nömmer <[email protected]>

Change issue reading test according to issues.list

c8de295

Signed-off-by: Raphael Nömmer <[email protected]>

Rename metrics, add parameter check in metrics

9672a70

Signed-off-by: Raphael Nömmer <[email protected]>

Rename util-metrics.R to util-network-metrics.R

b433115

Signed-off-by: Raphael Nömmer <[email protected]>

Raphael-N added 2 commits December 5, 2017 23:33

Rename metrics file

ddb93be

Signed-off-by: Raphael Nömmer <[email protected]>

Update filename in util-init.R

d4b53df

Signed-off-by: Raphael Nömmer <[email protected]>

clhunsen requested review from clhunsen and bockthom December 11, 2017 12:26

clhunsen added the enhancement label Dec 11, 2017

clhunsen added this to the v3.0 milestone Dec 11, 2017

clhunsen added 3 commits December 11, 2017 14:41

Fix indentation to avoid merge conflict

52dd2e3

To avoid a merge conflict in PR #78, we fix the indentation of a statement in the file 'util-read.R'. Signed-off-by: Claus Hunsen <[email protected]>

Merge remote-tracking branch 'upstream/dev' into hiwis/master

524718d

Merging the changes from se-passau/dev to avoid merge conflicts in #78. Signed-off-by: Claus Hunsen <[email protected]>

clhunsen added 7 commits December 12, 2017 17:44

Format networks-metrics module

45ec73f

This patch only applies some more readable code formatting to the networks-metrics module. Signed-off-by: Claus Hunsen <[email protected]>

Update README file

af2b3b6

Fix mistakes and add 'unify.date.ranges' documentation. Signed-off-by: Claus Hunsen <[email protected]>

Re-order NetworkConf and ProjectConf

1b86b1d

Now, in the README file and the configuration module, we have 'ProjectConf' documentation first and 'NetworkConf' documentation second. Signed-off-by: Claus Hunsen <[email protected]>

Move PaStA method in data class

61fc771

This is just a code movement of the method 'get.pasta.items' for better structure in the 'ProjectData' class. Signed-off-by: Claus Hunsen <[email protected]>

clhunsen previously approved these changes Dec 13, 2017

View reviewed changes

bockthom reviewed Dec 13, 2017

View reviewed changes

bockthom requested changes Dec 13, 2017

View reviewed changes

clhunsen added 2 commits December 13, 2017 15:12

Fix and update function documentation in reading module

389a63b

Fix typos in the roxygen documentation of some of the reading functions. Remove wrong documentation for the issue-reading function. Props to @bockthom for pointing this out. Signed-off-by: Claus Hunsen <[email protected]>

clhunsen dismissed their stale review via 389a63b December 13, 2017 14:14

Rename showcase file to 'showcase.R'

caa38c1

For better comprehensibility, the showcase file is renamed to 'showcase.R', as the name 'test.R' was misleading regarding the tests. Signed-off-by: Claus Hunsen <[email protected]>

bockthom approved these changes Dec 13, 2017

View reviewed changes

bockthom merged commit 86f5c3a into se-sic:dev Dec 13, 2017

bockthom mentioned this pull request Dec 13, 2017

Network metrics #73

Closed

clhunsen mentioned this pull request Dec 14, 2017

How to handle incomplete ranges resp. missing data? #38

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Data cutting and network metrics #78

Data cutting and network metrics #78

hechtlC commented Dec 11, 2017

clhunsen commented Dec 11, 2017 •

edited

Loading

ecklbarb commented Dec 12, 2017

clhunsen left a comment

bockthom Dec 13, 2017

bockthom left a comment

Data cutting and network metrics #78

Data cutting and network metrics #78

Conversation

hechtlC commented Dec 11, 2017

clhunsen commented Dec 11, 2017 • edited Loading

ecklbarb commented Dec 12, 2017

clhunsen left a comment

Choose a reason for hiding this comment

bockthom Dec 13, 2017

Choose a reason for hiding this comment

bockthom left a comment

Choose a reason for hiding this comment

clhunsen commented Dec 11, 2017 •

edited

Loading