Skip to content

Diffing Metrics

Jump to bottom
Vincenzo Musco edited this page Oct 9, 2018 · 2 revisions

The tool sa-metrics can be used to compute different evolution metrics between two versions of an app. It works on output archived produced by the sa-disassemble tool. Same inclusion/exclusion parameters can be passed to this function:

usage: sa-metrics [-h] [--verbose] [--onlyapppackage] [--fulllinesofcode]
                  [--aggregateoperators] [--include-unpackaged]
                  [--exclude-lists [EXCLUDE_LISTS [EXCLUDE_LISTS ...]]]
                  [--include-lists [INCLUDE_LISTS [INCLUDE_LISTS ...]]]
                  [--no-innerclasses-split]
                  smaliv1 smaliv2 pkg

Compute evolution metrics between two smali versions.

positional arguments:
  smaliv1               Version 1 folder containing smali files
  smaliv2               Version 2 folder containing smali files
  pkg                   The app package name

optional arguments:
  -h, --help            show this help message and exit
  --verbose, -v         Show metrics details
  --onlyapppackage, -P  Includes only classes in the app package specified
  --fulllinesofcode, -f
                        Show full lines instead of opcodes for differences
  --aggregateoperators, -a
                        Aggregate the operator by their first keywork.
  --include-unpackaged, -U
                        Includes classes which are not in a package
  --exclude-lists [EXCLUDE_LISTS [EXCLUDE_LISTS ...]], -e [EXCLUDE_LISTS [EXCLUDE_LISTS ...]]
                        Files containing excluded list
  --include-lists [INCLUDE_LISTS [INCLUDE_LISTS ...]], -i [INCLUDE_LISTS [INCLUDE_LISTS ...]]
                        Files containing included list
  --no-innerclasses-split, -I
                        Do not split metrics for inner/outer classes

Metrics reported by the tools includes:

  • Global class evolution metrics
    • Number of total classes on both versions (#C-, #C+);
    • Number of classes which contains only revisions and new methods (E);
    • Number of classes for which only their method body changed (B);
    • Number of classes for which their change are neither classified on E or B (C);
    • Number of classes which contains AT LEAST one method addition (A);
    • Number of classes which contains AT LEAST one method deletion (D);
  • Class metrics
    • Number of classes added (CA);
    • Number of classes deleted (CD);
    • Number of classes changed (CC);
  • Class method metrics
    • Number of method classes added (MA);
    • Number of method classes deleted (MD);
    • Number of method classes renamed (MR);
    • Number of method classes changed (MC);
    • Number of method classes revised (MRev, subset of MC);
  • Class fields metrics
    • Number of fields added (FA);
    • Number of fields deleted (FD);
    • Number of fields renamed (MR);
    • Number of fields changed (FC);
    • Number of fields revised (MRev, subset of MC);
  • Bytecode instructions metrics
    • List of smali operators added (addedLines);
    • List of smali operators deleted (removedLines);

By default, these metrics are shown for inner classes (IN) and outer classes (OUT). An aggregated mode is available using the --no-innerclasses-split option (resulting in merging all [IN|OUT]x into a x aggregated metrics).

The --verbose option allows to get a more verbose and visual output.

python metrics.py 495_2947185.apk.smali 495_5014602.apk.smali ""
===== IN CLASSES =====
v0 has 789 classes, v1 has 1585 classes.
E = 34. R = 48. C = 117.
Classes - Added:   901, Changed:   199, Deleted:   105.
Methods - Added:   476, Revised:   289, Changed:   305, Renamed:    17, Deleted:    69.
Fields - Added:    24, Changed:    37, Renamed:     9, Deleted:    24.
Added lines:
	- cmp-long
    (truncated output)
Removed lines:
	- cmp-long
    (truncated output)
===== OUT CLASSES =====
v0 has 752 classes, v1 has 1735 classes.
E = 61. R = 52. C = 165.
Classes - Added:  1000, Changed:   278, Deleted:    17.
Methods - Added:  1031, Revised:  1547, Changed:  1590, Renamed:    74, Deleted:   465.
Fields - Added:   480, Changed:   234, Renamed:    24, Deleted:   299.
Added lines:
	- "Landroid/support/v4/app/Fragment;",
    (truncated output)
Removed lines:
	- Ljava/net/UnknownHostException;,
    (truncated output)
Clone this wiki locally