Pli manager integration #17

jakob-zwiener · 2015-09-14T08:53:47Z

No description provided.

slightly slower than sequential, not two phase yet

Conflicts: algorithm_helper_benchmarks/src/main/java/de/metanome/algorithm_helper/data_structures/benchmarks/PLIBenchmarkRunner.java

Conflicts: algorithm_helper/src/main/java/de/metanome/algorithm_helper/data_structures/IntPair.java algorithm_helper/src/main/java/de/metanome/algorithm_helper/data_structures/PLIBuilder.java algorithm_helper/src/main/java/de/metanome/algorithm_helper/data_structures/PositionListIndex.java algorithm_helper/src/test/java/de/metanome/algorithm_helper/data_structures/LongPairTest.java algorithm_helper/src/test/java/de/metanome/algorithm_helper/data_structures/PLIBuilderFixture.java algorithm_helper/src/test/java/de/metanome/algorithm_helper/data_structures/PositionListIndexFixture.java algorithm_helper/src/test/java/de/metanome/algorithm_helper/data_structures/PositionListIndexTest.java algorithm_helper_benchmarks/src/test/java/de/metanome/algorithm_helper/data_structures/benchmarks/PLIGeneratorTest.java

Conflicts: algorithm_helper/src/main/java/de/metanome/algorithm_helper/data_structures/PositionListIndex.java

Conflicts: algorithm_helper/src/main/java/de/metanome/algorithm_helper/data_structures/PositionListIndex.java algorithm_helper/src/test/java/de/metanome/algorithm_helper/data_structures/PositionListIndexFixture.java algorithm_helper/src/test/java/de/metanome/algorithm_helper/data_structures/PositionListIndexTest.java

Conflicts: algorithm_helper/src/main/java/de/metanome/algorithm_helper/data_structures/PositionListIndex.java

taking 1 subset with lowest key error

Conflicts: algorithm_helper/src/main/java/de/metanome/algorithm_helper/data_structures/PLIManager.java algorithm_helper/src/test/java/de/metanome/algorithm_helper/data_structures/PLIManagerTest.java

(cherry picked from commit 7e9b12a)

Conflicts: algorithm_helper/src/main/java/de/metanome/algorithm_helper/data_structures/PositionListIndex.java algorithm_helper/src/test/java/de/metanome/algorithm_helper/data_structures/PositionListIndexFixture.java algorithm_helper/src/test/java/de/metanome/algorithm_helper/data_structures/PositionListIndexTest.java algorithm_helper_benchmarks/src/main/java/de/metanome/algorithm_helper/data_structures/benchmarks/PLISyntheticBenchmark.java

…tegration Conflicts: algorithm_helper/src/main/java/de/metanome/algorithm_helper/data_structures/PositionListIndex.java algorithm_helper/src/test/java/de/metanome/algorithm_helper/data_structures/PositionListIndexTest.java

fixed a performance glitch

jakob-zwiener · 2015-11-11T13:03:50Z

algorithm_helper/src/main/java/de/metanome/algorithm_helper/data_structures/PLIManager.java

+    pliSetTrie.add(columnCombination);
+
+    List<ColumnCombinationBitset> supersets = pliSetTrie.getExistingSupersets(columnCombination);
+


jakob-zwiener added 30 commits June 10, 2015 14:00

first spike for parallel map

25f6056

slightly slower than sequential, not two phase yet

fixde typoe

90f6099

Merge branch 'master' into parallel_map

8fdcaf2

wip parallelise updateMap

69bed5c

Merge branch 'master' into parallel_map

4a4fb6f

parallelised interim version (broken functionality)

50ba168

added PLISyntheticBenchmark.java

1af6ef6

pli intersect functional

5399e95

small syso and comment cleanups in PositionListIndex.java

be3343a

Merge branch 'master' into parallel_map

69c316f

Conflicts: algorithm_helper_benchmarks/src/main/java/de/metanome/algorithm_helper/data_structures/benchmarks/PLIBenchmarkRunner.java

moved PLISyntheticBenchmark.java to the correct package

5dc3c70

reformat

16ebe90

wip PLI materialization

02194de

Merge branch 'master' into parallel_materialization

4fb94ed

Conflicts: algorithm_helper/src/main/java/de/metanome/algorithm_helper/data_structures/PositionListIndex.java

Merge branch 'master' into parallel_materialization

02d29e1

renamed asList test

88cdd90

parallel materialization functional

81d58a1

improvements to the synthetic benchmarks

44a7cdf

added synthetic benchmarks (number of threads set to 1)

772d0b3

Merge branch 'benchmarks' into parallel_materialization

87fdf5e

increasing number of threads

a9344a8

fixed parallel updateMap

2662a50

Merge branch 'parallel_materialization' into parallel_intersect

c32f275

Conflicts: algorithm_helper/src/main/java/de/metanome/algorithm_helper/data_structures/PositionListIndex.java

added thread parameter to syso and file output

20551b7

added thread parameter to syso and file output

0dc43e6

implemented most basic functionality of the pli manager

65f323a

first subset implementation

fd97cd6

taking 1 subset with lowest key error

jakob-zwiener force-pushed the pli-manager-integration branch from 0ae6dde to 3cbcc0e Compare September 25, 2015 11:53

jakob-zwiener added 28 commits September 26, 2015 09:19

Merge branch 'lru-pli-cache' into pli-manager-integration

cca4395

Conflicts: algorithm_helper/src/main/java/de/metanome/algorithm_helper/data_structures/PLIManager.java algorithm_helper/src/test/java/de/metanome/algorithm_helper/data_structures/PLIManagerTest.java

integrating lru cache into pli manager

b502adc

Exposed cache size setting

840e173

Improved class doc string

9f6f51f

removed fulfilled todo

6f111ab

tested that the required PLI is added to the cache

3cc5e84

simplified getPli (multi) logic

15fa26e

Merge branch 'master' into pli-manager-integration

231410e

implemented traversal direction aware superset evitiction

b135a71

(cherry picked from commit 7e9b12a)

tested that existing PLIs are correctly added to the manager

7d32372

added docs

6facf2a

removed done todo

5812b3e

removed expansion parallelisation

17350d1

parallel expansion base

e9fc1ec

parallel map base

1e0572f

parallel map base

61c52ef

parallel map comparable to parallel expansion implementation

8bfa1cb

implemented grouped parallel update map

85d197a

implemented cutoff and avoided some rehashing

10d683d

implemented parallel intersect

92e6ca6

deactivated cutoff

d21e6b3

added cutoff, fewer threads

cb9d6a7

callable for probing

44eda40

merged new intra parallelisation

e939e57

buildPLI handles empty column combination requests

3ac44fc

fixed a performance glitch

made buildPli thread safe

3dafe26

jakob-zwiener reviewed Nov 11, 2015
View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pli manager integration #17

Pli manager integration #17

jakob-zwiener commented Sep 14, 2015

jakob-zwiener Nov 11, 2015

		pliSetTrie.add(columnCombination);

		List<ColumnCombinationBitset> supersets = pliSetTrie.getExistingSupersets(columnCombination);

Pli manager integration #17

Are you sure you want to change the base?

Pli manager integration #17

Conversation

jakob-zwiener commented Sep 14, 2015

jakob-zwiener Nov 11, 2015

Choose a reason for hiding this comment