Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mixed order operations FMSA #4

Open
wants to merge 71 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
3b1e07c
Function Merging
rcorcs Nov 19, 2020
2f517c3
Handling of attributes and flags
rcorcs Nov 20, 2020
55a1723
added filter for memory requirements vs available memory.
rcorcs Dec 3, 2020
76543ab
added other similarity metrics
rcorcs Dec 14, 2020
3aa495f
work on distance metrics
rcorcs Dec 17, 2020
96a5213
added no-reuse option; nullify operands before their correct assignment.
rcorcs Dec 20, 2020
095e12a
refactored candidates ranking
rcorcs Dec 24, 2020
b200d7b
add other function merging versions; conservative/greedy function mer…
rcorcs Jan 8, 2021
0048693
improved HyFM alignment strategy.
rcorcs Jan 29, 2021
5b0f146
latest HyFM version
rcorcs Mar 12, 2021
9144536
Merged much of Sean's code
ppetoumenos Mar 13, 2021
a145ef5
bug fixes
ppetoumenos Mar 14, 2021
0156524
Templating and fine-tuning
ppetoumenos Mar 14, 2021
5ca421c
probably not important performance fix
ppetoumenos Mar 14, 2021
333b228
LSH vs Fingerprint-based matching abstracted + fixes & optimisation
ppetoumenos Mar 26, 2021
0d784d9
mostly removing warnings
ppetoumenos Mar 26, 2021
bc510e6
minor
ppetoumenos Apr 12, 2021
eee399c
imported Rodrigo's fix for BlockAddress constants
ppetoumenos Apr 12, 2021
8db2ef9
some cleanup and clang-tidy checks
ppetoumenos May 6, 2021
22fcc81
minor out-of-bounds bug
ppetoumenos May 7, 2021
1b6566a
formatting mostly
ppetoumenos May 7, 2021
c71390f
Extra stats and a small performance optimisation
ppetoumenos May 10, 2021
8960386
timer-related fixes
ppetoumenos May 10, 2021
d9a4d0a
removed two stale alignment modes
ppetoumenos May 13, 2021
c7bdb21
Bug fixing and cleanup
ppetoumenos May 17, 2021
06da859
minor fixes
ppetoumenos May 18, 2021
9fe868b
changed the size of OpcodeFreq back to 32-bit. 64-bits have a signifi…
ppetoumenos May 19, 2021
a30f566
minor
ppetoumenos May 21, 2021
405414f
Removal of duplicate bands. Linear scans for small programs. Adaptive…
Seanst98 Jul 15, 2021
552202b
Merge pull request #1 from Seanst98/LSHfm
ppetoumenos Jul 15, 2021
8613029
minor fixes: dangling pointer; correct landingpad predecessor.
rcorcs Jul 15, 2021
f455461
minor changes
rcorcs Jul 15, 2021
9d8e756
Added report for alignment and distances of all function pairs
ppetoumenos Jul 16, 2021
dd7b313
minor bug fixed
ppetoumenos Jul 16, 2021
98dc328
Merge branch 'LSHfm' of https://github.com/ppetoumenos/llvm-project i…
rcorcs Jul 16, 2021
9e330d0
Updated command line arguments; Fix timer output.
rcorcs Jul 20, 2021
7a8009a
Added early return to instToInt
Seanst98 Jul 20, 2021
aecc35e
re-added a debug message
Seanst98 Jul 20, 2021
9927463
enabled label operand reordering
rcorcs Jul 20, 2021
9ccf898
Merge pull request #2 from Seanst98/LSHfm
rcorcs Jul 20, 2021
68712b9
MinHash RNG seeded with a constant now
ppetoumenos Jul 28, 2021
d1a8fe5
fully deterministic HSFM
ppetoumenos Jul 28, 2021
8d072c8
fixed bug when handling functions with inline assembly
ppetoumenos Jul 28, 2021
332f3df
Fixed issue related to inline assembly by mismatching all callbr.
rcorcs Jul 28, 2021
e257b83
Merge branch 'LSHfm' of https://github.com/ppetoumenos/llvm-project i…
rcorcs Jul 28, 2021
4aa01d2
moved fix to matchInstructions.
rcorcs Jul 28, 2021
70a9fb7
Merge pull request #3 from rcorcs/master
ppetoumenos Aug 6, 2021
01f04b0
fixed bugs introduced by the latest upstream fetch
ppetoumenos Aug 6, 2021
6b29025
merged with llvm upstream
ppetoumenos Aug 6, 2021
f5aacf3
Added per function pair timing measurements
ppetoumenos Aug 10, 2021
43ffe32
Added per pair timer for the update. Excluded more functions with pro…
ppetoumenos Aug 18, 2021
6544ed7
more timings stats
ppetoumenos Aug 19, 2021
aa18d7e
New adaptive policy
ppetoumenos Sep 1, 2021
cdd616c
Submitted version
ppetoumenos Sep 4, 2021
2a722ce
Ported to New Pass Manager.
rcorcs Oct 27, 2021
9e7212e
fixed issue with attributes
rcorcs Oct 27, 2021
b061760
Debugging helper options
ppetoumenos Oct 27, 2021
aadce29
fix linkage bug in replaceByCall
rcorcs Oct 27, 2021
522d3cb
Merge branch 'LSHfm' of github.com:ppetoumenos/llvm-project into LSHfm
rcorcs Oct 27, 2021
8a8e591
renaming some parameters
ppetoumenos Oct 28, 2021
08be7d5
minor fix
ppetoumenos Oct 28, 2021
7c8ca1a
couple of bug fixes
ppetoumenos Nov 3, 2021
6598026
manually backported https://reviews.llvm.org/rGc70fa6da9a0c6523f796f8…
ppetoumenos Nov 3, 2021
5f043b7
Updated list of publications
ppetoumenos Nov 8, 2021
62f2870
Some debug options removed
ppetoumenos Nov 11, 2021
e8bd165
Timers not stopped properly when producing a function merging report
ppetoumenos Dec 3, 2021
15fc75b
Timers not stopped properly when producing a function merging report …
ppetoumenos Dec 3, 2021
efbe0e7
Merge remote-tracking branch 'origin/master' into LSHfm
ppetoumenos Sep 19, 2023
9d2adc2
Updated Code
ppetoumenos Sep 19, 2023
8082184
typo
ppetoumenos Sep 20, 2023
5f02034
llvm.dbg.addr was removed
ppetoumenos Sep 21, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
renaming some parameters
  • Loading branch information
ppetoumenos committed Oct 28, 2021

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit 8a8e591f1285d4e66f2b56b8b6f442a6a93d84f0
43 changes: 13 additions & 30 deletions llvm/lib/Transforms/IPO/FunctionMerging.cpp
Original file line number Diff line number Diff line change
@@ -114,8 +114,8 @@
#include <unordered_set>
#include <vector>

#include <cstdlib>
#include <climits>
#include <cstdlib>
#include <ctime>

#ifdef __unix__
@@ -198,16 +198,16 @@ static cl::opt<bool> HyFMProfitability(
"hyfm-profitability", cl::init(true), cl::Hidden,
cl::desc("Try to reuse merged functions for another merge operation"));

static cl::opt<bool> EnableLSH(
"func-merging-lsh", cl::init(false), cl::Hidden,
cl::desc("Enable function pairing based on LSH"));
static cl::opt<bool> EnableF3M(
"func-merging-f3m", cl::init(false), cl::Hidden,
cl::desc("Enable function pairing based on MinHashes and LSH"));

static cl::opt<unsigned> LSHRows(
"hyfm-lsh-rows", cl::init(2), cl::Hidden,
"hyfm-f3m-rows", cl::init(2), cl::Hidden,
cl::desc("Number of rows in the LSH structure"));

static cl::opt<unsigned> LSHBands(
"hyfm-lsh-bands", cl::init(100), cl::Hidden,
"hyfm-f3m-bands", cl::init(100), cl::Hidden,
cl::desc("Number of bands in the LSH structure"));

static cl::opt<bool> ShingleCrossBBs(
@@ -1988,7 +1988,7 @@ template <class T, template<typename> class FPTy = Fingerprint> class MatcherFQ
CountCandidates++;
}
if (best_match.candidate != nullptr)
if (!EnableLSH || best_match.Distance < RankingDistance)
if (!EnableF3M || best_match.Distance < RankingDistance)
/*if (EnableThunkPrediction)
{
if (std::max(best_match.size, best_match.OtherSize) + EstimateThunkOverhead(it->candidate, best_match->candidate)) // Needs AlwaysPreserved
@@ -2010,7 +2010,7 @@ template <class T, template<typename> class FPTy = Fingerprint> class MatcherFQ
new_match.OtherSize = it->size;
new_match.OtherMagnitude = it->FP.magnitude;
new_match.Magnitude = entry.FP.magnitude;
if (!EnableLSH || new_match.Distance < RankingDistance)
if (!EnableF3M || new_match.Distance < RankingDistance)
matches.push_back(std::move(new_match));
if (RankingThreshold && (CountCandidates > RankingThreshold))
break;
@@ -2055,7 +2055,6 @@ template <class T> class MatcherLSH : public Matcher<T> {
SearchStrategy strategy;

std::list<MatcherEntry> candidates;
//tsl::robin_map<uint32_t, std::vector<MatcherIt>> lsh;
std::unordered_map<uint32_t, std::vector<MatcherIt>> lsh;
std::vector<std::pair<T, MatcherIt>> cache;
std::vector<MatchInfo<T>> matches;
@@ -3604,14 +3603,10 @@ bool FunctionMerging::runImpl(
size++;
}

//bool linearScan = size < 100 ? true : false;
bool linearScan = false;

// Create a threshold based on the application's size
if (AdaptiveThreshold || AdaptiveBands)
{
double x = std::log10(size) / 10;
//RankingDistance = (double) (1.0 / (2.0 + std::pow(x/(1.0-x),-3.0))) + 0.125;
RankingDistance = (double) (x - 0.3);
if (RankingDistance < 0.05)
RankingDistance = 0.05;
@@ -3622,16 +3617,6 @@ bool FunctionMerging::runImpl(
float target_probability = 0.9;
float offset = 0.1;
unsigned tempBands = std::ceil(std::log(1.0 - target_probability) / std::log(1.0 - std::pow(RankingDistance + offset, LSHRows)));
//size_t fingerprint_size = LSHRows * LSHBands;
//if (RankingDistance < 0.15)
// LSHRows = 1;
//else if (RankingDistance >= 0.15 && RankingDistance < 0.4)
// LSHRows = 2;
//else if (RankingDistance >= 0.4 && RankingDistance < 0.5)
// LSHRows = 4;
//else
// LSHRows = 5;
//LSHBands = fingerprint_size / LSHRows;
if (tempBands < LSHBands)
LSHBands = tempBands;

@@ -3647,12 +3632,10 @@ bool FunctionMerging::runImpl(
errs() << "LSHRows: " << LSHRows << "\n";
errs() << "LSHBands: " << LSHBands << "\n";

if (EnableLSH && !linearScan){
matcher = std::make_unique<MatcherLSH<Function *>>(FM, Options, LSHRows, LSHBands); errs() << "LSH MH\n";}
else if (EnableLSH && linearScan){
matcher = std::make_unique<MatcherFQ<Function *, FingerprintMH>>(FM, Options); errs() << "LIN SCAN MH\n";}
else{
matcher = std::make_unique<MatcherFQ<Function *>>(FM, Options); errs() << "LIN SCAN FP\n";}
if (EnableF3M)
matcher = std::make_unique<MatcherLSH<Function *>>(FM, Options, LSHRows, LSHBands); errs() << "LSH MH\n";
else
matcher = std::make_unique<MatcherFQ<Function *>>(FM, Options); errs() << "LIN SCAN FP\n";

std::unordered_set<std::string> AllowedSet(AllowedFunctionsList.begin(), AllowedFunctionsList.end());
std::unordered_set<std::string> ExcludedSet(ExcludedFunctionsList.begin(), ExcludedFunctionsList.end());
@@ -3744,7 +3727,7 @@ bool FunctionMerging::runImpl(
std::string F2Name(GetValueName(F2));

if (Verbose) {
if (EnableLSH) {
if (EnableF3M) {
Fingerprint<Function *> FP1(F1);
Fingerprint<Function *> FP2(F2);
OtherDistance = FP1.distance(FP2);