Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

draft: Universal stream join #59

Draft
wants to merge 96 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
96 commits
Select commit Hold shift + click to select a range
5657d94
1. change into time-based window
Mar 2, 2022
ea385fa
1. Add some doxgen template
Mar 2, 2022
979048c
1. Add some doxgen template
Mar 2, 2022
02c2c6c
1. Add some doxgen template
Mar 3, 2022
c3c3fbf
1. Add some doxgen template
Mar 3, 2022
3d89a1a
1. Change the readme.md
Mar 3, 2022
be51da4
1. Change the readme.md
Mar 3, 2022
526643c
1. Change the readme.md
Mar 3, 2022
10620c5
1. Change the readme.md
Mar 3, 2022
30bb924
1. Adjust the code structure into 4 folders
Mar 4, 2022
2956412
1. Adjust the code structure into 5 folders
Mar 9, 2022
c248f04
1. Reconstruct the legacy "JoinMethods" as "TestBench" in benchmark p…
Mar 9, 2022
d038232
1. edit the README.md to redirect to API docs
Mar 9, 2022
38c61b1
1. add the branch for lazy window slider
Mar 10, 2022
0900586
1. add the branch for lazy window slider
Mar 10, 2022
94d7444
issue #29
Mar 10, 2022
54e430f
issue #29:
Mar 15, 2022
512b08a
issue #29:
Mar 15, 2022
96ad018
issue #29:
Mar 15, 2022
c234587
remove the xxx build
Mar 15, 2022
7608d61
Lazy slider (#31)
TONYSKYZENG Mar 16, 2022
e995a7d
issue #29:
Mar 16, 2022
fc1f385
issue #30:
Mar 16, 2022
3ae5cc5
issue #30:
Mar 16, 2022
dd7bf68
issue #30:
Mar 17, 2022
36a41a1
delete commit scripts
Mar 17, 2022
9f38430
issue #30
Mar 17, 2022
ce8008d
Merge branch '#30_CellJoin' into universal_stream_join
Mar 17, 2022
48358ff
issue #30
Mar 17, 2022
8996baf
issue #30
Mar 17, 2022
892a15d
Merge pull request #34 from intellistream/#30_CellJoin
TONYSKYZENG Mar 17, 2022
3476647
issue #30
Mar 17, 2022
4ca9958
issue #35
Mar 17, 2022
cbe2e45
issue #35
Mar 17, 2022
8e9f090
issue #35
Mar 17, 2022
02c9f11
issue #35
Mar 17, 2022
b031275
issue #35
Mar 17, 2022
d2aa83e
Merge pull request #40 from intellistream/#35_verify
TONYSKYZENG Mar 17, 2022
0a085a9
#35 verify (#41)
TONYSKYZENG Mar 18, 2022
3b7cd01
#18 split join (#43)
TONYSKYZENG Mar 21, 2022
26005a5
close #19
Mar 24, 2022
ed67c8d
close #19
Mar 24, 2022
6c023aa
close #19
Mar 24, 2022
16fa3c2
Merge pull request #48 from intellistream/#19_intermediate_results
TONYSKYZENG Mar 24, 2022
7b9e155
#44 lwj right (#50)
TONYSKYZENG Apr 15, 2022
ca848ba
done.
Apr 26, 2022
657aec6
naming convention.
Apr 26, 2022
ef83d9f
Code Refactor.
Apr 26, 2022
9894757
Code Refactor.
Apr 26, 2022
52871aa
Remove log4xx. It is not compatible on Windows.
Aug 3, 2022
11820ce
code clean.
Aug 3, 2022
156d655
Docker file fails to install g++ 11.
Aug 4, 2022
e644180
The docker environment does not reconize standard header such as <fil…
Aug 6, 2022
9faa9ee
1. Creat helpers of docker (#53)
TONYSKYZENG Aug 21, 2022
e27e282
Compiles and runs on Ubuntu 20.04
Aug 24, 2022
4b38913
Rename to AllianceDB.
Aug 28, 2022
aad88bb
Add arguments.
Aug 29, 2022
f2c5c24
Code clean up.
Aug 29, 2022
84af3d7
Code clean up.
Aug 29, 2022
38c0c17
Code clean up.
Aug 30, 2022
1c9f0e9
Code clean up.
Aug 31, 2022
da3851c
feat: add gflags as submodule
wzru Dec 10, 2022
a367b28
feat: Param & Stream init
wzru Dec 11, 2022
3839130
feat: create_symplink instead of copy datasets
wzru Dec 11, 2022
bf3effb
feat: fix cmake.yml
wzru Dec 11, 2022
0511d3d
feat: report Tuples.size()
wzru Dec 11, 2022
3862957
feat: use count-based window
wzru Dec 11, 2022
751f142
feat: VerifyEngine
wzru Dec 16, 2022
e177997
fix: remove Types.cpp
wzru Dec 16, 2022
eba1353
feat: use std::tuple as JoinResult::Tuple
wzru Dec 17, 2022
bfa80f6
feat: use algo instead of engine
wzru Dec 17, 2022
afe1bd0
feat: add SystemeTest/Verify
wzru Dec 17, 2022
3067247
feat: wrong EagerEngine
wzru Dec 23, 2022
38f64f8
chore: add .clang-format
wzru Dec 24, 2022
4a4e22e
fix: add chmod +x to pre-commit
wzru Dec 24, 2022
f27f8d0
fix: .clang-format
wzru Dec 26, 2022
3aa4fd2
chore: mv EagerEngine to LaziestEngine
wzru Dec 29, 2022
1130cfa
feat: remove HashJoin & change API
wzru Dec 30, 2022
36c1f48
feat: wrong Handshake Join
wzru Dec 30, 2022
02f4a49
feat: unfinished Handshake Join
wzru Dec 30, 2022
70d1958
fix(ci): add libboost
wzru Dec 30, 2022
47de57a
ci: add clang-format-check.yml
wzru Jan 10, 2023
84f1951
feat: SplitJoin (#55)
westYY123 Jan 10, 2023
1f5033a
style: fix clang-format-check
wzru Jan 10, 2023
8737f02
feat: engine split windows (#56)
westYY123 Feb 8, 2023
2c0f16b
fix: SplitJoin works (#57)
westYY123 Feb 13, 2023
ba5d312
fix: handshake join works
wzru Feb 14, 2023
16bb914
refactor: Context & Param (#60)
wzru Feb 26, 2023
8faf470
feat: SplitJoinOrigin (#61)
westYY123 Mar 1, 2023
6f92767
feat: print info in benchmark (#62)
wzru Mar 1, 2023
59f694f
quick update
Jun 17, 2023
36840a9
quick update
Jun 17, 2023
e3ea02a
update the verify logic.
Jun 20, 2023
21d7ffc
update the verify logic.
Jun 20, 2023
62931a5
Code clean up.
Sep 5, 2023
66d824a
Code clean up.
Sep 6, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
57 changes: 57 additions & 0 deletions .clang-format
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
BasedOnStyle: Google

# Allow double brackets such as std::vector<std::vector<int>>.
Standard: Cpp11

# Indent 4 spaces at a time.
IndentWidth: 4

# Keep lines under 100 columns long.
ColumnLimit: 100

# Always break before braces
BreakBeforeBraces: Custom
BraceWrapping:
AfterCaseLabel: true
AfterClass: true
AfterControlStatement: true
AfterEnum: true
AfterFunction: true
AfterNamespace: true
AfterStruct: true
AfterUnion: true
BeforeCatch: true
BeforeElse: true
IndentBraces: false
SplitEmptyFunction: false
SplitEmptyRecord: false
SplitEmptyNamespace: false

# Keeps extern "C" blocks unindented.
AfterExternBlock: false

# Indent case labels.
IndentCaseLabels: false

# Right-align pointers and references
PointerAlignment: Right

# ANGLE likes to align things as much as possible.
AlignOperands: true
AlignConsecutiveAssignments: true

# Use 0 space negative offset for access modifiers
AccessModifierOffset: -4

AllowShortCaseLabelsOnASingleLine: true

# Useful for spacing out functions in classes
KeepEmptyLinesAtTheStartOfBlocks: true

# Indent nested PP directives.
IndentPPDirectives: AfterHash

# Include blocks style
IncludeBlocks: Preserve

KeepEmptyLinesAtTheStartOfBlocks: false
22 changes: 0 additions & 22 deletions .github/workflows/ccpp.yml

This file was deleted.

20 changes: 20 additions & 0 deletions .github/workflows/clang-format-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: clang-format Check
on: [pull_request]
jobs:
formatting-check:
name: Formatting Check
runs-on: ubuntu-latest
strategy:
matrix:
path:
- 'include'
- 'src'
- 'benchmark/src'
- 'test'
steps:
- uses: actions/checkout@v3
- name: Run clang-format style check for C/C++/Protobuf programs.
uses: jidicula/[email protected]
with:
clang-format-version: '13'
check-path: ${{ matrix.path }}
41 changes: 41 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: Tests

on:
push:
branches: [ universal_stream_join ]
pull_request:
branches: [ universal_stream_join ]

env:
# Customize the CMake build type here (Release, Debug, RelWithDebInfo, etc.)
BUILD_TYPE: Release

jobs:
build:
# The CMake configure and build commands are platform agnostic and should work equally
# well on Windows or Mac. You can convert this to a matrix build if you need
# cross-platform coverage.
# See: https://docs.github.com/en/free-pro-team@latest/actions/learn-github-actions/managing-complex-workflows#using-a-build-matrix
runs-on: ubuntu-22.04

steps:
- uses: actions/checkout@v3
with:
submodules: recursive
- name: Install g++
run: |
sudo apt update
sudo apt install -y gcc g++ libboost-all-dev
- name: Configure CMake
# Configure CMake in a 'build' subdirectory. `CMAKE_BUILD_TYPE` is only required if you are using a single-configuration generator such as make.
# See https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html?highlight=cmake_build_type
run: cmake -B ${{github.workspace}}/build -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}}

- name: Build
# Build your program with the given configuration
run: cmake --build ${{github.workspace}}/build --config ${{env.BUILD_TYPE}}
- name: Test
working-directory: ${{github.workspace}}/build
# Execute tests defined by the CMake configuration.
# See https://cmake.org/cmake/help/latest/manual/ctest.1.html for more detail
run: ctest -C ${{env.BUILD_TYPE}} --rerun-failed --output-on-failure
9 changes: 7 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,18 @@ sorting.zip

hashing/.idea/*
sorting/.idea/*

doc
*.o

cmake-build-*/

.clion*

venv/

build/
*build/
com_scripts
commit
.idea/
.vscode/
.DS_store
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "third_party/gflags"]
path = third_party/gflags
url = https://github.com/gflags/gflags
107 changes: 107 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# Copyright (C) 2021 by the INTELLI team (https://github.com/intellistream)

#Please INSTALL...
#On Debian/Ubuntu Linux, gflags can be installed using the following command:
#
#sudo apt-get install libgflags-dev
#On macOS, Homebrew includes a formula for gflags:
#
#brew install gflags

cmake_minimum_required(VERSION 3.14)
project(AllianceDB CXX)
# Judge the os system

#gcc options
if (DEFINED ENV{CC})
message(STATUS "you have set CC= " $ENV{CC})
set(CMAKE_C_COMPILER $ENV{CC})
endif ()
if (DEFINED ENV{CXX})
message(STATUS "you have set CXX= " $ENV{CXX})
set(CMAKE_CXX_COMPILER $ENV{CXX})
endif ()

# Custom CMake find instructions and macros
set(CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake;${CMAKE_MODULE_PATH}")
include(cmake/macros.cmake)
include(cmake/default.cmake)
#set(BOOST_REQUESTED_VERSION 1.61)
## on macOS, Use belows to specify Boost path
#SET(CMAKE_INCLUDE_PATH ${CMAKE_INCLUDE_PATH}"/usr/local/boost_1_81_0")
#SET(CMAKE_LIBRARY_PATH ${CMAKE_LIBRARY_PATH}"/usr/local/boost_1_81_0/lib")
#find_package(Boost COMPONENTS program_options system)
#include(cmake/FindBoost.cmake)

message(STATUS ${CMAKE_CXX_COMPILER})
# C++ Standard
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

# Set Optimization Flags
# -Werror=return-type -Werror=unused-variable -Werror=unused-parameter
set(CMAKE_CXX_FLAGS "-std=c++20 -lstdc++fs -Wall -g")
set(CMAKE_CXX_FLAGS_DEBUG "-g -O0 -DNO_RACE_CHECK -DAllianceDB_DEBUG_MODE=1")
set(CMAKE_CXX_FLAGS_RELEASE "-Wno-ignored-qualifiers -Wno-sign-compare -O3")
# Set LOGGING_LEVEL Flag
if (AllianceDB_LOGGING_LEVEL)
get_log_level_value(AllianceDB_LOGGING_VALUE)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -DAllianceDB_LOGGING_LEVEL=${AllianceDB_LOGGING_VALUE}")
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -DAllianceDB_LOGGING_LEVEL=${AllianceDB_LOGGING_VALUE}")
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -DAllianceDB_LOGGING_LEVEL=${AllianceDB_LOGGING_VALUE}")
else (AllianceDB_LOGGING_LEVEL)
message("---Everything will be logged")
endif (AllianceDB_LOGGING_LEVEL)

message(STATUS "CMAKE_CXX_FLAGS ${CMAKE_CXX_FLAGS}")
message(STATUS "CMAKE_CXX_FLAGS_DEBUG ${CMAKE_CXX_FLAGS_DEBUG}")
message(STATUS "CMAKE_CXX_FLAGS_RELEASE ${CMAKE_CXX_FLAGS_RELEASE}")

option(ENABLE_UNIT_TESTS "Enable unit tests" ON)
message(STATUS "Enable testing: ${ENABLE_UNIT_TESTS}")

if (ENABLE_UNIT_TESTS)
enable_testing()
# Google Test
include(FetchContent)

FetchContent_Declare(
googletest
GIT_REPOSITORY https://github.com/google/googletest.git
GIT_TAG release-1.11.0
)
set(gtest_force_shared_crt ON CACHE BOOL "" FORCE)
FetchContent_MakeAvailable(googletest)
include(GoogleTest)
endif ()

# Print all used include directories
message(STATUS "INCLUDE_DIRS:")
get_property(dirs DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} PROPERTY INCLUDE_DIRECTORIES)
foreach (dir ${dirs})
message(STATUS " - ${dir}")
endforeach ()

message(STATUS " Libraries: ${LIBRARIES}")

# Add Source Code
add_subdirectory(src)

# Add Library
get_sources(AllianceDB_SOURCE_FILES)
get_headers(AllianceDB_HEADER_FILES)
add_library(AllianceDB SHARED ${AllianceDB_SOURCE_FILES} ${AllianceDB_HEADER_FILES} ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_CURRENT_BINARY_DIR})
target_include_directories(AllianceDB PUBLIC "include")
target_include_directories(AllianceDB PUBLIC "${CMAKE_CURRENT_BINARY_DIR}")
target_link_libraries(AllianceDB PUBLIC ${LIBRARIES} -lpthread)

#Add benchmarks with command
#Main Entry.
add_subdirectory(benchmark)

#Add tests (based on GTest suits)
if (ENABLE_UNIT_TESTS)
add_subdirectory(test)
endif ()
install(DIRECTORY "include" DESTINATION "/AllianceDB" COMPONENT AllianceDB)

Loading