Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hyperledger Iroha v1.0 #61

Merged
merged 80 commits into from
May 6, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
ed619ba
Restore gitignored files and submodule
Apr 18, 2019
6ea50cb
Yac gate behaves itself when a commit message from the future arrives
Apr 17, 2019
3b31288
Fix asan issue for soci::use
luckychess Apr 17, 2019
4b4501e
Create GitHub issue template
LiraLemur Apr 18, 2019
f915dc7
update issue template
LiraLemur Apr 18, 2019
aaf19a4
Link to video on how to get an LFID
ryjones Apr 19, 2019
8cdc8ff
Fix several tsan issues in tests
luckychess Apr 19, 2019
8a14b58
lock-free behaviour access in FakePeer
MBoldyrev Apr 20, 2019
86347d4
renamed addInitilaPeer -> addFakePeer
MBoldyrev Apr 18, 2019
6f8c270
itf: getThisPeer() method
MBoldyrev Apr 18, 2019
64aef9e
add_peer_test
MBoldyrev Apr 18, 2019
dcf8f50
add_peer_test: RealPeerIsAdded
MBoldyrev Apr 18, 2019
8d3823a
prettify add_peer_test
MBoldyrev Apr 19, 2019
a12fec5
addPeer test extend & prettify
MBoldyrev Apr 22, 2019
3dea19b
Fix batch validator for use in block loader
Apr 23, 2019
dfc4932
Improve batches validation
Apr 24, 2019
fcd4094
Add dbg msg to validation in unsafeCreateBlock
Apr 24, 2019
d21740c
Fix review comments to batch validator
Apr 24, 2019
356610b
refactor Ansible role
bakhtin Apr 22, 2019
8fb2da3
add overlay network parameters
bakhtin Apr 25, 2019
a651e87
Fix consensus fuzzing test
Apr 18, 2019
19badb9
Fix yac transport and deserializer
Apr 18, 2019
d05f863
Switch from Oracle JDK to OpenJRE
bakhtin Apr 29, 2019
7819a5b
refactor iroha::expected::Result
MBoldyrev Apr 12, 2019
60cf440
Bump SOCI version
lebdron Apr 26, 2019
48d7776
Refactor libpq externalproject
lebdron Apr 27, 2019
d347343
Fix getting started docs [skip ci]
LiraLemur Apr 19, 2019
531f587
[ci skip] Fixing the fixing of getting started
LiraLemur Apr 19, 2019
28a2142
[ci skip] adding missing info on KEY parameter
LiraLemur Apr 19, 2019
9c158e2
Fix RST docs souces formatting [skip ci]
Apr 23, 2019
59a8bb0
Update docs/source/getting_started/python-guide.rst
lebdron Apr 24, 2019
b650a89
Apply suggestions from code review
lebdron Apr 24, 2019
edb1674
final fixes
LiraLemur Apr 29, 2019
3b97eac
Add tests fo batch pipeline
Apr 25, 2019
37340ae
Refactor batch pipeline test according to reviews
Apr 30, 2019
00b59f3
Fix mutex initialization issue in ITF
Apr 22, 2019
c1d581b
Add tests for font case sensivity of hex keys
Apr 22, 2019
06a8207
Fix tests descriptions for hex keys test
Apr 30, 2019
d03e6a0
Fix batch validator for use in block loader
Apr 30, 2019
4c11aa2
Fix issue with uncaught exception in logger
luckychess Apr 30, 2019
9975fcb
Catch fmt::v5::format_error exceptions in logger
luckychess Apr 30, 2019
9754ece
Fix log message formats over the whole project
luckychess Apr 30, 2019
95c9725
Add option to load peers from a file on startup
luckychess Apr 16, 2019
1a2a527
Fix review issues
luckychess Apr 23, 2019
c613d99
added missing header
MBoldyrev Apr 28, 2019
adb40f6
pass alternative initial peers as optional
MBoldyrev Apr 24, 2019
1cdeb8d
refactored JSON config loader with a class
MBoldyrev Apr 28, 2019
dc3b7c1
added initial peers to iroha config
MBoldyrev Apr 28, 2019
de84996
Fix build (move partial specializations)
Apr 30, 2019
c74930d
Fix build error
luckychess May 2, 2019
913da0a
Fix itf tests
luckychess May 2, 2019
1c44dc7
Fix review issues
luckychess May 3, 2019
853c17f
Fix tests build
luckychess May 3, 2019
c2c8d22
Add validation for Key and Writer fields for GetAccountDetail
luckychess May 3, 2019
8a2ebdd
Reworked initial peer presence with revering result methods and rewor…
muratovv May 4, 2019
1916a72
Remove vectors from BlockQuery interface
lebdron May 4, 2019
5a0f018
Bump rxcpp version to the latest available
luckychess May 3, 2019
3a0599e
Add functionality of reconnection to storage_impl
muratovv Apr 30, 2019
37d7bb3
* Add test for reconnection strategy
muratovv Apr 30, 2019
b6f9a1e
Add reconnection mechanism without soci reconnections
muratovv May 3, 2019
689bd73
Fix review issues
muratovv May 3, 2019
328c326
Fix doc string of FailoverCallback
muratovv May 4, 2019
8393f12
Fix review issues:
muratovv May 6, 2019
7879bae
Fix clang format
muratovv May 6, 2019
1f9c8c2
Fix markup in ansible docs [skip ci]
May 6, 2019
525fd0c
Fix relative references in docs [skip ci]
May 6, 2019
3b82d30
Add build guide for Windows [skip ci]
May 6, 2019
1367c04
Fix Irohad config parameters documentation [skip ci]
May 6, 2019
063c49a
Add docs for initial_peers config option
luckychess May 6, 2019
bd6fc4b
Change peers list description formulation
luckychess May 6, 2019
47e1d39
Fix initial_peers docs review issues [ci skip]
luckychess May 6, 2019
1e0bf2c
Refactor WsvRestorer to reuse existing block store
lebdron May 4, 2019
85cab25
Move BlockStorageStub to WsvRestorerImpl
lebdron May 6, 2019
fa91f4b
Explicitly inherit createMutableStorage in Storage
lebdron May 6, 2019
a937712
fixed json loader
MBoldyrev May 6, 2019
1a9a85a
Fix old events getting executed in ordering init
lebdron Apr 19, 2019
211e631
Fix processing with the same block round
lebdron Apr 24, 2019
f6fbff5
Use filter-map instead of flat_map
lebdron May 6, 2019
3a442dc
Fix Windows build with MSVC
lebdron May 6, 2019
b953c83
SOCI private API usage workaround
lebdron May 6, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/issue_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
## Hello!

It is great that you've decided to contribute to Hyperledger Iroha by telling us more about the issue you've encountered!
The thing is that we now create and store issues only in [Hyperledger JIRA](https://jira.hyperledger.org/projects/IR) so we would kindly ask you to create your issue there.
That way you will be able to track it in JIRA with your [LFID](https://www.youtube.com/watch?v=EEc4JRyaAoA) account.

Still, if it is more convenient for you to create an issue on GitHub, our Yozhik Bot will carefully transfer it to JIRA and close the issue here.
6 changes: 6 additions & 0 deletions cmake/Modules/Finded25519.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,12 @@ set_target_properties(ed25519 PROPERTIES
IMPORTED_LOCATION ${ed25519_LIBRARY}
)

if (MSVC)
set_target_properties(ed25519 PROPERTIES
INTERFACE_LINK_LIBRARIES bcrypt
)
endif()

if(ENABLE_LIBS_PACKAGING)
add_install_step_for_lib(${ed25519_LIBRARY})
endif()
20 changes: 11 additions & 9 deletions cmake/Modules/Findpq.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -41,22 +41,24 @@ if (NOT pq_FOUND)
GIT_REPOSITORY ${URL}
GIT_TAG ${VERSION}
CONFIGURE_COMMAND
./configure --without-readline
./configure --without-readline --prefix=${EP_PREFIX}
CC=${CMAKE_C_COMPILER}

BUILD_IN_SOURCE 1
BUILD_COMMAND ${MAKE} COPT=${CMAKE_C_FLAGS} -C ./src/bin/pg_config &&
${MAKE} COPT=${CMAKE_C_FLAGS} -C ./src/interfaces/libpq
BUILD_BYPRODUCTS ${EP_PREFIX}/src/postgres_postgres/src/interfaces/libpq/libpq.a
INSTALL_COMMAND "" # remove install step
${MAKE} COPT=${CMAKE_C_FLAGS} -C ./src/interfaces/libpq &&
${MAKE} COPT=${CMAKE_C_FLAGS} -C ./src/backend ../../src/include/utils/fmgroids.h
BUILD_BYPRODUCTS ${EP_PREFIX}/lib/libpq.a
INSTALL_COMMAND make -C src/bin/pg_config install &&
make -C src/interfaces/libpq install &&
make -C src/include install
TEST_COMMAND "" # remove test step
UPDATE_COMMAND "" # remove update step
)
externalproject_get_property(postgres_postgres source_dir)
set(postgres_INCLUDE_DIR ${source_dir}/src/include)
set(pq_INCLUDE_DIR ${source_dir}/src/interfaces/libpq)
set(pq_LIBRARY ${source_dir}/src/interfaces/libpq/libpq.a)
set(pg_config_EXECUTABLE ${source_dir}/src/bin/pg_config/pg_config)
set(postgres_INCLUDE_DIR ${EP_PREFIX}/include)
set(pq_INCLUDE_DIR ${EP_PREFIX}/include)
set(pq_LIBRARY ${EP_PREFIX}/lib/libpq.a)
set(pg_config_EXECUTABLE ${EP_PREFIX}/bin/pg_config)
file(MAKE_DIRECTORY ${pq_INCLUDE_DIR} ${postgres_INCLUDE_DIR})

add_dependencies(pg_config postgres_postgres)
Expand Down
2 changes: 1 addition & 1 deletion cmake/Modules/Findrxcpp.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ find_package_handle_standard_args(rxcpp DEFAULT_MSG


set(URL https://github.com/Reactive-Extensions/rxcpp.git)
set(VERSION 795587fa311fa41050111a830d91b183d8e53ff9)
set(VERSION f38fb8e3c30b01b07c57e08039b2942f1685224d)
set_target_description(rxcpp "Library for reactive programming" ${URL} ${VERSION})


Expand Down
11 changes: 7 additions & 4 deletions cmake/Modules/Findsoci.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,14 @@ mark_as_advanced(SOCI_INCLUDE_DIRS)

find_library(
SOCI_LIBRARY
NAMES soci_core soci_core_3_2
NAMES soci_core soci_core_4_0
HINTS ${SOCI_INCLUDE_DIR}/..
PATH_SUFFIXES lib${LIB_SUFFIX})
mark_as_advanced(SOCI_LIBRARY)

find_library(
SOCI_postgresql_PLUGIN
NAMES soci_postgresql soci_postgresql_3_2
NAMES soci_postgresql soci_postgresql_4_0
HINTS ${SOCI_INCLUDE_DIR}/..
PATH_SUFFIXES lib${LIB_SUFFIX})
mark_as_advanced(SOCI_postgresql_PLUGIN)
Expand All @@ -33,7 +33,7 @@ include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(soci DEFAULT_MSG ${_SOCI_REQUIRED_VARS})

set(URL https://github.com/SOCI/soci)
set(VERSION 111b50af8c3876ea392367640b4bd83b4f903ab8) # 3.2.3
set(VERSION 1f2343e32822549df85660c0eb7acea3f0e5ed8d)
set_target_description(soci "The C++ Database Access Library" ${URL} ${VERSION})

if (NOT soci_FOUND)
Expand All @@ -43,7 +43,7 @@ if (NOT soci_FOUND)
GIT_TAG ${VERSION}
CONFIGURE_COMMAND ${CMAKE_COMMAND}
-G${CMAKE_GENERATOR}
-H${EP_PREFIX}/src/soci_soci/src
-H${EP_PREFIX}/src/soci_soci
-B${EP_PREFIX}/src/soci_soci-build
-DCMAKE_INCLUDE_PATH=${pq_INCLUDE_DIR}
-DCMAKE_LIBRARY_PATH=${pq_INCLUDE_DIR}
Expand All @@ -53,6 +53,9 @@ if (NOT soci_FOUND)
-DCMAKE_C_FLAGS=${CMAKE_C_FLAGS}
-DCMAKE_CXX_FLAGS=${CMAKE_CXX_SOCI_FLAGS}
-DCMAKE_INSTALL_PREFIX=${EP_PREFIX}
-DSOCI_CXX_C11=ON
-DSOCI_TESTS=OFF
-DSOCI_EMPTY=OFF
-DWITH_BOOST=ON
-DWITH_DB2=OFF
-DWITH_FIREBIRD=OFF
Expand Down
196 changes: 196 additions & 0 deletions deploy/ansible/ansible.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,196 @@
# Configuration file for Ansible 2.2
# Documentation: http://docs.ansible.com/ansible/intro_configuration.html

[defaults]


########################################
# Display settings
########################################

# Output display
force_color = 1
nocows = 1


# Note: http://docs.ansible.com/ansible/intro_configuration.html#ansible-managed
ansible_managed = Ansible managed - ansible-skel
#ansible_managed = Ansible managed
#ansible_managed = Ansible managed - {file} on {host}


# Warn when ansible think it is better to use module.
# Note: http://docs.ansible.com/ansible/intro_configuration.html#id88
command_warnings = True

# Enable this to debug tasks calls
display_args_to_stdout = False
display_skipped_hosts = false


########################################
# Playbook settings
########################################


# Default strategy
strategy = free

# Make some actions faster with loops
#squash_actions = apk,apt,dnf,homebrew,package,pacman,pkgng,yum,zypper

# Set it to replace or merge
hash_behaviour = replace
#hash_behaviour = merge, but prefer use combine filter

# Polling
poll_interval = 15
internal_poll_interval=0.001

# Number of hosts processed in parallel
forks = 20

# This shouldn't be touched, but could be useful
error_on_undefined_vars = True
force_handlers = False



########################################
# Behaviour settings
########################################


# Make role variables private
retry_files_enabled = True

# Fact options
gathering = smart
#gathering = !all
#gathering = smart,network,hardware,virtual,ohai,facter
#gathering = network,!hardware,virtual,!ohai,!facter

# Locales
module_set_locale = True
module_lang = en_US.UTF-8

# Enable or disable logs :
# Note put to false in prod
no_log = False

# TODO Deprecated option ?
no_target_syslog = True


########################################
# Common destinations
########################################

inventory = inventory/default.ini
hostfile = inventory/default.ini
roles_path = roles:roles/local
retry_files_save_path = config/tmp/retry/
log_path = config/tmp/ansible.log
#vault_password_file = config/vault_password


########################################
# Plugins paths
########################################

action_plugins = plugins/action:~/.ansible/plugins/action_plugins/:/usr/share/ansible_plugins/action_plugins
callback_plugins = plugins/callback:~/.ansible/plugins/callback_plugins/:/usr/share/ansible_plugins/callback_plugins
connection_plugins = plugins/connection:~/.ansible/plugins/connection_plugins/:/usr/share/ansible_plugins/connection_plugins
filter_plugins = plugins/filter:~/.ansible/plugins/filter_plugins/:/usr/share/ansible_plugins/filter_plugins
lookup_plugins = plugins/lookup:~/.ansible/plugins/lookup_plugins/:/usr/share/ansible_plugins/lookup_plugins
vars_plugins = plugins/vars:~/.ansible/plugins/vars_plugins/:/usr/share/ansible_plugins/vars_plugins
strategy_plugins = plugins/strategy:~/.ansible/plugins/strategy_plugins/:/usr/share/ansible_plugins/strategy_plugins
library = plugins/modules:/usr/share/ansible


########################################
# Escalation settings
########################################

# Escalation
#become = False
#become_method = su
## su, pbrun, pfexec, doas, ksu
#become_user = root
become_ask_pass = True
#become_allow_same_user = False


# Sudo settings
#sudo_exe = sudo
#sudo_flags=-H -S -n
#sudo_user = root


########################################
# SSH settings
########################################

# if your private key has different name,
# you can either change its name here, either set it
# ansible-playbook -i <inventory> --private-key <key name> <playbook name>
private_key_file = ~/.ssh/id_rsa

host_key_checking = False # do not ask for ssh fingerprint
remote_port = 22
remote_user = root

timeout = 5
transport = smart
#transport = jail,ssh,local,chroot,paramiko,...



[ssh_connection]
# Note: when using "sudo:" operations you must first disable ‘requiretty’ in /etc/sudoers on all managed hosts
pipelining = True

# Enable only on old systems
# Note: http://docs.ansible.com/ansible/playbooks_acceleration.html
scp_if_ssh = False

[paramiko_connection]
record_host_keys = True
proxy_command = ssh -W "%h:%p" bastion



########################################
# Accelerate mode
########################################
#[accelerate]
#accelerate_port = 5099
#accelerate_timeout = 30
#accelerate_connect_timeout = 1.0
#accelerate_daemon_timeout = 30
#accelerate_multi_key = yes


########################################
# Various
########################################

[selinux]
show_custom_stats = True



########################################
# Color settings: TODO DEPRECATEDS ?
########################################

[colors]
#verbose = blue
#warn = bright purple
#error = red
#debug = dark gray
#deprecate = purple
#skip = cyan
#unreachable = red
#ok = green
#changed = yellow
1 change: 1 addition & 0 deletions deploy/ansible/kubespray
Submodule kubespray added at 7a2cfb
26 changes: 15 additions & 11 deletions deploy/ansible/roles/iroha-docker/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
##### Description
# Description
[This role](https://github.com/hyperledger/iroha/tree/master/deploy/ansible/roles/iroha-docker) deploys multiple replicas of Iroha containers (one Iroha peer per container) on remote hosts. Each Iroha peer can communicate with others in two ways:
- using public IP addresses or hostnames set in inventory list OR
- using private IP addresses of the Docker overlay network
Expand All @@ -7,11 +7,11 @@ The first one is easier to implement since it does not require preliminary confi

This option is enabled by default.

The second one can be used when there exists an overlay network between the hosts. In short, overlay network allows for Docker containers to communicate using a single subnet. Such that each container would have a unique IP address in that subnet. Learn more in official Docker documentation (https://docs.docker.com/network/overlay). We recommend to use Calico for setting up Docker overlay network since it can be used as a network plugin (https://docs.projectcalico.org/v1.5/getting-started/docker/tutorials/basic).
The second one can be used when there exists an overlay network between the hosts. In short, overlay network allows for Docker containers to communicate using a single subnet. Such that each container would have a unique IP address in that subnet. Learn more in official Docker documentation (https://docs.docker.com/network/overlay). Overlay network can be created if your instance is part of a Swarm cluster. Another method does not involve creating a Swarm cluster but requires a distributed key-value storage. [There is a guide](https://docker-k8s-lab.readthedocs.io/en/latest/docker/docker-etcd.html) on how to create such overlay network.

The second way is also suitable for local-only deployments.
This method is also suitable for local-only deployments and does not require any key-value storage.

##### Requirements
# Requirements
Tested on Ubuntu 16.04, 18.04
- Local:
- python3, python3-dev
Expand All @@ -26,7 +26,7 @@ The second way is also suitable for local-only deployments.
**Note:**
> `docker.io` package from Ubuntu repos will not work. Either use Ansible role or install Docker following official instructions for your OS flavor.

##### Quick Start
# Quick Start
1. Install Ansible
```
pip3 install ansible
Expand All @@ -49,11 +49,11 @@ It will also install Docker along with required python modules. If you want to s
**Note:**
> This command escalates privileges on a remote host during the run. It is required to be able to spin up Docker containers. We recommend to run the playbook using a passwordless remote sudo user.

##### Initial configuration
# Initial configuration

See `defaults/main.yml` file to get more details about available configuration options.

##### Examples
# Examples
**Example 1**
<!-- TODO: Cover more example cases -->
Deploying 6 Iroha peers on two remote hosts communicating using public IP addresses. With 2 and 4 replicas on each host respectively.
Expand All @@ -77,12 +77,12 @@ Deploying 6 Iroha peers on two remote hosts communicating using public IP addres

**192.168.122.109.yml**
```
replicas: 2
iroha_replicas: 2
```

**192.168.122.30.yml**
```
replicas: 4
iroha_replicas: 4
```

4. Run the playbook
Expand All @@ -95,5 +95,9 @@ Deploying 6 Iroha peers on two remote hosts communicating over overlay network (

**TBD**

##### Caveats
1. If `/usr/bin/python` does not exist on a remote host, Ansible will fail with the misleading message: `... Make sure this host can be reached over ssh`. This usually happens when Ansible uses Python 3. On Ubuntu systems `/usr/bin/python3` is not symlinked to `/usr/bin/python` which Ansible expects to find. The problem can be solved by setting `ansible_python_interpreter` variable to `/usr/bin/python3`.
**Caveats**

> If `/usr/bin/python` does not exist on a remote host, Ansible will fail with the misleading message: `... Make sure this host can be reached over ssh`.
This usually happens when Ansible uses Python 3.
On Ubuntu systems `/usr/bin/python3` is not symlinked to `/usr/bin/python` which Ansible expects to find.
The problem can be solved by setting `ansible_python_interpreter` variable to `/usr/bin/python3`.
Loading