Ref !3/peerplays-network#376: Graphene Updates

This adds the most important updates to Graphene from BitShares. Most notably,
bitshares/bitshares-core#1506

Second most notably, it updates Peerplays' FC to be in sync with BitShares FC.

This is a squash commit of several subcommits. The subcommit messages are
reproduced below:

Replace fc::uint128 with boost::multiprecision::uint128_t

replace smart_ref with shared_ptr

Fixes/Remove Unused

Remove NTP time

Remove old macro

This macro is now in FC, so no need to define it here anymore

Replaced fc::array with std::array

Separate exception declaration and implementation

Adapted to fc promise changes

Fixes

Add back in some of Peter's fixes that got lost in the cherry pick

_hash endianness fixes

Remove all uses of fc/smart_ref

It's gone, can't use it anymore

Replace improper static_variant operator overloads with comparators

Fixes

Remove boost::signals from build system; it's header-only so it's not
listed in cmake anymore.

Also remove some unused hashing code

Impl. pack/unpack functions for extension class

Ref #1506: Isolate chain/protocol to its own library

Ref #1506: Add object_downcast_t

Allows the more concise expression `object_downcast_t<xyz>` instead of
the old `typename object_downcast<xyz>::type`

Ref #1506: Move ID types from db to protocol

The ID types, object_id and object_id_type, were defined in the db
library, and the protocol library depends on db to get these types.
Technically, the ID types are defined by the protocol and used by the
database, and not vice versa. Therefore these types should be in the
protocol library, and db should depend on protocol to get them.

This commit makes it so.

Ref #1506: Isolate chain/protocol to its own library

Remove commented-out index code

Wrap overlength line

Remove unused key types

Probably fix Docker build

Fix build after rebase

Ref #1506/#1737: Some requested changes

Ref #1506/#1737: Macro-fy ID type definitions

Define macros to fully de-boilerplate ID type definitions.

Externalities:
 - Rename transaction_object -> transaction_history_object
 - Rename impl_asset_dynamic_data_type ->
impl_asset_dynamic_data_object_type
 - Rename impl_asset_bitasset_data_type ->
impl_asset_bitasset_data_object_type

The first is to avoid a naming collision on transaction_id_type, and the
other two are to maintain consistency with the naming of the other
types.

Ref #1506/#1737: Fix clean_name()

Ref #1506/#1737: Oops

Fix .gitignore

Externalized serialization in protocol library

Fix compile sets

Delete a couple of ghost files that were in the tree but not part
of the project (I accidentally added them to CMakeLists while
merging, but they're broken and not part of the Peerplays code), and
add several files that got dropped from the build during merge.

General fixes

Fix warnings, build issues, unused code, etc.

Fix #1772 by decprecating cli_wallet -H

More fixes

Fix errors and warnings and generally coax it to build

Fix test

I'm pretty sure this didn't break from what I did... But I can't build
the original code, so I can't tell. Anyways, this one now passes...
Others still fail...

Small fix

Fix crash in auth checks

Final fixes

Last round of fixes following the rebase to Beatrice

Rename project in CMakeLists.txt

The CMakeLists.txt declared this project as BitShares and not Peerplays,
which makes it confusing in IDEs. Rename it to be clear which project is
open.

Resolve peerplays-network#374

Replace all object refs in macros with IDs, and fix affected tests to look
up objects by ID rather than using invalidated refs.

A full audit of all tests should be performed to eliminate any further
usage of invalidated object references.

Resolve peerplays-network#373: Add object notifiers

Various fixes

Fixes to various issues, primarily reflections, that cropped up
during merge conflict resolution

Fix startup bug in Bookie plugin

Bookie plugin was preventing the ndoe from starting up because it
registered its secondary indexes to create objects in its own primary
indexes to track objects being created in other primary indexes, and did
so during its `initialize()` step, which is to say, before the database
was loaded from disk at startup. This caused the secondary indexes to
create tracker objects when the observed indexes were loading objects
from disk. This then caused a failure when these tracker indexes were
later loaded from disk, and the first object IDs collided.

This is fixed by refraining from defining secondary indexes until the
`startup()` stage rather than the `initialize()` stage. Primary indexes
are registered in `initialize()`, secondary indexes are registered in
`startup()`.

I have no idea how this was working before I got here...

Fix egenesis install

Fixes after updates

Rebase on updated develop branch and fix conflicts

.gitmodules

@@ -4,6 +4,6 @@
     ignore = dirty
 [submodule "libraries/fc"]
     path = libraries/fc
-    url = https://github.com/peerplays-network/peerplays-fc.git
+    url = https://github.com/nathanhourt/peerplays-fc
     branch = latest-fc
     ignore = dirty

CMakeLists.txt

@@ -1,5 +1,5 @@
-# Defines BitShares library target.
-project( BitShares )
+# Defines Peerplays library target.
+project( Peerplays )
 cmake_minimum_required( VERSION 2.8.12 )
 
 set( BLOCKCHAIN_NAME "BitShares" )
@@ -8,6 +8,7 @@ set( CLI_CLIENT_EXECUTABLE_NAME graphene_client )
 set( GUI_CLIENT_EXECUTABLE_NAME BitShares )
 set( CUSTOM_URL_SCHEME "gcs" )
 set( INSTALLER_APP_ID "68ad7005-8eee-49c9-95ce-9eed97e5b347" )
+set( CMAKE_CXX_STANDARD 14 )
 
 # http://stackoverflow.com/a/18369825
 if("${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU")
@@ -46,7 +47,6 @@ LIST(APPEND BOOST_COMPONENTS thread
                              system
                              filesystem
                              program_options
-                             signals
                              serialization
                              chrono
                              unit_test_framework
@@ -113,11 +113,11 @@ else( WIN32 ) # Apple AND Linux
     if( APPLE )
         # Apple Specific Options Here
         message( STATUS "Configuring BitShares on OS X" )
-        set( CMAKE_CXX_FLAGS "${CMAKE_C_FLAGS} -std=c++11 -stdlib=libc++ -Wall" )
+        set( CMAKE_CXX_FLAGS "${CMAKE_C_FLAGS} -stdlib=libc++ -Wall" )
     else( APPLE )
         # Linux Specific Options Here
         message( STATUS "Configuring BitShares on Linux" )
-        set( CMAKE_CXX_FLAGS "${CMAKE_C_FLAGS} -std=c++11 -Wall" )
+        set( CMAKE_CXX_FLAGS "${CMAKE_C_FLAGS} -Wall" )
         set( rt_library rt )
         #set( pthread_library pthread)
         set(CMAKE_LINKER_FLAGS "-pthread" CACHE STRING "Linker Flags" FORCE)

libraries/CMakeLists.txt

@@ -4,9 +4,8 @@ add_subdirectory( db )
 add_subdirectory( chain )
 add_subdirectory( egenesis )
 add_subdirectory( net )
-#add_subdirectory( p2p )
-add_subdirectory( time )
 add_subdirectory( utilities )
 add_subdirectory( app )
 add_subdirectory( plugins )
 add_subdirectory( wallet )
+add_subdirectory( protocol )

libraries/app/CMakeLists.txt

@@ -12,8 +12,7 @@ add_library( graphene_app
            )
 
 # need to link graphene_debug_witness because plugins aren't sufficiently isolated #246
-#target_link_libraries( graphene_app graphene_market_history graphene_account_history graphene_chain fc graphene_db graphene_net graphene_utilities graphene_debug_witness )
-target_link_libraries( graphene_app graphene_market_history graphene_account_history graphene_accounts_list graphene_affiliate_stats graphene_chain fc graphene_db graphene_net graphene_time graphene_utilities graphene_debug_witness graphene_bookie graphene_elasticsearch peerplays_sidechain )
+target_link_libraries( graphene_app graphene_market_history graphene_account_history graphene_accounts_list graphene_affiliate_stats graphene_chain fc graphene_db graphene_net graphene_utilities graphene_debug_witness graphene_bookie graphene_elasticsearch peerplays_sidechain )
 target_include_directories( graphene_app
                             PUBLIC "${CMAKE_CURRENT_SOURCE_DIR}/include"
                             "${CMAKE_CURRENT_SOURCE_DIR}/../egenesis/include" )

libraries/app/api.cpp

@@ -29,16 +29,16 @@
 #include <graphene/chain/database.hpp>
 #include <graphene/chain/get_config.hpp>
 #include <graphene/utilities/key_conversion.hpp>
-#include <graphene/chain/protocol/fee_schedule.hpp>
+#include <graphene/protocol/fee_schedule.hpp>
 #include <graphene/chain/confidential_object.hpp>
 #include <graphene/chain/market_object.hpp>
-#include <graphene/chain/transaction_object.hpp>
+#include <graphene/chain/transaction_history_object.hpp>
 #include <graphene/chain/withdraw_permission_object.hpp>
 #include <graphene/chain/worker_object.hpp>
 #include <graphene/chain/tournament_object.hpp>
 
+#include <fc/crypto/base64.hpp>
 #include <fc/crypto/hex.hpp>
-#include <fc/smart_ref_impl.hpp>
 #include <fc/rpc/api_connection.hpp>
 #include <fc/thread/future.hpp>
 
@@ -170,7 +170,7 @@ namespace graphene { namespace app {
              {
                 auto block_num = b.block_num();
                 auto& callback = _callbacks.find(id)->second;
-                fc::async( [capture_this,this,id,block_num,trx_num,trx,callback]() {
+                fc::async( [capture_this,id,block_num,trx_num,trx,callback]() {
                    callback( fc::variant( transaction_confirmation{ id, block_num, trx_num, trx },
                                           GRAPHENE_MAX_NESTED_OBJECTS ) );
                 } );
@@ -192,8 +192,8 @@ namespace graphene { namespace app {
     {
        _app.chain_database()->check_tansaction_for_duplicated_operations(trx);
 
-       fc::promise<fc::variant>::ptr prom( new fc::promise<fc::variant>() );
-       broadcast_transaction_with_callback( [=]( const fc::variant& v ){
+       fc::promise<fc::variant>::ptr prom = fc::promise<fc::variant>::create();
+       broadcast_transaction_with_callback( [prom]( const fc::variant& v ){
         prom->set_value(v);
        }, trx );
 

libraries/app/application.cpp

@@ -26,8 +26,10 @@
 #include <graphene/app/application.hpp>
 #include <graphene/app/plugin.hpp>
 
-#include <graphene/chain/protocol/fee_schedule.hpp>
-#include <graphene/chain/protocol/types.hpp>
+#include <graphene/chain/db_with.hpp>
+#include <graphene/chain/genesis_state.hpp>
+#include <graphene/protocol/fee_schedule.hpp>
+#include <graphene/protocol/types.hpp>
 
 #include <graphene/egenesis/egenesis.hpp>
 
@@ -37,8 +39,6 @@
 #include <graphene/utilities/key_conversion.hpp>
 #include <graphene/chain/worker_evaluator.hpp>
 
-#include <fc/smart_ref_impl.hpp>
-
 #include <fc/io/fstream.hpp>
 #include <fc/rpc/api_connection.hpp>
 #include <fc/rpc/websocket_api.hpp>
@@ -80,7 +80,7 @@ namespace detail {
       auto nathan_key = fc::ecc::private_key::regenerate(fc::sha256::hash(string("nathan")));
       dlog("Allocating all stake to ${key}", ("key", utilities::key_to_wif(nathan_key)));
       genesis_state_type initial_state;
-      initial_state.initial_parameters.current_fees = fee_schedule::get_default();//->set_all_fees(GRAPHENE_BLOCKCHAIN_PRECISION);
+      initial_state.initial_parameters.current_fees = std::make_shared<fee_schedule>(fee_schedule::get_default());
       initial_state.initial_active_witnesses = GRAPHENE_DEFAULT_MIN_WITNESS_COUNT;
       initial_state.initial_timestamp = time_point_sec(time_point::now().sec_since_epoch() /
             initial_state.initial_parameters.block_interval *
@@ -380,7 +380,6 @@ namespace detail {
             _chain_db->enable_standby_votes_tracking( _options->at("enable-standby-votes-tracking").as<bool>() );
          }
 
-         bool replay = false;
          std::string replay_reason = "reason not provided";
 
          if( _options->count("replay-blockchain") )
@@ -554,7 +553,7 @@ namespace detail {
          _chain_db->push_transaction( transaction_message.trx );
       } FC_CAPTURE_AND_RETHROW( (transaction_message) ) }
 
-      virtual void handle_message(const message& message_to_process) override
+      virtual void handle_message(const message&) override
       {
          // not a transaction, not a block
          FC_THROW( "Invalid Message Type" );

libraries/app/config_util.cpp

@@ -210,7 +210,7 @@ static void load_config_file(const fc::path& config_ini_path, const bpo::options
 {
    deduplicator dedup;
    bpo::options_description unique_options("Graphene Witness Node");
-   for( const boost::shared_ptr<bpo::option_description> opt : cfg_options.options() )
+   for( const boost::shared_ptr<bpo::option_description>& opt : cfg_options.options() )
    {
       const boost::shared_ptr<bpo::option_description> od = dedup.next(opt);
       if( !od ) continue;
@@ -259,7 +259,7 @@ static void create_new_config_file(const fc::path& config_ini_path, const fc::pa
    deduplicator dedup(modify_option_defaults);
    std::ofstream out_cfg(config_ini_path.preferred_string());
    std::string plugin_header_surrounding( 78, '=' );
-   for( const boost::shared_ptr<bpo::option_description> opt : cfg_options.options() )
+   for( const boost::shared_ptr<bpo::option_description>& opt : cfg_options.options() )
    {
       const boost::shared_ptr<bpo::option_description> od = dedup.next(opt);
       if( !od ) continue;

libraries/app/database_api.cpp

@@ -26,19 +26,19 @@
 #include <graphene/chain/get_config.hpp>
 #include <graphene/chain/tournament_object.hpp>
 #include <graphene/chain/account_object.hpp>
-#include <graphene/chain/protocol/address.hpp>
-#include <graphene/chain/pts_address.hpp>
+#include <graphene/chain/hardfork.hpp>
+
+#include <graphene/protocol/address.hpp>
+#include <graphene/protocol/pts_address.hpp>
 
 #include <fc/bloom_filter.hpp>
-#include <fc/smart_ref_impl.hpp>
 
 #include <fc/crypto/hex.hpp>
 #include <fc/rpc/api_connection.hpp>
 #include <fc/uint128.hpp>
 
 #include <boost/range/iterator_range.hpp>
 #include <boost/rational.hpp>
-#include <boost/multiprecision/cpp_int.hpp>
 
 #include <cctype>
 
@@ -2571,10 +2571,9 @@ graphene::app::gpos_info database_api_impl::get_gpos_info(const account_id_type
 #endif
 
    vector<vesting_balance_object> account_vbos;
-   const time_point_sec now = _db.head_block_time();
    auto vesting_range = _db.get_index_type<vesting_balance_index>().indices().get<by_account>().equal_range(account);
    std::for_each(vesting_range.first, vesting_range.second,
-                  [&account_vbos, now](const vesting_balance_object& balance) {
+                  [&account_vbos](const vesting_balance_object& balance) {
                      if(balance.balance.amount > 0 && balance.balance_type == vesting_balance_type::gpos
                         && balance.balance.asset_id == asset_id_type())
                         account_vbos.emplace_back(balance);

libraries/app/include/graphene/app/api.hpp

@@ -25,8 +25,8 @@
 
 #include <graphene/app/database_api.hpp>
 
-#include <graphene/chain/protocol/types.hpp>
-#include <graphene/chain/protocol/confidential.hpp>
+#include <graphene/protocol/types.hpp>
+#include <graphene/protocol/confidential.hpp>
 
 #include <graphene/market_history/market_history_plugin.hpp>
 #include <graphene/accounts_list/accounts_list_plugin.hpp>

libraries/app/include/graphene/app/database_api.hpp

@@ -25,7 +25,7 @@
 
 #include <graphene/app/full_account.hpp>
 
-#include <graphene/chain/protocol/types.hpp>
+#include <graphene/protocol/types.hpp>
 
 #include <graphene/chain/database.hpp>
 

libraries/app/include/graphene/app/full_account.hpp

@@ -26,6 +26,8 @@
 #include <graphene/chain/account_object.hpp>
 #include <graphene/chain/vesting_balance_object.hpp>
 #include <graphene/chain/market_evaluator.hpp>
+#include <graphene/chain/market_object.hpp>
+#include <graphene/chain/proposal_object.hpp>
 #include <graphene/chain/withdraw_permission_object.hpp>
 
 namespace graphene { namespace app {
@@ -67,7 +69,6 @@ FC_REFLECT( graphene::app::full_account,
             (limit_orders)
             (call_orders)
             (settle_orders)
-            (proposals)
             (assets)
             (withdraws)
             (proposals) 

libraries/app/plugin.cpp

@@ -23,7 +23,7 @@
  */
 
 #include <graphene/app/plugin.hpp>
-#include <graphene/chain/protocol/fee_schedule.hpp>
+#include <graphene/protocol/fee_schedule.hpp>
 
 namespace graphene { namespace app {
 

libraries/chain/CMakeLists.txt

@@ -5,8 +5,7 @@ set_source_files_properties( "${CMAKE_CURRENT_BINARY_DIR}/include/graphene/chain
 
 add_dependencies( build_hardfork_hpp cat-parts )
 
-file(GLOB HEADERS "include/graphene/chain/*.hpp")
-file(GLOB PROTOCOL_HEADERS "include/graphene/chain/protocol/*.hpp")
+file(GLOB HEADERS "include/graphene/chain/*.hpp" "${CMAKE_CURRENT_BINARY_DIR}/include/graphene/chain/hardfork.hpp")
 
 if( GRAPHENE_DISABLE_UNITY_BUILD )
    set( GRAPHENE_DB_FILES
@@ -36,39 +35,9 @@ add_library( graphene_chain
              ${GRAPHENE_DB_FILES}
              fork_database.cpp
 
-             protocol/types.cpp
-             protocol/address.cpp
-             protocol/authority.cpp
-             protocol/asset.cpp
-             protocol/assert.cpp
-             protocol/account.cpp
-             protocol/transfer.cpp
-             protocol/committee_member.cpp
-             protocol/witness.cpp
-             protocol/market.cpp
-             protocol/proposal.cpp
-             protocol/withdraw_permission.cpp
-             protocol/asset_ops.cpp
-             protocol/lottery_ops.cpp
-             protocol/memo.cpp
-             protocol/worker.cpp
-             protocol/custom.cpp
-             protocol/operations.cpp
-             protocol/transaction.cpp
-             protocol/block.cpp
-             protocol/fee_schedule.cpp
-             protocol/confidential.cpp
-             protocol/vote.cpp
-             protocol/tournament.cpp
-             protocol/small_ops.cpp
-             protocol/custom_permission.cpp
-             protocol/custom_account_authority.cpp
-             protocol/offer.cpp
-
              genesis_state.cpp
              get_config.cpp
-
-             pts_address.cpp
+             exceptions.cpp
 
              evaluator.cpp
              balance_evaluator.cpp
@@ -89,8 +58,9 @@ add_library( graphene_chain
              withdraw_permission_evaluator.cpp
              worker_evaluator.cpp
              confidential_evaluator.cpp
-             special_authority.cpp
+             special_authority_evaluation.cpp
              buyback.cpp
+             account_role_evaluator.cpp
 
              account_object.cpp
              asset_object.cpp
@@ -103,15 +73,11 @@ add_library( graphene_chain
 
              is_authorized_asset.cpp
 
-             protocol/sport.cpp
              sport_evaluator.cpp
-             protocol/event_group.cpp
              event_group_evaluator.cpp
              event_group_object.cpp
-             protocol/event.cpp
              event_evaluator.cpp
              event_object.cpp
-             protocol/betting_market.cpp
              betting_market_evaluator.cpp
              betting_market_object.cpp
              betting_market_group_object.cpp
@@ -123,9 +89,6 @@ add_library( graphene_chain
              offer_object.cpp
              offer_evaluator.cpp
              nft_evaluator.cpp
-             protocol/nft.cpp
-             protocol/account_role.cpp
-             account_role_evaluator.cpp
 
              son_evaluator.cpp
              son_object.cpp
@@ -138,12 +101,11 @@ add_library( graphene_chain
              sidechain_transaction_evaluator.cpp
 
              ${HEADERS}
-             ${PROTOCOL_HEADERS}
              "${CMAKE_CURRENT_BINARY_DIR}/include/graphene/chain/hardfork.hpp"
            )
 
 add_dependencies( graphene_chain build_hardfork_hpp )
-target_link_libraries( graphene_chain fc graphene_db )
+target_link_libraries( graphene_chain fc graphene_db graphene_protocol )
 target_include_directories( graphene_chain
                             PUBLIC "${CMAKE_CURRENT_SOURCE_DIR}/include" "${CMAKE_CURRENT_BINARY_DIR}/include" )
 
@@ -159,4 +121,3 @@ INSTALL( TARGETS
    ARCHIVE DESTINATION lib
 )
 INSTALL( FILES ${HEADERS} DESTINATION "include/graphene/chain" )
-INSTALL( FILES ${PROTOCOL_HEADERS} DESTINATION "include/graphene/chain/protocol" )

libraries/chain/account_evaluator.cpp

@@ -22,16 +22,14 @@
  * THE SOFTWARE.
  */
 
-#include <fc/smart_ref_impl.hpp>
-
 #include <graphene/chain/account_evaluator.hpp>
 #include <graphene/chain/buyback.hpp>
 #include <graphene/chain/buyback_object.hpp>
 #include <graphene/chain/database.hpp>
 #include <graphene/chain/exceptions.hpp>
 #include <graphene/chain/hardfork.hpp>
 #include <graphene/chain/internal_exceptions.hpp>
-#include <graphene/chain/special_authority.hpp>
+#include <graphene/chain/special_authority_evaluation.hpp>
 #include <graphene/chain/special_authority_object.hpp>
 #include <graphene/chain/worker_object.hpp>
 
@@ -216,7 +214,7 @@ object_id_type account_create_evaluator::do_apply( const account_create_operatio
    if( dynamic_properties.accounts_registered_this_interval % global_properties.parameters.accounts_per_fee_scale == 0
          && global_properties.parameters.account_fee_scale_bitshifts != 0 )
    {
-      d.modify(global_properties, [&dynamic_properties](global_property_object& p) {
+      d.modify(global_properties, [](global_property_object& p) {
          p.parameters.current_fees->get<account_create_operation>().basic_fee <<= p.parameters.account_fee_scale_bitshifts;
       });
    }