Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mon 22554 fix tests #924

Merged
merged 84 commits into from
Oct 23, 2023
Merged
Show file tree
Hide file tree
Changes from 66 commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
8599dfa
fix(broker): little fixes in broker
bouda1 Oct 12, 2023
3cf454f
fix(tests): fixes in various tests
bouda1 Oct 12, 2023
5c3480c
fix(tests): cleanup and little fixes
bouda1 Oct 12, 2023
55f9ad4
fix(tests): scheduler test is split into 2 tests, retry check and the…
bouda1 Oct 11, 2023
f1c61a4
fix(tests): external-commands2 contained an issue in its last check
bouda1 Oct 11, 2023
ce4c012
fix(tests): several tests should be fixed
bouda1 Oct 12, 2023
73cc685
fix(tests): services-and-bulk-stmt.robot
bouda1 Oct 12, 2023
8fd685d
fix(tests): forced_checks fixed
bouda1 Oct 12, 2023
5228000
fix(tests): Tests on forced checks improved
bouda1 Oct 12, 2023
b0d51c5
fix(tests): coding style better followed
bouda1 Oct 13, 2023
8097c52
fix(tests): some bench tests disabled on arm
bouda1 Oct 13, 2023
08e70f7
fix(ci): attempt to disable some bench tests
bouda1 Oct 13, 2023
e01e042
wip(ci): attempt to get information about arm cpu
bouda1 Oct 13, 2023
cfa2804
wip(ci)
bouda1 Oct 13, 2023
e8ed4a1
wip(ci)
bouda1 Oct 13, 2023
9d6441c
wip(ci): attempt to reduce the artifact
bouda1 Oct 13, 2023
4796c22
fix(tests): restoring db_variables.robot with its weird values
bouda1 Oct 13, 2023
06b1776
wip(ci)
bouda1 Oct 13, 2023
c3b1c12
wip(tests)
bouda1 Oct 14, 2023
0df35be
fix(ci)
bouda1 Oct 14, 2023
cf99e31
fix(broker/feeder): feeder can be not stopped when switched off
bouda1 Oct 14, 2023
e287359
fix(tests): cleanup of some keywords
bouda1 Oct 14, 2023
d2ef8c9
wip(tests): oops
bouda1 Oct 14, 2023
53dc6e6
wip(ci)
bouda1 Oct 14, 2023
d1d4db2
wip(tests)
bouda1 Oct 14, 2023
b45efc3
enh(tests): unstable feature added
bouda1 Oct 16, 2023
004e543
wip(ci)
bouda1 Oct 16, 2023
fcfab31
wip(ci): dev-23.10 added
bouda1 Oct 16, 2023
f8a3f11
fix(ci): output.xml restored
bouda1 Oct 16, 2023
a0ddc1b
fix(tests): unused tag removed
bouda1 Oct 16, 2023
80f0b50
fix(cmake): issue with python 3.x, x < 11
bouda1 Oct 16, 2023
b905159
cleanup(tests): unused buggy function commented before it is really used
bouda1 Oct 16, 2023
c1fd455
wip(tests): changes after review
bouda1 Oct 16, 2023
de62a4a
fix(tests): changes after review
bouda1 Oct 16, 2023
2ddd147
fix(tests): no more develop branch in benches
bouda1 Oct 16, 2023
b4ad132
fix(tests/reverse_bam): the process should be better killed
bouda1 Oct 16, 2023
8a344d3
fix(tests): map process better killed
bouda1 Oct 16, 2023
91e8e6f
cleanup(tests): Changes to satisfay the Architect
bouda1 Oct 17, 2023
3928df9
fix(tests): wip
bouda1 Oct 17, 2023
b9cd780
wip(tests)
bouda1 Oct 18, 2023
5fa8a6a
fix(broker): engine is directly accessible from muxers
bouda1 Oct 18, 2023
5963f43
fix(ci)
bouda1 Oct 18, 2023
350b132
fix(broker): mysql_connection finish() may finish too late
bouda1 Oct 18, 2023
0f55341
fix(broker/mysql_connection): issue when the connection thread is sto…
bouda1 Oct 18, 2023
692e322
fix(tests/bam)
bouda1 Oct 18, 2023
9a94bc8
fix(tests): reverse-connection, db cleanup added
bouda1 Oct 18, 2023
c721d21
fix(ci): DEBUG_ROBOT set to OFF
bouda1 Oct 18, 2023
4a19314
wip(tests): on bench
bouda1 Oct 19, 2023
6641f0d
enh(broker): shared_from_this no more used
bouda1 Oct 19, 2023
073b315
tests(bam): bam_pb has more checks
bouda1 Oct 19, 2023
40b128a
fix(tests)
bouda1 Oct 19, 2023
af86f64
fix(tests)
bouda1 Oct 19, 2023
6647e35
enh(broker): test_robot not always enabled
bouda1 Oct 19, 2023
c32bbc7
fix(broker): unused header removed
bouda1 Oct 19, 2023
05c73f1
cleanup(tests/broker): robot simplified and one mutex unused removed
bouda1 Oct 20, 2023
60ebea3
fix(broker): coding style fixed
bouda1 Oct 20, 2023
86e6642
enh(tests): new keyword applied everywhere
bouda1 Oct 20, 2023
8383e5e
fix(cmake)
bouda1 Oct 20, 2023
8e0982f
fix(broker): muxer/engine again
bouda1 Oct 20, 2023
dd9109a
fix(tests): an issue when the tuple is empty fixed
bouda1 Oct 20, 2023
d7e9687
fix(cmake)
bouda1 Oct 20, 2023
ed2a950
fix(ci): debug_robot is back
bouda1 Oct 20, 2023
a5e245a
fix(broker): another one
bouda1 Oct 20, 2023
1208231
fix(ci)
bouda1 Oct 20, 2023
f2483bb
fix(broker): bad initialization
bouda1 Oct 21, 2023
fc4c31d
fix(broker): unused code removed
bouda1 Oct 21, 2023
935c3ce
Update tests/resources/Common.py
bouda1 Oct 23, 2023
76512ee
Update tests/connector_perl/connector_perl.robot
bouda1 Oct 23, 2023
b4b0341
Update tests/connector_perl/connector_perl.robot
bouda1 Oct 23, 2023
b6e215c
Update tests/connector_perl/connector_perl.robot
bouda1 Oct 23, 2023
77a9232
Update tests/broker/bbdo-server-client.robot
bouda1 Oct 23, 2023
6e6d8c8
Update tests/broker-engine/acknowledgement.robot
bouda1 Oct 23, 2023
14080ec
Update tests/broker-engine/acknowledgement.robot
bouda1 Oct 23, 2023
0b58fe0
Update tests/broker-engine/acknowledgement.robot
bouda1 Oct 23, 2023
1054662
Update tests/broker-engine/acknowledgement.robot
bouda1 Oct 23, 2023
933d755
cleanup(broker)
bouda1 Oct 23, 2023
7d38b08
Update tests/broker-engine/acknowledgement.robot
bouda1 Oct 23, 2023
df0aabd
Update tests/broker-engine/acknowledgement.robot
bouda1 Oct 23, 2023
fcb93c4
Update tests/broker-engine/acknowledgement.robot
bouda1 Oct 23, 2023
0c0daa7
cleanup(robot)
bouda1 Oct 23, 2023
220e400
Update tests/broker-engine/log-v2_engine.robot
bouda1 Oct 23, 2023
946a914
cleanup(tests)
bouda1 Oct 23, 2023
1d04c74
cleanup(robot)
bouda1 Oct 23, 2023
e72d791
fix(robot): typo
bouda1 Oct 23, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .github/scripts/collect-test-robot.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,11 @@ database_type=$2
distrib=${ID}
distrib=$(echo $distrib | tr '[:lower:]' '[:upper:]')

#cpu=$(lscpu | awk '$1 ~ "Architecture" { print $2 }')
if [[ "$test_file" =~ "unstable" ]] ; then
exit 0
fi

if [ ${database_type} == 'mysql' ] && [ ! -f tests/${test_file}.mysql ]; then
echo > tests/log.html
echo '<?xml version="1.0" encoding="UTF-8"?>' > tests/output.xml
Expand Down Expand Up @@ -106,4 +111,4 @@ cd tests
./init-proto.sh

echo "####################### Run Centreon Collect Robot Tests #######################"
robot $test_file
robot -e unstable $test_file
1 change: 1 addition & 0 deletions .github/workflows/package-collect.yml
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ jobs:

sudo $CMAKE \
-G "Ninja" \
-DDEBUG_ROBOT=OFF \
-DWITH_TESTING=OFF \
-DWITH_BENCH=ON \
-DWITH_MODULE_SIMU=OFF \
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/robot-nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@ jobs:
uses: actions/checkout@v4

- run: |
gh workflow run robot-nightly.yml -r "dev-23.10.x"
gh workflow run robot-nightly.yml -r "dev-23.04.x"
gh workflow run robot-nightly.yml -r "dev-22.10.x"
gh workflow run robot-nightly.yml -r "dev-22.04.x"
gh workflow run robot-nightly.yml -r "dev-21.10.x"
shell: bash
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand Down
11 changes: 5 additions & 6 deletions .github/workflows/robot-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -88,19 +88,18 @@ jobs:
AWS_BUCKET: centreon-collect-robot-report

- name: Move reports
if: ${{ !cancelled() }}
if: ${{ failure() }}
run: |
mkdir reports
FILE_PREFIX=`echo "${{ matrix.feature }}" | sed -e "s#/#__#g"`-${{inputs.database_type}}
mv tests/log.html reports/$FILE_PREFIX-log.html
mv tests/output.xml reports/$FILE_PREFIX-output.xml
mv tests/report.html reports/$FILE_PREFIX-report.html
if [ -d tests/failed ] ; then
mv tests/log.html reports/$FILE_PREFIX-log.html
mv tests/output.xml reports/$FILE_PREFIX-output.xml
mv tests/failed reports/$FILE_PREFIX-failed
fi

- name: Upload Test Results
if: ${{ !cancelled() }}
if: ${{ failure() }}
uses: actions/upload-artifact@v3
with:
name: robot-test-reports-${{inputs.database_type}}-${{ inputs.distrib }}-${{ inputs.arch }}
Expand All @@ -109,7 +108,7 @@ jobs:

robot-test-report:
needs: [robot-test]
if: ${{ !cancelled() }}
if: ${{ failure() }}
runs-on: ubuntu-22.04

steps:
Expand Down
5 changes: 5 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ endif()
add_definitions("-D_GLIBCXX_USE_CXX11_ABI=1")

option(NG "C++17 build." OFF)
option(DEBUG_ROBOT OFF)

if(NG)
set(CMAKE_CXX_STANDARD 17)
Expand Down Expand Up @@ -128,6 +129,10 @@ set(COLLECT_PATCH 0)
set(COLLECT_VERSION "${COLLECT_MAJOR}.${COLLECT_MINOR}.${COLLECT_PATCH}")
add_definitions(-DCENTREON_CONNECTOR_VERSION=\"${COLLECT_VERSION}\")

if (DEBUG_ROBOT)
add_definitions(-DDEBUG_ROBOT)
endif()

# ########### CONSTANTS ###########
set(USER_BROKER centreon-broker)
set(USER_ENGINE centreon-engine)
Expand Down
2 changes: 1 addition & 1 deletion broker/bam/inc/com/centreon/broker/bam/computable.hh
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ class computable {
* @brief This method is used by the dump() method. It gives a summary of this
* computable main informations.
*
* @return A multiline strings with various informations.
* @return A multiline string with various informations.
*/
virtual std::string object_info() const = 0;
/**
Expand Down
2 changes: 1 addition & 1 deletion broker/bam/src/bool_constant.cc
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright 2016, 2023 Centreon
* Copyright 2014, 2023 Centreon
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand Down
5 changes: 5 additions & 0 deletions broker/bam/src/bool_operation.cc
Original file line number Diff line number Diff line change
Expand Up @@ -110,14 +110,19 @@ std::string bool_operation::object_info() const {
switch (_type) {
case addition:
op = "PLUS";
break;
case substraction:
op = "MINUS";
break;
case multiplication:
op = "MUL";
break;
case division:
op = "DIV";
break;
case modulo:
op = "MODULO";
break;
default:
return "unknown operation";
}
Expand Down
3 changes: 2 additions & 1 deletion broker/bam/src/connector.cc
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,8 @@ static constexpr multiplexing::muxer_filter _monitoring_stream_filter = {
neb::downtime::static_type(), neb::pb_downtime::static_type(),
bam::ba_status::static_type(), bam::pb_ba_status::static_type(),
bam::kpi_status::static_type(), bam::pb_kpi_status::static_type(),
inherited_downtime::static_type(), pb_inherited_downtime::static_type()};
inherited_downtime::static_type(), pb_inherited_downtime::static_type(),
extcmd::pb_ba_info::static_type()};

static constexpr multiplexing::muxer_filter _reporting_stream_filter = {
bam::kpi_event::static_type(),
Expand Down
1 change: 1 addition & 0 deletions broker/bam/src/monitoring_stream.cc
Original file line number Diff line number Diff line change
Expand Up @@ -581,6 +581,7 @@ int monitoring_stream::write(std::shared_ptr<io::data> const& data) {
_write_external_command(cmd);
} break;
case extcmd::pb_ba_info::static_type(): {
log_v2::bam()->info("BAM: dump BA");
extcmd::pb_ba_info const& e =
*std::static_pointer_cast<const extcmd::pb_ba_info>(data);
auto& obj = e.obj();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ class global_cache : public std::enable_shared_from_this<global_cache> {

void allocation_exception_handler();

virtual void managed_map(bool create){};
virtual void managed_map(bool create) {}

public:
using pointer = std::shared_ptr<global_cache>;
Expand Down
8 changes: 8 additions & 0 deletions broker/core/inc/com/centreon/broker/misc/misc.hh
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,16 @@ std::string dump_filters(const multiplexing::muxer_filter& filters);
std::list<perfdata> parse_perfdata(uint32_t host_id,
uint32_t service_id,
const char* str);
#if DEBUG_ROBOT
void debug(const std::string& content);
#endif
} // namespace misc

#if DEBUG_ROBOT
#define DEBUG(content) misc::debug(content)
#else
#define DEBUG(content)
#endif
CCB_END()

#endif // !CCB_MISC_MISC_HH
2 changes: 2 additions & 0 deletions broker/core/inc/com/centreon/broker/processing/feeder.hh
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ class feeder : public stat_visitable,

protected:
feeder(const std::string& name,
const std::shared_ptr<multiplexing::engine>& parent,
std::shared_ptr<io::stream>& client,
const multiplexing::muxer_filter& read_filters,
const multiplexing::muxer_filter& write_filters);
Expand All @@ -84,6 +85,7 @@ class feeder : public stat_visitable,
public:
static std::shared_ptr<feeder> create(
const std::string& name,
const std::shared_ptr<multiplexing::engine>& parent,
std::shared_ptr<io::stream>& client,
const multiplexing::muxer_filter& read_filters,
const multiplexing::muxer_filter& write_filters);
Expand Down
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
/*
** Copyright 2009-2012,2015,2019-2021 Centreon
**
** Licensed under the Apache License, Version 2.0 (the "License");
** you may not use this file except in compliance with the License.
** You may obtain a copy of the License at
**
** http://www.apache.org/licenses/LICENSE-2.0
**
** Unless required by applicable law or agreed to in writing, software
** distributed under the License is distributed on an "AS IS" BASIS,
** WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
** See the License for the specific language governing permissions and
** limitations under the License.
**
** For more information : [email protected]
*/
* Copyright 2009-2012,2015,2019-2021 Centreon
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* For more information : [email protected]
*/

#ifndef CCB_MULTIPLEXING_ENGINE_HH
#define CCB_MULTIPLEXING_ENGINE_HH
Expand Down Expand Up @@ -63,7 +63,7 @@ class callback_caller;
*
* @see muxer
*/
class engine : public std::enable_shared_from_this<engine> {
class engine {
static std::mutex _load_m;
static std::shared_ptr<engine> _instance;

Expand All @@ -82,8 +82,7 @@ class engine : public std::enable_shared_from_this<engine> {
std::deque<std::shared_ptr<io::data>> _kiew;

// Subscriber.
std::vector<std::shared_ptr<muxer>> _muxers;
std::mutex _muxers_m;
std::vector<std::weak_ptr<muxer>> _muxers;

// Statistics.
EngineStats* _stats;
Expand Down Expand Up @@ -111,8 +110,8 @@ class engine : public std::enable_shared_from_this<engine> {
void publish(const std::deque<std::shared_ptr<io::data>>& to_publish);
void start();
void stop();
void subscribe(const std::shared_ptr<muxer>& subscriber);
void unsubscribe(const muxer* subscriber);
void subscribe(std::shared_ptr<muxer>& subscriber);
void unsubscribe_muxer(const muxer* subscriber);
};
} // namespace multiplexing

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ class muxer : public io::stream {
static uint32_t _event_queue_max_size;

const std::string _name;
std::shared_ptr<engine> _engine;
const std::string _queue_file_name;
multiplexing::muxer_filter _read_filter;
multiplexing::muxer_filter _write_filter;
Expand All @@ -83,6 +84,7 @@ class muxer : public io::stream {
void _update_stats(void) noexcept;

muxer(std::string name,
const std::shared_ptr<engine>& parent,
const muxer_filter& r_filter,
const muxer_filter& w_filter,
bool persistent = false);
Expand All @@ -94,6 +96,7 @@ class muxer : public io::stream {
static uint32_t event_queue_max_size() noexcept;

static std::shared_ptr<muxer> create(std::string name,
const std::shared_ptr<engine>& parent,
const muxer_filter& r_filter,
const muxer_filter& w_filter,
bool persistent = false);
Expand Down Expand Up @@ -121,6 +124,7 @@ class muxer : public io::stream {
void set_read_filter(const muxer_filter& w_filter);
void set_write_filter(const muxer_filter& w_filter);
void clear_read_handler();
void unsubscribe();
};

/**
Expand Down
21 changes: 14 additions & 7 deletions broker/core/multiplexing/src/engine.cc
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
#include "com/centreon/broker/config/applier/state.hh"
#include "com/centreon/broker/io/events.hh"
#include "com/centreon/broker/log_v2.hh"
#include "com/centreon/broker/misc/misc.hh"
#include "com/centreon/broker/multiplexing/muxer.hh"
#include "com/centreon/broker/pool.hh"

Expand Down Expand Up @@ -236,19 +237,20 @@ void engine::stop() {
EngineStats::STOPPED);
}
log_v2::core()->debug("multiplexing: engine stopped");
DEBUG(fmt::format("STOP engine {:p}", static_cast<void*>(this)));
}

/**
* Subscribe a muxer to the multiplexing engine if not already subscribed.
*
* @param[in] subscriber A muxer.
*/
void engine::subscribe(const std::shared_ptr<muxer>& subscriber) {
void engine::subscribe(std::shared_ptr<muxer>& subscriber) {
log_v2::config()->debug("engine: muxer {} subscribes to engine",
subscriber->name());
std::lock_guard<std::mutex> l(_engine_m);
for (auto& m : _muxers)
if (m == subscriber) {
if (m.lock() == subscriber) {
log_v2::config()->debug("engine: muxer {} already subscribed",
subscriber->name());
return;
Expand All @@ -261,10 +263,11 @@ void engine::subscribe(const std::shared_ptr<muxer>& subscriber) {
*
* @param[in] subscriber Subscriber.
*/
void engine::unsubscribe(const muxer* subscriber) {
void engine::unsubscribe_muxer(const muxer* subscriber) {
std::lock_guard<std::mutex> l(_engine_m);
for (auto it = _muxers.begin(); it != _muxers.end(); ++it) {
if (it->get() == subscriber) {
auto w = it->lock();
if (!w || w.get() == subscriber) {
log_v2::config()->debug("engine: muxer {} unsubscribes to engine",
subscriber->name());
_muxers.erase(it);
Expand All @@ -281,12 +284,16 @@ engine::engine()
_stats{stats::center::instance().register_engine()},
_unprocessed_events{0u},
_sending_to_subscribers{false} {
DEBUG(fmt::format("CONSTRUCTOR engine {:p}", static_cast<void*>(this)));
stats::center::instance().update(&EngineStats::set_mode, _stats,
EngineStats::NOT_STARTED);
}

engine::~engine() noexcept {
/* Muxers should be unsubscribed before arriving here. */
assert(_muxers.empty());
log_v2::core()->debug("core: cbd engine destroyed.");
DEBUG(fmt::format("DESTRUCTOR engine {:p}", static_cast<void*>(this)));
}

/**
Expand Down Expand Up @@ -380,8 +387,8 @@ bool engine::_send_to_subscribers(send_to_mux_callback_type&& callback) {
// it will be destroyed at the end of the scope of this function and at the
// end of lambdas posted
cb = std::make_shared<detail::callback_caller>(std::move(callback),
shared_from_this());
last_muxer = *_muxers.rbegin();
_instance);
last_muxer = _muxers.rbegin()->lock();
if (_muxers.size() > 1) {
/* Since the sending is parallelized, we use the thread pool for this
* purpose except for the last muxer where we use this thread. */
Expand All @@ -392,7 +399,7 @@ bool engine::_send_to_subscribers(send_to_mux_callback_type&& callback) {
/* We use the thread pool for the muxers from the first one to the
* second to last */
for (auto it = _muxers.begin(); it != it_last; ++it) {
pool::io_context().post([kiew, m = *it, cb]() {
pool::io_context().post([kiew, m = it->lock(), cb]() {
try {
m->publish(*kiew);
} // pool threads protection
Expand Down
Loading