Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Environment Variable Support #83

Merged
merged 9 commits into from
May 20, 2024
Merged

Environment Variable Support #83

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
726ac9d
Adding `derive_configuration` to allow for config file overrides to c…
fosterbrereton May 17, 2024
ecbccc7
adding new tests to dogfood
fosterbrereton May 17, 2024
d1bf6a7
testing a failure
fosterbrereton May 17, 2024
d266e60
better test reporting
fosterbrereton May 17, 2024
84ca0b3
tweak
fosterbrereton May 18, 2024
f3f1828
tweak
fosterbrereton May 18, 2024
c3a15b6
tweak
fosterbrereton May 20, 2024
20baa98
tweak
fosterbrereton May 20, 2024
b7669a0
improvements to graceful exit logic
fosterbrereton May 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .github/workflows/build-and-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,12 @@ jobs:
id: build-orc-orc_dogfood
continue-on-error: true
run: |
xcodebuild -project ./build/orc.xcodeproj -scheme orc_dogfood -configuration Debug
ORC_OUTPUT_FILE=./output.json ORC_OUTPUT_FILE_MODE=json xcodebuild -json -project ./build/orc.xcodeproj -scheme orc_dogfood -configuration Debug -hideShellScriptEnvironment
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A new test that leverages the environment variables to cause a JSON file to get dumped when running orc_dogfood (which is a CMake target that uses ORC to build ORC). The JSON is then traversed with a couple jq queries to make sure some boilerplate keys are in there and filled in with something meaningful.

cat output.json
if [ `cat output.json | jq '.["synopsis"]["violations"]'` == 0 ]; then echo "SUCCESS"; else echo "FAILURE"; exit 1; fi
if [ `cat output.json | jq '.["synopsis"]["dies_skipped"]'` != 0 ]; then echo "SUCCESS"; else echo "FAILURE"; exit 1; fi
if [ `cat output.json | jq '.["synopsis"]["unique_symbols"]'` != 0 ]; then echo "SUCCESS"; else echo "FAILURE"; exit 1; fi
if [ `cat output.json | jq '.["synopsis"]["object_files_scanned"]'` != 0 ]; then echo "SUCCESS"; else echo "FAILURE"; exit 1; fi
- name: 🛠️ orc release
id: build-orc-release
continue-on-error: true
Expand Down
4 changes: 3 additions & 1 deletion include/orc/str.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,9 @@ std::string format_size(std::size_t x, format_mode mode = format_mode::binary);
std::string format_pct(float x);

inline std::string format_pct(float x, float total) {
return format_pct(x / total);
return format_pct(total ? x / total : 0);
}

std::string toupper(std::string&& s);

/**************************************************************************************************/
215 changes: 126 additions & 89 deletions src/main.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,42 @@ void open_output_file(const std::string& a, const std::string& b) {

/**************************************************************************************************/

void process_orc_config_file(const char* bin_path_string) {
template <typename T>
T parse_enval(std::string&&) = delete;

template <>
std::string parse_enval(std::string&& x) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why move semantics instead of const std::string&?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. the value coming from getenv is a const char*, which is then converted to a std::string for this routine. For the std::string specialization of parse_enval, we can move the incoming string to the result with no additional changes. If the incoming string were const std::string&, it would cost us a copy to make a new one to return by value.

assert(!x.empty());
return x;
}

template <>
bool parse_enval(std::string&& x) {
assert(!x.empty());
return x != "0" && toupper(std::move(x)) != "FALSE";
}

template <>
std::size_t parse_enval(std::string&& x) {
assert(!x.empty());
return std::max<int>(std::atoi(x.c_str()), 0);
}

template <typename T>
T derive_configuration(const char* key,
const toml::parse_result& settings,
T&& fallback) {
T result = settings[key].value_or(fallback);
std::string envar = toupper(std::string("ORC_") + key);
if (const char* enval = std::getenv(envar.c_str())) {
result = parse_enval<T>(enval);
}
return result;
}
Comment on lines +108 to +118
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This routine is the heart of the PR. Instead of taking values from the settings file directly, derive_configuration will also look at the associated environment variable and use that if found.


/**************************************************************************************************/

void process_orc_configuration(const char* bin_path_string) {
std::filesystem::path pwd(std::filesystem::current_path());
std::filesystem::path bin_path(pwd / bin_path_string);
std::filesystem::path config_path;
Expand Down Expand Up @@ -114,104 +149,106 @@ void process_orc_config_file(const char* bin_path_string) {
bin_path = parent;
}

toml::parse_result settings;

if (exists(config_path)) {
try {
auto settings = toml::parse_file(config_path.string());
auto& app_settings = settings::instance();

app_settings._graceful_exit = settings["graceful_exit"].value_or(false);
app_settings._max_violation_count = settings["max_error_count"].value_or(0);
app_settings._forward_to_linker = settings["forward_to_linker"].value_or(true);
app_settings._standalone_mode = settings["standalone_mode"].value_or(false);
app_settings._dylib_scan_mode = settings["dylib_scan_mode"].value_or(false);
app_settings._parallel_processing = settings["parallel_processing"].value_or(true);
app_settings._filter_redundant = settings["filter_redundant"].value_or(true);
app_settings._print_object_file_list = settings["print_object_file_list"].value_or(false);
app_settings._relative_output_file = settings["relative_output_file"].value_or("");

if (settings["output_file"]) {
std::string fn = *settings["output_file"].value<std::string>();
open_output_file("", fn);
}
settings = toml::parse_file(config_path.string());
} catch (const toml::parse_error& err) {
cerr_safe([&](auto& s) { s << "Parsing failed:\n" << err << "\n"; });
throw std::runtime_error("configuration parsing error");
}
} else {
cerr_safe([&](auto& s) { s << "ORC config file: not found\n"; });
}

if (auto log_level = settings["log_level"].value<std::string>()) {
const auto& level = *log_level;
if (level == "silent") {
app_settings._log_level = settings::log_level::silent;
} else if (level == "warning") {
app_settings._log_level = settings::log_level::warning;
} else if (level == "info") {
app_settings._log_level = settings::log_level::info;
} else if (level == "verbose") {
app_settings._log_level = settings::log_level::verbose;
} else {
// not a known value. Switch to verbose!
app_settings._log_level = settings::log_level::verbose;
cout_safe(
[&](auto& s) { s << "warning: Unknown log level (using verbose)\n"; });
}
}
auto& app_settings = settings::instance();

app_settings._graceful_exit = derive_configuration("graceful_exit", settings, false);
app_settings._max_violation_count = derive_configuration("max_error_count", settings, std::size_t(0));
app_settings._forward_to_linker = derive_configuration("forward_to_linker", settings, true);
app_settings._standalone_mode = derive_configuration("standalone_mode", settings, false);
app_settings._dylib_scan_mode = derive_configuration("dylib_scan_mode", settings, false);
app_settings._parallel_processing = derive_configuration("parallel_processing", settings, true);
app_settings._filter_redundant = derive_configuration("filter_redundant", settings, true);
app_settings._print_object_file_list = derive_configuration("print_object_file_list", settings, false);
app_settings._relative_output_file = derive_configuration("relative_output_file", settings, std::string());

const std::string log_level = derive_configuration("log_level", settings, std::string("warning"));
const std::string output_file = derive_configuration("output_file", settings, std::string());
const std::string output_file_mode = derive_configuration("output_file_mode", settings, std::string("text"));

// Do this early so we can log the ensuing output if it happens.
if (!output_file.empty()) {
open_output_file(std::string(), output_file);
}

if (app_settings._standalone_mode && app_settings._dylib_scan_mode) {
throw std::logic_error(
"Both standalone and dylib scanning mode are enabled. Pick one.");
}
// Do this early, too, so we can _suppress_ the output if we need to.
if (output_file_mode == "text") {
app_settings._output_file_mode = settings::output_file_mode::text;
} else if (output_file_mode == "json") {
app_settings._output_file_mode = settings::output_file_mode::json;
} else if (log_level_at_least(settings::log_level::warning)) {
cout_safe([&](auto& s) {
s << "warning: unknown output_file_mode '" << output_file_mode << "'; using text\n";
});
}

if (app_settings._dylib_scan_mode) {
app_settings._forward_to_linker = false;
}
if (log_level == "silent") {
app_settings._log_level = settings::log_level::silent;
} else if (log_level == "warning") {
app_settings._log_level = settings::log_level::warning;
} else if (log_level == "info") {
app_settings._log_level = settings::log_level::info;
} else if (log_level == "verbose") {
app_settings._log_level = settings::log_level::verbose;
} else {
// not a known value. Switch to verbose!
app_settings._log_level = settings::log_level::verbose;
cout_safe(
[&](auto& s) { s << "warning: unknown log_level '" << log_level << "'; using verbose\n"; });
}

auto read_string_list = [&_settings = settings](const char* name) {
std::vector<std::string> result;
if (auto* array = _settings.get_as<toml::array>(name)) {
for (const auto& entry : *array) {
if (auto symbol = entry.value<std::string>()) {
result.push_back(*symbol);
}
}
}
std::sort(result.begin(), result.end());
return result;
};

app_settings._symbol_ignore = read_string_list("symbol_ignore");
app_settings._violation_report = read_string_list("violation_report");
app_settings._violation_ignore = read_string_list("violation_ignore");

if (!app_settings._violation_report.empty() &&
!app_settings._violation_ignore.empty()) {
if (log_level_at_least(settings::log_level::warning)) {
cout_safe([&](auto& s) {
s << "warning: Both `violation_report` and `violation_ignore` lists found\n";
s << "warning: `violation_report` will be ignored in favor of `violation_ignore`\n";
});
}
}
if (app_settings._standalone_mode && app_settings._dylib_scan_mode) {
throw std::logic_error(
"Both standalone and dylib scanning mode are enabled. Pick one.");
}

if (settings["output_file_mode"]) {
const std::string mode = *settings["output_file_mode"].value<std::string>();
if (mode == "text") {
app_settings._output_file_mode = settings::output_file_mode::text;
} else if (mode == "json") {
app_settings._output_file_mode = settings::output_file_mode::json;
} else if (log_level_at_least(settings::log_level::warning)) {
cout_safe([&](auto& s) {
s << "warning: unknown `output_file_mode`: " << mode << "\n";
});
if (app_settings._dylib_scan_mode) {
app_settings._forward_to_linker = false;
}

auto read_string_list = [&_settings = settings](const char* name) {
std::vector<std::string> result;
if (auto* array = _settings.get_as<toml::array>(name)) {
for (const auto& entry : *array) {
if (auto symbol = entry.value<std::string>()) {
result.push_back(*symbol);
}
}
}
std::sort(result.begin(), result.end());
return result;
};

if (log_level_at_least(settings::log_level::info)) {
cout_safe([&](auto& s) {
s << "info: ORC config file: " << config_path.string() << "\n";
});
}
} catch (const toml::parse_error& err) {
cerr_safe([&](auto& s) { s << "Parsing failed:\n" << err << "\n"; });
throw std::runtime_error("configuration parsing error");
app_settings._symbol_ignore = read_string_list("symbol_ignore");
app_settings._violation_report = read_string_list("violation_report");
app_settings._violation_ignore = read_string_list("violation_ignore");

if (!app_settings._violation_report.empty() &&
!app_settings._violation_ignore.empty()) {
if (log_level_at_least(settings::log_level::warning)) {
cout_safe([&](auto& s) {
s << "warning: Both `violation_report` and `violation_ignore` lists found\n";
s << "warning: `violation_report` will be ignored in favor of `violation_ignore`\n";
});
}
} else {
cerr_safe([&](auto& s) { s << "ORC config file: not found\n"; });
}

if (log_level_at_least(settings::log_level::info)) {
cout_safe([&](auto& s) {
s << "info: ORC config file: " << config_path.string() << "\n";
});
}
}

Expand Down Expand Up @@ -483,7 +520,7 @@ int main(int argc, char** argv) try {

signal(SIGINT, interrupt_callback_handler);

process_orc_config_file(argv[0]);
process_orc_configuration(argv[0]);

cmdline_results cmdline = process_command_line(argc, argv);

Expand Down
4 changes: 2 additions & 2 deletions src/orc.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -644,7 +644,7 @@ std::string to_json(const std::vector<odrv_report>& reports) {
constexpr auto spaces_k = 0; // no pretty-printing in release builds
#endif

nlohmann::json violations;
nlohmann::json violations = nlohmann::json::object_t();

for (const auto& report : reports) {
assert(violations.count(report._symbol) == 0);
Expand All @@ -657,7 +657,7 @@ std::string to_json(const std::vector<odrv_report>& reports) {
synopsis["object_files_scanned"] = g._object_file_count.load();
synopsis["dies_processed"] = g._die_processed_count.load();
synopsis["dies_skipped"] = g._die_skipped_count.load();
synopsis["dies_skipped_pct"] = g._die_skipped_count * 100. / g._die_processed_count;
synopsis["dies_skipped_pct"] = g._die_processed_count ? (g._die_skipped_count * 100. / g._die_processed_count) : 0;
synopsis["unique_symbols"] = g._unique_symbol_count.load();

nlohmann::json result = nlohmann::json::object_t {
Expand Down
9 changes: 9 additions & 0 deletions src/str.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,3 +90,12 @@ std::string format_pct(float x) {
}

/**************************************************************************************************/

std::string toupper(std::string&& s) {
std::transform(s.begin(), s.end(), s.begin(), [](auto c){
return std::toupper(c);
});
return s;
}

/**************************************************************************************************/
Loading