-
Notifications
You must be signed in to change notification settings - Fork 39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[reconfigurator] Retrieve keeper lgif information #6549
Changes from 15 commits
e3a40bf
78520f5
40ae8f2
9959fd7
bc1705a
bb3a272
3c8ada0
83c7869
481aad8
0866bbe
6c442d0
c8a1f89
380139f
3b5bb45
7a6619b
8d1104d
6371954
55f59bd
5cbbae2
bd3465d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,130 @@ | ||
// This Source Code Form is subject to the terms of the Mozilla Public | ||
// License, v. 2.0. If a copy of the MPL was not distributed with this | ||
// file, You can obtain one at https://mozilla.org/MPL/2.0/. | ||
|
||
use anyhow::Result; | ||
use camino::Utf8PathBuf; | ||
use clickhouse_admin_types::Lgif; | ||
use dropshot::HttpError; | ||
use illumos_utils::{output_to_exec_error, ExecutionError}; | ||
use slog::Logger; | ||
use slog_error_chain::{InlineErrorChain, SlogInlineError}; | ||
use std::ffi::OsStr; | ||
use std::io; | ||
use std::net::SocketAddrV6; | ||
use tokio::process::Command; | ||
|
||
#[derive(Debug, thiserror::Error, SlogInlineError)] | ||
pub enum ClickhouseCliError { | ||
#[error("failed to run `clickhouse {subcommand}`")] | ||
Run { | ||
description: &'static str, | ||
subcommand: String, | ||
#[source] | ||
err: io::Error, | ||
}, | ||
#[error(transparent)] | ||
ExecutionError(#[from] ExecutionError), | ||
#[error("failed to parse command output")] | ||
Parse { | ||
description: &'static str, | ||
stdout: String, | ||
stderr: String, | ||
#[source] | ||
err: anyhow::Error, | ||
}, | ||
} | ||
|
||
impl From<ClickhouseCliError> for HttpError { | ||
fn from(err: ClickhouseCliError) -> Self { | ||
match err { | ||
ClickhouseCliError::Run { .. } | ||
| ClickhouseCliError::Parse { .. } | ||
| ClickhouseCliError::ExecutionError(_) => { | ||
let message = InlineErrorChain::new(&err).to_string(); | ||
HttpError { | ||
status_code: http::StatusCode::INTERNAL_SERVER_ERROR, | ||
error_code: Some(String::from("Internal")), | ||
external_message: message.clone(), | ||
internal_message: message, | ||
} | ||
} | ||
} | ||
} | ||
} | ||
|
||
#[derive(Debug)] | ||
pub struct ClickhouseCli { | ||
/// Path to where the clickhouse binary is located | ||
pub binary_path: Utf8PathBuf, | ||
/// Address on where the clickhouse keeper is listening on | ||
pub listen_address: SocketAddrV6, | ||
pub log: Option<Logger>, | ||
} | ||
|
||
impl ClickhouseCli { | ||
pub fn new(binary_path: Utf8PathBuf, listen_address: SocketAddrV6) -> Self { | ||
Self { binary_path, listen_address, log: None } | ||
} | ||
|
||
pub fn with_log(mut self, log: Logger) -> Self { | ||
self.log = Some(log); | ||
self | ||
} | ||
|
||
pub async fn lgif(&self) -> Result<Lgif, ClickhouseCliError> { | ||
self.keeper_client_non_interactive( | ||
"lgif", | ||
"Retrieve logically grouped information file", | ||
Lgif::parse, | ||
self.log.clone().unwrap(), | ||
) | ||
.await | ||
} | ||
|
||
async fn keeper_client_non_interactive<'a, F, T>( | ||
karencfv marked this conversation as resolved.
Show resolved
Hide resolved
|
||
&self, | ||
query: &str, | ||
subcommand_description: &'static str, | ||
parse: F, | ||
log: Logger, | ||
) -> Result<T, ClickhouseCliError> | ||
where | ||
F: FnOnce(&Logger, &[u8]) -> Result<T>, | ||
{ | ||
let mut command = Command::new(&self.binary_path); | ||
command | ||
.arg("keeper-client") | ||
.arg("--host") | ||
.arg(&format!("[{}]", self.listen_address.ip())) | ||
.arg("--port") | ||
.arg(&format!("{}", self.listen_address.port())) | ||
.arg("--query") | ||
.arg(query); | ||
|
||
let output = command.output().await.map_err(|err| { | ||
let args: Vec<&OsStr> = command.as_std().get_args().collect(); | ||
let args_parsed: Vec<String> = args | ||
.iter() | ||
.map(|&os_str| os_str.to_str().unwrap().to_owned()) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sorry if I wasn't clear. I don't think we should panic here if the string isn't UTF8. Instead we should return an error, especially since this function already returns a There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sorry again. For some reason your comments didn't show up on github last I looked. I realize this is unwrapping now because it's arguments we pass. Nonetheless I'd rather be on the safe side and return an error if possible. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So, Maybe I'm misunderstanding, but it feels a bit strange to return an error because of a malformed error? It seems simpler to collect all the arguments, however they were passed and return a single error with that information like it was in the beginning with There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. TBH, we don't even need these Unless this information isn't useful as part of the error message? In that case I could remove them altogether 🤔 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok, looks like I totally misinterpreted what was happening here. Sorry about that @karencfv! I didn't realize this was just for the error message. You were right using There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 😄 nw! I've changed the names of the variables, so it's clearer what those are for |
||
.collect(); | ||
let args_str = args_parsed.join(" "); | ||
ClickhouseCliError::Run { | ||
description: subcommand_description, | ||
subcommand: args_str, | ||
err, | ||
} | ||
})?; | ||
|
||
if !output.status.success() { | ||
return Err(output_to_exec_error(command.as_std(), &output).into()); | ||
} | ||
|
||
parse(&log, &output.stdout).map_err(|err| ClickhouseCliError::Parse { | ||
description: subcommand_description, | ||
stdout: String::from_utf8_lossy(&output.stdout).to_string(), | ||
stderr: String::from_utf8_lossy(&output.stdout).to_string(), | ||
err, | ||
}) | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this command may be good for debugging/introspection aside from the inventory we need for reconfiguration and we should keep it.
I just want to point out that for inventory we'll likely not use this API command directly, but rather use the underlying keeper request along with another request to
/keeper/config
to get the current committed configuration. That will eliminate an extra network round trip.Unfortunately, keeper doesn't seem to provide a way to get the configuration and its log index together in one call. If it did that we could use that for inventory. I think that would be a good patch to make to keeper and the keeper cli, but until then we'll need to make two separate calls. And because of the inherent race condition of two calls, we'll likely have to call at least one endpoint twice to see that the config hasn't changed while polling. We would want to do that local to the sled-agent without a round trip from nexus.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, yeah definitely! I realise there would be another endpoint to collect all the information we need for inventory itself. My reasoning was that the lgif command is useful for many things, and in the spirit of keeping PRs as small and easily digestible as possible, I was going to make a separate PR with that endpoint. On hindsight I should have written that on the PRs description. Sorry about that!