Filtering & Dumping binary data

[pabloko]

Hi, i was looking for a way to dump POST&PUT data on a production instance using jakarta to serve a java servlet, with the particularity that i also need binary dumping, and also filtering the hook to some urls due to high memory needed on some binary request i had to log (~300k) also ive choosed to dump binary data as hex string, so buffer has to be duplicated to allocate all buffer. I wanted to avoid
state->buffer = apr_palloc(state->mp, cfg->max_size + 1);
on non filtered urls, so added this at the begginning of the input filter

apr_status_t dumpost_input_filter (ap_filter_t *f, apr_bucket_brigade *bb, ap_input_mode_t mode, apr_read_type_e block, apr_off_t readbytes) {

		apr_status_t ret;
		if (strstr(f->r->the_request, "/file/to/log1/") != NULL || strstr(f->r->the_request, "/file/to/log2") != NULL) {

		}  else {
			if ((ret = ap_get_brigade(f->next, bb, mode, block, readbytes)) != APR_SUCCESS)
				return ret;
			return APR_SUCCESS;
		}

Ive never worked with APR or apache so i dont know if this is good or not, but for sure, i should had to create a new AP_INIT_ITERATE cmd on the config to add filtered urls on config file instead hardcode them. As i wont need it and due to lazyness i will keep this...

Then i foundout that the way dumpit is doing the copy to the buffer truncates the data on nulls [strncpy(buf, ibuf, nbytes);] so ive replaced with byte to byte copy

/*strncpy(buf, ibuf, nbytes);*/
				
				for (int kk = 0; kk < nbytes; kk++) {
					buf[kk]=ibuf[kk];
				}
                *current_size += nbytes;

Then ive added a simple routine to perform binary to hex string conversion

void hexArrayToStr(unsigned char* info, unsigned int infoLength, char **buffer, unsigned int start) {
    const char* pszNibbleToHex = {"0123456789ABCDEF"};
    int nNibble, i;
            for (i = 0; i < infoLength; i++) {
					nNibble = info[i] >> 4;
					buffer[0][2 * i + start] = pszNibbleToHex[nNibble];
					nNibble = info[i] & 0x0F;
					buffer[0][2 * i + 1 + start] = pszNibbleToHex[nNibble];
            }
}

and edited logit to test the buffer to find binary bytes and set a proper output, i know i should check more bytes than null, for example (<0x20 && >0x7e && !=0x0A && !=0x0D) but in my case i encounter lots of 0x00 on my binary buffer so it will be fine as is.

apr_status_t logit(ap_filter_t *f) {
    request_state *state = f->ctx;
    request_rec *r = f->r;

    if (state == NULL || state->log_size == 0) return -1;
    state->buffer[state->log_size] = 0;
///ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r, "mod_dumpost: wat");

    if (state->fd == NULL) {
      // no file to write to, write to error log
      // data is truncated to MAX_STRING_LEN ~ 8192 in apache
      ap_log_rerror(APLOG_MARK, APLOG_INFO, 0, r, "\"%s\" %s", r->the_request, state->buffer);
    } else {
      // need to manually get the time and ip address -- too lazy to make these cusomizable
      char *time = apr_palloc(r->pool, 50);
      apr_ctime(time, r->request_time);
      char *ip = r->connection->client_ip;
      apr_size_t nbytes_written;
      char *text = apr_psprintf(r->pool, "[%s] %s \"%s\" ",time, ip, r->the_request);
	  
	  //test bin
	  	  int not_bin=1;
		  for (int i = 0; i < state->log_size; i++)
		  {
			 if (state->buffer[i]==0) not_bin=0;
		  }
	  
	 int jj=strlen(text);
	 if (not_bin==0) {
		 char *text2 = apr_palloc(r->pool, (state->log_size*2) + 2 + jj);
		 sprintf(text2, "%s", text);
		 hexArrayToStr(state->buffer, state->log_size, &text2, jj);
		 jj=jj+(state->log_size*2);
		 text2[jj]='\n';
		 jj++;
		 text2[jj]='\0';
		 apr_status_t rc = apr_file_write_full(state->fd, text2, jj, &nbytes_written);

		  if (rc != APR_SUCCESS) {
			ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r, "mod_dumpost: error while writing to log");
			return rc;
		  }
		  apr_file_close(state->fd);
	 } else {
		 char *text2 = apr_palloc(r->pool, (state->log_size) + 1 + jj);
		 sprintf(text2, "%s%s\n", text, state->buffer);
		 jj=strlen(text2);
		 apr_status_t rc = apr_file_write_full(state->fd, text2, jj, &nbytes_written);

		  if (rc != APR_SUCCESS) {
			ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r, "mod_dumpost: error while writing to log");
			return rc;
		  }
		  apr_file_close(state->fd);
	 }
	 
      
    }
    return APR_SUCCESS;
}

I do not know if i missed any requeriment by apr to handle those buffers but seems to be working nice and not leaking any memory but i didnt tested that properly...

[danghvu]

Hi, interesting work,

Nothing wrong I can see immediately, if you can cleanup and make a PR, I will look at it more closely. Maybe the hex-string output can be enabled with an option, which will be useful to other people I can imagine.

Vu