forked from OpenTSDB/opentsdb.net
-
Notifications
You must be signed in to change notification settings - Fork 0
/
http-api.html
474 lines (429 loc) · 18.7 KB
/
http-api.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
<!DOCTYPE html>
<html lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
<title>HTTP API - OpenTSDB - A Distributed, Scalable Monitoring System</title>
<link rel="stylesheet" href="css/style.css" type="text/css" media="screen"/>
<!--[if lte IE 8]>
<link rel="stylesheet" href="css/ie.css" type="text/css" media="screen"/>
<![endif]-->
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-18339382-1']);
_gaq.push(['_setDomainName', 'none']);
_gaq.push(['_setAllowLinker', true]);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>
</head>
<body>
<header><div id="headerbar">
<h1><a href="index.html">OpenTSDB</a></h1>
<nav><ul id="navbar">
<li><a href="overview.html">Overview</a></li>
<li><a href="getting-started.html">Getting Started</a></li>
<li><a href="manual.html">Manual</a></li>
<li><a href="faq.html">FAQ</a></li>
</ul></nav>
</div></header>
<!--[if lte IE 8]>
<div class="iesucks">Warning: You're using an unsupported, archaic browser.
Get a better, modern browsing experience with
<a href="http://www.google.com/chrome">Chrome</a> or
<a href="http://www.mozilla.com/firefox">Firefox</a>.</div>
<![endif]-->
<section id="content">
<section id="httpapi">
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js" type="text/javascript"></script>
<script src="misc/toc.js" type="text/javascript"></script>
<h2>HTTP API</h2>
This document is a draft specification of the HTTP API exposed by the TSD
(Time Series Daemon). OpenTSDB is still in beta, so this specification is
subject to changes. This specification is current as of
<a href="https://github.com/OpenTSDB/opentsdb/commit/f33d77416017802bb4c3001aa9581db7feced3d6">December 4, 2010</a>.
The excerpts of HTTP queries in this document have been intentionally
simplified, e.g. by removing some headers in the responses or not specifying
the HTTP protocol version in the requests (HTTP 1.1 is assumed throughout).
<div id="toc"></div>
<h3>Generalities</h3>
The code that dispatches all requests is in
<a href="https://github.com/OpenTSDB/opentsdb/blob/b85f7602872ecefda7b2a8b35e79681473f74130/src/tsd/RpcHandler.java">
<code>RpcHandler.java</code></a>. Some HTTP end-points can return results
in 3 different formats:
<ul>
<li>The default format (either HTML or plain text)</li>
<li>JSON, if the URL contains the <code>json</code> query string parameter.</li>
<li>PNG, if the URL contains the <code>png</code> query string parameter.</li>
</ul>
<table>
<thead style="font-family:monospace;white-space:pre-wrap"><tr>
<th>GET /invalid</th><th>GET /invalid?json</th><th>GET /invalid?png</th>
</tr></thead>
<tbody><tr>
<td class="code">HTTP/1.1 404 Not Found
Content-Type: text/html; charset=UTF-8
Content-Length: 964
<!DOCTYPE HTML>
<html><head>
<title>Page Not Found</title>
[...]
</td><td class="code">HTTP/1.1 404 Not Found
Content-Type: application/json
Content-Length: 24
{"err":"Page Not Found"}</td><td class="code">HTTP/1.1 404 Not Found
Content-Type: image/png
Content-Length: 3524
?PNG
[...]</td>
</tr></tbody>
</table>
Note that the value of the query strings parameters <code>json</code> and
<code>png</code> are ignored, so we just don't pass a value at all.
<code>GET /foo?png</code> is thus equivalent to
<code>GET /foo?png=ignored</code>.
<p>
The rest of this document explains the behavior of each HTTP end point.
<h3 id="/">/</h3>
This returns the home page for the TSD's UI (User Interface). All query
string parameters are ignored.
<h3 id="/aggregators">/aggregators</h3>
Returns the list of available aggregation functions, in JSON.
<div class="code"><i>GET /aggregators</i>
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 25
["min","sum","max","avg"]
</div>
<h3 id="/diediedie">/diediedie</h3>
Accessing this end point causes the TSD to perform a graceful shutdown and
exit. A graceful shutdown prevents data loss by flushing all the buffered
edits to HBase before exiting.
<h3 id="/dropcaches">/dropcaches</h3>
Accessing this endpoint causes the TSD to drop all it's in-memory caches.
<h3 id="/logs">/logs</h3>
Returns the last N messages logged by the TSD internally, most recent messages
first. Can also be used to change the logging level at runtime (see below).
The default output is in plain text, each field is tab delimited:
<div class="code"><i>GET /logs</i>
HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 245715
1297563030 INFO New I/O server worker #1-1 net.opentsdb.core.TsdbQuery TsdbQuery(start_time=1297559430, end_time=1297563030, <i>[...]</i>
1297563030 INFO New I/O server worker #1-2 net.opentsdb.graph.Plot Wrote Gnuplot script to /tmp/tsd-read/04/81/DE2CDAE94659.gnuplot
1297563020 INFO New I/O server worker #1-2 net.opentsdb.tsd.HttpQuery [id: 0x5ef114fa, /127.0.0.1:40362 => /127.0.0.1:4243] HTTP /q?<i>[...]</i>
<i>[...]</i>
</div>
Output is also available in JSON, where each line above is converted to a JSON
strings and all the strings are returned in an array.
<p>
The number of log messages returned depends on the configuration of the
cyclic buffer in
<a href="https://github.com/OpenTSDB/opentsdb/blob/master/src/logback.xml"><code>src/logback.xml</code></a>.
<p>
Passing a <code>level</code> query string parameter will instead change the
logging level of the TSD.
<div class="code"><i>GET /logs?level=debug</i>
HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 42
Set the log level to DEBUG on 51 loggers.
</div>
In addition, you can change the logging level of a single logger by specifying
its name (either the name of a package or the name of a class):
<div class="code"><i>GET /logs?level=debug&logger=org.hbase.async.RegionClient</i>
HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 40
Set the log level to DEBUG on 1 logger.
</div>
Valid
<a href="http://logback.qos.ch/apidocs/ch/qos/logback/classic/Level.html">levels</a>
are (from the most verbose to the least): <code>ALL TRACE DEBUG INFO WARN
ERROR OFF</code> (names are case insensitive).
<h3 id="/q">/q</h3>
This end point is used to query the time series database. It can produce
results in HTML, JSON, PNG or plain text.
<h4>/q Parameters</h4>
The <code>/q</code> end-point accepts a number of different query string
parameters. At least <code>start</code> and one <code>m</code> parameters are
required.
<table>
<thead>
<tr><th>Name</th><th>Definition</th></tr>
</thead>
<tbody>
<tr><td><code>start</code></td><td>The query's start date.</td></tr>
<tr><td><code>end</code></td><td>The query's end date.</td></tr>
<tr><td><code>m</code></td><td>The query itself.</td></tr>
<tr><td><code>o</code></td><td>Rendering options.</td></tr>
<tr><td><code>wxh</code></td><td>The dimensions of the graph.</td></tr>
<tr><td><code>yrange</code></td><td>The range of the left Y axis.</td></tr>
<tr><td><code>y2range</code></td><td>The range of the right Y axis.</td></tr>
<tr><td><code>ylabel</code></td><td>Label for the left Y axis.</td></tr>
<tr><td><code>y2label</code></td><td>Label for the right Y axis.</td></tr>
<tr><td><code>yformat</code></td><td>Format string for the left Y axis.</td></tr>
<tr><td><code>y2format</code></td><td>Format string for the right Y axis.</td></tr>
<tr><td><code>ylog</code></td><td>Enables log scale for the left Y axis.</td></tr>
<tr><td><code>y2log</code></td><td>Enables log scale for the right Y axis.</td></tr>
<tr><td><code>key</code></td><td>Options for the key (legend) of the graph.</td></tr>
<tr><td><code>nokey</code></td><td>Removes the key (legend) from the graph.</td></tr>
<tr><td><code>nocache</code></td><td>Forces TSD to ignore cache and fetch results from HBase.</td></tr>
</tbody>
</table>
<p>
Dates are always expected to be in one of two forms: absolute (e.g.
<code>end=2011/02/12-19:42:51</code>) or relative (e.g.
<code>start=2h-ago</code>). Dates are always assumed to be in the local time
of the TSD. Relative dates can use the suffix <code>s</code> for seconds,
<code>m</code> for minutes, <code>h</code> for hours, <code>d</code> for days,
<code>w</code> for weeks, <code>y</code> for years. If no <code>end</code>
date is specified, the TSD will use the current time as the end date.
<p>
Queries are expressed in the <code>m</code> parameters. There can be multiple
<code>m</code> parameters. Each <code>m</code> parameter is made of multiple
components, separated by colons (:). Here are some examples and explanations
of well formed <code>m</code> parameters:
<table>
<thead>
<tr><th>Query</th><th>Meaning</th></tr>
</thead>
<tbody>
<tr>
<td><code>m=sum:proc.stat.cpu</code></td>
<td>Sum the value of <code>proc.stat.cpu</code> across all time series for
this metric.</td>
</tr>
<tr>
<td><code>m=sum:rate:proc.stat.cpu</code></td>
<td>Sum the rate of change of <code>proc.stat.cpu</code> across all time
series for this metric.</td>
</tr>
<tr>
<td><code>m=sum:rate:proc.stat.cpu{host=foo}</code></td>
<td>Sum the rate of change of <code>proc.stat.cpu</code> across all time
series for this metric but only for the host <code>foo</code>.</td>
</tr>
<tr>
<td><code>m=sum:rate:proc.stat.cpu{host=foo,type=idle}</code></td>
<td>Idle CPU cycles for host <code>foo</code>.</td>
</tr>
<tr>
<td><code>m=sum:rate:proc.stat.cpu{host=foo,type=user|system}</code></td>
<td>User and system CPU cycles for host <code>foo</code>.</td>
</tr>
<tr>
<td><code>m=sum:rate:proc.stat.cpu{host=foo,type=*}</code></td>
<td>Show how CPU cycles are spent on host <code>foo</code>. Note that
<code>type=*</code> is roughly equivalent to <code>GROUP BY type</code>
in SQL.</td>
</tr>
<tr>
<td><code>m=avg:rate:proc.stat.cpu{type=idle}</code></td>
<td>Average number of idle CPU cycles across the board.</td>
</tr>
<tr>
<td><code>m=avg:10m-avg:rate:proc.stat.cpu{type=idle}</code></td>
<td>Downsample the datapoints using the 10-minute average, and show the average number of idle CPU cycles across the
board.</td> </tr>
</tbody>
</tbody>
</table>
<p>
As you can infer from the examples above, a query always has at least an
aggregation function first, and a metric name last. There's optional qualifiers
before the metric to indicate that a metric is a rate or to do downsampling of
datapoints. Tag queries can be optionally
specified in curly braces after the name of the metric. The overall syntax
is (square brackets indicate an optional part):
<pre>AGG:[interval-DOWNSAMPLE:][rate:]metric[{tag1=value1[,tag2=value2...]}]</pre>
<p>
The <code>o</code> parameter is optional and is used to pass options to
Gnuplot to alter the way it renders each line on the graph. It is typically
used to move a line on the right axis (<code>o=axis x1y2</code>). If an
<code>o</code> parameter is specified, then each <code>m</code> parameter must
be followed by its <code>o</code> parameter.
<p>
The <code>wxh</code> parameter specifies the dimensions of the graph, e.g.
<code>wxh=640x480</code> for an image of 640 pixels large and 480 pixels high.
<h4>/q Output formats</h4>
The default output format is HTML. Right now it is not very useful, it simply
generates a page that fetches the graph as an image. The image is served when
the <code>png</code> query string parameter is present.
<p>
The plain text output format is produced when the <code>ascii</code> query
string parameter is specified. This format is the same as the one used during
data imports:
<div class="code"><i>GET /q?start=30s-ago&m=sum:rate:proc.stat.cpu{host=foo,type=*}&ascii</i>
HTTP/1.1 200 OK
Content-Type: text/plain
Cache-Control: no-cache
Content-Length: 23094
proc.stat.cpu 1297574486 54.2 host=foo type=user
proc.stat.cpu 1297574501 39.6 host=foo type=user
<i>[...]</i>
proc.stat.cpu 1297574486 2.66 host=foo type=nice
proc.stat.cpu 1297574501 0.06 host=foo type=nice
<i>[...]</i>
proc.stat.cpu 1297574486 5.0 host=foo type=system
proc.stat.cpu 1297574501 1.6 host=foo type=system
<i>[...]</i>
</div>
The TSD may return data for a larger time interval than the one requested.
This is done in order to help third-party graphing tools to properly plot
values at the edges of the graph.
<p>
Note that in this example, the <code>Cache-Control</code> header asks clients
to not cache the response. This is because we requested the last 30 seconds
of data, and this data is too fresh to be cached (collectors are still in the
process of updating it).
<p>
The JSON output contains meta-data instead of the actual results of the query.
This is primarily for the benefit of AJAXy UIs, such as the one of the TSD
itself.
<div class="code"><i>GET /q?start=30s-ago&m=sum:rate:proc.stat.cpu{host=foo}&json</i>
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 58
{"plotted":2,"points":387,"etags":[["type"]],"timing":109}
</div>
Typically a UI makes a request to the <code>&json</code> URL first, and
in case of success then requests the same URL again but with
<code>&png</code> instead to fetch the graph. The graph is actually
generated durnig the first request, the second one will simply hit the cache.
The meta-data in the example above means:
<ul>
<li>Only 2 data points ended up being plotted on the graph.</li>
<li>The TSD processed 387 data points. The most likely reason as to why only
2 data points ended up on the graph is that they were aggregated together by
sum.</li>
<li>The query is implicitely aggregating a tag, <code>type</code>.</li>
<li>The TSD self-timed the query and claims that it took 109 milliseconds to
run (including the time to generate the graph).</li>
</ul>
<h3 id="/s">/s</h3>
Serves static files, such as JavaScript generated by the GWT compiler or
<code>favicon.ico</code>. The TSD needs a <code>--staticroot</code> argument
to start. This argument is the path to a directory that contains the files
served by this end point.
<p>
When a request for <code>GET /s/queryui.nocache.js</code> comes in, for
instance, the file <code>${staticroot}/queryui.nocache.js</code> is sent to
the browser.
<p>
Note: The TSD will allow clients to cache static files for 1 year by default,
and will report the age of the file on disk. If the file name contains
<code>nocache</code>, then the TSD will tell clients to not cache the file
(this idiom is used by GWT).
<h3 id="/stats">/stats</h3>
Serves self-reported stats about the TSD itself. The default format is
<code>text/plain</code>:
<div class="code"><i>GET /stats</i>
HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 2796
tsd.connectionmgr.connections 1297547003 3388 host=foo
tsd.connectionmgr.exceptions 1297547003 372 host=foo
tsd.rpc.received 1297547003 2027270 type=telnet host=foo
tsd.rpc.received 1297547003 124758 type=http host=foo
tsd.rpc.exceptions 1297547003 16 host=foo
<i>[...]</i>
</div>
Note that the format produced is suitable to be imported directly into
OpenTSDB. Alternatively, the stats can be retrieved as a JSON array:
<div class="code"><i>GET /stats?json</i>
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 2887
["tsd.connectionmgr.connections 1297547030 3388 host=foo",
"tsd.connectionmgr.exceptions 1297547030 372 host=foo",
"tsd.rpc.received 1297547030 2027514 type=telnet host=foo",
"tsd.rpc.received 1297547030 124763 type=http host=foo",
"tsd.rpc.exceptions 1297547030 16 host=foo", <i>[...]</i>]
</div>
<h4 id="self-metrics">Self-reported metrics</h4>
<ul>
<li><code>tsd.compaction.count</code>: Number of rows
<a href="schema.html#reduced-overhead">compacted</a> so far.</li>
<li><code>tsd.connectionmgr.connections</code>: Number of TCP connections TSD
accepted.</li>
<li><code>tsd.connectionmgr.exceptions</code>: Number of uncaught exceptions
that forced TSD to close an accepted connection.</li>
<li><code>tsd.rpc.received</code>: Number of RPCs received.</li>
<li><code>tsd.rpc.exceptions</code>: Number of exceptions caught while serving
RPCs.</li>
<li><code>tsd.http.graph.requests</code>: Number of graphs served.</li>
<li><code>tsd.query.count</code>: Number of read requests served.</li>
<li><code>tsd.rpc.errors</code>: Number of errors during write requests.</li>
<li><code>tsd.uid.cache-hit</code>: Number of cache hits in the in-memory
UID cache.</li>
<li><code>tsd.uid.cache-miss</code>: Number of cache misses in the in-memory
UID cache.</li>
<li><code>tsd.uid.cache-size</code>: Number of entries in the in-memory UID
cache.</li>
<li><code>tsd.jvm.ramfree</code>: Number of bytes reported as free by the
JVM's <code>Runtime.freeMemory()</code>.</li>
<li><code>tsd.jvm.ramused</code>: Number of bytes reported as used by the
JVM's <code>Runtime.totalMemory()</code>.</li>
<li><code>tsd.hbase.root_lookups</code>: Number of times a lookup had to be
performed in the <code>-ROOT-</code> table.</li>
<li><code>tsd.hbase.meta_lookups</code>: Number of times a lookup had to be
performed in the <code>.META.</code> table.</li>
</ul>
<h3 id="/suggest">/suggest</h3>
Provides suggestions for metric names, tag names, tag values. Results are
served in JSON, as a single list.
<p>
This end point requires two query string parameters:
<ul>
<li><code>type</code>, the type of suggestion to provide.
The type parameter must have one of these three values:
<ul>
<li><code>metrics</code>: Provide suggestions for metric names.</li>
<li><code>tagk</code>: Provide suggestions for tag names.</li>
<li><code>tagv</code>: Provide suggestions for tag values.</li>
</ul></li>
<li><code>q</code>, a search string base the suggestions on.</li>
</ul>
</p>
<p>
Optionally, this endpoint may also take the following parameter:
<ul>
<li><code>max</code>, the maximum number of suggestions to return.
Defaults to 25.</li>
</ul>
</p>
<div class="code"><i>GET /suggest?type=metrics&q=df</i>
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 111
["df.1kblocks.free","df.1kblocks.total","df.1kblocks.used","df.inodes.free","df.inodes.total","df.inodes.used"]
</div>
<h3 id="/version">/version</h3>
Returns the version string of the TSD, in plain text by default.
<div class="code"><i>GET /version</i>
HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 115
net.opentsdb built at revision 17a68a2 (MODIFIED)
Built on 2011/02/07 06:18:06 +0000 by tsuna@foo:/opt/tsdb-read
</div>
JSON output is also supported:
<div class="code"><i>GET /version?json</i>
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 189
{"short_revision":"17a68a2","full_revision":"17a68a247e7bdd1d57cfdd39e99da5011d788800",
"timestamp":1297059486,"repo_status":"MODIFIED","user":"tsuna","host":"foo", "repo":"/opt/tsdb-read"}
</div>
For more information regarding the version string or the build data available,
refer to the javadoc of
<a href="http://tsunanet.net/~tsuna/opentsdb/api/net/opentsdb/BuildData.html"><code>net.opentsdb.BuildData</code></a>.
</section>
<footer>
© 2010–2013 The OpenTSDB Authors.
</footer>
</section></body></html>