YAPC::EU 2014 Lightning Talk

OHAI

Introduction to Hijk

Kang-min Liu @booking.com @gugod

Hijk… ?

Small, specialized HTTP client

• thin client
• pure perl. 200 line of code, 250 line of POD
• HTTP/1.1 server
  • persistent connection by default
• deal with read/connect client timeout

“Specialized”, meaning it does only one job.

Low-level API

my $res = Hijk::request({ method => “GET”, host => “example.com”, port => “80”, path => “/flower”, query_string => “color=red” });

if (exists $res->{error} and $res->{error} & Hijk::Error::TIMEOUT) { die “Oh noes we had some sort of timeout”; }

It does not handle these…

• no Location - 30x redirects
• no Transfer-Encoding
• no Proxy
• no SSL
• no Chunked encodng (but working in progress)
• no cookie jars

really, low-level api :)

ElasticSearch response

> curl -D - http://localhost:9200/

HTTP/1.1 200 OK Content-Type: application/json; charset=UTF-8 Content-Length: 300

{ “status” : 200, “name” : “Ravage 2099”, “version” : { “number” : “1.3.2”, “build_hash” : “dee175dbe2f254f3f26992f5d7591939aaefd12f”, “build_timestamp” : “2014-08-13T14:29:30Z”, “build_snapshot” : false, “lucene_version” : “4.9” }, “tagline” : “You Know, for Search” }

Use Cases

• ElasticSearch
  • used by Search::Elasticsearch distribution
• ShardedKV (key-value store interface)
• couchdb / mongodb … (only with very trivial test cases)
• HBase (after chunked encoding is )
• nginx (need some server config)

Bascially when you control the server.

Client timeouts (other)

eval { $ua->request(…); 1; } or do { my $error = $@ || “UNKNOWN”;

if ($error =~ /timeout/i) { … } };

Client timeouts (Hijk)

my $res = Hijk::request({…});

if (exists $res->{error} and $res->{error} & Hijk::Error::TIMEOUT) { die “Oh noes we had some sort of timeout”; }

Simple Benchmark Numbers

curl -XGET http://localhost:9200/

~/src/Hijk > perl examples/bench-elasticsearch.pl Rate lwp____ tiny___ hijk pp lwp____ 928/s – -77% -93% tiny___ 4016/s 333% – -69% hijk pp 13158/s 1318% 228% –

N* Other benchmark

From: http://www.martin-evans.me.uk/node/169

HTTP::Tiny keep_alive=1: 6 wallclock secs ( 2.95 usr + 0.59 sys = 3.54 CPU) @ 1412.43/s (n=5000) HTTP::Tiny keep_alive=0: 13 wallclock secs ( 7.04 usr + 2.51 sys = 9.55 CPU) @ 523.56/s (n=5000) LWP: 20 wallclock secs (14.35 usr + 2.71 sys = 17.06 CPU) @ 293.08/s (n=5000) LWPGHHTP: 13 wallclock secs ( 8.11 usr + 2.12 sys = 10.23 CPU) @ 488.76/s (n=5000) GHTTP: 6 wallclock secs ( 1.13 usr + 1.64 sys = 2.77 CPU) @ 1805.05/s (n=5000) curl: 4 wallclock secs ( 0.88 usr + 0.40 sys = 1.28 CPU) @ 3906.25/s (n=5000) furl: 6 wallclock secs ( 2.95 usr + 0.64 sys = 3.59 CPU) @ 1392.76/s (n=5000) hijk: 3 wallclock secs ( 0.75 usr + 0.20 sys = 0.95 CPU) @ 5263.16/s (n=5000)

Comment from a viking

“LWP is a fat pig”

– Ævar Arnfjörð Bjarmason

(but it is general purposes.)

Summary

• Hijk - Specialized HTTP Client in 200 loc
• Small and performant
• Talks to your data servers.
• @gugod
• https://github.com/gugod/Hijk
• https://metacpan.org/pod/Hijk

OKTHXBYE

Thank your for listening.

@gugod https://github.com/gugod/Hijk