atom.xml

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">

  <title><![CDATA[csfam]]></title>
  <link href="http://csfam.github.com/atom.xml" rel="self"/>
  <link href="http://csfam.github.com/"/>
  <updated>2012-04-09T01:49:11-04:00</updated>
  <id>http://csfam.github.com/</id>
  <author>
    <name><![CDATA[csfam]]></name>
    
  </author>
  <generator uri="http://octopress.org/">Octopress</generator>

  
  <entry>
    <title type="html"><![CDATA[Cross-document messaging saves the day]]></title>
    <link href="http://csfam.github.com/blog/2012/04/08/cross-document-messaging-saves-the-day/"/>
    <updated>2012-04-08T07:51:00-04:00</updated>
    <id>http://csfam.github.com/blog/2012/04/08/cross-document-messaging-saves-the-day</id>
    <content type="html"><![CDATA[<p>We&#8217;ve been distracted and haven&#8217;t posted here in a while, but I wanted to share something I had to wrestle with: cross-domain ajax uploads. So what&#8217;s the problem?</p>

<p>Well, we&#8217;ve divided the system into two separate parts: a frontend and a RESTful API (maybe a subject for a future post but in the meantime <a href="https://plus.google.com/112678702228711889851/posts/eVeouesvaVX">check this out</a>). These two parts sit on different domains. This caused me endless headaches when trying to get an image upload mechanism working.</p>

<p><img class="center" src="http://csfam.github.com/images/posts/arch.png" title="udunit general architecture" ></p>

<p>The first problem I had was getting an ajax file upload to work. The gist is that file uploads via XMLHttpRequests aren&#8217;t allowed (for security reasons). One workaround involves using an iframe as the upload target, so the response from the POST request is loaded asynchronously into the iframe, giving the impression of an ajax-y upload. (See <a href="http://www.alfajango.com/blog/ajax-file-uploads-with-the-iframe-method/">this post</a> for a much more in-depth discussion of this method.) That wasn&#8217;t terribly painful actually.</p>

<p>But at this point there are two domains involved: the frontend domain of the main page where we uploaded from and the API domain of the iframe that we uploaded to. I needed to access the response data within the iframe in order to determine whether the upload was successful and to obtain the uploaded image&#8217;s URL. However trying to access the data within the iframe resulted in a browser security exception (cross-domain). The problem is that browsers will not trust XMLHttpRequests&#8217; response data coming from a domain different than the page from which the request is made (hence cross-domain). Trying to access the data throws an error and returns no data.</p>

<p>How do we fix this? It took several days of banging my head against the wall and a great little website called <a href="http://caniuse.com">caniuse.com</a>, where you can discover all the cool features that web browsers implement, are implementing and should be implementing. Enter cross-document messaging.</p>

<p>Cross-document messaging (documented quite well on the <a href="https://developer.mozilla.org/en/DOM/window.postMessage">Mozilla Developer Network</a>) allows you to post a message from one DOM window to another DOM window. The message that is sent can be any arbitrary data. Thus we can send the response data from the iframe (which has its own DOM window) to the parent or container window. The only question I ran into was how to trigger the iframe to fire off the <code>window.postMessage</code> function after the response from the upload was returned. The solution I came up with was to actually return an HTML page as the upload response; the page contains a section of javascript that executes as soon as the page is loaded (in the invisible iframe). The javascript calls the <code>window.postMessage</code> function with the target window being the parent window and the data being the metadata from the upload response. Definitely a hack but it gets the job done &#8211; the returned HTML looks something like this:</p>

<pre><code>&lt;html&gt;
  &lt;body&gt;
    &lt;script type='text/javascript'&gt;
      window.parent.postMessage({ 'data': 'response data' },'targetWindow');
    &lt;/script&gt;
  &lt;/body&gt;
&lt;/html&gt;
</code></pre>

<p>where the <code>{ 'data': 'response data' }</code> represents the actual data we wanted to return in the response and <code>targetWindow</code> is the parent window. Then you just need to register an event listener for the &#8220;message&#8221; event on the parent window (since <code>window.postMessage</code> triggers this event), and the data will be passed to the event handler, therefore allowing you to pass the upload response data from the iframe to the parent window. This allows the upload to be performed asynchronously (ajax-y) and to the upload service sitting on another domain. And ta-da, thanks to cross-document messaging we have image upload working just as we wanted it.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[The Tech Behind Udunit]]></title>
    <link href="http://csfam.github.com/blog/2012/01/25/the-tech-behind-udunit/"/>
    <updated>2012-01-25T18:10:00-05:00</updated>
    <id>http://csfam.github.com/blog/2012/01/25/the-tech-behind-udunit</id>
    <content type="html"><![CDATA[<p>One of the first problems we encountered in building Udunit was choosing the technologies we wanted to use.  Given three people with relatively strong opinions, agreeing on technologies to use was no easy task.  Luckily, after discussing at length our various options, we were able to persuade one another to a more or less unanimous conclusion.</p>

<h3>Hosting</h3>

<p>We were all leaning towards <a href="http://www.heroku.com">Heroku</a> but spent some time discussing the benefits of <a href="http://aws.amazon.com/ec2">Amazon EC2</a>.  We liked the idea of having the freedom and control over our servers, but ultimately wanted something that was easy to deploy to and required little maintenance.  Heroku gave us the most value for the least amount of work so that was that.</p>

<h3>Database</h3>

<p>After watching one of <a href="http://www.10gen.com/presentations/mongonyc-2011/foursquare">Foursquare&#8217;s tech talks</a>, we became enamored with <a href="http://www.mongodb.org">Mongodb</a> and wanted to use it in Udunit.  The parts that particularly appealed to us were the JSON style storage, auto-scaling/sharding and full index support.  We were a little wary about using NoSQL, considering all of us were more used to relational databases like MySQL, but we were excited about the possibilities.  Plus, despite our familiarity with MySQL, we were also frustrated with the hassle in scaling, the restriction in schema design and the performance.</p>

<h3>API Framework</h3>

<p>A few of us also pushed for using <a href="http://liftweb.net/">Lift/Scala</a> for our API framework (hint: Foursquare <a href="https://docs.google.com/present/view?id=dcbpz3ck_25czcns2c2&amp;revision=_latest&amp;start=0&amp;theme=blank&amp;cwj=true">uses</a> it).  Honestly, the biggest selling point here for us was the novelty of using a new language that reduces the verbosity of Java and encourages functional programming.</p>

<h3>Web Framework</h3>

<p>We had a variety of options here.  A couple of us have used Django, GWT and Struts before so we had some stronger opinions (one of which was absolutely no GWT).  However, considering we were using Lift for our API framework, we ultimately decided that Lift on the frontend would accelerate our learning curve and also simplify the number of technologies we had to learn.  We were also intrigued by the idea behind Snippets which seemed like a completely new way of thinking about frontend design.</p>

<h3>Conclusions thus far..</h3>

<p>We&#8217;ve been using these technologies for a couple weeks now.  Scala has proven to be an extraodinary language with a bit of a painful learning curve.  But, so far, no complaints.  We&#8217;re excited to be learning these technologies and having fun while doing it!</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Measuring Heroku's concurrent request performance for long-running requests]]></title>
    <link href="http://csfam.github.com/blog/2012/01/17/measuring-herokus-concurrent-request-performance/"/>
    <updated>2012-01-17T00:32:00-05:00</updated>
    <id>http://csfam.github.com/blog/2012/01/17/measuring-herokus-concurrent-request-performance</id>
    <content type="html"><![CDATA[<p>I&#8217;ve been meaning to write some posts here for a while, and I&#8217;m glad I&#8217;m finally getting around to it. The first thing I wanted to write about is some testing we did a couple months back with the number of concurrent user requests that could be handled by <a href="http://www.heroku.com">Heroku</a> using a variety of backend technologies. But first off, in case you&#8217;ve managed to miss it &#8211; what is Heroku, why are we using it, and how does it work?</p>

<ul>
<li>It is a cloud-based application platform with built-in scalability and management tools.</li>
<li>It makes things dead simple from deployment to scaling up.</li>
<li>It&#8217;s free as long as you stay under a monthly traffic limit, which is perfect if you&#8217;re just starting up your idea.</li>
<li>Essentially dynos handle web requests within the Heroku architecture, and scaling up your application is just a matter of increasing the number of dynos you are using. Piece of cake.</li>
</ul>


<p>One of the other nice things about Heroku is it provides developers with a good amount of freedom in terms of what languages and frameworks can be used. Heroku will support JVM-based languages (Java, Scala, etc.), Ruby, Node.js, and Python. We were considering a number of options for the language and framework to use for our latest project, <a href="http://demo.udunit.com">udunit</a>, and we had narrowed our list down to <a href="http://scala-lang.org">Scala</a> and the <a href="http://liftweb.net/">Lift framework</a>, <a href="http://www.python.org">Python</a> and the <a href="https://www.djangoproject.com/">Django framework</a>, or <a href="http://nodejs.org">Node.js</a> and the <a href="http://expressjs.com">Express framework</a>.</p>

<h2>The Tests</h2>

<p>We set up very simple Heroku projects &#8211; one for each of the three frameworks mentioned above &#8211; that would result in a long-running request (i.e. the server was &#8220;busy&#8221; generating the response for ~2 seconds). Using the <a href="http://httpd.apache.org/docs/2.4/programs/ab.html">Apache Bench</a> tool, we were able to do some load testing of the various frameworks running on Heroku. What we really wanted to test was how well a single Heroku dyno would handle concurrent requests. So we ran tests via Apache Bench with increasingly more concurrent requests: 1, 2, 3, 4,5, 10, 20. The tests each hit a page which would trigger a long-running request. The tests were executed with a command like</p>

<pre><code> ab -n 200 -c 20
</code></pre>

<p>for 200 total requests with 20 requests at a time. Apache Bench provides data on the requests sent, such as the average number of requests handled per second (see graph below) or the min/max time taken to handle a single request. Using this data we were able to get an idea of what one Heroku dyno could handle.</p>

<h2>The Results</h2>

<p>Here we have a chart showing the number of requests handled per second for each of the three frameworks. Everyone was able to keep up while the number of concurrent requests was small; at 5 concurrent requests, all the frameworks were handling ~2.5 requests per second &#8211; so pretty close to ideal (5 requests / 2 second long-running request = 2.5 requests per second).</p>

<script type="text/javascript" src="http://csfam.github.com//ajax.googleapis.com/ajax/static/modules/gviz/1.0/chart.js"> {"dataSourceUrl":"//docs.google.com/spreadsheet/tq?key=0AlrawxK1noHLdExqRUhLWFp6YXkwaFVyOENseE1TZkE&transpose=0&headers=1&merge=COLS&range=B2%3AB9%2CH2%3AH9%2CH14%3AH21%2CH26%3AH33%2CI2%3AI9&gid=0&pub=1","options":{"vAxes":[{"title":"requests per second handled","minValue":null,"viewWindowMode":"pretty","viewWindow":{"min":null,"max":null},"maxValue":null},{"viewWindowMode":"pretty","viewWindow":{}}],"series":{"1":{"color":"#109618"},"3":{"color":"#dc3912"}},"booleanRole":"certainty","title":"Concurrent requests handled via Heroku","animation":{"duration":500},"vAxis":{"format":""},"useFirstColumnAsDomain":true,"hAxis":{"title":"concurrent requests","format":""},"isStacked":false,"width":600,"height":371},"state":{},"view":{"columns":[{"calc":"stringify","type":"string","sourceColumn":0},1,2,3,4]},"chartType":"ColumnChart","chartName":"Chart 1"} </script>


<p>When the load started getting higher, performance began dropping off (mainly in the python-django setup, somewhat in the scala-lift setup, hardly at all for node.js and express). Moving up to 50 concurrent requests, both Django and Lift became hardly useable, though surprisingly Express maintained a very high level of performance. Node.js and Express were the clear winners here, though it should be noted that we were going with the default configuration for Lift and Django. With better tuning of the server configuration, we could probably get higher performance from these two frameworks.</p>

<p>In the end we decided to go with the Lift framework. It&#8217;s performance was still strong and increasing system responsiveness while under high load is just a matter of increasing the number of Heroku dynos being used. Lift also provides a much fuller and more powerful framework than Express does. In the end Lift&#8217;s strong performance and rich set of features won us over. But I&#8217;m definitely going to push for some Node.js projects in the future ;)</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[in the beginning]]></title>
    <link href="http://csfam.github.com/blog/2012/01/05/in-the-beginning/"/>
    <updated>2012-01-05T23:53:00-05:00</updated>
    <id>http://csfam.github.com/blog/2012/01/05/in-the-beginning</id>
    <content type="html"><![CDATA[<p>There was this group of people, who got to know each other in the basement of a computer lab.  They were a talented bunch, during their time in the lab they built interesting projects and had fun working with each other.  They made a good team.  Then the good times were over and they had to graduate.  They all went their separate ways, but many of them later, during cold rainy nights sitting in front of computers in cubicles working on other people&#8217;s work, would remember those good times and wish they could be part of something like that sometime again.</p>

<p>Many years later, they get laid-off from their jobs at the big corporations to be replaced by new dream-filled youth &#8211; youth just as ready to give up their dreams, creativity and energy in exchange for job security.   As they leave the building and watch the new hires take their places they wonder if it could have been different.</p>

<p>…</p>

<p>Wait a sec!  It can be different!  We are all talented, energetic, and good at what we do!  We are creative.  We love applying ourselves to new projects.  There is no reason not to harness this power!  We are powerful!  We love cs and graphics and making things.  We are scientists and inventors, artists and artisans, and we are funny and fun.  We play flash games and we waste time on tv shows, and we create and deliver.</p>

<p>CSFAM &#8211; is the chance for all of us to do what we want, what we&#8217;re interested in, to create the work that we love with the people we love working with.</p>
]]></content>
  </entry>
  
</feed>