Implement NNA as a NameNode ServicePlugin

[pjeli]

This is a massive sweeping issue that involves re-architecting NNA to allow it to run on top of a real Standby NameNode as a ServicePlugin implementation.

Documentation available here: https://docs.google.com/document/d/1nlF7OrhGTqCFjSyAx3qf6sG0bXgzpvzS5jMO0IO-bKs/edit?usp=sharing

[pjeli]

So I have done some experimenting with NNA as a NameNode ServicePlugin and have gotten it to boot on a Standby NameNode as one. Of course with an NNA jar required in the classpath to get it to boot too. But the jar had to have SparkJava and the other dependencies in it too.

I think the approach I am going to take is to refactor and try to minimize dependencies -- possibly even removing SparkJava in favor for NameNode's "HttpServer2" / Jetty.

But at least we know this is possible.

[pjeli]

So I actually have this done right now. Uploading a patch here for anyone using the trunk of Apache Hadoop. I will continue to massage the current iteration of NNA until I can make it simple enough for Apache to accept.
NNA_Apache.000.patch.txt

[pjeli]

Made even more progress here but bumping up against a major issue. It seems we will be unable to support running NNA on Jersey 1.9 (the HTTP server used by Hadoop 2.x) due to its dependency on asm 3.x which is incompatible with Java 8.

We CAN however, support NNA in Hadoop 3.x where the asm dependency has been updated to support Java 8.

We cannot try to downgrade to Java 7 in NNA though because a lot of the magic that makes NNA happen is in the lamdas and Stream API usage that is provided by 8. It would be a major effort to try to create the same functionality in 7 just for NNA's sake...

So I think the way to proceed here is going to be that NNA as Plugin will come when we drop support for all Hadoop 2.x builds and move onto Hadoop 3.x only builds.

[pjeli]

Attaching new patch for NNA Apache migration.
NNA_Apache.001.patch.txt

[pjeli]

The work involved for this is mostly done and in the master branch today. You can find it as NamenodeAnalyticsMethods.java. All that is missing is the NameNodePlugin piece and some bridging between what is GitHub NNA today and what will be Apache HDFS NNA later.