Skip to content

Commit

Permalink
Merge pull request #13 from haoch/master
Browse files Browse the repository at this point in the history
Added Social Network Information of Eagle
  • Loading branch information
armanoharan committed Oct 27, 2015
2 parents 9a06d8a + 72a2af3 commit 4fc1bde
Show file tree
Hide file tree
Showing 26 changed files with 788 additions and 48 deletions.
13 changes: 5 additions & 8 deletions _config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ url: "http://goeagle.io" # the base hostname & protocol for your site

# Build settings
markdown: kramdown
permalink: date

documentations:
- category: Getting Started
Expand Down Expand Up @@ -50,11 +51,7 @@ documentations:
url: /docs/tutorial/classification.html
- category: Advanced
links:
- title: Policy API Design
url: /docs/alert-api.html
- title: Alert API Design
url: /docs/alert-api.html
- title: Query API Design
url: /docs/userprofile.html
- title: User Profile Design
url: /docs/userprofile.html
- title: Eagle Architecture Highlights
url: /docs/architecture-highlights.html
- title: User Profile Machine Learning
url: /docs/user-profile-ml.html
3 changes: 3 additions & 0 deletions _includes/head.html
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@

<link rel="stylesheet" href="{{ "/css/animate.css" | prepend: site.baseurl }}">
<link rel="stylesheet" href="{{ "/css/bootstrap.min.css" | prepend: site.baseurl }}">

<link rel="stylesheet" href="{{ "/css/font-awesome.min.css" | prepend: site.baseurl }}">

<link rel="stylesheet" href="{{ "/css/misc.css" | prepend: site.baseurl }}">
<link rel="stylesheet" href="{{ "/css/style.css" | prepend: site.baseurl }}">
<link rel="stylesheet" href="{{ "/css/styles.css" | prepend: site.baseurl }}">
Expand Down
13 changes: 7 additions & 6 deletions _includes/header.html
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,16 @@
<!-- Collect the nav links, forms, and other content for toggling -->
<!-- <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
<ul class="nav navbar-nav navbar-right" id="top-menu">
<li><a class="menu" href="{{ "/#home_page" | prepend: site.baseurl }}">HOME</a></li> -->
<!-- <li><a class="menu" href="{{ "/docs/" | prepend: site.baseurl }}">DOCS</a></li> -->
<!-- <li><a class="menu" href="{{ "/#about_page" | prepend: site.baseurl }}">ABOUT</a></li>
<li><a class="menu" href="{{ "/#home_page" | prepend: site.baseurl }}">HOME</a></li>
<li><a class="menu" href="{{ "/docs/" | prepend: site.baseurl }}">DOCS</a></li>
<li><a class="menu" href="{{ "/#about_page" | prepend: site.baseurl }}">ABOUT</a></li>
<li><a class="menu" href="{{ "/#diagram_page" | prepend: site.baseurl }}">ARCHITECTURE</a></li>
<li><a class="menu" href="{{ "/#modules_page" | prepend: site.baseurl }}">MODULES</a></li>
<li><a class="menu" href="{{ "/#usecase_page" | prepend: site.baseurl }}">USE CASES</a></li>
</ul>
</div> -->
<!-- /.navbar-collapse -->
<li>
</li>
</ul> -->
</div>
</div>
<!-- /.container-fluid -->
</nav>
Expand Down
7 changes: 7 additions & 0 deletions _layouts/page.html
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,13 @@
</div>
<div class="hometext">
<h2>Secure Hadoop in Real Time</h2>
<div class="social-buttons">
<!-- <a href="https://github.com/ebay/eagle"><i class="fa fa-github"></i></a>
<a href="http://twitter.com/TheApacheEagle"><i class="fa fa-twitter"></i></a>
<a href="https://www.facebook.com/TheApacheEagle/"><i class="fa fa-facebook"></i></a>
<a href="#"><i class="fa fa-weixin"></i></a> -->
<!-- <a href="https://www.weibo.com/ApacheEagle/"><i class="fa fa-weibo"></i></a> -->
</div>
</div>
</div>
</div>
Expand Down
2 changes: 1 addition & 1 deletion _posts/2015-09-23-welcome-to-jekyll.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: post
title: "Eagle"
date: 2015-09-23 19:24:33
categories: documentation
categories: post
---

You’ll find this post in your `_posts` directory. Go ahead and edit it and re-build the site to see your changes. You can rebuild the site in many different ways, but the most common way is to run `jekyll serve`, which launches a web server and auto-regenerates your site when a file is updated.
Expand Down
29 changes: 29 additions & 0 deletions architecture-highlights.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
layout: doc
title: "Eagle Architecture Highlights"
permalink: /docs/architecture-highlights.html
---

> Eagle is an open-source Data Activity Monitoring solution for Hadoop to instantly detect access to sensitive data or malicious activities, and to take appropriate actions.
![](/images/docs/eagle-highlights.png)

### Data Collection and Storage

Eagle provides programming API for extending Eagle to integrate any data source into Eagle policy evaluation framework. For example, Eagle hdfs audit monitoring collects data from Kafka which is populated from namenode log4j appender or from logstash agent. Eagle hive monitoring collects hive query logs from running job through YARN API, which is designed to be scalable and fault-tolerant.

Eagle uses HBase as storage for storing metadata and metrics data, and also supports relational database through configuration change.

### Data Processing

**Stream processing API**: Eagle provides a stream processing API which is an abstraction on Apache Storm, but is also extensible to other streaming engines. This abstraction allows developers to easily assemble data transformation, filtering, external data join, etc. without being physically bound to a specific streaming platform. The Eagle streaming API also allows developers to easily integrate business logic with the Eagle policy engine. Internally, the Eagle framework compiles business logic execution DAG into program primitives of the underlying stream infrastructure—for example, Apache Storm.

**Alerting framework**: The Eagle alerting framework includes a stream metadata API, a policy engine provider API for extensibility, and a policy partitioner interface for scalability.

**Machine-learning module**: Eagle provides capabilities to define user activity patterns or user profiles for Hadoop users based on the user behavior in the platform. The idea is to provide anomaly detection capability without setting hard thresholds in the system. The user profiles generated by our system are modeled using machine-learning algorithms and used for detection of anomalous user activities, where users’ activity pattern differs from their pattern history. Currently Eagle uses two algorithms for anomaly detection: Eigen-Value Decomposition and Density Estimation. The algorithms read data from HDFS audit logs, slice and dice data, and generate models for each user in the system. Once models are generated, Eagle uses the Storm framework for near-real-time anomaly detection to determine if current user activities are suspicious or not with respect to their model. The block diagram below shows the current pipeline for user profile training and online detection.

### Eagle Service
**Policy Manager**: Eagle Policy Manager provides a UI and Restful API for users to define policies. The Eagle user interface makes it easy to manage policies with a few clicks, mark or import sensitivity metadata, perform HDFS or Hive resource browsing, access alert dashboards, etc.

**Query Service**: Eagle provides a SQL-like service API to support comprehensive computation for huge sets of data—comprehensive filtering, aggregation, histogram, sorting, top, arithmetical expression, pagination, etc. Although Eagle supports HBase for data storage as a first-class citizen, a relational database is supported as well. For HBase storage, the Eagle query framework compiles a user-provided SQL-like query into HBase native filter objects, and then executes it through the HBase coprocessor on the fly.

10 changes: 6 additions & 4 deletions community.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ permalink: /docs/community.html

### Issue Tracker

[https://github.com/eBay/Eagle/issues](https://github.com/eBay/Eagle/issues)
`Coming soon ... `

_Currently please use [https://github.com/eBay/Eagle/issues](https://github.com/eBay/Eagle/issues) for issue tracking_
<br/>

### Events and Meetups
Expand All @@ -24,15 +26,15 @@ permalink: /docs/community.html

**Initial Commiters**

* [Adunuthula, Seshu](https://github.com/seshuad)
* [Arun Manoharan](https://github.com/armanoharan)
* [Edward Zhang (张勇)](https://github.com/yonzhang)
* [Hao Chen(陈浩)](https://github.com/haoch)
* [Chaitali Gupta](https://github.com/chaitaligupta)
* [Libin Sun(孙立斌)](https://github.com/sunlibin)
* [Jilin Jiang(将吉麟)](https://github.com/zombiej)
* [Qingwen Zhao(赵晴雯)](https://github.com/qingwen220)
* [Hemanth Dendukuri](https://github.com/hdendukuri)
* [Arun Manoharan](https://github.com/armanoharan)
* [Adunuthula, Seshu](https://github.com/seshuad)
<br/>

### Project History
Expand All @@ -41,4 +43,4 @@ permalink: /docs/community.html

### Credits

* Thanks [eBay Inc.](http://www.ebay.com) to donated this project to open source community, first announement at [eBay Techblog](#).
* Thanks [eBay Inc.](http://www.ebay.com) to donated this project to open source community, first announement at [eBay Techblog](http://www.ebaytechblog.com/2015/10/23/eagle-is-your-hadoop-data-secured/).
4 changes: 4 additions & 0 deletions css/font-awesome.min.css

Large diffs are not rendered by default.

13 changes: 13 additions & 0 deletions css/styles.css
Original file line number Diff line number Diff line change
Expand Up @@ -107,4 +107,17 @@ html[xmlns] .clearfix{display:block;}

.page-main-content h1{
text-align: left
}

.social-buttons > a{
color:white;
margin:auto 2px;
}

.social-buttons > a:hover{
color: #126acb;
}

.social-buttons > a > i{
font-size: 1.2em;
}
14 changes: 13 additions & 1 deletion ecosystem.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,16 @@ title: "Ecosystem"
permalink: /docs/ecosystem.html
---

What's the unique value of eagle in the ecosystem? `TODO`
### Eagle Core

The core of eagle is the distributed real-time policy framework, including highly abstracted Streaming Programing API, CEP-based Policy Execution Engine, SQL-Like REST API based on HBase, Alert Deduplication, Alert Notification Framework and so on.

### Extension

Eagle supports pluggable policy execution engined like siddhi, machine learning and more, could support any types of data sources.

### Integration
Eagle provides Ambari plugin for easily installation, can be easily intergrated into existing hadoop cluster.

### User Interface
Eagle provide highly user-friendly UI for policy management
Binary file added fonts/FontAwesome.otf
Binary file not shown.
Binary file added fonts/fontawesome-webfont.eot
Binary file not shown.
Loading

0 comments on commit 4fc1bde

Please sign in to comment.