Skip to content

Commit

Permalink
Update the documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
@alessandrogario
alessandrogario committed Jul 18, 2018
1 parent 97874ba commit 5d61f03
Show file tree
Hide file tree
Showing 6 changed files with 71 additions and 160 deletions.
13 changes: 5 additions & 8 deletions CMakeLists.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,6 @@ function(main)
message(STATUS "================================")
endfunction()

function(GetTableName descriptor)
string(FIND "${descriptor}" ":" table_name_separator)
string(SUBSTRING "${descriptor}" 0 ${table_name_separator} table_name)

set(GetTableName_OUTPUT "${table_name}" PARENT_SCOPE)
endfunction()

function(ImportLibraries)
message("Importing libraries from '${CMAKE_CURRENT_SOURCE_DIR}/libraries/${library}'")

Expand All @@ -64,6 +57,10 @@ function(ImportExtensions)

file(GLOB extension_list RELATIVE "${CMAKE_CURRENT_SOURCE_DIR}" "${CMAKE_CURRENT_SOURCE_DIR}/*")
foreach(extension ${extension_list})
if("${extension}" STREQUAL ".vscode")
continue()
endif()

set(full_extension_path "${CMAKE_CURRENT_SOURCE_DIR}/${extension}")
if(NOT IS_DIRECTORY "${full_extension_path}")
continue()
Expand All @@ -87,7 +84,7 @@ function(IsFolderEnabled relative_path)
set(platforms_file_path "${CMAKE_CURRENT_SOURCE_DIR}/${relative_path}/.platforms")

if(NOT EXISTS "${platforms_file_path}")
message(" ! Missing platform file: ${platforms_file_path}")
message(" ! Missing '.platforms` file: ${relative_path}. Skipping...")

set(IsFolderEnabled_OUTPUT false PARENT_SCOPE)
return()
Expand Down
77 changes: 58 additions & 19 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,50 +12,89 @@ This repository includes [osquery](https://osquery.io/) [extensions](https://osq
| ntfs_forensics | Provides osquery with NTFS-specific forensic information for incident responders. | Windows |
| (more to come) | ... | ... |

## Dependencies

##### All platforms

The full Boost package is required to build the extensions. Unfortunately the version provided with osquery does not come with all the components, and using a separate library will most likely introduce linking errors.

We have submitted the following PR to fix the issue: https://github.com/facebook/osquery/pull/4339

For the time being, it is best to cherry-pick the last 3 commits from that branch inside the master branch of the osquery repository; this will automatically take care of everything for Linux and macOS.

For Windows, the package must be rebuilt due to a bug in the binaries that have been uploaded to the S3 repository. It is best to work inside a folder in the root of the drive like `C:\Projects\osquery` because the script will generate many folders (otherwise you risk hitting the path size limit, which is not always captured by Chocolatey).

1. Run the following script once: `.\tools\make-win64-dev-env.bat`
2. Uninstall the boost-msvc14 package: `choco uninstall boost-msvc14`
3. Build the Boost package from scratch: `cd osquery`, `.\tools\provision\chocolatey\boost-msvc14.ps1`
4. Enter the folder where the package has been created (should be somewhere under `build\chocolatey`) and run: `choco install -s . .\boost-msvc14.1.66.0-r1.nupkg`

##### macOS
* macOS, user with sudo (to run the osquery build dependencies install script)
* Xcode
* openssl and curl (install from Homebrew: `brew install openssl curl`)

## Running the automated tests

Once osquery has been built with tests enabled (i.e.: *without* the SKIP_TESTS variable), enter the build/<platform_name> folder and run the following command: `make trailofbits_extensions_tests`. Note that tests are not supported on Windows.

## Building

1. Clone the osquery repository
2. Symlink the extensions you want to build into the external osquery directory. Use the following link name: "extension_\<name\>".
3. Build osquery
4. Run 'make externals'
2. Clone the osquery-extensions repository
3. Symlink the osquery-extensions folder to `osquery/externals/external_trailofbits`
4. Build osquery
5. Build the extensions

### Example
Here's an example

```
cd /src
git clone https://github.com/facebook/osquery.git
git clone https://github.com/trailofbits/osquery-extensions.git
cd /src/osquery-extensions
ln -s /src/osquery-extensions/efigy /src/osquery/external/extension_efigy
git clone https://github.com/facebook/osquery.git /src/osquery
git clone https://github.com/trailofbits/osquery-extensions.git /src/osquery-extensions
# Use mklink on Windows
cd /src/osquery
ln -s /src/osquery-extensions /src/osquery/external/extension_trailofbits
# On Windows, just run `.\tools\make-win64-dev-env.bat` from a PowerShell
# instance with Administrator privileges
make sysprep
make deps
make -j `nproc` # If using macOS, replace `nproc` with `sysctl -n hw.ncpu`
# On Windows, just run `.\tools\make-win64-binaries.bat` from a PowerShell
# instance with Administrator privileges
#
# If using macOS, replace `nproc` with `sysctl -n hw.ncpu`
make -j `nproc`
# On macOS and Linux make will usually also build the extension; on
# Windows you always have to do it manually
#
# For Windows run
# `cd build\windows10 && cmake --build . --config Release --target trailofbits_osquery_extensions`
make externals
# Run make again to ensure osquery recognizes the extension to build
make -j `nproc`
```

If you see the following warning, it can be ignored: `-- Cannot find Doxygen executable in path`

The extension should be in a subfolder of `/src/osquery/build` once the second make command completes successfully. Using `find . -name "efigy.ext"` can help you locate it quickly.
This is where the extension should be available once it has been built:
* Windows: `osquery/build/windows10/external/Release/trailofbits_osquery_extensions.ext.exe`
* Linux: `osquery/build/linux/external/trailofbits_osquery_extensions.ext`
* macOS: `osquery/build/darwin/external/trailofbits_osquery_extensions.ext`

## Usage

To quickly test an extension, you can either start it from the osqueryi shell, or launch it manually and wait for it to connect to the running osquery instance.
To quickly test the extension, you can either start it from the osqueryi shell, or launch it manually and wait for it to connect to the running osquery instance.

Consider either changing the ownership of `efigy.ext` to root or running osquery with the `--allow_unsafe` flag.
Consider either changing the ownership of `trailofbits_osquery_extensions.ext` to root or running osquery with the `--allow_unsafe` flag.

> osqueryi --extension /path/to/efigy.ext
> osqueryi --extension /path/to/trailofbits_osquery_extensions.ext
```
$ sudo osqueryi --extension osquery-facebook/build/darwin10.12/external/extension_efigy/efigy.ext
$ sudo osqueryi --extension osquery/build/darwin/external/trailofbits_osquery_extensions.ext
Using a virtual database. Need help, type '.help'
osquery> select * from efigy;
osquery> SELECT * FROM efigy;
+--------------------+-----------------+--------------------+-------------------+------------+---------------------+
| latest_efi_version | efi_version | efi_version_status | latest_os_version | os_version | build_number_status |
+--------------------+-----------------+--------------------+-------------------+------------+---------------------+
Expand Down
46 changes: 4 additions & 42 deletions efigy/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,56 +2,18 @@

[EFIgy](http://efigy.io/) is a service by Duo Labs that helps Apple Mac users determine if they are running the expected EFI firmware version given their Mac hardware and OS build version. This extension integrates osquery with the EFIgy API, so you can determine if all of the EFI firmware on your Mac fleet is up-to-date.

## Dependencies

Requirements:
* macOS, user with sudo (to run the osquery build dependencies install script)
* Xcode
* openssl and curl (install from Homebrew: `brew install openssl curl`)

## Building

1. Clone the osquery repository
2. Symlink this extension into the external osquery directory. Use the following link name: "extension_santa".
3. Build osquery
4. Run 'make externals'

```
cd /src
git clone https://github.com/facebook/osquery.git
git clone https://github.com/trailofbits/osquery-extensions.git
cd /src/osquery-extensions
ln -s efigy /src/osquery/external/extension_efigy
cd /src/osquery
make sysprep
make deps
make -j `sysctl -n hw.ncpu` # how you parallelize the build when on macOS
make externals
# Run make again to ensure osquery recognizes the extension to build
make -j `sysctl -n hw.ncpu`
```

If you see the following warning, it can be ignored: `-- Cannot find Doxygen executable in path`

The extension should be in a subfolder of `/src/osquery/build` once the second make command completes successfully.
Using `find . -name "efigy.ext"` can help you locate it quickly.

## Usage

To quickly test an extension, you can either start it from the osqueryi shell, or launch it manually and wait for it
to connect to the running osquery instance.

Consider either changing the ownership of `efigy.ext` to root, or running osquery with the `--allow_unsafe` flag.
Consider either changing the ownership of `trailofbits_osquery_extensions.ext` to root, or running osquery with the `--allow_unsafe` flag.

`osqueryi --extension /path/to/efigy.ext`
`osqueryi --extension /path/to/trailofbits_osquery_extensions.ext`
```
$ sudo osqueryi --extension osquery-facebook/build/darwin10.12/external/extension_efigy/efigy.ext
$ sudo osqueryi --extension osquery/build/darwin/external/trailofbits_osquery_extensions.ext
Using a virtual database. Need help, type '.help'
osquery> select * from efigy;
osquery> SELECT * FROM efigy;
+--------------------+-----------------+--------------------+-------------------+------------+---------------------+
| latest_efi_version | efi_version | efi_version_status | latest_os_version | os_version | build_number_status |
+--------------------+-----------------+--------------------+-------------------+------------+---------------------+
Expand Down
20 changes: 0 additions & 20 deletions fwctl/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,26 +2,6 @@

The fwctl extension provides osquery with the ability to view and _manage_ the OS-native firewall rules and `/etc/hosts` file (port and host blocking). Verify what your endpoints are blocking, and add new blocking rules as needed.

## Building

1. Install Boost
2. Clone the osquery repository
3. Symlink this extension into the external osquery directory. Use the following link name: "extension_fwctl".
4. Build osquery (see the [official building guide](https://osquery.readthedocs.io/en/latest/development/building/))
5. Run 'make externals'

## Installing Boost

macOS: `brew install boost`

Ubuntu: `apt install boost -y`

Windows: Open the official [Boost download page](http://www.boost.org/users/download/) and download `boost_1_66_0-msvc-14.0-64.exe` under [Boost - Third party downloads](https://dl.bintray.com/boostorg/release/1.66.0/binaries/).

## Running the automated tests

Once osquery has been built with tests enabled (i.e.: *without* the SKIP_TESTS variable), enter the build/<platform_name> folder and run the following command: `make fwctl_tests`.

## Installation

### macOS
Expand Down
31 changes: 1 addition & 30 deletions ntfs_forensics/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,41 +2,12 @@

This extension uses the [Sleuthkit](http://www.sleuthkit.org/) library to print forensic information about an NTFS filesystem. The library and headers are bundled with this repository, so there is no additional step to build or install Sleuthkit.

## Dependencies

Requirements:
* Windows

## Building

1. Clone the osquery repository
2. Symlink this extension into the external osquery directory. Use the following link name: "extension_ntfs".
3. Build osquery
4. From the build directory, run 'cmake --build . --config Release --target'

```
cd /src
git clone https://github.com/facebook/osquery.git
git clone https://github.com/trailofbits/osquery-extensions.git
cd /src/osquery/external
cmd.exe /c mklink /D extension_ntfs ..\osquery-extensions\ntfs_forensics\
cd /src/osquery
.\tools\make-win64-dev-env.bat
.\tools\make-win64-binaries.bat
cd build\windows10
cmake --build . --config Release --target
```

The extension should be in a subfolder of `/src/osquery/build` once the make commands complete successfully.

## Usage

To quickly test an extension, you can either start it from the osqueryi shell, or launch it manually and wait for it
to connect to the running osquery instance.

`osqueryi --extension /path/to/ntfs_forensics.ext.exe`
`osqueryi --extension /path/to/trailofbits_osquery_extensions.ext.exe`

Example:

Expand Down
44 changes: 3 additions & 41 deletions santa/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,57 +4,19 @@
This extension for osquery enables the osquery user to read the log of `DENY` events that Santa generated on the host
with a table called `santa_events`, and to remotely view and *create* new rules for Santa (with or without the use of a Santa sync server or Upvote server) with a table called `santa_rules`.

## Dependencies

Requirements:
* macOS
* Xcode
* Boost (can be installed with Homebrew: `brew install boost`)

## Building

1. Clone the osquery repository
2. Symlink this extension into the external osquery directory. Use the following link name: "extension_santa".
3. Build osquery
4. Run 'make externals'

```
cd /src
git clone https://github.com/facebook/osquery.git
git clone https://github.com/trailofbits/osquery-extensions.git
cd /src/osquery-extensions
ln -s /src/osquery-extensions/santa /src/osquery/external/extension_santa
cd /src/osquery
make sysprep
make deps
make -j `sysctl -n hw.ncpu` # how you parallelize the build when on macOS
make externals
# Run make again to ensure osquery recognizes the extension to build
make -j `sysctl -n hw.ncpu`
```

If you see the following warning, it can be ignored: `-- Cannot find Doxygen executable in path`

The extension should be in a subfolder of `/src/osquery/build` once the second make command completes successfully.
Using `find . -name "santa.ext"` can help you locate it quickly.

## Usage

To quickly test an extension, you can either start it from the osqueryi shell, or launch it manually and wait for it
to connect to the running osquery instance.

Consider either changing the ownership of `santa.ext` to root, or running osquery with the `--allow_unsafe` flag.
Consider either changing the ownership of `trailofbits_osquery_extensions.ext` to root, or running osquery with the `--allow_unsafe` flag.

`osqueryi --extension /path/to/santa.ext`
`osqueryi --extension /path/to/trailofbits_osquery_extensions.ext`

Example:

```
$ sudo ./build/darwin10.13/osquery/osqueryi --extension ./build/darwin10.13/external/extension_santa/santa.ext
$ sudo ./build/darwin10.13/osquery/osqueryi --extension osquery/build/darwin/external/trailofbits_osquery_extensions.ext
Using a virtual database. Need help, type '.help'
osquery> .schema santa_rules
CREATE TABLE santa_rules(`shasum` TEXT, `state` TEXT, `type` TEXT);
Expand Down

0 comments on commit 5d61f03

Please sign in to comment.