Merge pull request #703 from BC-SECURITY/release/5.7.3

v5.7.3 into main

.github/CONTRIBUTING.md

@@ -46,18 +46,14 @@ The `main` branch in `BC-SECURITY/Empire` automatically syncs.
 ### Code Formatting and Linting
 
 * We are using [psf/black](https://github.com/psf/black) for code formatting.
-    * Black is a Python code formatter that helps to keep the codebase uniform and easy to read
-* We are using [PyCQA/isort](https://github.com/PyCQA/isort)
-    * Isort is a Python utility that sorts and formats imports.
 * We are using [charliermarsh/ruff](https://github.com/charliermarsh/ruff) for linting.
-    * Ruff is a python linter that helps identify common bugs and style issues.
+    * We are using the E, W, F, I, UP, and B rulesets.
 * After implementing your changes:
-    1. run `black .` (or `poetry run black .`).
-    2. run `isort .` (or `poetry run isort .`).
-    3. run `ruff . --fix` (or `poetry run ruff . --fix`). 
+    1. run `ruff . --fix` (or `poetry run ruff . --fix`).
+    2. run `black .` (or `poetry run black .`).
 * The repo is also configured to use [pre-commit](https://pre-commit.com/) to automatically format code.
   * Once you have pre-commit installed, you can run `pre-commit install` to install the pre-commit hooks.
-  * Then pre-commit will execute black, isort, and ruff automatically before committing.
+  * Then pre-commit will execute black and ruff automatically before committing.
 
 ### Tests
 

.github/workflows/lint-and-test.yml

@@ -12,10 +12,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v3
-      - uses: psf/[email protected]
-      - uses: isort/isort-action@master
-        with:
-          isort-version: 5.12.0
+      - uses: psf/[email protected]
       - name: Run ruff
         run: |
           pip install ruff==0.0.283
@@ -76,12 +73,22 @@ jobs:
           poetry install
       - name: Run test suite - mysql
         run: |
-          DATABASE_USE=mysql poetry run pytest . -v --runslow
+            set -o pipefail
+            if [ "${{ matrix.python-version }}" = "3.11" ]; then
+                DATABASE_USE=mysql poetry run pytest -v --runslow --cov=empire/server --junitxml=pytest.xml --cov-report=term-missing:skip-covered . | tee pytest-coverage.txt
+            else
+                DATABASE_USE=mysql poetry run pytest -v --runslow .
+            fi
       - name: Run test suite - sqlite
         if: ${{ startsWith(github.head_ref, 'release/') || contains(github.event.pull_request.labels.*.name, 'test-sqlite') }}
         run: |
           DATABASE_USE=sqlite poetry run pytest . -v --runslow
-
+      - name: Pytest coverage comment
+        if: ${{ matrix.python-version == '3.11' }}
+        uses: MishaKav/[email protected]
+        with:
+          pytest-coverage-path: ./pytest-coverage.txt
+          junitxml-path: ./pytest.xml
   test_image:
     # To save CI time, only run these tests on the release PRs
     if: ${{ startsWith(github.head_ref, 'release/') }}

.pre-commit-config.yaml

@@ -1,18 +1,21 @@
 repos:
--   repo: https://github.com/psf/black
-    rev: 23.7.0
+-   repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.4.0
     hooks:
-    - id: black
-      language_version: python3.9
-
--   repo: https://github.com/pycqa/isort
-    rev: 5.12.0
-    hooks:
-      - id: isort
-        name: isort (python)
+    - id: trailing-whitespace
+    - id: check-json
+    - id: check-yaml
+    - id: check-merge-conflict
+    - id: end-of-file-fixer
 
 - repo: https://github.com/astral-sh/ruff-pre-commit
   rev: v0.0.283
   hooks:
     - id: ruff
       args: [--fix, --exit-non-zero-on-fix]
+
+-   repo: https://github.com/psf/black-pre-commit-mirror
+    rev: 23.9.1
+    hooks:
+    - id: black
+      language_version: python3.9

CHANGELOG.md

@@ -7,6 +7,32 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [5.7.3] - 2023-10-17
+
+-   Updated Starkiller to v2.6.1
+-   Fixed global obfuscation not working on modules (@Cx01N)
+-   Added bypass module in PowerShell to run bypasses after agent is staged (@Cx01N)
+-   Fixed IronPython and Python stagers not getting obfuscation applied (@Cx01N)
+
+## [5.7.2] - 2023-09-28
+
+-   Updated Dropbox C2 to use new API endpoints (@Cx01N)
+-   Standardized Kill Date and Working Hours for PowerShell Agents (@Cx01N)
+-   Apply fixes for future Python 3.12 compatibility (@Vinnybod)
+-   Add additional rulesets to ruff linting (@Vinnybod)
+
+## [5.7.1] - 2023-09-25
+
+## [5.7.0] - 2023-09-17
+
+-   Add avatars to users (@Vinnybod)
+-   Update plugin documentation, update embedded plugins to not abuse notifications (@Vinnybod)
+-   Add additional pre-commit hooks for code cleanup (@Vinnybod)
+-   Report test coverage on pull requests (@Vinnybod)
+-   Fixed issue with multiple parameters not executing in IronPython for C# tasks (@Cx01N)
+-   Fix for spawnas not generating bat file (@wizquaza)
+-   Fixed taskings for OneDrive listener (@Hubbl3)
+
 ## [5.6.4] - 2023-09-08
 
 -   Added Stix2 to dependency list for Advanced Reports (@Cx01N)
@@ -84,7 +110,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 -   Added python module for Polkit (CVE-2021-3560) (@Cx01N)
 -   Fixed safecheck error for python module sudo spawn (@Cx01N)
 -   Fixed file error in Invoke-Shellcode (@Cx01N)
--   Removed duplicate modules between languages (@Cx01N) 
+-   Removed duplicate modules between languages (@Cx01N)
     -   Removed .NET Core modules due to errors
     -   Removed redundant C# lateral movement modules
     -   Removed Covenant Mimikatz in favor of Invoke-Mimikatz
@@ -117,7 +143,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [5.4.0] - 2023-05-22
 
 -   Remove Starkiller as a submodule, treat it as a normal directory (@Vinnybod)
-    -   Everything should 'just work', but if you have issues after pulling these latest changes, try deleting the Starkiller directory before running the server `rm -r empire/server/api/v2/starkiller`. 
+    -   Everything should 'just work', but if you have issues after pulling these latest changes, try deleting the Starkiller directory before running the server `rm -r empire/server/api/v2/starkiller`.
 -   Some improvements to the release flow after starkiller submodule removal (@Vinnybod)
 
 ## [5.3.0] - 2023-05-17
@@ -584,7 +610,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 -   Updated shellcoderdi to newest version (@Cx01N)
 -   Added a Nim launcher (@Hubbl3)
 
-[Unreleased]: https://github.com/BC-SECURITY/Empire-Sponsors/compare/v5.6.4...HEAD
+[Unreleased]: https://github.com/BC-SECURITY/Empire-Sponsors/compare/v5.7.3...HEAD
+
+[5.7.3]: https://github.com/BC-SECURITY/Empire-Sponsors/compare/v5.7.2...v5.7.3
+
+[5.7.2]: https://github.com/BC-SECURITY/Empire-Sponsors/compare/v5.7.1...v5.7.2
+
+[5.7.1]: https://github.com/BC-SECURITY/Empire-Sponsors/compare/v5.7.0...v5.7.1
+
+[5.7.0]: https://github.com/BC-SECURITY/Empire-Sponsors/compare/v5.6.4...v5.7.0
 
 [5.6.4]: https://github.com/BC-SECURITY/Empire-Sponsors/compare/v5.6.3...v5.6.4
 

docs/plugins/hooks-and-filters.md

@@ -11,7 +11,9 @@ Hooks and filters are a function that a developer can implement that will be cal
 A minimal hook implementation.
 
 ```python
-from empire.server.common.hooks import hooks
+from sqlalchemy.orm import Session
+from empire.server.core.hooks import hooks
+from empire.server.core.db import models
 
 def my_hook(db: Session, agent: models.Agent):
     """
@@ -26,15 +28,17 @@ hooks.register_hook(hooks.AFTER_AGENT_CHECKIN_HOOK, 'checkin_logger_hook', my_ho
 A minimal filter implementation.
 
 ```python
-from empire.server.common.hooks import hooks
+from sqlalchemy.orm import Session
+from empire.server.core.hooks import hooks
+from empire.server.core.db import models
 
-def my_filter(db: Session, tasking: models.Tasking):
+def my_filter(db: Session, task: models.AgentTask):
     """
     Reverses the output string of a tasking.
     """
-    tasking.output = tasking.output[::-1]
+    task.output = task.output[::-1]
 
-    return tasking
+    return task
 
 
 hooks.register_filter(hooks.BEFORE_TASKING_RESULT_FILTER, 'reverse_filter', my_filter)

docs/plugins/plugin-development.md

@@ -1,8 +1,7 @@
 # Plugin Development
 
-
 ## Execute Function
-The execute function is the entry point for the plugin. It is called when the plugin is executed via the CLI or the API. The execute function is passed the following arguments:
+The execute function is the entry point for the plugin. It is called when the plugin is executed via the API. The execute function is passed the following arguments:
 
 * command - A dict of the command arguments, already parsed and validated by the core Empire code
 * kwargs - Additional arguments that may be passed in by the core Empire code. Right now there are only two.
@@ -11,35 +10,45 @@ The execute function is the entry point for the plugin. It is called when the pl
 
 If the plugin doesn't have `**kwargs`, then no kwargs will be sent. This is to ensure backwards compatibility with plugin pre-5.2.
 
+### Response
+
+Before the plugin's execute function is called, the core Empire code will validate the command arguments. If the arguments are invalid, the API will return a 400 error with the error message.
+
+The execute function can return a String, a Boolean, or a Tuple of (Any, String)
+
+* None - The execution will be considered successful.
+* String - The string will be displayed to the user executing the plugin and the execution will be considered successful.
+* Boolean - If the boolean is True, the execution will be considered successful. If the boolean is False, the execution will be considered failed.
+* Tuple - The tuple must be a tuple of (Any, String). The second value in the tuple represents an error message. The string will be displayed to the user executing the plugin and the execution will be considered failed.
 
 ```python
 def execute(self, command, **kwargs):
-    user = kwargs.get('user', None)
-    db = kwargs.get('db', None)
+    ...
 
-    return "Execution complete"
+    # Successful execution
+    # return None
+    # return "Execution complete"
+    # return True
+
+    # Failed execution
+    # return False, "Execution failed"
 ```
 
+### Custom Exceptions
 
-## Using the database
-If objects are being retrieved from the database, the service functions require a database session be passed in.
-Effectively, this means that you must handle your database session in the plugin. Using the Context Manager syntax
-ensures the db session gets cleaned up when done.
+If the plugin raises a `PluginValidationException`, the API will return a 400 error with the error message.
 
 ```python
-from empire.server.core.db.base import SessionLocal
+from empire.server.core.exceptions import PluginValidationException
 
 def execute(self, command, **kwargs):
-    user = kwargs.get('user', None)
-    db = kwargs.get('db', None)
+    ...
 
-    agents = self.main_menu.agentsv2.get_all(db)
-
-    return "Execution complete"
+    raise PluginValidationException("This is a validation error")
 ```
 
 ## Plugin Tasks
-Plugins can now store tasks. The data model looks pretty close to Agent tasks. This is for agent executions that:
+Plugins can store tasks. The data model looks pretty close to Agent tasks. This is for agent executions that:
 
 1. Want to attach a file result
 2. Need to display a lot of output, where notifications don't quite work
@@ -63,12 +72,68 @@ def execute(self, command, **kwargs):
     )
 
     db.add(plugin_task)
-    db.commit()
 ```
 
-
 For an example of using plugin tasks and attaching files, see the [basic_reporting plugin](https://github.com/BC-SECURITY/Empire/blob/main/server/plugins/basic_reporting/basic_reporting.plugin).
 
+## Notifications
+Notifications are meant for time sensitive information that the user should be aware of.
+In Starkiller, these get displayed immediately, so it is important not to spam them.
+
+To send a notification, use the `plugin_service`.
+
+```python
+def register(self, mainMenu):
+    self.plugin_service = mainMenu.pluginsv2
+
+def execute(self, command, **kwargs):
+    # Do something
+
+    self.plugin_service.plugin_socketio_message(
+        self.info["Name"], "Helo World!"
+    )
+```
+
+## Using the database
+Execute functions and hooks/filters are sent a SQLAlchemy database session. This does not need to be
+opened or closed, as the calling code handles that. The database session is passed in
+as a keyword argument.
+
+```python
+from sqlalchemy.orm import Session
+
+def execute(self, command, **kwargs):
+    user = kwargs.get('user', None)
+    db: Session = kwargs.get('db', None)
+
+    agents = self.main_menu.agentsv2.get_all(db)
+
+    return "Execution complete"
+```
+
+It is important not to close the database session, as it will be used by the calling code and sent to other hooks/filters.
+
+```python
+from sqlalchemy.orm import Session
+from empire.server.core.db import models
+
+def on_agent_checkin(self, db: Session, agent: models.Agent):
+    # Do something
+    pass
+```
+
+When executing code outside of the execute function or hooks/filters, you will need to open a database session.
+This means that you must handle your database session in the plugin. Using the Context Manager syntax
+ensures the db session commits and closes properly.
+```python
+from empire.server.core.db.base import SessionLocal
+
+def do_something():
+    with SessionLocal.begin() as db:
+        # Do the things with the db session
+        pass
+```
+
 ## Event-based functionality (hooks and filters)
 This is outlined in [Hooks and Filters](./hooks-and-filters.md).
 

empire/client/client.py

@@ -118,7 +118,7 @@ class CliExitException(BaseException):
     pass
 
 
-class EmpireCli(object):
+class EmpireCli:
     def __init__(self) -> None:
         self.completer = MyCustomCompleter(self)
         self.menus: Dict[Menu] = {
@@ -443,7 +443,7 @@ def parse_command_line(self, text: str, cmd_line: List[str], resource_file=False
                     args = self.strip(docopt(func.__doc__, argv=cmd_line[1:]))
                     new_args = {}
                     # todo casting for type hinted values?
-                    for key, hint in get_type_hints(func).items():
+                    for key in get_type_hints(func).keys():
                         if key != "return":
                             new_args[key] = args[key]
                     func(**new_args)

empire/client/src/EmpireCliConfig.py

@@ -7,7 +7,7 @@
 log = logging.getLogger(__name__)
 
 
-class EmpireCliConfig(object):
+class EmpireCliConfig:
     def __init__(self):
         self.yaml: Dict = {}
         if "--config" in sys.argv:
@@ -20,7 +20,7 @@ def __init__(self):
 
     def set_yaml(self, location: str):
         try:
-            with open(location, "r") as stream:
+            with open(location) as stream:
                 self.yaml = yaml.safe_load(stream)
         except yaml.YAMLError as exc:
             log.error(exc)

empire/client/src/EmpireCliState.py

@@ -24,7 +24,7 @@
     pass
 
 
-class EmpireCliState(object):
+class EmpireCliState:
     def __init__(self):
         self.host = ""
         self.port = ""
@@ -427,7 +427,7 @@ def get_agents(self):
         self.agents = {x["name"]: x for x in response.json()["records"]}
 
         # Whenever agents are refreshed, add socketio listeners for taskings.
-        for name, agent in self.agents.items():
+        for _name, agent in self.agents.items():
             session_id = agent["session_id"]
             self.sio.on(f"agents/{session_id}/task", self.add_to_cached_results)
 
@@ -664,7 +664,7 @@ def get_active_plugins(self):
         )
 
         self.plugins = {x["name"]: x for x in response.json()["records"]}
-        for name, plugin in self.plugins.items():
+        for _name, plugin in self.plugins.items():
             plugin_name = plugin["name"]
             self.sio.on(f"plugins/{plugin_name}/notifications", self.add_plugin_cache)
         return self.plugins