diff --git a/.gitmodules b/.gitmodules index 03d3f832e..13322c63d 100644 --- a/.gitmodules +++ b/.gitmodules @@ -46,3 +46,6 @@ [submodule "plugins/emu"] path = plugins/emu url = https://github.com/mitre/emu.git +[submodule "plugins/magma"] + path = plugins/magma + url = https://github.com/mitre/magma.git diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9854e5a85..7b3cea311 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -40,7 +40,7 @@ python -m pytest ``` This will run all unit tests in your current development environment. Depending on the level of the change, you might need to run the test suite on various versions of Python. The unit testing pipeline will run the entire suite across multiple Python versions that we support when you submit your PR. -We utilize `tox` to test CALDERA in multiple versions of Python. This will only run if the interpreter is present on your system. To run tox, execute: +We utilize `tox` to test Caldera in multiple versions of Python. This will only run if the interpreter is present on your system. To run tox, execute: ``` tox ``` diff --git a/README.md b/README.md index 9e57963bf..e16b8f0e3 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ The framework consists of two components: an asynchronous command-and-control (C2) server with a REST API and a web interface. 2) **Plugins**. These repositories expand the core framework capabilities and providing additional functionality. Examples include agents, reporting, collections of TTPs and more. -## Resources and Socials +## Resources & Socials * 📜 [Documentation, training, and use-cases](https://caldera.readthedocs.io/en/latest/) * ✍️ [Caldera's blog](https://medium.com/@mitrecaldera/welcome-to-the-official-mitre-caldera-blog-page-f34c2cdfef09) * 🌐 [Homepage](https://caldera.mitre.org) @@ -37,6 +37,7 @@ These plugins are supported and maintained by the Caldera team. - **[Fieldmanual](https://github.com/mitre/fieldmanual)** (documentation) - **[GameBoard](https://github.com/mitre/gameboard)** (visualize joint red and blue operations) - **[Human](https://github.com/mitre/human)** (create simulated noise on an endpoint) +- **[Magma](https://github.com/mitre/magma)** (VueJS UI for Caldera v5) - **[Manx](https://github.com/mitre/manx)** (shell functionality and reverse shell payloads) - **[Response](https://github.com/mitre/response)** (incident response) - **[Sandcat](https://github.com/mitre/sandcat)** (default agent) @@ -59,6 +60,7 @@ These requirements are for the computer running the core framework: * Python 3.8+ (with Pip3) * Recommended hardware to run on is 8GB+ RAM and 2+ CPUs * Recommended: GoLang 1.17+ to dynamically compile GoLang-based agents. +* NodeJS (v16+ recommended for v5 VueJS UI) ## Installation @@ -67,7 +69,7 @@ Concise installation steps: git clone https://github.com/mitre/caldera.git --recursive cd caldera pip3 install -r requirements.txt -python3 server.py --insecure +python3 server.py --insecure --build ``` Full steps: @@ -84,11 +86,28 @@ pip3 install -r requirements.txt Finally, start the server. ```Bash -python3 server.py --insecure +python3 server.py --insecure --build ``` - +The --build flag automatically installs any VueJS UI dependencies, bundles the UI into a dist directory, and is served by the Caldera server. You will only have to use the --build flag again if you add any plugins or make any changes to the UI. Once started, log into http://localhost:8888 using the default credentials red/admin. Then go into Plugins -> Training and complete the capture-the-flag style training course to learn how to use Caldera. +If you prefer to not use the new VueJS UI, revert to Caldera v4.2.0. Correspondingly, do not use the `--build` flag for earlier versions as not required. + +### User Interface Development + +If you'll be developing the UI, there are a few more additional installation steps. + +**Requirements** +* NodeJS (v16+ recommended) + +**Setup** + +1. Add the Magma submodule if you haven't already: `git submodule add https://gitlab.mitre.org/caldera/other/magma` +1. Install NodeJS dependencies: `cd plugins/magma && npm install && cd ..` +1. Start the Caldera server with an additional flag: `python3 server.py --uidev localhost` + +Your Caldera server is available at http://localhost:8888 as usual, but there will now be a hot-reloading development server for the VueJS front-end available at http://localhost:3000. Both logs from the server and the front-end will display in the terminal you launched the server from. + ## Docker Deployment To build a Caldera docker image, ensure you have docker installed and perform the following actions: ```Bash diff --git a/SECURITY.md b/SECURITY.md index f05eb93f1..929c9d02d 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -28,14 +28,14 @@ Under this policy, "research" means activities in which you: ## Reporting a vulnerability -Information submitted under this policy will be used for defensive purposes only, i.e. to mitigate or remediate vulnerabilities. Since CALDERA is run by a not-for-profit and is open source by nature, by +Information submitted under this policy will be used for defensive purposes only, i.e. to mitigate or remediate vulnerabilities. Since Caldera is run by a not-for-profit and is open source by nature, by submitting a vulnerability, you acknowledge that you have no expectation of payment. However, we will ensure that credit is given to the bug finder. ## What we would like to see from you To help us triage and prioritize submissions, please include the following in your report: -- Affected version of CALDERA (committed hash or version number), operating system used, and python version. +- Affected version of Caldera (committed hash or version number), operating system used, and python version. - Describe the location the vulnerability was discovered and the potential impact of exploitation. @@ -49,7 +49,7 @@ When you choose to share your contact information with us, we commit to coordina - Within ***10 business days***, we will acknowledge that your report has been received. -- After notifying the CALDERA team, we will open reported issues to the public within ***90 days***, or after a fix is released (whichever comes first). +- After notifying the Caldera team, we will open reported issues to the public within ***90 days***, or after a fix is released (whichever comes first). - To the best of our ability, we will confirm the existence of the vulnerability to you and be as transparent as possible about what steps we are taking during the remediation process, including on issues or challenges that may delay resolution. diff --git a/app/api/rest_api.py b/app/api/rest_api.py index 9ae59dc40..2d52069d4 100644 --- a/app/api/rest_api.py +++ b/app/api/rest_api.py @@ -7,7 +7,7 @@ import marshmallow as ma from aiohttp import web -from aiohttp_jinja2 import template, render_template +from aiohttp_jinja2 import render_template from app.api.packs.advanced import AdvancedPack from app.api.packs.campaign import CampaignPack @@ -29,12 +29,13 @@ def __init__(self, services): asyncio.get_event_loop().create_task(AdvancedPack(services).enable()) async def enable(self): + self.app_svc.application.router.add_static('/assets', 'plugins/magma/dist/assets/', append_version=True) + # TODO: only serve static files in legacy plugin mode self.app_svc.application.router.add_static('/gui', 'static/', append_version=True) # unauthorized GUI endpoints - self.app_svc.application.router.add_route('*', '/', self.landing) - self.app_svc.application.router.add_route('*', '/enter', self.validate_login) - self.app_svc.application.router.add_route('*', '/logout', self.logout) - self.app_svc.application.router.add_route('GET', '/login', self.login) + self.app_svc.application.router.add_route('GET', '/', self.landing) + self.app_svc.application.router.add_route('POST', '/enter', self.validate_login) + self.app_svc.application.router.add_route('POST', '/logout', self.logout) # unauthorized API endpoints self.app_svc.application.router.add_route('*', '/file/download', self.download_file) self.app_svc.application.router.add_route('POST', '/file/upload', self.upload_file) @@ -42,31 +43,19 @@ async def enable(self): self.app_svc.application.router.add_route('*', '/api/rest', self.rest_core) self.app_svc.application.router.add_route('GET', '/api/{index}', self.rest_core_info) self.app_svc.application.router.add_route('GET', '/file/download_exfil', self.download_exfil_file) - - @template('login.html', status=401) - async def login(self, request): - return dict() + self.app_svc.application.router.add_route('GET', '/{tail:(?!plugin/).*}', self.handle_catch) async def validate_login(self, request): return await self.auth_svc.login_user(request) - @template('login.html') async def logout(self, request): await self.auth_svc.logout_user(request) async def landing(self, request): - access = await self.auth_svc.get_permissions(request) - if not access: - # If user doesn't have access, server will attempt to redirect to login. - return await self.auth_svc.login_redirect(request) - plugins = await self.data_svc.locate('plugins', {'access': tuple(access), **dict(enabled=True)}) - data = dict(plugins=[p.display for p in plugins], errors=self.app_svc.errors + self._request_errors(request)) - template_name = access[0].name - if template_name == "RED": - template_name = "core_red" - elif template_name == "BLUE": - template_name = "core_blue" - return render_template(f"{template_name}.html", request, data) + return render_template("index.html", request, {}) + + async def handle_catch(self, request): + return render_template("index.html", request, {}) @check_authorization async def rest_core(self, request): diff --git a/app/api/v2/__init__.py b/app/api/v2/__init__.py index 8c167ccf9..1a9c40465 100644 --- a/app/api/v2/__init__.py +++ b/app/api/v2/__init__.py @@ -3,10 +3,11 @@ def make_app(services): from .responses import json_request_validation_middleware - from .security import authentication_required_middleware_factory + from .security import authentication_required_middleware_factory, pass_option_middleware app = web.Application( middlewares=[ + pass_option_middleware, authentication_required_middleware_factory(services['auth_svc']), json_request_validation_middleware ] @@ -54,4 +55,7 @@ def make_app(services): from .handlers.contact_api import ContactApi ContactApi(services).add_routes(app) + from .handlers.payload_api import PayloadApi + PayloadApi(services).add_routes(app) + return app diff --git a/app/api/v2/handlers/contact_api.py b/app/api/v2/handlers/contact_api.py index c9facb1c2..6e2e10ac6 100644 --- a/app/api/v2/handlers/contact_api.py +++ b/app/api/v2/handlers/contact_api.py @@ -15,6 +15,7 @@ def add_routes(self, app: web.Application): router = app.router router.add_get('/contacts/{name}', self.get_contact_report) router.add_get('/contacts', self.get_available_contact_reports) + router.add_get('/contactlist', self.get_contact_list) @aiohttp_apispec.docs(tags=['contacts'], summary='Retrieve a List of Beacons made by Agents to the specified Contact', @@ -43,3 +44,7 @@ async def get_contact_report(self, request: web.Request): async def get_available_contact_reports(self, request: web.Request): contacts = self._api_manager.get_available_contact_reports() return web.json_response(contacts) + + async def get_contact_list(self, request: web.Request): + contacts = [dict(name=c.name, description=c.description) for c in self._api_manager.contact_svc.contacts] + return web.json_response(contacts) diff --git a/app/api/v2/handlers/health_api.py b/app/api/v2/handlers/health_api.py index c15c2635e..c9229ca22 100644 --- a/app/api/v2/handlers/health_api.py +++ b/app/api/v2/handlers/health_api.py @@ -19,15 +19,17 @@ def add_routes(self, app: web.Application): router.add_get('/health', security.authentication_exempt(self.get_health_info)) @aiohttp_apispec.docs(tags=['health'], - summary='Health endpoints returns the status of CALDERA', - description='Returns the status of CALDERA and additional details including versions of system components') + summary='Health endpoints returns the status of Caldera', + description='Returns the status of Caldera and additional details including versions of system components') @aiohttp_apispec.response_schema(CalderaInfoSchema, 200, description='Includes all loaded plugins and system components.') async def get_health_info(self, request): loaded_plugins_sorted = sorted(self._app_svc.get_loaded_plugins(), key=operator.attrgetter('name')) + access = await self._auth_svc.get_permissions(request) mapping = { - 'application': 'CALDERA', + 'application': 'Caldera', 'version': app.get_version(), + 'access': access[0].name, 'plugins': loaded_plugins_sorted } diff --git a/app/api/v2/handlers/operation_api.py b/app/api/v2/handlers/operation_api.py index d9e9d60f2..e8881d0f8 100644 --- a/app/api/v2/handlers/operation_api.py +++ b/app/api/v2/handlers/operation_api.py @@ -8,7 +8,7 @@ from app.api.v2.responses import JsonHttpNotFound from app.api.v2.schemas.base_schemas import BaseGetAllQuerySchema, BaseGetOneQuerySchema from app.api.v2.schemas.link_result_schema import LinkResultSchema -from app.objects.c_operation import Operation, OperationSchema, OperationOutputRequestSchema +from app.objects.c_operation import Operation, OperationSchema, OperationSchemaAlt, OperationOutputRequestSchema from app.objects.secondclass.c_link import LinkSchema @@ -21,6 +21,7 @@ def __init__(self, services): def add_routes(self, app: web.Application): router = app.router router.add_get('/operations', self.get_operations) + router.add_get('/operations/summary', self.get_operations_summary) router.add_get('/operations/{id}', self.get_operation_by_id) router.add_post('/operations', self.create_operation) router.add_patch('/operations/{id}', self.update_operation) @@ -37,7 +38,7 @@ def add_routes(self, app: web.Application): @aiohttp_apispec.docs(tags=['operations'], summary='Retrieve operations', - description='Retrieve all CALDERA operations from memory. Use fields from the ' + description='Retrieve all Caldera operations from memory. Use fields from the ' '`BaseGetAllQuerySchema` in the request body to filter.') @aiohttp_apispec.querystring_schema(BaseGetAllQuerySchema) @aiohttp_apispec.response_schema(OperationSchema(many=True, partial=True), @@ -48,7 +49,7 @@ async def get_operations(self, request: web.Request): @aiohttp_apispec.docs(tags=['operations'], summary='Retrieve an operation by operation id', - description='Retrieve one CALDERA operation from memory based on the operation id (String ' + description='Retrieve one Caldera operation from memory based on the operation id (String ' 'UUID). Use fields from the `BaseGetOneQuerySchema` in the request body to add ' '`include` and `exclude` filters.', parameters=[{ @@ -66,8 +67,28 @@ async def get_operation_by_id(self, request: web.Request): return web.json_response(operation) @aiohttp_apispec.docs(tags=['operations'], - summary='Create a new CALDERA operation record', - description='Create a new CALDERA operation using the format provided in the ' + summary='Retrieve operations (alternate)', + description='Retrieve all Caldera operations from memory, with an alternate selection' + ' of properties. Use fields from the `BaseGetAllQuerySchema` in the request' + ' body to filter.') + @aiohttp_apispec.querystring_schema(BaseGetAllQuerySchema) + @aiohttp_apispec.response_schema(OperationSchemaAlt(many=True, partial=True), + description='The response is a list of all operations.') + async def get_operations_summary(self, request: web.Request): + remove_props = ['chain', 'host_group', 'source', 'visibility'] + operations = await self.get_all_objects(request) + operations_mod = [] + for op in operations: + op['agents'] = self._api_manager.get_agents(op) + op['hosts'] = await self._api_manager.get_hosts(op) + for prop in remove_props: + op.pop(prop, None) + operations_mod.append(op) + return web.json_response(operations_mod) + + @aiohttp_apispec.docs(tags=['operations'], + summary='Create a new Caldera operation record', + description='Create a new Caldera operation using the format provided in the ' '`OperationSchema`. Required schema fields are as follows: "name", ' '"adversary.adversary_id", "planner.id", and "source.id"') @aiohttp_apispec.request_schema(OperationSchema) @@ -79,7 +100,7 @@ async def create_operation(self, request: web.Request): @aiohttp_apispec.docs(tags=['operations'], summary='Update fields within an operation', - description='Update one CALDERA operation in memory based on the operation id (String ' + description='Update one Caldera operation in memory based on the operation id (String ' 'UUID). The `state`, `autonomous` and `obfuscator` fields in the operation ' 'object may be edited in the request body using the `OperationSchema`.', parameters=[{ @@ -98,7 +119,7 @@ async def update_operation(self, request: web.Request): @aiohttp_apispec.docs(tags=['operations'], summary='Delete an operation by operation id', - description='Delete one CALDERA operation from memory based on the operation id (String ' + description='Delete one Caldera operation from memory based on the operation id (String ' 'UUID).', parameters=[{ 'in': 'path', diff --git a/app/api/v2/handlers/payload_api.py b/app/api/v2/handlers/payload_api.py new file mode 100644 index 000000000..6ffe14274 --- /dev/null +++ b/app/api/v2/handlers/payload_api.py @@ -0,0 +1,43 @@ +import itertools +import pathlib + +import aiohttp_apispec +from aiohttp import web +import marshmallow as ma + +from app.api.v2.handlers.base_api import BaseApi +from app.api.v2.schemas.base_schemas import BaseGetAllQuerySchema + + +class PayloadSchema(ma.Schema): + payloads = ma.fields.List(ma.fields.String()) + + +class PayloadApi(BaseApi): + def __init__(self, services): + super().__init__(auth_svc=services['auth_svc']) + self.data_svc = services['data_svc'] + self.file_svc = services['file_svc'] + + def add_routes(self, app: web.Application): + router = app.router + router.add_get('/payloads', self.get_payloads) + + @aiohttp_apispec.docs(tags=['payloads'], + summary='Retrieve payloads', + description='Retrieves all stored payloads.') + @aiohttp_apispec.querystring_schema(BaseGetAllQuerySchema) + @aiohttp_apispec.response_schema(PayloadSchema(), + description='Returns a list of all payloads in PayloadSchema format.') + async def get_payloads(self, request: web.Request): + cwd = pathlib.Path.cwd() + payload_dirs = [cwd / 'data' / 'payloads'] + payload_dirs.extend(cwd / 'plugins' / plugin.name / 'payloads' + for plugin in await self.data_svc.locate('plugins') if plugin.enabled) + payloads = { + self.file_svc.remove_xored_extension(p.name) + for p in itertools.chain.from_iterable(p_dir.glob('[!.]*') for p_dir in payload_dirs) + if p.is_file() + } + + return web.json_response(list(payloads)) diff --git a/app/api/v2/handlers/planner_api.py b/app/api/v2/handlers/planner_api.py index 1493036b8..f109907a8 100644 --- a/app/api/v2/handlers/planner_api.py +++ b/app/api/v2/handlers/planner_api.py @@ -20,7 +20,7 @@ def add_routes(self, app: web.Application): @aiohttp_apispec.docs(tags=['planners'], summary='Retrieve planners', - description='Retrieve CALDERA planners by criteria. Supply fields from the `PlannerSchema` ' + description='Retrieve Caldera planners by criteria. Supply fields from the `PlannerSchema` ' 'to the `include` and `exclude` fields of the `BaseGetAllQuerySchema` in the ' 'request body to filter retrieved planners.') @aiohttp_apispec.querystring_schema(BaseGetAllQuerySchema) @@ -32,7 +32,7 @@ async def get_planners(self, request: web.Request): @aiohttp_apispec.docs(tags=['planners'], summary='Retrieve a planner by planner id', - description='Retrieve one CALDERA planner based on the planner id (String `UUID`). ' + description='Retrieve one Caldera planner based on the planner id (String `UUID`). ' 'Supply fields from the `PlannerSchema` to the `include` and `exclude` fields ' 'of the `BaseGetOneQuerySchema` in the request body to filter retrieved ' 'planners.', diff --git a/app/api/v2/managers/operation_api_manager.py b/app/api/v2/managers/operation_api_manager.py index d5eece127..ada8be28f 100644 --- a/app/api/v2/managers/operation_api_manager.py +++ b/app/api/v2/managers/operation_api_manager.py @@ -206,6 +206,63 @@ async def get_agent(self, operation: Operation, data: dict): raise JsonHttpNotFound(f'Agent {data["paw"]} was not found.') return agent + def get_agents(self, operation: dict): + agents = {} + chain = operation.get('chain', []) + for link in chain: + paw = link.get('paw') + if paw and paw not in agents: + tmp_agent = self.find_object('agents', {'paw': paw}).display + tmp_agent['links'] = [] + agents[paw] = tmp_agent + agents[paw]['links'].append(link) + return agents + + async def get_hosts(self, operation: dict): + hosts = {} + chain = operation.get('chain', []) + for link in chain: + host = link.get('host') + if not host: + continue + if host not in hosts: + tmp_agent = self.find_object('agents', {'host': host}).display + tmp_host = { + 'host': tmp_agent.get('host'), + 'host_ip_addrs': tmp_agent.get('host_ip_addrs'), + 'platform': tmp_agent.get('platform'), + 'reachable_hosts': await self.get_reachable_hosts(agent=tmp_agent) + } + hosts[host] = tmp_host + return hosts + + async def get_reachable_hosts(self, agent: dict = None, operation: dict = None): + """ + NOTE: When agent is supplied, only hosts discovered by agent + are retrieved. + """ + trait_names = BaseWorld.get_config('reachable_host_traits') or [] + paws = () + + if agent is not None: + paws = paws + (agent.get('paw'),) + else: + for agent in operation.get('host_group', []): + paw = agent.get('paw') + if paw: + paws = paws + (paw,) + + hosts = [] + for trait in trait_names: + fqdns = await self.services['knowledge_svc'].get_facts({ + 'trait': trait, + 'collected_by': paws, + }) + for name in fqdns: + hosts.append(name.value) + + return hosts + def build_executor(self, data: dict, agent: Agent): if not data.get('timeout'): data['timeout'] = 60 diff --git a/app/api/v2/schemas/caldera_info_schemas.py b/app/api/v2/schemas/caldera_info_schemas.py index e813e5e65..c7c1fa7ba 100644 --- a/app/api/v2/schemas/caldera_info_schemas.py +++ b/app/api/v2/schemas/caldera_info_schemas.py @@ -7,6 +7,7 @@ class CalderaInfoSchema(schema.Schema): application = fields.String() version = fields.String() + access = fields.String() plugins = fields.List(fields.Nested(Plugin.display_schema)) class Meta: diff --git a/app/api/v2/security.py b/app/api/v2/security.py index 9cf86214a..bf0220960 100644 --- a/app/api/v2/security.py +++ b/app/api/v2/security.py @@ -67,3 +67,13 @@ async def authentication_required_middleware(request, handler): raise web.HTTPUnauthorized() return await handler(request) return authentication_required_middleware + +"""Allow all 'OPTIONS' request to the server to return 200 + +This mitigates CORS issues while developing the UI. +""" +@web.middleware +async def pass_option_middleware(request, handler): + if request.method == 'OPTIONS': + raise web.HTTPOk() + return await handler(request) diff --git a/app/ascii_banner.py b/app/ascii_banner.py new file mode 100644 index 000000000..074d668cb --- /dev/null +++ b/app/ascii_banner.py @@ -0,0 +1,48 @@ +import os + + +RED = "\33[1m" +DARK_RED = "\x1b[38;5;1m" +BLUE = "\33[94m" +DARK_BLUE = "\x1b[38;5;20m" +GREEN = "\033[32m" +YELLOW = "\033[93m" +PURPLE = '\033[0;35m' +DARK_PURPLE = '\x1b[38;5;92m' +CYAN = "\033[36m" +END = "\033[0m" + + +_BANNER = """ + ██████╗ █████╗ ██╗ ██████╗ ███████╗██████╗ █████╗ +██╔════╝██╔══██╗██║ ██╔══██╗██╔════╝██╔══██╗██╔══██╗ +██║ ███████║██║ ██║ ██║█████╗ ██████╔╝███████║ +██║ ██╔══██║██║ ██║ ██║██╔══╝ ██╔══██╗██╔══██║ +╚██████╗██║ ██║███████╗██████╔╝███████╗██║ ██║██║ ██║ + ╚═════╝╚═╝ ╚═╝╚══════╝╚═════╝ ╚══════╝╚═╝ ╚═╝╚═╝ ╚═╝ +""" + + +_BANNER_SECTION_1 = "\n\ + ██████╗ █████╗ ██╗ ██████╗ ███████╗██████╗ █████╗\n\ +██╔════╝██╔══██╗██║ ██╔══██╗██╔════╝██╔══██╗██╔══██╗\n\ +" + + +_BANNER_SECTION_2 = "\ +██║ ███████║██║ ██║ ██║█████╗ ██████╔╝███████║\n\ +██║ ██╔══██║██║ ██║ ██║██╔══╝ ██╔══██╗██╔══██║\n\ +" + + +BANNER_SECTION_3 = "\ +╚██████╗██║ ██║███████╗██████╔╝███████╗██║ ██║██║ ██║\n\ + ╚═════╝╚═╝ ╚═╝╚══════╝╚═════╝ ╚══════╝╚═╝ ╚═╝╚═╝ ╚═╝\n\ +" + + +if int(os.environ.get('NO_COLOR', 0)) == 1: + ASCII_BANNER = _BANNER +else: + ASCII_BANNER = f"{DARK_BLUE}{_BANNER_SECTION_1}{DARK_PURPLE}{_BANNER_SECTION_2}{DARK_RED}{BANNER_SECTION_3}{END}" + diff --git a/app/objects/c_operation.py b/app/objects/c_operation.py index 9648b86cb..d7b553773 100644 --- a/app/objects/c_operation.py +++ b/app/objects/c_operation.py @@ -70,6 +70,23 @@ def build_operation(self, data, **kwargs): return None if kwargs.get('partial') is True else Operation(**data) +class HostSchema(ma.Schema): + display_name = ma.fields.String(dump_only=True) + host = ma.fields.String() + host_ip_addrs = ma.fields.List(ma.fields.String(), allow_none=True) + platform = ma.fields.String() + reachable_hosts = ma.fields.List(ma.fields.String(), allow_none=True) + + +class OperationSchemaAlt(OperationSchema): + chain = property(lambda: AttributeError) + host_group = property(lambda: AttributeError) + source = property(lambda: AttributeError) + visibility = property(lambda: AttributeError) + agents = ma.fields.Dict(keys=ma.fields.String(), values=ma.fields.Nested(AgentSchema())) + hosts = ma.fields.Dict(keys=ma.fields.String(), values=ma.fields.Nested(HostSchema())) + + class Operation(FirstClassObjectInterface, BaseObject): EVENT_EXCHANGE = 'operation' EVENT_QUEUE_STATE_CHANGED = 'state_changed' diff --git a/app/service/app_svc.py b/app/service/app_svc.py index 7843b74ad..7f280a8f0 100644 --- a/app/service/app_svc.py +++ b/app/service/app_svc.py @@ -122,7 +122,7 @@ async def load(p): asyncio.get_event_loop().create_task(load(plug)) templates = ['plugins/%s/templates' % p.lower() for p in self.get_config('plugins')] - templates.append('templates') + templates.append('plugins/magma/dist') aiohttp_jinja2.setup(self.application, loader=jinja2.FileSystemLoader(templates)) async def retrieve_compiled_file(self, name, platform, location=''): diff --git a/app/service/auth_svc.py b/app/service/auth_svc.py index 03fccb649..17b2c1aea 100644 --- a/app/service/auth_svc.py +++ b/app/service/auth_svc.py @@ -86,7 +86,7 @@ async def create_user(self, username, password, group): @staticmethod async def logout_user(request): await forget(request, web.Response()) - raise web.HTTPFound('/login') + raise web.HTTPFound('/') async def login_user(self, request): """Log a user in and save the session diff --git a/app/service/file_svc.py b/app/service/file_svc.py index 753fd2d71..01cf2b0ec 100644 --- a/app/service/file_svc.py +++ b/app/service/file_svc.py @@ -243,7 +243,7 @@ def _read(self, filename): try: buf = self.encryptor.decrypt(buf[len(FILE_ENCRYPTION_FLAG):]) except InvalidToken: - self.log.error('Failed to decrypt saved CALDERA state due to incorrect encryption key.\n' + self.log.error('Failed to decrypt saved Caldera state due to incorrect encryption key.\n' ' - If attempting to restore secure backup, verify that conf/local.yml exists with ' 'correct encryption_key value, and that the server is being run without --insecure.\n' ' - If attempting to restore insecure backup, verify that conf/default.yml exists ' diff --git a/app/service/login_handlers/default.py b/app/service/login_handlers/default.py index 45bb5fef1..1770194b4 100644 --- a/app/service/login_handlers/default.py +++ b/app/service/login_handlers/default.py @@ -32,7 +32,7 @@ async def handle_login(self, request, **kwargs): raise Exception('Auth service not available.') await auth_svc.handle_successful_login(request, username) self.log.debug('%s failed login attempt: ', username) - raise web.HTTPFound('/login') + raise web.HTTPFound('/') async def handle_login_redirect(self, request, **kwargs): """Handle login redirect. diff --git a/app/utility/base_world.py b/app/utility/base_world.py index 96e460c59..ab88253f7 100644 --- a/app/utility/base_world.py +++ b/app/utility/base_world.py @@ -58,7 +58,11 @@ def encode_string(s): @staticmethod def jitter(fraction): i = fraction.split('/') - return randint(int(i[0]), int(i[1])) + min, max = int(i[0]), int(i[1]) + if min > max: + logging.warn(f'Jitter range max value (max={max}) less than min value (min={min}). Using min={max} and max={min}.') + min, max = max, min + return randint(min, max) @staticmethod def create_logger(name): diff --git a/app/utility/payload_encoder.py b/app/utility/payload_encoder.py index 8cb969fed..9e8f2a8a1 100644 --- a/app/utility/payload_encoder.py +++ b/app/utility/payload_encoder.py @@ -2,7 +2,7 @@ This module contains helper functions for encoding and decoding payload files. If AV is running on the server host, then it may sometimes flag, quarantine, or delete -CALDERA payloads. To help prevent this, encoded payloads can be used to prevent AV +Caldera payloads. To help prevent this, encoded payloads can be used to prevent AV from breaking the server. The convention expected by the server is that encoded payloads will be XOR'ed with the DEFAULT_KEY contained in the payload_encoder.py module. diff --git a/app/utility/rule_set.py b/app/utility/rule_set.py index 52a365572..cd5d0ef28 100644 --- a/app/utility/rule_set.py +++ b/app/utility/rule_set.py @@ -80,9 +80,9 @@ async def _is_ip_rule_match(self, rule, fact): | DENY: 127.0.0.0/24 | 127.0.0.0/23 | ------------------------------------- In the above case, we do not match on this fact, since the fact is a supernet of the rule (it "contains" the - rule subnet). Therefore, the rule subnet is only a portion of the fact subnet. Thus, CALDERA would ignore the + rule subnet). Therefore, the rule subnet is only a portion of the fact subnet. Thus, Caldera would ignore the DENY rule and scan /23 anyway. But this would include a denied subnet range, which is undesired behavior. - This being the case, CALDERA does not match on non-equivalent subnets. + This being the case, Caldera does not match on non-equivalent subnets. """ if rule.match != '.*': is_fact_address = await self._is_ip_address(fact.value) diff --git a/conf/default.yml b/conf/default.yml index 5fd4da373..afae89d9d 100644 --- a/conf/default.yml +++ b/conf/default.yml @@ -26,6 +26,9 @@ objects.planners.default: atomic crypt_salt: REPLACE_WITH_RANDOM_VALUE encryption_key: ADMIN123 exfil_dir: /tmp/caldera +reachable_host_traits: +- remote.host.fqdn +- remote.host.ip host: 0.0.0.0 plugins: - access diff --git a/plugins/access b/plugins/access index 0e677767b..775181bf0 160000 --- a/plugins/access +++ b/plugins/access @@ -1 +1 @@ -Subproject commit 0e677767bf0b841a3b79e091314d5d9051501d16 +Subproject commit 775181bf056b2f1cbf09c4f12d1f069b03953812 diff --git a/plugins/atomic b/plugins/atomic index 9e2c9587d..4f489bb69 160000 --- a/plugins/atomic +++ b/plugins/atomic @@ -1 +1 @@ -Subproject commit 9e2c9587d3e1ef1ebd2c89a22d550b7131ee4099 +Subproject commit 4f489bb69371f3300f71bfd659a8d2a0d011f88a diff --git a/plugins/builder b/plugins/builder index f2ce67cab..de24b683b 160000 --- a/plugins/builder +++ b/plugins/builder @@ -1 +1 @@ -Subproject commit f2ce67cabe917988b423b761659c4d0b483d21f3 +Subproject commit de24b683b3b0f696c72fc2ba18f0c244e88db157 diff --git a/plugins/compass b/plugins/compass index fb88e0274..85e2ebf2f 160000 --- a/plugins/compass +++ b/plugins/compass @@ -1 +1 @@ -Subproject commit fb88e0274a79ab87ad0965595aaea645a5ce0b62 +Subproject commit 85e2ebf2f9486973db5824bfed27c290c93a7b66 diff --git a/plugins/debrief b/plugins/debrief index cc5d3f52d..eba44d5de 160000 --- a/plugins/debrief +++ b/plugins/debrief @@ -1 +1 @@ -Subproject commit cc5d3f52d3d2f663295e8b3f7cd939dcdab504b9 +Subproject commit eba44d5de9eed8a620bf3c23279bf3885d9d327e diff --git a/plugins/emu b/plugins/emu index 02a0f3eb5..72157680e 160000 --- a/plugins/emu +++ b/plugins/emu @@ -1 +1 @@ -Subproject commit 02a0f3eb5a9cb7cab7804c398d8bee6f25d736f3 +Subproject commit 72157680e3e2ba9a98f64678a937abfadb2ea1d2 diff --git a/plugins/fieldmanual b/plugins/fieldmanual index 856eee10a..cb53f2b70 160000 --- a/plugins/fieldmanual +++ b/plugins/fieldmanual @@ -1 +1 @@ -Subproject commit 856eee10a4c4e023a6052dc52f6aa4f116f76c43 +Subproject commit cb53f2b70df91ab156c19a48dda2df07dacabc95 diff --git a/plugins/gameboard b/plugins/gameboard index 3d98c3231..dacb6ce23 160000 --- a/plugins/gameboard +++ b/plugins/gameboard @@ -1 +1 @@ -Subproject commit 3d98c32316d0ee248249353ee8736b36391b17b4 +Subproject commit dacb6ce234dd917438883d38c65317eb3cf95d3d diff --git a/plugins/human b/plugins/human index 4368dea17..0e344ea6f 160000 --- a/plugins/human +++ b/plugins/human @@ -1 +1 @@ -Subproject commit 4368dea178f7ec0aef7514b10adb1a86e22f41f8 +Subproject commit 0e344ea6f244e70923afe121174cc9c4c5bae7cc diff --git a/plugins/magma b/plugins/magma new file mode 160000 index 000000000..5c3622652 --- /dev/null +++ b/plugins/magma @@ -0,0 +1 @@ +Subproject commit 5c3622652d1ffd89071e975708a9478a6edf23f1 diff --git a/plugins/manx b/plugins/manx index e7205ea45..a7e83a6ab 160000 --- a/plugins/manx +++ b/plugins/manx @@ -1 +1 @@ -Subproject commit e7205ea454eef8b9f0a8fbf9be35438bf3234575 +Subproject commit a7e83a6ab555bc7182077f215e4f25fe0399c7c4 diff --git a/plugins/response b/plugins/response index 889213af1..8ea6ab5e0 160000 --- a/plugins/response +++ b/plugins/response @@ -1 +1 @@ -Subproject commit 889213af15ce9ab9382375784408e06225d9530c +Subproject commit 8ea6ab5e0f341aba8facf29ba0301643e3d6a3b4 diff --git a/plugins/sandcat b/plugins/sandcat index 7c326bda5..b408f3f9b 160000 --- a/plugins/sandcat +++ b/plugins/sandcat @@ -1 +1 @@ -Subproject commit 7c326bda523d10e9aa29d7fd634b9dda9d9789db +Subproject commit b408f3f9b3df98819e4a0392dc7665cc12fb7381 diff --git a/plugins/ssl b/plugins/ssl index ca7706342..1ae843111 160000 --- a/plugins/ssl +++ b/plugins/ssl @@ -1 +1 @@ -Subproject commit ca7706342d2a60e71b7206f2addd425dfa185ab4 +Subproject commit 1ae843111919b5a6e58a03917bd9be6b852ecb1f diff --git a/plugins/stockpile b/plugins/stockpile index 01331a302..fe0c5b7bb 160000 --- a/plugins/stockpile +++ b/plugins/stockpile @@ -1 +1 @@ -Subproject commit 01331a302c277bf8e5dc3cd957917ef405282539 +Subproject commit fe0c5b7bbd5dcdde4f90b5b6e493c2b0459d03e8 diff --git a/plugins/training b/plugins/training index b058b6773..37f998f42 160000 --- a/plugins/training +++ b/plugins/training @@ -1 +1 @@ -Subproject commit b058b67736e34125b87cb6bc2254456967674451 +Subproject commit 37f998f4203c84b3ef96e82103e12c5045cdea6e diff --git a/requirements.txt b/requirements.txt index eea409f15..2d1b44ccc 100644 --- a/requirements.txt +++ b/requirements.txt @@ -12,7 +12,6 @@ sphinx_rtd_theme==1.3.0 myst-parser==2.0.0 marshmallow==3.20.1 dirhash==0.2.1 -donut-shellcode==1.0.2 marshmallow-enum==1.5.1 ldap3==2.9.1 lxml~=4.9.1 # debrief diff --git a/server.py b/server.py old mode 100755 new mode 100644 index ee86c5d64..a1b44ecc4 --- a/server.py +++ b/server.py @@ -4,6 +4,7 @@ import os import sys import warnings +import subprocess import aiohttp_apispec from aiohttp_apispec import validation_middleware @@ -11,8 +12,10 @@ import app.api.v2 from app import version +from app.ascii_banner import ASCII_BANNER from app.api.rest_api import RestApi from app.api.v2.responses import apispec_request_validation_middleware +from app.api.v2.security import pass_option_middleware from app.objects.c_agent import Agent from app.objects.secondclass.c_executor import Executor from app.objects.secondclass.c_link import Link @@ -32,11 +35,13 @@ def setup_logger(level=logging.DEBUG): - logging.basicConfig(level=level, - format='%(asctime)s - %(levelname)-5s (%(filename)s:%(lineno)s %(funcName)s) %(message)s', - datefmt='%Y-%m-%d %H:%M:%S') + logging.basicConfig( + level=level, + format="%(asctime)s - %(levelname)-5s (%(filename)s:%(lineno)s %(funcName)s) %(message)s", + datefmt="%Y-%m-%d %H:%M:%S", + ) for logger_name in logging.root.manager.loggerDict.keys(): - if logger_name in ('aiohttp.server', 'asyncio'): + if logger_name in ("aiohttp.server", "asyncio"): continue else: logging.getLogger(logger_name).setLevel(100) @@ -45,22 +50,32 @@ def setup_logger(level=logging.DEBUG): async def start_server(): - await auth_svc.apply(app_svc.application, BaseWorld.get_config('users')) + await auth_svc.apply(app_svc.application, BaseWorld.get_config("users")) runner = web.AppRunner(app_svc.application) await runner.setup() - await web.TCPSite(runner, BaseWorld.get_config('host'), BaseWorld.get_config('port')).start() + await web.TCPSite( + runner, BaseWorld.get_config("host"), BaseWorld.get_config("port") + ).start() -def run_tasks(services): +def run_tasks(services, run_vue_server=False): loop = asyncio.get_event_loop() loop.create_task(app_svc.validate_requirements()) loop.run_until_complete(data_svc.restore_state()) loop.run_until_complete(knowledge_svc.restore_state()) - loop.run_until_complete(RestApi(services).enable()) loop.run_until_complete(app_svc.register_contacts()) loop.run_until_complete(app_svc.load_plugins(args.plugins)) - loop.run_until_complete(data_svc.load_data(loop.run_until_complete(data_svc.locate('plugins', dict(enabled=True))))) - loop.run_until_complete(app_svc.load_plugin_expansions(loop.run_until_complete(data_svc.locate('plugins', dict(enabled=True))))) + loop.run_until_complete( + data_svc.load_data( + loop.run_until_complete(data_svc.locate("plugins", dict(enabled=True))) + ) + ) + loop.run_until_complete( + app_svc.load_plugin_expansions( + loop.run_until_complete(data_svc.locate("plugins", dict(enabled=True))) + ) + ) + loop.run_until_complete(RestApi(services).enable()) loop.run_until_complete(auth_svc.set_login_handlers(services)) loop.create_task(app_svc.start_sniffer_untrusted_agents()) loop.create_task(app_svc.resume_operations()) @@ -68,63 +83,134 @@ def run_tasks(services): loop.create_task(learning_svc.build_model()) loop.create_task(app_svc.watch_ability_files()) loop.run_until_complete(start_server()) - loop.run_until_complete(event_svc.fire_event(exchange='system', queue='ready')) + loop.run_until_complete(event_svc.fire_event(exchange="system", queue="ready")) + if run_vue_server: + loop.run_until_complete(start_vue_dev_server()) try: - logging.info('All systems ready.') + logging.info("All systems ready.") + logging.info(ASCII_BANNER) loop.run_forever() except KeyboardInterrupt: - loop.run_until_complete(services.get('app_svc').teardown(main_config_file=args.environment)) + loop.run_until_complete( + services.get("app_svc").teardown(main_config_file=args.environment) + ) def init_swagger_documentation(app): """Makes swagger documentation available at /api/docs for any endpoints marked for aiohttp_apispec documentation. """ - warnings.filterwarnings( - "ignore", - message="Multiple schemas resolved to the name" - ) + warnings.filterwarnings("ignore", message="Multiple schemas resolved to the name") aiohttp_apispec.setup_aiohttp_apispec( app=app, - title='CALDERA', + title="Caldera", version=version.get_version(), - swagger_path='/api/docs', - url='/api/docs/swagger.json', - static_path='/static/swagger' + swagger_path="/api/docs", + url="/api/docs/swagger.json", + static_path="/static/swagger", ) app.middlewares.append(apispec_request_validation_middleware) app.middlewares.append(validation_middleware) -if __name__ == '__main__': +async def enable_cors(request, response): + response.headers["Access-Control-Allow-Origin"] = ( + "http://" + args.uiDevHost + ":3000" + ) + response.headers["Access-Control-Allow-Credentials"] = "true" + response.headers["Access-Control-Allow-Methods"] = ( + "GET, POST, PUT, PATCH, DELETE, OPTIONS, HEAD" + ) + response.headers["Access-Control-Allow-Headers"] = ( + "Access-Control-Allow-Headers, Origin, Accept, X-Requested-With, Content-Type, Access-Control-Request-Method, Access-Control-Request-Headers" + ) + + +async def start_vue_dev_server(): + await asyncio.create_subprocess_shell( + "npm run dev", stdout=sys.stdout, stderr=sys.stderr, cwd="./plugins/magma/" + ) + logging.info("VueJS development server is live.") + + +def _get_parser(): + def list_str(values): - return values.split(',') - sys.path.append('') - parser = argparse.ArgumentParser('Welcome to the system') - parser.add_argument('-E', '--environment', required=False, default='local', help='Select an env. file to use') - parser.add_argument("-l", "--log", dest="logLevel", choices=['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'], - help="Set the logging level", default='INFO') - parser.add_argument('--fresh', action='store_true', required=False, default=False, - help='remove object_store on start') - parser.add_argument('-P', '--plugins', required=False, default=os.listdir('plugins'), - help='Start up with a single plugin', type=list_str) - parser.add_argument('--insecure', action='store_true', required=False, default=False, - help='Start caldera with insecure default config values. Equivalent to "-E default".') + return values.split(",") + + parser = argparse.ArgumentParser("Welcome to the system") + parser.add_argument( + "-E", + "--environment", + required=False, + default="local", + help="Select an env. file to use", + ) + parser.add_argument( + "-l", + "--log", + dest="logLevel", + choices=["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"], + help="Set the logging level", + default="INFO", + ) + parser.add_argument( + "--fresh", + action="store_true", + required=False, + default=False, + help="remove object_store on start", + ) + parser.add_argument( + "-P", + "--plugins", + required=False, + default=os.listdir("plugins"), + help="Start up with a single plugin", + type=list_str, + ) + parser.add_argument( + "--insecure", + action="store_true", + required=False, + default=False, + help='Start caldera with insecure default config values. Equivalent to "-E default".', + ) + parser.add_argument( + "--uidev", + dest="uiDevHost", + help="Start VueJS dev server for front-end alongside the caldera server. Provide hostname, i.e. localhost.", + ) + parser.add_argument( + "--build", + action="store_true", + required=False, + default=False, + help="Build the VueJS front-end to serve it from the caldera server.", + ) + return parser + +if __name__ == '__main__': + sys.path.append("") + + parser = _get_parser() args = parser.parse_args() setup_logger(getattr(logging, args.logLevel)) if args.insecure: - logging.warning('--insecure flag set. Caldera will use the default.yml config file.') - args.environment = 'default' - elif args.environment == 'local': + logging.warning( + "--insecure flag set. Caldera will use the default.yml config file." + ) + args.environment = "default" + elif args.environment == "local": ensure_local_config() - main_config_path = 'conf/%s.yml' % args.environment - BaseWorld.apply_config('main', BaseWorld.strip_yml(main_config_path)[0]) - logging.info('Using main config from %s' % main_config_path) - BaseWorld.apply_config('agents', BaseWorld.strip_yml('conf/agents.yml')[0]) - BaseWorld.apply_config('payloads', BaseWorld.strip_yml('conf/payloads.yml')[0]) + main_config_path = "conf/%s.yml" % args.environment + BaseWorld.apply_config("main", BaseWorld.strip_yml(main_config_path)[0]) + logging.info("Using main config from %s" % main_config_path) + BaseWorld.apply_config("agents", BaseWorld.strip_yml("conf/agents.yml")[0]) + BaseWorld.apply_config("payloads", BaseWorld.strip_yml("conf/payloads.yml")[0]) data_svc = DataService() knowledge_svc = KnowledgeService() @@ -134,7 +220,7 @@ def list_str(values): Executor, Agent, Link, - AppConfigGlobalVariableIdentifier + AppConfigGlobalVariableIdentifier, ] ) rest_svc = RestService() @@ -143,13 +229,32 @@ def list_str(values): learning_svc = LearningService() event_svc = EventService() - app_svc = AppService(application=web.Application(client_max_size=5120**2)) - app_svc.register_subapp('/api/v2', app.api.v2.make_app(app_svc.get_services())) + app_svc = AppService( + application=web.Application( + client_max_size=5120**2, middlewares=[pass_option_middleware] + ) + ) + app_svc.register_subapp("/api/v2", app.api.v2.make_app(app_svc.get_services())) init_swagger_documentation(app_svc.application) + if args.uiDevHost: + if not os.path.exists("./plugins/magma/dist"): + logging.info("Building VueJS front-end.") + subprocess.run(["npm", "run", "build"], cwd="plugins/magma", check=True) + logging.info("VueJS front-end build complete.") + app_svc.application.on_response_prepare.append(enable_cors) + + if args.build: + logging.info("Building VueJS front-end.") + subprocess.run(["npm", "install"], cwd="plugins/magma", check=True) + subprocess.run(["npm", "run", "build"], cwd="plugins/magma", check=True) + logging.info("VueJS front-end build complete.") if args.fresh: - logging.info("Fresh startup: resetting server data. See %s directory for data backups.", DATA_BACKUP_DIR) + logging.info( + "Fresh startup: resetting server data. See %s directory for data backups.", + DATA_BACKUP_DIR, + ) asyncio.get_event_loop().run_until_complete(data_svc.destroy()) asyncio.get_event_loop().run_until_complete(knowledge_svc.destroy()) - run_tasks(services=app_svc.get_services()) + run_tasks(services=app_svc.get_services(), run_vue_server=args.uiDevHost) \ No newline at end of file diff --git a/templates/abilities.html b/templates/abilities.html index cd33a576b..bbcefebe9 100644 --- a/templates/abilities.html +++ b/templates/abilities.html @@ -5,7 +5,7 @@

Abilities

- An ability is a specific ATT&CK tactic/technique implementation which can be executed on running agents. Abilities will include the command(s) to run, the platforms / executors the commands can run on (ex: Windows / PowerShell), payloads to include, and a reference to a module to parse the output on the CALDERA server. + An ability is a specific ATT&CK tactic/technique implementation which can be executed on running agents. Abilities will include the command(s) to run, the platforms / executors the commands can run on (ex: Windows / PowerShell), payloads to include, and a reference to a module to parse the output on the Caldera server.


diff --git a/templates/core.html b/templates/core.html index fe091eaca..a9a971026 100644 --- a/templates/core.html +++ b/templates/core.html @@ -27,7 +27,7 @@
{{ user_name }} @@ -156,14 +156,14 @@

Startup Messages