More cleanup and bugfixes

.gitlab-ci.yml

@@ -13,14 +13,14 @@ validate_challenges:
 build_basic:
 #    stage: build_basic
     script:
-        - ./ctf-tool.py --address "ctf.cyberatuc.org" CTF.BASE.zip challenges/*
+        - ./ctf-tool.py --address "ctf.cyberatuc.org" challenges/*
 
 
 build_install:
 #    stage: build_install
     script:
         - apt install cron -y
-        - ./ctf-tool.py --install --address "ctf.cyberatuc.org" CTF.BASE.zip challenges/test-challenge-pack/
+        - ./ctf-tool.py --install --address "ctf.cyberatuc.org" challenges/test-challenge-pack/
         - ls -lR /home/
         # TODO: assert correct permissions and that no extra files are added
         # TODO: note that v2 will use services which are easier to check

README-v2.md

@@ -0,0 +1,61 @@
+# CTF-TOOL
+A tool for building modular CTFs with minimum effort.
+
+
+# Concepts
+## Challenges:
+| Challenges Type    | Definition |
+| ------------------ | ---------- |
+| Standard           | Standard challenges present the player with a question and optionally a set of files. The player is expected to input a flag to solve the problem.
+| Service-Required   | Service-required challenges are similar to standard challenges but require a service to be running that the player can connect to (typically via netcat).
+| Machine-Required   | Machine-required challenges are similar to both standard and service-required problems but instead require a dedicated machine to be presented to the players. This type is planned for future and not currently supported.
+| Container-required | Same as Machine-required but use containers instead of machines. Also planned for future and not currently supported.
+
+## Challenge Packs:
+ctf-tool's design revolves around the concept of challenge packs which are in essence directories with the subdirectories that represent individual challenges. The reasoning of using challenge packs as a building block is that it allows problems to be neatly sorted, reused, and even allows the CTF builder to adjust the difficulty for the audience's proficiecy level. The actual content and structure of challenge packs is unfortunately still volatile as ctf-tool is developed into version 1.
+
+## Flags:
+For the sake of simplicity, ctf-tool only currently supports a single static flag per problem. CTFd supports multiple flags and regex flags which is a planned feature for ctf-tool's future. Additionally, static flag randomization and per-player flags are planned for the future.
+
+## CTF Host / Challenge Host:
+The CTF Host is the machine that the players will be directed to in order to submit flags for points. In the current version, ctf-tool expects the CTF host to be an instance of CTFd, but in future versions, FBCTD support is planned.
+
+The Challenge Host is the machine that ctf-tool expects to install services on for players to be able to connect to. This is the machine that the tool is expected to be run on.
+
+ctf-tool considers the challenge host and the ctf host to be two seperate machines, although this is not inforced in any way and using one machine for both should be perfectly fine.
+
+# Supported Systems
+*Linux:* Currently, only Ubuntu 18.04 is considered stable, although other debian-based distros should work. Really any host all the dependency packages installed should work. 
+
+*Windows:* No offical support currently
+
+*Mac:* No offical support currently
+
+## Setup for ubuntu 18.04 machines
+1. Clone this repo
+2. Install docker-ce
+3. sudo scripts/install_required_packages.sh
+4. run the tool with the --install flag unless you don't need services for challenges
+
+# Tool instructions
+## ctf-tool.py
+Builds a CTFd config from a problem set and optionally installs the problem set onto the local machine for socat connections. Meant to be a plug and play clone of the capability seen at Battelle/Hack Ohio 2017 CTF wiht major extentions.
+
+# Development Roadmap
+## Features
+* add the challenge host setup script (default behaviour)
+* Add prereq challenges ability
+* some kind of more interactive way to setup
+* FBCTF output format
+* checkout some vulnerable vm generators and vulnhub for a self-randomizing event every time
+* [x] Need to support flags that are hardcoded in challenges (different flag value for running server than in distributed zip)
+
+## Challenge packs/integrations
+* OTW challenges
+* Various vulnhub challenges
+* Battelle.org/cyber-challenge - This is a good med/hard/hard pack
+* Advent of Code clone?
+* Google CTF beginners ctf
+* Lock pick challenge?
+* Juice Shop
+

Vagrantfile

@@ -0,0 +1,64 @@
+# -*- mode: ruby -*-
+# vi: set ft=ruby :
+
+# All Vagrant configuration is done below. The "2" in Vagrant.configure
+# configures the configuration version (we support older styles for
+# backwards compatibility). Please don't change it unless you know what
+# you're doing.
+Vagrant.configure("2") do |config|
+  # The most common configuration options are documented and commented below.
+  # For a complete reference, please see the online documentation at
+  # https://docs.vagrantup.com.
+
+  # Every Vagrant development environment requires a box. You can search for
+  # boxes at https://vagrantcloud.com/search.
+  config.vm.box = "ubuntu/bionic64"
+
+  # Disable automatic box update checking. If you disable this, then
+  # boxes will only be checked for updates when the user runs
+  # `vagrant box outdated`. This is not recommended.
+  # config.vm.box_check_update = false
+
+  # Create a forwarded port mapping which allows access to a specific port
+  # within the machine from a port on the host machine. In the example below,
+  # accessing "localhost:8080" will access port 80 on the guest machine.
+  # config.vm.network "forwarded_port", guest: 80, host: 8080
+
+  # Create a private network, which allows host-only access to the machine
+  # using a specific IP.
+  # config.vm.network "private_network", ip: "192.168.33.10"
+
+  # Create a public network, which generally matched to bridged network.
+  # Bridged networks make the machine appear as another physical device on
+  # your network.
+  # config.vm.network "public_network"
+
+  # Share an additional folder to the guest VM. The first argument is
+  # the path on the host to the actual folder. The second argument is
+  # the path on the guest to mount the folder. And the optional third
+  # argument is a set of non-required options.
+  # config.vm.synced_folder "../data", "/vagrant_data"
+
+  # Provider-specific configuration so you can fine-tune various
+  # backing providers for Vagrant. These expose provider-specific options.
+  # Example for VirtualBox:
+  #
+  # config.vm.provider "virtualbox" do |vb|
+  #   # Display the VirtualBox GUI when booting the machine
+  #   vb.gui = true
+  #
+  #   # Customize the amount of memory on the VM:
+  #   vb.memory = "1024"
+  # end
+  #
+  # View the documentation for the provider you are using for more
+  # information on available options.
+
+  # Enable provisioning with a shell script. Additional provisioners such as
+  # Puppet, Chef, Ansible, Salt, and Docker are also available. Please see the
+  # documentation for more information about their specific syntax and use.
+  # config.vm.provision "shell", inline: <<-SHELL
+  #   apt-get update
+  #   apt-get install -y apache2
+  # SHELL
+end

ctf-tool.py

@@ -35,8 +35,7 @@ def get_challenge_list(args):
                     chal = Challenge(os.path.join(problem_dir, category, challenge))
                     if chal.name in challenge_names:
                         print(f"Two or more challenges named {chal.name}")
-                        # This is because of the zip file names for challenge.zip not being
-                        # hash based. This may also be a limitation of CTFd #TODO
+                        # TODO: v2 This is because of the zip file names for challenge.zip not being hash based. This may also be a limitation of CTFd, need to investigate
                         quit(1)
                     challenges.append(chal)
                     challenge_names.append(chal.name)
@@ -191,12 +190,8 @@ def install_on_current_machine(challenge, new_user_home, address):
     shutil.copy2(challenge.requires_server_path, new_user_home)
     challenge.requires_server_path = os.path.join(new_user_home, "requires-server")
     # copy everything to new user's home dir
-    # TODO: We should probably only copy a smaller zip to the user's home
-    # then run some predefined script per challenge
-    # Clif: "Agreed", doing it with the server.zip and requires-server now
-    # copytree(challenge.directory, new_user_home)
 
-    # flag.txt perms are different
+    # note that flag.txt perms are different than the executable
 
     os.system(f"chown -R root:root {new_user_home}")
     os.system(f"chmod -R 040 {new_user_home}")
@@ -219,8 +214,7 @@ def install_on_current_machine(challenge, new_user_home, address):
         print("")
         return
     try:
-        # I might just change this to pass in the whole challenge object
-        #setup_listener(**required_vars)
+        # Setup crontab and file permissions
         setup_listener(challenge)
         os.system(f"chown root:{challenge.username} {new_user_home}/flag.txt")
         os.system(f"chmod 020 {new_user_home}/flag.txt")
@@ -271,10 +265,12 @@ def create_challenge_docker_env(path, challenges):
 
 def main():
     # CLI Parser
+    # TODO: v2 make this like the range-master parser so it's easier to extend
     parser = argparse.ArgumentParser()
-    parser.add_argument("basezip",
+    parser.add_argument("--basezip",
                         help="Zip file to pull ctfd metadata from, use a fresh CTFd instance export if you need one",
-                        nargs=1)
+                        nargs=1,
+                        default=["resources/ctfd.base.zip"])
     parser.add_argument("directory", help="Directories of challenge packs to load", nargs="+")
     parser.add_argument("--force", action="store_true", help="ignore challenge pack validation errors")
     parser.add_argument("--install", action="store_true", help="use the local machine as the challenge host")
@@ -283,8 +279,7 @@ def main():
     parser.add_argument("--name", default="ctf-tool", help="Name of the output zip file")
     args = parser.parse_args()
 
-    # TODO:determine action(s) vs running a blob script
-    # this is a note for v2 to use the new parser/shell system in range-master
+    # TODO:determine action(s) vs running a blob script this is a note for v2 to use the new parser/shell system in range-master
 
     # Validate the problem set
     validate_challenge_bundles(args)

docs/DIR-STRUCTURE.MD

@@ -6,7 +6,7 @@
 			flag.txt      - the challenge's flag
 			message.txt   - the message to show on ctfd for the problem
 			value.txt     - the point value of the challenge
-			challenge.zip - zip file for users to download -- TODO: rename to the problem name
+			challenge.zip - zip file for users to download, is renamed to <problem name>.zip when added so users can track files easier
 			server.zip    - zip file that contains all of the relevant files for running a problem as a nc server. A copy of the binary and flag should be in here.
 			solution.txt  - (unimp) details on how to solve the challenge
 			hint_<x>.txt  - (unimp) Hint file where x is the order the hints should be given

docs/SETUP-CHECKLIST.md

@@ -1,3 +1,6 @@
+# SETUP CHECKLIST
+a simple checklist for hosting a real ctfd instance with this tool, do these post-zip upload
+
 # TBD items
 * SSL/DNS for the CTFd instance
 * IP for the Challenge host, can probably go static and have ACM pay for it
@@ -10,6 +13,7 @@
 
 ## Accounts
 1. Verify emails is enabled
+2. Change default admin account from root:root
 
 ## Settings
 1. Challenge Visibility is public
@@ -28,7 +32,8 @@
 1. Make sure the challenges list has been imported successfully
 
 # Challenge host setup
-TODO:
+1. Make sure the challenges are up with `sudo netstat -plant`
+2. Make sure flags have permissions `-----r---- root:challenge_name`
 
 # CTF run crew checklist
 1. Verify everyone who is on the run crew has admin access to CTFd and the challenge host
@@ -39,4 +44,3 @@ TODO:
 1. Make sure the challenge host has been stopped and deleted one hour after the event ends
 2. Make sure the CTFd instance is stopped one day after the event
 3. Make sure the dns name (if any) has been redirected properly to cyberatuc.org
-

docs/chris-notes.md

@@ -1 +1,3 @@
-my docker ctfd image id is 319a1904748e, it seems like one of the ones pushed leading up to the rev uc broke
+1. my docker ctfd image id is 319a1904748e, it seems like one of the ones pushed leading up to the rev uc broke
+2. zip testo.zip -X -r db/ uploads/*/* , for some reason ctfd does not like empty dirs being added to the zip file
+