README.md

Awesome GPT + Security

A curated list of awesome security tools, experimental case or other interesting things with LLM or GPT.

Contents

• Tools
  • Integrated
  • Audit
  • Reconnaissance
  • Offensive
  • Detecting
  • Preventing
  • Social Engineering
  • Reverse Engineering
  • Investigation
  • Fix
  • Assessment
• Cases
  • Experimental
  • Academic
  • Blogs
  • Fun
• GPT Security
  • Standard
  • Bypass Security Policy
  • Bug Bounty
  • Crack
  • Plugin Security
• Contributing

Attention

Here is A nice tool to Finetune ALL LLMs with ALL Adapeters on ALL Platforms!

Tools

🧰

Integrated

• SecGPT - SecGPT aims to make further contributions to network security by combining LLM, including penetration testing, red-blue confrontations, CTF competitions, and other aspects.
• AutoAudit - An LLM for Cyber Security
• secgpt - Cyber security LLM(Lora finetuned with baichuan-13B using some material of cyber security)
• HackerGPT-2.0 - HackerGPT is your indispensable digital companion in the world of hacking.

Audit

• SourceGPT - prompt manager and source code analyzer built on top of ChatGPT as the oracle
• vulnhuntr - Zero shot vulnerability discovery using LLMs
• ChatGPTScanner - A white box code scan powered by ChatGPT
• chatgpt-code-analyzer - ChatGPT Code Analyzer for Visual Studio Code
• hacker-ai - An online tool using AI to detect vulnerabilities in source code
• audit_gpt - Fine-tuning GPT for Smart Contract Auditing
• vulchatgpt - Use IDA PRO HexRays decompiler with OpenAI(ChatGPT) to find possible vulnerabilities in binaries
• Ret2GPT - Advanced AI-powered binary analysis tool leveraging OpenAI's LangChain technology, revolutionizing CTF Pwners' experience in binary file interpretation and vulnerability detection.

Reconnaissance

• CensysGPT Beta - The tool enables users to quickly and easily gain insights into hosts on the internet, streamlining the process and allowing for more proactive threat hunting and exposure management
• GPT_Vuln-analyzer - Uses ChatGPT API, Python-Nmap, DNS Recon modules and uses the GPT3 model to create vulnerability reports based on Nmap scan data, and DNS scan information. It can also perform subdomain enumeration to a great extent
• SubGPT - SubGPT looks at subdomains you have already discovered for a domain and uses BingGPT to find more.
• Navi - A QA based Reconnaissance Tool with GPT
• ChatCVE - The ChatCVE Lang Chain App is an AI-powered devSecOps application 🔍, for oganizations triaging and aggregating CVE (Common Vulnerabilities and Exposures) information.
• ZoomeyeGPT - ZoomEyeGPT browser extension is a GPT-based Chrome browser extension designed to bring AI-assisted search experience to ZoomEye users.
• uncover-turbo - Realize a general-purpose natural language surveying and mapping engine, and open up the last mile from natural language to surveying and mapping grammar.
• DevOpsGPT - AI-Driven Software Development Automation Solution

Offensive

• PentestGPT - A GPT-empowered penetration testing tool
• burpgpt - A Burp Suite extension that integrates OpenAI's GPT to perform an additional passive scan for discovering highly bespoke vulnerabilities, and enables running traffic-based analysis of any type.
• ReconAIzer - A Burp Suite extension to add OpenAI (GPT) on Burp and help you with your Bug Bounty recon to discover endpoints, params, URLs, subdomains and more!
• CodaMOSA - CodaMOSA is the paper code of CodaMOSA: Escaping Coverage Plateaus in Test Generation with Pre-trained Large Language Models. It implements a fuzzer combined with OpenAI API, aiming to alleviate the problem of stagnant coverage in traditional fuzz.
• PassGAN - A Deep Learning Approach for Password Guessing. HomeSecurityHeroes land a Product, and you can test how much time an AI would need to crack your password here.
• nuclei-ai-extension - Official by Nuclei Team. Browser Extension for Rapid Nuclei Template Generation.
• nuclei_gpt - Only need to submit the relevant Request and Response and the description of the vulnerability to generate a Nuclei PoC.
• Nuclei Templates AI Generator -- Create Nuclei templates by textual description (e.g., vulnerability scanners by PoC).
• hackGPT - Leverage OpenAI and ChatGPT to do hackerish things

Detecting

• k8sgpt - a tool for scanning your Kubernetes clusters, diagnosing, and triaging issues in simple English.
• cloudgpt - Vulnerability scanner for AWS customer managed policies using ChatGPT
• IATelligence - About IATelligence is a Python script that will extract the IAT of a PE file and request GPT to get more information about the API and the ATT&CK matrix related
• rebuff - Prompt Injection Detector.
• Callisto - An Intelligent Automated Binary Vulnerability Analysis Tool.
• LLMFuzzer - LLMFuzzer is the first open-source fuzzing framework specifically designed for Large Language Models (LLMs), especially for their integrations in applications via LLM APIs.
• Vigil - Prompt injection detection and LLM prompt security scanner

Preventing

Social Engineering

• ChatGPT-Web-Setting-Funny-Abuse - Play with ChatGPT-Web and found the HTML rendering in description settings.

Reverse Engineering

• LLM4Decompile - Reverse Engineering: Decompiling Binary Code with Large Language Models
• Gepetto - About IDA plugin which queries OpenAI's gpt-3.5-turbo language model to speed up reverse-engineering
• gpt-wpre - Whole-Program Reverse Engineering with GPT-3
• G-3PO - A Script that Solicits GPT-3 for Comments on Decompiled Code

Investigation

• beelzebub - Go-Based Low-Code Honeypot Framework with Enhanced Security, Leveraging GPT-3 for System Virtualization

Fix

• wolverine - Auto fix the bugs in your Python Script/Code

Assessment

• falco-gpt - AI-generated remediations for Falco audit events
• selefra - an open-source policy-as-code software that provides analytics for multi-cloud and SaaS.
• openai-cti-summarizer - openai-cti-summarizer is a tool for generating threat intelligence summary reports based on OpenAI's GPT-3.5 and GPT-4 API

---

Cases

🌰

Experimental

• Lost in ChatGPT's memories: escaping ChatGPT-3.5 memory issues to write CVE PoCs
• I built a Zero Day virus with undetectable exfiltration using only ChatGPT prompts
• Experimenting with GPT-3 for Detecting Security Vulnerabilities in Code
• We put GPT-4 in Semgrep to point out false positives & fix code
• A Practical, AI-Generated Phishing PoC With ChatGPT
• Capturing the Flag with GPT-4
• I Used GPT-3 to Find 213 Security Vulnerabilities in a Single Codebase
• Using ChatGPT to generate encoder and supporting WebShell
• Using OpenAI Chat to Generate Phishing Campaigns -- Include Phishing Platform
• Chat4GPT Experiments for Security
• GPT-3 use cases for Cybersecurity
• AI-Powered Fuzzing: Breaking the Bug Hunting Barrier

Academic

• GPT-4 Technical Report -- OpenAI's own security assessment and mitigation of the model
• Ignore Previous Prompt: Attack Techniques For Language Models -- Pioneering work of Prompt Injection
• More than you've asked for: A Comprehensive Analysis of Novel Prompt Injection Threats to Application-Integrated Large Language Models
• RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
• Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks
• Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
• Can We Generate Shellcodes via Natural Language? An Empirical Study

Blogs

• Dissecting redis CVE-2023-28425 with chatGPT as assistant
• Security Code Review With ChatGPT
• ChatGPT happy to write ransomware, just really bad at it
• Create ATT&CK Groups Knowledge Base
• Model Confusion - Weaponizing ML models for red teams and bounty hunters
• Using LLMs to reverse JavaScript variable name minification

Fun

• shortest prompt that will enable GPT to protect the secret key
• a CTF-like game that teaches how to bypass LLM using language hacks
• ai-goat - Learn AI security through a series of vulnerable LLM CTF challenges.

---

GPT Security

🚨

Standard

• modelscan - Protection against Model Serialization Attacks
• ATT&CK for LLM Apps
• The OWASP Top 10 for Large Language Model Applications project
• Google AI Red Team
• PurpleLlama - Empowering developers, advancing safety, and building an open ecosystem
• agentic_security - Agentic LLM Vulnerability Scanner
• garak - LLM vulnerability scanner
• inspect_ai - Inspect: A framework for large language model evaluations

Bypass Security Policy

• Chat GPT "DAN" (and other "Jailbreaks")
• ChatGPT Prompts for Bug Bounty & Pentesting
• promptmap - automatically tests prompt injection attacks on ChatGPT instances
• Use "Typoglycemia" to Bypass the LLM's Security Policy
• Universal and Transferable Adversarial Attacks on Aligned Language Models
• promptbench - A robustness evaluation framework for large language models on adversarial prompts
• jailbreak_llms - A dataset consists of 15,140 ChatGPT prompts from Reddit, Discord, websites, and open-source datasets (including 1,405 jailbreak prompts).

Bug Bounty

• Building A Virtual Machine inside ChatGPT - deprecated but interesting
• LangChain vulnerable to code injection -- CVE-2023-29374
• ai-exploits - A collection of real world AI/ML exploits for responsibly disclosed vulnerabilities

Crack

• gpt4free - Just API's from some language model sites.
• EdgeGPT - Reverse engineered API of Microsoft's Bing Chat AI
• GPTs - leaked prompts of GPTs

Plugin Security

• SecureGPT – Dynamically test the security of your ChatGPT Plugins APIs (Free DAST for ChatGPT Plugins).

Contributing

Your contributions are always welcome! Please take a look at the contribution guidelines first.

---

If you have any question about this opinionated list, do not hesitate to open an issue on GitHub.

Thanks again for your contribution and keeping this community vibrant. ❤️