Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deserialize and serialize JSON to and from MsgPack format #19

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
106 changes: 106 additions & 0 deletions convert/Export-Raw-Selection/Export-Raw-Selection.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
{
"description": "Exports raw content to a file with original format",
"edition": 2,
"graph": {
"edges": [
{
"source": {
"exec_alias": "exec",
"node_id": 0
},
"target": {
"exec_alias": "exec",
"node_id": 2
}
},
{
"source": {
"exec_alias": "exec",
"node_id": 2
},
"target": {
"exec_alias": "exec",
"node_id": 1
}
}
],
"nodes": [
{
"alias": "convert_start",
"definition_id": "caido/convert-start",
"display": {
"x": -10,
"y": -100
},
"id": 0,
"inputs": [],
"name": "Convert Start",
"version": "0.1.0"
},
{
"alias": "convert_end",
"definition_id": "caido/convert-end",
"display": {
"x": -20,
"y": 190
},
"id": 1,
"inputs": [
{
"alias": "data",
"value": {
"data": "$shell.data",
"kind": "ref"
}
}
],
"name": "Convert End",
"version": "0.1.0"
},
{
"alias": "shell",
"definition_id": "caido/shell-cmd",
"display": {
"x": -20,
"y": 10
},
"id": 2,
"inputs": [
{
"alias": "shell",
"value": {
"data": "bash",
"kind": "string"
}
},
{
"alias": "code",
"value": {
"data": "# Create exports directory\nexport_dir=\"$HOME/caido_exports\"\nmkdir -p \"$export_dir\"\n\n# Create temp file first to analyze content\ntmp_file=$(mktemp)\ntrap 'rm -f \"$tmp_file\"' EXIT\n\n# Save input to temp file first\ncat - > \"$tmp_file\"\n\n# Get file info\nfile_info=$(/usr/bin/file -b \"$tmp_file\")\n\n# Generate readable timestamp (Dec-20-2024_15-30-45)\ntimestamp=$(date \"+%b-%d-%Y_%H-%M-%S\")\n\n# Set output filename - always .bin for unrecognized types\noutput_file=\"$export_dir/raw_${timestamp}.bin\"\n\n# Copy from temp file to final location\ncp \"$tmp_file\" \"$output_file\"\n\n# Get file size\nfile_size=$(stat -f%z \"$output_file\" 2>/dev/null || stat -c%s \"$output_file\")\n\n# Output results\necho \"File exported successfully:\"\necho \"Location: $output_file\"\necho \"Size: $file_size bytes\"\necho \"Type: $file_info\"\n",
"kind": "string"
}
},
{
"alias": "timeout",
"value": {
"data": 5000,
"kind": "integer"
}
},
{
"alias": "data",
"value": {
"data": "$convert_start.data",
"kind": "ref"
}
}
],
"name": "Shell",
"version": "0.1.0"
}
]
},
"kind": "convert",
"name": "Export Raw Content",
"id": "export-raw-content"
}
6 changes: 6 additions & 0 deletions convert/Export-Raw-Selection/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@

# Deserialize MessagePack to json

Author: craftysecurity

Workflow to export selected content to a file in raw format, useful for raw formats that cannot be copy pasted.
225 changes: 225 additions & 0 deletions convert/GZIP Compress/GZIP Compress.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,225 @@
{

"description": "Compresses data using GZIP",

"edition": 2,

"graph": {

"edges": [

{

"source": {

"exec_alias": "exec",

"node_id": 0

},

"target": {

"exec_alias": "exec",

"node_id": 2

}

},

{

"source": {

"exec_alias": "exec",

"node_id": 2

},

"target": {

"exec_alias": "exec",

"node_id": 1

}

}

],

"nodes": [

{

"alias": "convert_start",

"definition_id": "caido/convert-start",

"display": {

"x": -10,

"y": -100

},

"id": 0,

"inputs": [],

"name": "Convert Start",

"version": "0.1.0"

},

{

"alias": "convert_end",

"definition_id": "caido/convert-end",

"display": {

"x": -20,

"y": 190

},

"id": 1,

"inputs": [

{

"alias": "data",

"value": {

"data": "$shell.data",

"kind": "ref"

}

}

],

"name": "Convert End",

"version": "0.1.0"

},

{

"alias": "shell",

"definition_id": "caido/shell-cmd",

"display": {

"x": -20,

"y": 10

},

"id": 2,

"inputs": [

{

"alias": "shell",

"value": {

"data": "bash",

"kind": "string"

}

},

{

"alias": "code",

"value": {

"data": "# Create temp file\ntmp_file=$(mktemp)\ntmp_out=$(mktemp)\ndebug_info=\"\"\ntrap 'rm -f \"$tmp_file\" \"$tmp_out\"' EXIT\n\n# Save input to file\ncat - > \"$tmp_file\"\n\n# Try Python's gzip module with configurable compression\npython3 - \"$tmp_file\" <<'EOF'\nimport sys\nimport gzip\nfrom io import BytesIO\n\ntry:\n with open(sys.argv[1], 'rb') as f:\n data = f.read()\n \n # Compress with highest speed setting\n compressed = gzip.compress(data, compresslevel=1)\n sys.stdout.buffer.write(compressed)\n sys.exit(0)\n\nexcept Exception as e:\n print(f\"Error during compression: {e}\", file=sys.stderr)\n sys.exit(1)\nEOF\n\n# If Python script failed, try gzip command\nif [ $? -ne 0 ]; then\n if gzip -c -1 \"$tmp_file\" > \"$tmp_out\" 2>/dev/null; then\n cat \"$tmp_out\"\n exit 0\n else\n echo \"Failed to compress data\"\n exit 1\n fi\nfi\n",

"kind": "string"

}

},

{

"alias": "init",

"value": {

"data": "",

"kind": "string"

}

},

{

"alias": "timeout",

"value": {

"data": 5000,

"kind": "integer"

}

},

{

"alias": "data",

"value": {

"data": "$convert_start.data",

"kind": "ref"

}

}

],

"name": "Shell",

"version": "0.1.0"

}

]

},

"id": "gzip-compress-workflow",

"kind": "convert",

"name": "GZIP Compress"

}
11 changes: 11 additions & 0 deletions convert/GZIP Compress/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Compress to GZIP

Author: craftysecurity

Workflow to compress data to gzip for linux using shell and pyton.

### Requirements
pip3 install gzip

# Important
The gzip data format cannot be reliably copy and pasted as it contains non-printable characters and binary data that text editors and clipboards do not support properly. You should use 'replace' feature instead so the binary data is directly placed into caido intact.
Loading