Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] DIY parsing #41

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

[WIP] DIY parsing #41

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1b499c9
test
Simplychee Dec 14, 2024
a371738
sidebar?
Simplychee Dec 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions docs/logz/diy-parser.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
id: diy-parser
title: "DIY parse logs with a Sawmill pipeline"
description: "Parse logs with a Sawmill pipeline (https://github.com/logzio/sawmill/wiki)."
sidebar_label: "DIY parsing logs with Sawmill"
hide_title: true
hide_table_of_contents: true
api: eJztWOtv4zYS/1cG3A95nGw5vvZ2YXQXTbPpXbB5Ic6214sCdCyNJTYUqZKUvd4g//thSMlxXrvA9cv1cAmQUHwMZ4bz+M3cCtOQRS+NPirERJTkp7ispVKX5PzlqiGRiIJcbmXDm8REnJOdG1s7aNA6qUswc3BYN4pAmdLBUvoKEEq5IA0dMWhkQ0pqgu3K+8ZN0rSUvmpnw9zUqTLlZ2lSF/emS3kjd4awf39IOkDIjVKUMxN8Y3+5bRU58AZmBPSJ8tZTAVIDgmsol3OZg7EFWVhWZAl8ReBW2uMnQF3AvNWBIirpVzA3Spll2FO2sgh3O76MZ3pJlJxZtKsh7O6eGk+T3V3wlXRAumiM1J6Z1cZD66hgvnJL6AmMhbYpeNRxnsCs9TA3Fjw5z6I0rW2M4xu1Wg0h0+eK0BGQdi2zbiCvUJdRCEsla0Lq8PXx4pjXa/R5BSvTWsA8N632W67bOcy0SITH0onJlTiPPIjrRDRosSZPlhduheQnbtBXIhEaaxIToUz5rCHwZK8dZUqYUZACraNiCJeslBw1PwxqoE8ySun51DYqS1iswJH2zPmxKT8PpdlhPSFoWnb74sM+2hW01qlxZygSQZ+C/TFP91br8opqFJNbwaTERDhvWea7RFj6vZWWCjHxtqW76zhDzv9gihWfyI32pD0PsWmUzIOHpL85Fvz2KWkz+41yLxLRWPYnL8nxKhvwsdT0nuZSy6i2x+xscn8LmXCeGpeJCVwBf94seMyj3Oi5LPuvuSRV8EcmKlLKZCLpZ6eNkj4uQZxeoGppY/ptnJY6V21BH2jVXZgJL2uKi4oWpOKwdmUm4Bruwu813LEOo8sfm9JtyITW4kokQnqq3df1E1e/pJHe9u4SMW+VOiHnsKSvU66fbHyGeCWZblDflzeyVt5mYjwajwej14Px+HL0erL3ZjJ+869MQNDUW6nnBmpXvs3EobXB3ZXBgk1+LhVBukCbKjlLl1IrU84IfcqeP2SjmsAPbX5D/iL4ajg/AalzYy3lvnPhJHjaLGzso4zUsJWJ1g0InR/sbWVivbkyzoMsJvC3b+bv34wzIe7WP09cgCdcY7SL6huPRv+JG/TP/yhOhHAVqUNjzUIWHLIrgmCXHEJ+vbemXwHnntb+PXzZmu4euE4mTsxCUiYmGd94gt7KT5lIMjG3WNakfVhilYfZ7/lRnce6CfPj0XhvMHoz2Pv2cm882Xs9+ebb4eu/7v1lNBqNwv68ovwmbA1ch7nOysLsDJWsMUwzm5GPLhqF2Y/7g4IWMo9LZ74im4k7EV7jocIET9XkK8MpuTEu2DfH5IlIF3vrRKlMOeC7Bn2iTO9DtSO76GN6axVbe5d4sZFDFQPpk6f6OIVDdB62T41lBjX8JG0ptcQdcZc8Q2mA7YvE9p1EOMeYhLenq0LT6iUyOb5I5gA1FgjbB6S9RfUSBXqZkcOWYwNs/2hR38xb61+iodWLNH4m56EndEqsHIW6cC+Ram++ys6x0YXRLxFYvqyRwMvHKYxh+2d0ldSlD3Su+dnz1kq/Cu/+z8H++dHg8uzD4amYXF1vrk/ZdaOvP9h1++iqX0wbkniNGkuK0GL//Ai8uSHtYG5NHTz5qk/N94vX2/fiNL0s6au0QFfNDNoideQZErg0Uh/Ecyz9DjRY0jDTmd64DS31uGbQo7shMItLRmaaIuKaBThSBhzoTYeWFC1Qe0YQcBJu44jAaGN6dHjSU+XTC0kRAm5e7JzJJTK0DOhW+sDapXlBL0nAQ09v76+ROp7oldYo9IypE8iVzG8iAiW0PbzzphlYWVYeatJtEqCrI0bDcLW7e2mMcvCuk6vn+d0GP7u7f+gxgrBHjCVl3RjrMWrqhqiJcnRXBuOi4abmcrR2BY2VC6mojEi9xhvqoGz4bh1ZF2Tq9OMScAbknInDjJSkRcCQa7pQoYMZkYbc1I01tXRUJFCQIk8gfdSQpUZhzt99TRJwZSDQv4DUnspY/zhONX0ia+QHCkCGnaAiLMjeA+JNj+FQzfnw4h5AHvZJ6XkA+OdBeQ9B3tU6/95nmUeYbI26GFqtkdX/Dny6vmOLmJuncfIVHGlvTdGGcjLTofphe5UOCpO3HG2ogDbUrMzH7u5ZQyG4jYej3d11tRpMkasngn7DdHMpaoMLYuUMXOECpcKZolATFWapWWvXX6iy+d8AG5nOlJmlNTpPNu1wlOs2DZp2pmTO24arWu1Apl/BfutNjVG8Pi1gnCOoWVtmDl0cdJ2k/Hl1SdYih7ce/Nl7/gqTu3U42nTG1Pen0hh/XsF+Fz270pddnNXz8eI4hOK51MWD2nf9lE6WsVK+ryD5sDLmBtCvC+gu2GJRWHIOZmhDcrH9LSHqhAYC1gR+aQaKPEPV3BRclKMPAcsRPUOK+w8h5mzZjfQUJPvRWKiN5VMsb5A/CWSuHor8gto4fg5CwyLtA2g/GHTFf+VrtVZj6yvSvrMnNlWCzezNtrW2Km/g3EZ1HWpPtrGSQbxCDa6dsQfMyLqYhNkgStLcTqJwoovHG+mgD7truNDx+cfT0yu44GuVrCXvi7ghR6W6VNCVH8xbBBJhZwQMe6MR5EbnrbUMCvhk1w9w0NDanoZwaWBBVs5XUQq7vtExkujObKa2sI21lUATmzlcUmHuHxpqlMkyETab7xByhc695eDtyeamHihsdV5lAipL87eZqFEqbyYVqeb7Tl+ZeHfQOm9qsjBt85wt75Kw/i7Fd0PW0AEnS3IuPPvGBz+5axtO7VR0+spNXZMuqAiBhfXYK9QFPZAO9pFvUOGNUXddJZmwAwSJYmct9Os6JcWcOsn0P8IATrGmCbs4NX5wqHMTQn+3+BNntgkUNFfoKYHys2yeSDC9vDg7/fvxL3BxeHB2cnJ4+v7wfeBpa0po82orSLY1za1RaitEqQSKNti4r0haaAxXuxKVWoFCW26UrU5+pggQpA/15obH8CzZ2p3Np2S5wNtA8r0p48y0ftCGsFa7gZkPWkepSERnD5xRiN80wI37NxXJo9LggedzCaxkTtqF9NvBk/0G84o4rzw4PUnT5XI5xLA6NLZMu6MuPT46ODydHg7Gw1EIFUyXq8e+HE1CEVqj3riFW4j0XOP3ccv3cfmy0Vb4fy/5z9xLjmDQ0yefNgqlZjsJ5nbbNSuuxGIvgMggrUjEk4ZFnAtg8joRDLn41O3tDB19tOrujqd/b8lyQXudiAVayXEnoNFCOh4XYjJH5egLZrZ90XW7duC/p2n9rP76+kNz8REAPTtgIm5otdkNvQ59y1CVsCbi6kGUd9Bh8/70k84d9xviiRhuv7j3eqMTdX42vRSJmHVN8toUfMbiUiTh70RkImNuTXiD0F0I87dCoS7bWBpEuvzzb4JzLSY=
sidebar_class_name: "post api-method"
info_path: docs/logz/logz-io-api
custom_edit_url: null
hide_send_button: true
---

import ApiTabs from "@theme/ApiTabs";
import DiscriminatorTabs from "@theme/DiscriminatorTabs";
import MethodEndpoint from "@theme/ApiExplorer/MethodEndpoint";
import SecuritySchemes from "@theme/ApiExplorer/SecuritySchemes";
import MimeTabs from "@theme/MimeTabs";
import ParamsItem from "@theme/ParamsItem";
import ResponseSamples from "@theme/ResponseSamples";
import SchemaItem from "@theme/SchemaItem";
import SchemaTabs from "@theme/SchemaTabs";
import TabItem from "@theme/TabItem";

<h1 className={"openapi__heading"}>DIY parse logs with a Sawmill pipeline</h1>

<MethodEndpoint method={"post"} path={"/v1/sawmill/diy-parser/logType"}></MethodEndpoint>



Creates a new Sawmill pipeline in your account. Pipelines operate on incoming logs, parsing and transforming them into structured attributes for easier querying.
**Note:** Only admin users can access this endpoint. Use an application key created by an admin.

## Request

<details style={{"marginBottom":"1rem"}} className={"openapi-markdown__details"} data-collapsed={false} open={true}><summary style={{}}><h3 className={"openapi-markdown__details-summary-header-params"}>Path Parameters</h3></summary><div><ul><ParamsItem className={"paramsItem"} param={{"in":"path","name":"logType","description":"Type of the log being parsed. This can be an existing type (already sent to Logz.io) or a new type (to be sent to Logz.io for parsing).","example":"TestType","schema":{"type":"string"},"required":true}}></ParamsItem></ul></div></details><MimeTabs className={"openapi-tabs__mime"}><TabItem label={"application/json"} value={"application/json-schema"}><details style={{}} className={"openapi-markdown__details mime"} data-collapsed={false} open={true}><summary style={{}} className={"openapi-markdown__details-summary-mime"}><h3 className={"openapi-markdown__details-summary-header-body"}>Body</h3><strong className={"openapi-schema__required"}>required</strong></summary><div style={{"textAlign":"left","marginLeft":"1rem"}}></div><ul style={{"marginLeft":"1rem"}}><SchemaItem collapsible={false} name={"pipeLineDefinition"} required={false} schemaName={"string"} qualifierMessage={undefined} schema={{"type":"string","example":"{ \"steps\": [ { \"kv\": { \"config\": { \"field\": \"hello\", \"fieldSplit\": \" \", \"valueSplit\": \"=\", \"includeKeys\": [ \"time\", \"level\", \"msg\" ] } } } ] }"}}></SchemaItem><SchemaItem collapsible={true} className={"schemaItem"}><details style={{}} className={"openapi-markdown__details"}><summary style={{}}><span className={"openapi-schema__container"}><strong className={"openapi-schema__property"}>sampleLogs</strong><span className={"openapi-schema__name"}> object[]</span></span></summary><div style={{"marginLeft":"1rem"}}><li><div style={{"fontSize":"var(--ifm-code-font-size)","opacity":"0.6","marginLeft":"-.5rem","paddingBottom":".5rem"}}>Array [</div></li><SchemaItem collapsible={false} name={"type"} required={false} schemaName={"string"} qualifierMessage={undefined} schema={{"type":"string","example":"logType"}}></SchemaItem><SchemaItem collapsible={true} className={"schemaItem"}><details style={{}} className={"openapi-markdown__details"}><summary style={{}}><span className={"openapi-schema__container"}><strong className={"openapi-schema__property"}>fullMessage</strong><span className={"openapi-schema__name"}> object</span></span></summary><div style={{"marginLeft":"1rem"}}><SchemaItem collapsible={false} name={"message"} required={false} schemaName={"string"} qualifierMessage={undefined} schema={{"type":"string","example":"hi"}}></SchemaItem><SchemaItem collapsible={false} name={"hello"} required={false} schemaName={"string"} qualifierMessage={undefined} schema={{"type":"string","example":"time=\"2022-07-22T07:18:28Z\" level=info msg=\"Error uploading file /var/lib/winlogbeat/test.json: BucketRegionError: incorrect region, the bucket is not in '\"us-east-1'\" region, host id: 64fD82\""}}></SchemaItem></div></details></SchemaItem><li><div style={{"fontSize":"var(--ifm-code-font-size)","opacity":"0.6","marginLeft":"-.5rem"}}>]</div></li></div></details></SchemaItem></ul></details></TabItem></MimeTabs><div><div><ApiTabs><TabItem label={"200"} value={"200"}><div></div><div><MimeTabs className={"openapi-tabs__mime"} schemaType={"response"}><TabItem label={"application/json"} value={"application/json"}><SchemaTabs className={"openapi-tabs__schema"}><TabItem label={"Schema"} value={"Schema"}><details style={{}} className={"openapi-markdown__details response"} data-collapsed={false} open={true}><summary style={{}} className={"openapi-markdown__details-summary-response"}><strong>Schema</strong></summary><div style={{"textAlign":"left","marginLeft":"1rem"}}></div><ul style={{"marginLeft":"1rem"}}><li><div style={{"fontSize":"var(--ifm-code-font-size)","opacity":"0.6","marginLeft":"-.5rem","paddingBottom":".5rem"}}>Array [</div></li><div style={{"marginTop":".5rem","marginBottom":".5rem","marginLeft":"1rem"}}>

object

</div><li><div style={{"fontSize":"var(--ifm-code-font-size)","opacity":"0.6","marginLeft":"-.5rem"}}>]</div></li></ul></details></TabItem></SchemaTabs></TabItem></MimeTabs></div></TabItem></ApiTabs></div></div>
Loading