diff --git a/LICENSES/Apache-2.0.txt b/LICENSES/Apache-2.0.txt new file mode 100644 index 00000000..137069b8 --- /dev/null +++ b/LICENSES/Apache-2.0.txt @@ -0,0 +1,73 @@ +Apache License +Version 2.0, January 2004 +http://www.apache.org/licenses/ + +TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + +1. Definitions. + +"License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. + +"Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. + +"Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. + +"You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. + +"Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. + +"Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. + +"Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). + +"Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. + +"Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." + +"Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. + +2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. + +3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. + +4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: + + (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. + + You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. + +5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. + +6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. + +7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. + +8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. + +9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. + +END OF TERMS AND CONDITIONS + +APPENDIX: How to apply the Apache License to your work. + +To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. + +Copyright [yyyy] [name of copyright owner] + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. diff --git a/LICENSES/LicenseRef-Go119-BSD-Patentgrant.txt b/LICENSES/LicenseRef-Go119-BSD-Patentgrant.txt deleted file mode 100644 index fa1aad86..00000000 --- a/LICENSES/LicenseRef-Go119-BSD-Patentgrant.txt +++ /dev/null @@ -1,51 +0,0 @@ -Copyright (c) 2009 The Go Authors. All rights reserved. - -Redistribution and use in source and binary forms, with or without -modification, are permitted provided that the following conditions are -met: - - * Redistributions of source code must retain the above copyright -notice, this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above -copyright notice, this list of conditions and the following disclaimer -in the documentation and/or other materials provided with the -distribution. - * Neither the name of Google Inc. nor the names of its -contributors may be used to endorse or promote products derived from -this software without specific prior written permission. - -THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS -"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT -LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR -A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT -OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, -SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT -LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, -DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY -THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT -(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - - -Additional IP Rights Grant (Patents) - -"This implementation" means the copyrightable works distributed by -Google as part of the Go project. - -Google hereby grants to You a perpetual, worldwide, non-exclusive, -no-charge, royalty-free, irrevocable (except as stated in this section) -patent license to make, have made, use, offer to sell, sell, import, -transfer and otherwise run, modify and propagate the contents of this -implementation of Go, where such license applies only to those patent -claims, both currently owned or controlled by Google and acquired in -the future, licensable by Google that are necessarily infringed by this -implementation of Go. This grant does not include claims that would be -infringed only as a consequence of further modification of this -implementation. If you or your agent or exclusive licensee institute or -order or agree to the institution of patent litigation against any -entity (including a cross-claim or counterclaim in a lawsuit) alleging -that this implementation of Go or any code incorporated within this -implementation of Go constitutes direct or contributory patent -infringement, or inducement of patent infringement, then any patent -rights granted to you under this License for this implementation of Go -shall terminate as of the date such litigation is filed. diff --git a/Makefile b/Makefile index 19e31c7a..b4b3964b 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,7 @@ -# This file is Free Software under the MIT License -# without warranty, see README.md and LICENSES/MIT.txt for details. +# This file is Free Software under the Apache-2.0 License +# without warranty, see README.md and LICENSES/Apache-2.0.txt for details. # -# SPDX-License-Identifier: MIT +# SPDX-License-Identifier: Apache-2.0 # # SPDX-FileCopyrightText: 2021 German Federal Office for Information Security (BSI) # Software-Engineering: 2021 Intevation GmbH diff --git a/README.md b/README.md index 69601cd6..bc9ae2a8 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,18 @@ + + # csaf_distribution -An implementation of a [CSAF](https://csaf.io/) -[2.0 Spec](https://docs.oasis-open.org/csaf/csaf/v2.0/os/csaf-v2.0-os.html) -([Errata](https://docs.oasis-open.org/csaf/csaf/v2.0/csaf-v2.0.html)) +Implements a [CSAF](https://csaf.io/) +([specification v2.0](https://docs.oasis-open.org/csaf/csaf/v2.0/os/csaf-v2.0-os.html) +and its [errata](https://docs.oasis-open.org/csaf/csaf/v2.0/csaf-v2.0.html)) trusted provider, checker, aggregator and downloader. Includes an uploader command line tool for the trusted provider. @@ -59,7 +69,7 @@ Download the binaries from the most recent release assets on Github. ### Build from sources -- A recent version of **Go** (1.20+) should be installed. [Go installation](https://go.dev/doc/install) +- A recent version of **Go** (1.21+) should be installed. [Go installation](https://go.dev/doc/install) - Clone the repository `git clone https://github.com/csaf-poc/csaf_distribution.git ` @@ -90,7 +100,7 @@ For further details of the development process consult our [development page](./ ## License -- `csaf_distribution` is licensed as Free Software under MIT License. +- `csaf_distribution` is licensed as Free Software under the terms of the [Apache License, Version 2.0](./LICENSES/Apache-2.0.txt). - See the specific source files for details, the license itself can be found in the directory `LICENSES/`. diff --git a/cmd/csaf_aggregator/client.go b/cmd/csaf_aggregator/client.go index deb108a8..8200d34e 100644 --- a/cmd/csaf_aggregator/client.go +++ b/cmd/csaf_aggregator/client.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH diff --git a/cmd/csaf_aggregator/config.go b/cmd/csaf_aggregator/config.go index 57a619d7..adb3c627 100644 --- a/cmd/csaf_aggregator/config.go +++ b/cmd/csaf_aggregator/config.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH @@ -12,7 +12,7 @@ import ( "crypto/tls" "errors" "fmt" - "log" + "log/slog" "net/http" "os" "runtime" @@ -178,9 +178,11 @@ func (p *provider) ageAccept(c *config) func(time.Time) bool { } if c.Verbose { - log.Printf( - "Setting up filter to accept advisories within time range %s to %s\n", - r[0].Format(time.RFC3339), r[1].Format(time.RFC3339)) + slog.Debug( + "Setting up filter to accept advisories within time range", + "from", r[0].Format(time.RFC3339), + "to", r[1].Format(time.RFC3339), + ) } return r.Contains } @@ -393,6 +395,17 @@ func (c *config) setDefaults() { } } +// prepareLogging sets up the structured logging. +func (c *config) prepareLogging() error { + ho := slog.HandlerOptions{ + Level: slog.LevelDebug, + } + handler := slog.NewTextHandler(os.Stdout, &ho) + logger := slog.New(handler) + slog.SetDefault(logger) + return nil +} + // compileIgnorePatterns compiles the configured patterns to be ignored. func (p *provider) compileIgnorePatterns() error { pm, err := filter.NewPatternMatcher(p.IgnorePattern) diff --git a/cmd/csaf_aggregator/files.go b/cmd/csaf_aggregator/files.go index adf04aa1..18ccbb60 100644 --- a/cmd/csaf_aggregator/files.go +++ b/cmd/csaf_aggregator/files.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH diff --git a/cmd/csaf_aggregator/full.go b/cmd/csaf_aggregator/full.go index 600c650f..9ec9812a 100644 --- a/cmd/csaf_aggregator/full.go +++ b/cmd/csaf_aggregator/full.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH @@ -11,7 +11,7 @@ package main import ( "errors" "fmt" - "log" + "log/slog" "os" "path/filepath" "strings" @@ -29,11 +29,13 @@ type fullJob struct { err error } -// setupProviderFull fetches the provider-metadate.json for a specific provider. +// setupProviderFull fetches the provider-metadata.json for a specific provider. func (w *worker) setupProviderFull(provider *provider) error { - log.Printf("worker #%d: %s (%s)\n", - w.num, provider.Name, provider.Domain) - + w.log.Info("Setting up provider", + "provider", slog.GroupValue( + slog.String("name", provider.Name), + slog.String("domain", provider.Domain), + )) w.dir = "" w.provider = provider @@ -55,7 +57,7 @@ func (w *worker) setupProviderFull(provider *provider) error { "provider-metadata.json has %d validation issues", len(errors)) } - log.Printf("provider-metadata: %s\n", w.loc) + w.log.Info("Using provider-metadata", "url", w.loc) return nil } @@ -79,7 +81,7 @@ func (w *worker) fullWork(wg *sync.WaitGroup, jobs <-chan *fullJob) { func (p *processor) full() error { if p.cfg.runAsMirror() { - log.Println("Running in aggregator mode") + p.log.Info("Running in aggregator mode") // check if we need to setup a remote validator if p.cfg.RemoteValidatorOptions != nil { @@ -96,16 +98,18 @@ func (p *processor) full() error { }() } } else { - log.Println("Running in lister mode") + p.log.Info("Running in lister mode") } queue := make(chan *fullJob) var wg sync.WaitGroup - log.Printf("Starting %d workers.\n", p.cfg.Workers) + p.log.Info("Starting workers...", "num", p.cfg.Workers) + for i := 1; i <= p.cfg.Workers; i++ { wg.Add(1) w := newWorker(i, p) + go w.fullWork(&wg, queue) } @@ -135,12 +139,22 @@ func (p *processor) full() error { for i := range jobs { j := &jobs[i] if j.err != nil { - log.Printf("error: '%s' failed: %v\n", j.provider.Name, j.err) + p.log.Error("Job execution failed", + slog.Group("job", + slog.Group("provider"), + "name", j.provider.Name, + ), + "err", j.err, + ) continue } if j.aggregatorProvider == nil { - log.Printf( - "error: '%s' does not produce any result.\n", j.provider.Name) + p.log.Error("Job did not produce any result", + slog.Group("job", + slog.Group("provider"), + "name", j.provider.Name, + ), + ) continue } diff --git a/cmd/csaf_aggregator/indices.go b/cmd/csaf_aggregator/indices.go index 69954bd0..272d25b4 100644 --- a/cmd/csaf_aggregator/indices.go +++ b/cmd/csaf_aggregator/indices.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH @@ -12,7 +12,6 @@ import ( "bufio" "encoding/csv" "fmt" - "log" "os" "path/filepath" "sort" @@ -377,7 +376,7 @@ func (w *worker) writeIndices() error { } for label, summaries := range w.summaries { - log.Printf("%s: %d\n", label, len(summaries)) + w.log.Debug("Writing indices", "label", label, "summaries.num", len(summaries)) if err := w.writeInterims(label, summaries); err != nil { return err } diff --git a/cmd/csaf_aggregator/interim.go b/cmd/csaf_aggregator/interim.go index bdd5ebcd..023c9c42 100644 --- a/cmd/csaf_aggregator/interim.go +++ b/cmd/csaf_aggregator/interim.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH @@ -17,7 +17,6 @@ import ( "errors" "fmt" "io" - "log" "net/http" "os" "path/filepath" @@ -102,12 +101,12 @@ func (w *worker) checkInterims( // XXX: Should we return an error here? for _, e := range errors { - log.Printf("validation error: %s: %v\n", url, e) + w.log.Error("validation error", "url", url, "err", e) } // We need to write the changed content. - // This will start the transcation if not already started. + // This will start the transaction if not already started. dst, err := tx.Dst() if err != nil { return nil, err @@ -159,8 +158,7 @@ func (w *worker) checkInterims( // setupProviderInterim prepares the worker for a specific provider. func (w *worker) setupProviderInterim(provider *provider) { - log.Printf("worker #%d: %s (%s)\n", - w.num, provider.Name, provider.Domain) + w.log.Info("Setting up worker", provider.Name, provider.Domain) w.dir = "" w.provider = provider @@ -262,7 +260,7 @@ func (p *processor) interim() error { queue := make(chan *interimJob) var wg sync.WaitGroup - log.Printf("Starting %d workers.\n", p.cfg.Workers) + p.log.Info("Starting workers...", "num", p.cfg.Workers) for i := 1; i <= p.cfg.Workers; i++ { wg.Add(1) w := newWorker(i, p) diff --git a/cmd/csaf_aggregator/lazytransaction.go b/cmd/csaf_aggregator/lazytransaction.go index a2b1e94d..606d892f 100644 --- a/cmd/csaf_aggregator/lazytransaction.go +++ b/cmd/csaf_aggregator/lazytransaction.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH @@ -9,7 +9,7 @@ package main import ( - "log" + "log/slog" "os" "path/filepath" @@ -85,7 +85,8 @@ func (lt *lazyTransaction) commit() error { os.RemoveAll(lt.dst) return err } - log.Printf("Move %q -> %q\n", symlink, lt.src) + + slog.Debug("Moving directory", "from", symlink, "to", lt.src) if err := os.Rename(symlink, lt.src); err != nil { os.RemoveAll(lt.dst) return err diff --git a/cmd/csaf_aggregator/lister.go b/cmd/csaf_aggregator/lister.go index a3bfd29f..4d758e49 100644 --- a/cmd/csaf_aggregator/lister.go +++ b/cmd/csaf_aggregator/lister.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH diff --git a/cmd/csaf_aggregator/main.go b/cmd/csaf_aggregator/main.go index 6a85fe2f..68907d30 100644 --- a/cmd/csaf_aggregator/main.go +++ b/cmd/csaf_aggregator/main.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH @@ -11,6 +11,7 @@ package main import ( "fmt" + "log/slog" "os" "path/filepath" @@ -44,8 +45,9 @@ func lock(lockFile *string, fn func() error) error { func main() { _, cfg, err := parseArgsConfig() - options.ErrorCheck(err) - options.ErrorCheck(cfg.prepare()) - p := processor{cfg: cfg} - options.ErrorCheck(lock(cfg.LockFile, p.process)) + cfg.prepareLogging() + options.ErrorCheckStructured(err) + options.ErrorCheckStructured(cfg.prepare()) + p := processor{cfg: cfg, log: slog.Default()} + options.ErrorCheckStructured(lock(cfg.LockFile, p.process)) } diff --git a/cmd/csaf_aggregator/mirror.go b/cmd/csaf_aggregator/mirror.go index 3acb48e0..6bf72a33 100644 --- a/cmd/csaf_aggregator/mirror.go +++ b/cmd/csaf_aggregator/mirror.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH @@ -16,7 +16,7 @@ import ( "encoding/json" "fmt" "io" - "log" + "log/slog" "net/http" "net/url" "os" @@ -47,7 +47,7 @@ func (w *worker) mirror() (*csaf.AggregatorCSAFProvider, error) { if err != nil && w.dir != "" { // If something goes wrong remove the debris. if err := os.RemoveAll(w.dir); err != nil { - log.Printf("error: %v\n", err) + w.log.Error("Could not remove directory", "path", w.dir, "err", err) } } return result, err @@ -166,7 +166,7 @@ func (w *worker) writeProviderMetadata() error { {Expr: `$.public_openpgp_keys`, Action: util.ReMarshalMatcher(&pm.PGPKeys)}, }, w.metadataProvider); err != nil { // only log the errors - log.Printf("extracting data from orignal provider failed: %v\n", err) + w.log.Error("Extracting data from original provider failed", "err", err) } // We are mirroring the remote public keys, too. @@ -196,11 +196,11 @@ func (w *worker) mirrorPGPKeys(pm *csaf.ProviderMetadata) error { for i := range pm.PGPKeys { pgpKey := &pm.PGPKeys[i] if pgpKey.URL == nil { - log.Printf("ignoring PGP key without URL: %s\n", pgpKey.Fingerprint) + w.log.Warn("Ignoring PGP key without URL", "fingerprint", pgpKey.Fingerprint) continue } if _, err := hex.DecodeString(string(pgpKey.Fingerprint)); err != nil { - log.Printf("ignoring PGP with invalid fingerprint: %s\n", *pgpKey.URL) + w.log.Warn("Ignoring PGP key with invalid fingerprint", "url", *pgpKey.URL) continue } @@ -344,7 +344,7 @@ func (w *worker) doMirrorTransaction() error { // Check if there is a sysmlink already. target := filepath.Join(w.processor.cfg.Folder, w.provider.Name) - log.Printf("target: '%s'\n", target) + w.log.Debug("Checking for path existance", "path", target) exists, err := util.PathExists(target) if err != nil { @@ -359,7 +359,7 @@ func (w *worker) doMirrorTransaction() error { } } - log.Printf("sym link: %s -> %s\n", w.dir, target) + w.log.Debug("Creating sym link", "from", w.dir, "to", target) // Create a new symlink if err := os.Symlink(w.dir, target); err != nil { @@ -368,7 +368,7 @@ func (w *worker) doMirrorTransaction() error { } // Move the symlink - log.Printf("Move: %s -> %s\n", target, webTarget) + w.log.Debug("Moving sym link", "from", target, "to", webTarget) if err := os.Rename(target, webTarget); err != nil { os.RemoveAll(w.dir) return err @@ -499,14 +499,14 @@ func (w *worker) mirrorFiles(tlpLabel csaf.TLPLabel, files []csaf.AdvisoryFile) u, err := url.Parse(file.URL()) if err != nil { - log.Printf("error: %s\n", err) + w.log.Error("Could not parse advisory file URL", "err", err) continue } // Should we ignore this advisory? if w.provider.ignoreURL(file.URL(), w.processor.cfg) { if w.processor.cfg.Verbose { - log.Printf("Ignoring %s: %q\n", w.provider.Name, file.URL()) + w.log.Info("Ignoring advisory", slog.Group("provider", "name", w.provider.Name), "file", file) } continue } @@ -514,7 +514,7 @@ func (w *worker) mirrorFiles(tlpLabel csaf.TLPLabel, files []csaf.AdvisoryFile) // Ignore not conforming filenames. filename := filepath.Base(u.Path) if !util.ConformingFileName(filename) { - log.Printf("Not conforming filename %q. Ignoring.\n", filename) + w.log.Warn("Ignoring advisory because of non-conforming filename", "filename", filename) continue } @@ -531,19 +531,18 @@ func (w *worker) mirrorFiles(tlpLabel csaf.TLPLabel, files []csaf.AdvisoryFile) } if err := downloadJSON(w.client, file.URL(), download); err != nil { - log.Printf("error: %v\n", err) + w.log.Error("Error while downloading JSON", "err", err) continue } // Check against CSAF schema. errors, err := csaf.ValidateCSAF(advisory) if err != nil { - log.Printf("error: %s: %v", file, err) + w.log.Error("Error while validating CSAF schema", "err", err) continue } if len(errors) > 0 { - log.Printf("CSAF file %s has %d validation errors.\n", - file, len(errors)) + w.log.Error("CSAF file has validation errors", "num.errors", len(errors), "file", file) continue } @@ -551,29 +550,27 @@ func (w *worker) mirrorFiles(tlpLabel csaf.TLPLabel, files []csaf.AdvisoryFile) if rmv := w.processor.remoteValidator; rmv != nil { rvr, err := rmv.Validate(advisory) if err != nil { - log.Printf("Calling remote validator failed: %s\n", err) + w.log.Error("Calling remote validator failed", "err", err) continue } if !rvr.Valid { - log.Printf( - "CSAF file %s does not validate remotely.\n", file) + w.log.Error("CSAF file does not validate remotely", "file", file.URL()) continue } } sum, err := csaf.NewAdvisorySummary(w.expr, advisory) if err != nil { - log.Printf("error: %s: %v\n", file, err) + w.log.Error("Error while creating new advisory", "file", file, "err", err) continue } if util.CleanFileName(sum.ID) != filename { - log.Printf("ID %q does not match filename %s", - sum.ID, filename) + w.log.Error("ID mismatch", "id", sum.ID, "filename", filename) } if err := w.extractCategories(label, advisory); err != nil { - log.Printf("error: %s: %v\n", file, err) + w.log.Error("Could not extract categories", "file", file, "err", err) continue } @@ -624,7 +621,7 @@ func (w *worker) downloadSignatureOrSign(url, fname string, data []byte) error { if err != nil { if err != errNotFound { - log.Printf("error: %s: %v\n", url, err) + w.log.Error("Could not find signature URL", "url", url, "err", err) } // Sign it our self. if sig, err = w.sign(data); err != nil { diff --git a/cmd/csaf_aggregator/processor.go b/cmd/csaf_aggregator/processor.go index ccd5062e..9f10a774 100644 --- a/cmd/csaf_aggregator/processor.go +++ b/cmd/csaf_aggregator/processor.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH @@ -10,14 +10,14 @@ package main import ( "fmt" - "log" + "log/slog" "os" "path/filepath" - "github.com/ProtonMail/gopenpgp/v2/crypto" - "github.com/csaf-poc/csaf_distribution/v3/csaf" "github.com/csaf-poc/csaf_distribution/v3/util" + + "github.com/ProtonMail/gopenpgp/v2/crypto" ) type processor struct { @@ -26,6 +26,9 @@ type processor struct { // remoteValidator is a globally configured remote validator. remoteValidator csaf.RemoteValidator + + // log is the structured logger for the whole processor. + log *slog.Logger } type summary struct { @@ -48,6 +51,7 @@ type worker struct { dir string // Directory to store data to. summaries map[string][]summary // the summaries of the advisories. categories map[string]util.Set[string] // the categories per label. + log *slog.Logger // the structured logger, supplied with the worker number. } func newWorker(num int, processor *processor) *worker { @@ -55,6 +59,7 @@ func newWorker(num int, processor *processor) *worker { num: num, processor: processor, expr: util.NewPathEval(), + log: processor.log.With(slog.Int("worker", num)), } } @@ -86,9 +91,10 @@ func (w *worker) locateProviderMetadata(domain string) error { if w.processor.cfg.Verbose { for i := range lpmd.Messages { - log.Printf( - "Loading provider-metadata.json of %q: %s\n", - domain, lpmd.Messages[i].Message) + w.log.Info( + "Loading provider-metadata.json", + "domain", domain, + "message", lpmd.Messages[i].Message) } } @@ -141,7 +147,7 @@ func (p *processor) removeOrphans() error { fi, err := entry.Info() if err != nil { - log.Printf("error: %v\n", err) + p.log.Error("Could not retrieve file info", "err", err) continue } @@ -153,13 +159,13 @@ func (p *processor) removeOrphans() error { d := filepath.Join(path, entry.Name()) r, err := filepath.EvalSymlinks(d) if err != nil { - log.Printf("error: %v\n", err) + p.log.Error("Could not evaluate symlink", "err", err) continue } fd, err := os.Stat(r) if err != nil { - log.Printf("error: %v\n", err) + p.log.Error("Could not retrieve file stats", "err", err) continue } @@ -169,18 +175,18 @@ func (p *processor) removeOrphans() error { } // Remove the link. - log.Printf("removing link %s -> %s\n", d, r) + p.log.Info("Removing link", "path", fmt.Sprintf("%s -> %s", d, r)) if err := os.Remove(d); err != nil { - log.Printf("error: %v\n", err) + p.log.Error("Could not remove symlink", "err", err) continue } // Only remove directories which are in our folder. if rel, err := filepath.Rel(prefix, r); err == nil && rel == filepath.Base(r) { - log.Printf("removing directory %s\n", r) + p.log.Info("Remove directory", "path", r) if err := os.RemoveAll(r); err != nil { - log.Printf("error: %v\n", err) + p.log.Error("Could not remove directory", "err", err) } } } diff --git a/cmd/csaf_checker/config.go b/cmd/csaf_checker/config.go index 25d0ab4a..a31910d5 100644 --- a/cmd/csaf_checker/config.go +++ b/cmd/csaf_checker/config.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH diff --git a/cmd/csaf_checker/links.go b/cmd/csaf_checker/links.go index 57844898..0456acea 100644 --- a/cmd/csaf_checker/links.go +++ b/cmd/csaf_checker/links.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH diff --git a/cmd/csaf_checker/links_test.go b/cmd/csaf_checker/links_test.go index 3229511e..8abf4e6a 100644 --- a/cmd/csaf_checker/links_test.go +++ b/cmd/csaf_checker/links_test.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH diff --git a/cmd/csaf_checker/main.go b/cmd/csaf_checker/main.go index e636ab93..920a59be 100644 --- a/cmd/csaf_checker/main.go +++ b/cmd/csaf_checker/main.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH diff --git a/cmd/csaf_checker/processor.go b/cmd/csaf_checker/processor.go index 8f3a6c11..451a315c 100644 --- a/cmd/csaf_checker/processor.go +++ b/cmd/csaf_checker/processor.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2021 German Federal Office for Information Security (BSI) // Software-Engineering: 2021 Intevation GmbH diff --git a/cmd/csaf_checker/report.go b/cmd/csaf_checker/report.go index 8c652120..5d230f7f 100644 --- a/cmd/csaf_checker/report.go +++ b/cmd/csaf_checker/report.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2021 German Federal Office for Information Security (BSI) // Software-Engineering: 2021 Intevation GmbH diff --git a/cmd/csaf_checker/reporters.go b/cmd/csaf_checker/reporters.go index c707a144..016d3713 100644 --- a/cmd/csaf_checker/reporters.go +++ b/cmd/csaf_checker/reporters.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH diff --git a/cmd/csaf_checker/roliecheck.go b/cmd/csaf_checker/roliecheck.go index 94b1c2f0..53d11500 100644 --- a/cmd/csaf_checker/roliecheck.go +++ b/cmd/csaf_checker/roliecheck.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH diff --git a/cmd/csaf_checker/rules.go b/cmd/csaf_checker/rules.go index 6981b6b1..eadbbb24 100644 --- a/cmd/csaf_checker/rules.go +++ b/cmd/csaf_checker/rules.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH diff --git a/cmd/csaf_downloader/config.go b/cmd/csaf_downloader/config.go index 3a25cbcd..1448ecc8 100644 --- a/cmd/csaf_downloader/config.go +++ b/cmd/csaf_downloader/config.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH @@ -13,13 +13,12 @@ import ( "fmt" "io" "log" + "log/slog" "net/http" "os" "path/filepath" "time" - "golang.org/x/exp/slog" - "github.com/csaf-poc/csaf_distribution/v3/internal/certs" "github.com/csaf-poc/csaf_distribution/v3/internal/filter" "github.com/csaf-poc/csaf_distribution/v3/pkg/models" @@ -58,6 +57,8 @@ type Config struct { IgnorePattern []string `long:"ignore_pattern" short:"i" description:"Do not download files if their URLs match any of the given PATTERNs" value-name:"PATTERN" toml:"ignore_pattern"` ExtraHeader http.Header `long:"header" short:"H" description:"One or more extra HTTP header fields" toml:"header"` + EnumeratePMDOnly bool `long:"enumerate_pmd_only" description:"If this flag is set to true, the downloader will only enumerate valid provider metadata files, but not download documents" toml:"enumerate_pmd_only"` + RemoteValidator string `long:"validator" description:"URL to validate documents remotely" value-name:"URL" toml:"validator"` RemoteValidatorCache string `long:"validator_cache" description:"FILE to cache remote validations" value-name:"FILE" toml:"validator_cache"` RemoteValidatorPresets []string `long:"validator_preset" description:"One or more PRESETS to validate remotely" value-name:"PRESETS" toml:"validator_preset"` diff --git a/cmd/csaf_downloader/downloader.go b/cmd/csaf_downloader/downloader.go index c197c847..d61ed34f 100644 --- a/cmd/csaf_downloader/downloader.go +++ b/cmd/csaf_downloader/downloader.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022, 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2022, 2023 Intevation GmbH @@ -18,6 +18,7 @@ import ( "fmt" "hash" "io" + "log/slog" "net/http" "net/url" "os" @@ -28,8 +29,6 @@ import ( "sync" "time" - "golang.org/x/exp/slog" - "github.com/ProtonMail/gopenpgp/v2/crypto" "golang.org/x/time/rate" @@ -169,6 +168,36 @@ func httpLog(who string) func(string, string) { } } +func (d *Downloader) enumerate(domain string) error { + client := d.httpClient() + + loader := csaf.NewProviderMetadataLoader(client) + lpmd := loader.Enumerate(domain) + + docs := []any{} + + for _, pmd := range lpmd { + if d.cfg.verbose() { + for i := range pmd.Messages { + slog.Debug("Enumerating provider-metadata.json", + "domain", domain, + "message", pmd.Messages[i].Message) + } + } + + docs = append(docs, pmd.Document) + } + + // print the results + doc, err := json.MarshalIndent(docs, "", " ") + if err != nil { + slog.Error("Couldn't marshal PMD document json") + } + fmt.Println(string(doc)) + + return nil +} + func (d *Downloader) download(ctx context.Context, domain string) error { client := d.httpClient() @@ -776,3 +805,14 @@ func (d *Downloader) Run(ctx context.Context, domains []string) error { } return nil } + +// runEnumerate performs the enumeration of PMDs for all the given domains. +func (d *Downloader) RunEnumerate(domains []string) error { + defer d.stats.log() + for _, domain := range domains { + if err := d.enumerate(domain); err != nil { + return err + } + } + return nil +} diff --git a/cmd/csaf_downloader/forwarder.go b/cmd/csaf_downloader/forwarder.go index bfe21609..a2365973 100644 --- a/cmd/csaf_downloader/forwarder.go +++ b/cmd/csaf_downloader/forwarder.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH @@ -12,14 +12,13 @@ import ( "bytes" "crypto/tls" "io" + "log/slog" "mime/multipart" "net/http" "os" "path/filepath" "strings" - "golang.org/x/exp/slog" - "github.com/csaf-poc/csaf_distribution/v3/internal/misc" "github.com/csaf-poc/csaf_distribution/v3/util" ) diff --git a/cmd/csaf_downloader/forwarder_test.go b/cmd/csaf_downloader/forwarder_test.go index 2cd0a430..abfed42f 100644 --- a/cmd/csaf_downloader/forwarder_test.go +++ b/cmd/csaf_downloader/forwarder_test.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH @@ -14,6 +14,7 @@ import ( "encoding/json" "errors" "io" + "log/slog" "mime" "mime/multipart" "net/http" @@ -22,8 +23,6 @@ import ( "strings" "testing" - "golang.org/x/exp/slog" - "github.com/csaf-poc/csaf_distribution/v3/pkg/options" "github.com/csaf-poc/csaf_distribution/v3/util" ) diff --git a/cmd/csaf_downloader/main/main.go b/cmd/csaf_downloader/main/main.go index 19f70e26..dced3ef2 100644 --- a/cmd/csaf_downloader/main/main.go +++ b/cmd/csaf_downloader/main/main.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH @@ -11,11 +11,10 @@ package main import ( "context" + "log/slog" "os" "os/signal" - "golang.org/x/exp/slog" - "github.com/csaf-poc/csaf_distribution/v3/cmd/csaf_downloader" "github.com/csaf-poc/csaf_distribution/v3/pkg/options" ) @@ -42,6 +41,11 @@ func run(cfg *csaf_downloader.Config, domains []string) error { d.Forwarder = f } + // If the enumerate-only flag is set, enumerate found PMDs, + // else use the normal load method + if cfg.EnumeratePMDOnly { + return d.RunEnumerate(domains) + } return d.Run(ctx, domains) } diff --git a/cmd/csaf_downloader/stats.go b/cmd/csaf_downloader/stats.go index d8c9e154..c0c30828 100644 --- a/cmd/csaf_downloader/stats.go +++ b/cmd/csaf_downloader/stats.go @@ -1,14 +1,14 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH package csaf_downloader -import "golang.org/x/exp/slog" +import "log/slog" // stats contains counters of the downloads. type stats struct { diff --git a/cmd/csaf_downloader/stats_test.go b/cmd/csaf_downloader/stats_test.go index 822c7cb9..be11415d 100644 --- a/cmd/csaf_downloader/stats_test.go +++ b/cmd/csaf_downloader/stats_test.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH @@ -11,9 +11,8 @@ package csaf_downloader import ( "bytes" "encoding/json" + "log/slog" "testing" - - "golang.org/x/exp/slog" ) func TestStatsAdd(t *testing.T) { diff --git a/cmd/csaf_provider/actions.go b/cmd/csaf_provider/actions.go index 54d4e24c..8f385e6e 100644 --- a/cmd/csaf_provider/actions.go +++ b/cmd/csaf_provider/actions.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH diff --git a/cmd/csaf_provider/config.go b/cmd/csaf_provider/config.go index af99cc1e..49a72047 100644 --- a/cmd/csaf_provider/config.go +++ b/cmd/csaf_provider/config.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2021 German Federal Office for Information Security (BSI) // Software-Engineering: 2021 Intevation GmbH diff --git a/cmd/csaf_provider/controller.go b/cmd/csaf_provider/controller.go index c8680ff9..7f64fe29 100644 --- a/cmd/csaf_provider/controller.go +++ b/cmd/csaf_provider/controller.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2021 German Federal Office for Information Security (BSI) // Software-Engineering: 2021 Intevation GmbH diff --git a/cmd/csaf_provider/create.go b/cmd/csaf_provider/create.go index 8e882a50..56893c69 100644 --- a/cmd/csaf_provider/create.go +++ b/cmd/csaf_provider/create.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2021 German Federal Office for Information Security (BSI) // Software-Engineering: 2021 Intevation GmbH diff --git a/cmd/csaf_provider/files.go b/cmd/csaf_provider/files.go index 0b3c5ed4..39a97e36 100644 --- a/cmd/csaf_provider/files.go +++ b/cmd/csaf_provider/files.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2021 German Federal Office for Information Security (BSI) // Software-Engineering: 2021 Intevation GmbH diff --git a/cmd/csaf_provider/indices.go b/cmd/csaf_provider/indices.go index a7ecd3b3..805371bb 100644 --- a/cmd/csaf_provider/indices.go +++ b/cmd/csaf_provider/indices.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2021 German Federal Office for Information Security (BSI) // Software-Engineering: 2021 Intevation GmbH diff --git a/cmd/csaf_provider/main.go b/cmd/csaf_provider/main.go index 22646769..8740e814 100644 --- a/cmd/csaf_provider/main.go +++ b/cmd/csaf_provider/main.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2021 German Federal Office for Information Security (BSI) // Software-Engineering: 2021 Intevation GmbH diff --git a/cmd/csaf_provider/mux.go b/cmd/csaf_provider/mux.go index 34b7e2e1..021c074e 100644 --- a/cmd/csaf_provider/mux.go +++ b/cmd/csaf_provider/mux.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2021 German Federal Office for Information Security (BSI) // Software-Engineering: 2021 Intevation GmbH diff --git a/cmd/csaf_provider/rolie.go b/cmd/csaf_provider/rolie.go index ea48480c..98448bd3 100644 --- a/cmd/csaf_provider/rolie.go +++ b/cmd/csaf_provider/rolie.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2021 German Federal Office for Information Security (BSI) // Software-Engineering: 2021 Intevation GmbH diff --git a/cmd/csaf_provider/tmpl/create.html b/cmd/csaf_provider/tmpl/create.html index 74fef6d6..0b06f6f6 100644 --- a/cmd/csaf_provider/tmpl/create.html +++ b/cmd/csaf_provider/tmpl/create.html @@ -1,8 +1,8 @@ diff --git a/docs/csaf_provider.md b/docs/csaf_provider.md index b02165b2..81a45fa2 100644 --- a/docs/csaf_provider.md +++ b/docs/csaf_provider.md @@ -100,22 +100,12 @@ The following example file documents all available configuration options: #tlps = ["csaf", "white", "amber", "green", "red"] # Make the provider create a ROLIE service document. -#create_service_document = true +#create_service_document = false # Make the provider create a ROLIE category document from a list of strings. # If a list item starts with `expr:` # the rest of the string is used as a JsonPath expression # to extract a string from the incoming advisories. -# If the result of the expression is a string this string -# is used. If the result is an array each element of -# this array is tested if it is a string or an array. -# If this test fails the expression fails. If the -# test succeeds the rules are applied recursively to -# collect all strings in the result. -# Suggested expressions are: -# - vendor, product family and product names: "expr:$.product_tree..branches[?(@.category==\"vendor\" || @.category==\"product_family\" || @.category==\"product_name\")].name" -# - CVEs: "expr:$.vulnerabilities[*].cve" -# - CWEs: "expr:$.vulnerabilities[*].cwe.id" # Strings not starting with `expr:` are taken verbatim. # By default no category documents are created. # This example provides an overview over the syntax, diff --git a/docs/scripts/DNSConfigForItest.sh b/docs/scripts/DNSConfigForItest.sh index aa9a8ee6..f7b85f04 100755 --- a/docs/scripts/DNSConfigForItest.sh +++ b/docs/scripts/DNSConfigForItest.sh @@ -1,9 +1,9 @@ #!/usr/bin/env bash # -# This file is Free Software under the MIT License -# without warranty, see README.md and LICENSES/MIT.txt for details. +# This file is Free Software under the Apache-2.0 License +# without warranty, see README.md and LICENSES/Apache-2.0.txt for details. # -# SPDX-License-Identifier: MIT +# SPDX-License-Identifier: Apache-2.0 # # SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) # Software-Engineering: 2022 Intevation GmbH diff --git a/docs/scripts/Readme.md b/docs/scripts/Readme.md index 65c5260f..a3b932d6 100644 --- a/docs/scripts/Readme.md +++ b/docs/scripts/Readme.md @@ -1,6 +1,7 @@ -Scripts for assisting the Integration tests. They are written on Ubuntu 20.04 TLS amd64. +Scripts for assisting the Integration tests. +They were written on Ubuntu 20.04 TLS amd64 and also tested with 24.04 TLS. -- `prepareUbunutForITest.sh` installs the required packages for the csaf_distribution integration tests on a naked ubuntu 20.04 LTS amd64. +- `prepareUbuntuInstanceForITests.sh` installs the required packages for the csaf_distribution integration tests on a naked ubuntu LTS amd64. - `TLSConfigsForITest.sh` generates a root CA and webserver cert by running `createRootCAForITest.sh` and `createWebserverCertForITest.sh` and configures nginx for serving TLS connections. diff --git a/docs/scripts/TLSClientConfigsForITest.sh b/docs/scripts/TLSClientConfigsForITest.sh index c9c64a9c..1f941175 100755 --- a/docs/scripts/TLSClientConfigsForITest.sh +++ b/docs/scripts/TLSClientConfigsForITest.sh @@ -1,9 +1,9 @@ #!/usr/bin/env bash -# This file is Free Software under the MIT License -# without warranty, see README.md and LICENSES/MIT.txt for details. +# This file is Free Software under the Apache-2.0 License +# without warranty, see README.md and LICENSES/Apache-2.0.txt for details. # -# SPDX-License-Identifier: MIT +# SPDX-License-Identifier: Apache-2.0 # # SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) # Software-Engineering: 2022 Intevation GmbH diff --git a/docs/scripts/TLSConfigsForITest.sh b/docs/scripts/TLSConfigsForITest.sh index 9b1f183f..c1a5420a 100644 --- a/docs/scripts/TLSConfigsForITest.sh +++ b/docs/scripts/TLSConfigsForITest.sh @@ -1,7 +1,7 @@ -# This file is Free Software under the MIT License -# without warranty, see README.md and LICENSES/MIT.txt for details. +# This file is Free Software under the Apache-2.0 License +# without warranty, see README.md and LICENSES/Apache-2.0.txt for details. # -# SPDX-License-Identifier: MIT +# SPDX-License-Identifier: Apache-2.0 # # SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) # Software-Engineering: 2022 Intevation GmbH diff --git a/docs/scripts/createCCForITest.sh b/docs/scripts/createCCForITest.sh index 091dad7e..eaeb247c 100644 --- a/docs/scripts/createCCForITest.sh +++ b/docs/scripts/createCCForITest.sh @@ -1,7 +1,7 @@ -# This file is Free Software under the MIT License -# without warranty, see README.md and LICENSES/MIT.txt for details. +# This file is Free Software under the Apache-2.0 License +# without warranty, see README.md and LICENSES/Apache-2.0.txt for details. # -# SPDX-License-Identifier: MIT +# SPDX-License-Identifier: Apache-2.0 # # SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) # Software-Engineering: 2022 Intevation GmbH diff --git a/docs/scripts/createRootCAForITest.sh b/docs/scripts/createRootCAForITest.sh index cf7cd156..47f3af71 100755 --- a/docs/scripts/createRootCAForITest.sh +++ b/docs/scripts/createRootCAForITest.sh @@ -1,9 +1,9 @@ #!/usr/bin/env bash -# This file is Free Software under the MIT License -# without warranty, see README.md and LICENSES/MIT.txt for details. +# This file is Free Software under the Apache-2.0 License +# without warranty, see README.md and LICENSES/Apache-2.0.txt for details. # -# SPDX-License-Identifier: MIT +# SPDX-License-Identifier: Apache-2.0 # # SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) # Software-Engineering: 2022 Intevation GmbH diff --git a/docs/scripts/createWebserverCertForITest.sh b/docs/scripts/createWebserverCertForITest.sh index 2cfabd46..781b68d0 100644 --- a/docs/scripts/createWebserverCertForITest.sh +++ b/docs/scripts/createWebserverCertForITest.sh @@ -1,7 +1,7 @@ -# This file is Free Software under the MIT License -# without warranty, see README.md and LICENSES/MIT.txt for details. +# This file is Free Software under the Apache-2.0 License +# without warranty, see README.md and LICENSES/Apache-2.0.txt for details. # -# SPDX-License-Identifier: MIT +# SPDX-License-Identifier: Apache-2.0 # # SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) # Software-Engineering: 2022 Intevation GmbH diff --git a/docs/scripts/downloadExamples.sh b/docs/scripts/downloadExamples.sh index e8b3ac05..9d939590 100755 --- a/docs/scripts/downloadExamples.sh +++ b/docs/scripts/downloadExamples.sh @@ -2,10 +2,10 @@ # # Desc: Tries getting csaf 2.0 examples from api.github. Do not run too often! # -# This file is Free Software under the MIT License -# without warranty, see README.md and LICENSES/MIT.txt for details. +# This file is Free Software under the Apache-2.0 License +# without warranty, see README.md and LICENSES/Apache-2.0.txt for details. # -# SPDX-License-Identifier: MIT +# SPDX-License-Identifier: Apache-2.0 # # SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) # Software-Engineering: 2022 Intevation GmbH diff --git a/docs/scripts/prepareUbuntuInstanceForITests.sh b/docs/scripts/prepareUbuntuInstanceForITests.sh index f99bc268..ea88fc42 100755 --- a/docs/scripts/prepareUbuntuInstanceForITests.sh +++ b/docs/scripts/prepareUbuntuInstanceForITests.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash set -e -# This script prepares a naked Ubuntu 20.04 LTS amd64 +# This script prepares a naked Ubuntu LTS amd64 # for the csaf_distribution integration tests # by installing the required packages. @@ -14,19 +14,17 @@ curl -O https://dl.google.com/go/$latest_go rm -rf /usr/local/go # be sure that we do not have an old installation tar -C /usr/local -xzf $latest_go -# Install newer Node.js version from nodesource +# Install a current Node.js version from nodesource # as needed for https://github.com/secvisogram/csaf-validator-service # Instructions from # https://github.com/nodesource/distributions/blob/master/README.md#debmanual KEYRING=/usr/share/keyrings/nodesource.gpg -curl -fsSL https://deb.nodesource.com/gpgkey/nodesource.gpg.key | gpg --dearmor > "$KEYRING" +curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor > "$KEYRING" gpg --no-default-keyring --keyring "$KEYRING" --list-keys chmod a+r /usr/share/keyrings/nodesource.gpg -VERSION=node_16.x -DISTRO="$(lsb_release -s -c)" -echo "deb [signed-by=$KEYRING] https://deb.nodesource.com/$VERSION $DISTRO main" | tee /etc/apt/sources.list.d/nodesource.list -echo "deb-src [signed-by=$KEYRING] https://deb.nodesource.com/$VERSION $DISTRO main" | tee -a /etc/apt/sources.list.d/nodesource.list +NODE_MAJOR=20 +echo "deb [signed-by=/usr/share/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list apt-get update apt-get install -y nodejs diff --git a/docs/scripts/setupProviderForITest.sh b/docs/scripts/setupProviderForITest.sh index 86cbe236..1a57f1e1 100755 --- a/docs/scripts/setupProviderForITest.sh +++ b/docs/scripts/setupProviderForITest.sh @@ -1,9 +1,9 @@ #!/usr/bin/env bash # -# This file is Free Software under the MIT License -# without warranty, see README.md and LICENSES/MIT.txt for details. +# This file is Free Software under the Apache-2.0 License +# without warranty, see README.md and LICENSES/Apache-2.0.txt for details. # -# SPDX-License-Identifier: MIT +# SPDX-License-Identifier: Apache-2.0 # # SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) # Software-Engineering: 2022 Intevation GmbH diff --git a/docs/scripts/setupValidationService.sh b/docs/scripts/setupValidationService.sh index a0e4f6e4..d6f8ba7e 100755 --- a/docs/scripts/setupValidationService.sh +++ b/docs/scripts/setupValidationService.sh @@ -1,9 +1,9 @@ #!/usr/bin/env bash # -# This file is Free Software under the MIT License -# without warranty, see README.md and LICENSES/MIT.txt for details. +# This file is Free Software under the Apache-2.0 License +# without warranty, see README.md and LICENSES/Apache-2.0.txt for details. # -# SPDX-License-Identifier: MIT +# SPDX-License-Identifier: Apache-2.0 # # SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) # Software-Engineering: 2022 Intevation GmbH diff --git a/docs/scripts/testAggregator.sh b/docs/scripts/testAggregator.sh index 13d1df07..366ac075 100755 --- a/docs/scripts/testAggregator.sh +++ b/docs/scripts/testAggregator.sh @@ -1,9 +1,9 @@ #!/usr/bin/env bash -# This file is Free Software under the MIT License -# without warranty, see README.md and LICENSES/MIT.txt for details. +# This file is Free Software under the Apache-2.0 License +# without warranty, see README.md and LICENSES/Apache-2.0.txt for details. # -# SPDX-License-Identifier: MIT +# SPDX-License-Identifier: Apache-2.0 # # SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) # Software-Engineering: 2022 Intevation GmbH diff --git a/docs/scripts/testChecker.sh b/docs/scripts/testChecker.sh index 8c680d4f..cb45aad6 100755 --- a/docs/scripts/testChecker.sh +++ b/docs/scripts/testChecker.sh @@ -1,9 +1,9 @@ #!/usr/bin/env bash -# This file is Free Software under the MIT License -# without warranty, see README.md and LICENSES/MIT.txt for details. +# This file is Free Software under the Apache-2.0 License +# without warranty, see README.md and LICENSES/Apache-2.0.txt for details. # -# SPDX-License-Identifier: MIT +# SPDX-License-Identifier: Apache-2.0 # # SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) # Software-Engineering: 2022 Intevation GmbH diff --git a/docs/scripts/testDownloader.sh b/docs/scripts/testDownloader.sh index 3596399c..c4b9bced 100755 --- a/docs/scripts/testDownloader.sh +++ b/docs/scripts/testDownloader.sh @@ -1,9 +1,9 @@ #!/usr/bin/env bash -# This file is Free Software under the MIT License -# without warranty, see README.md and LICENSES/MIT.txt for details. +# This file is Free Software under the Apache-2.0 License +# without warranty, see README.md and LICENSES/Apache-2.0.txt for details. # -# SPDX-License-Identifier: MIT +# SPDX-License-Identifier: Apache-2.0 # # SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) # Software-Engineering: 2022 Intevation GmbH diff --git a/docs/scripts/uploadToProvider.sh b/docs/scripts/uploadToProvider.sh index e3aac283..151201c5 100755 --- a/docs/scripts/uploadToProvider.sh +++ b/docs/scripts/uploadToProvider.sh @@ -2,10 +2,10 @@ # # Desc: Call ./downloadExamples.sh and then try csaf_uploader. # -# This file is Free Software under the MIT License -# without warranty, see README.md and LICENSES/MIT.txt for details. +# This file is Free Software under the Apache-2.0 License +# without warranty, see README.md and LICENSES/Apache-2.0.txt for details. # -# SPDX-License-Identifier: MIT +# SPDX-License-Identifier: Apache-2.0 # # SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) # Software-Engineering: 2022 Intevation GmbH diff --git a/go.mod b/go.mod index 315d27f7..8fed8041 100644 --- a/go.mod +++ b/go.mod @@ -15,7 +15,6 @@ require ( github.com/stretchr/testify v1.9.0 go.etcd.io/bbolt v1.3.10 golang.org/x/crypto v0.23.0 - golang.org/x/exp v0.0.0-20231110203233-9a3e6036ecaa golang.org/x/term v0.20.0 golang.org/x/time v0.5.0 ) diff --git a/go.sum b/go.sum index dd0a4216..38b5c888 100644 --- a/go.sum +++ b/go.sum @@ -52,8 +52,6 @@ golang.org/x/crypto v0.3.1-0.20221117191849-2c476679df9a/go.mod h1:hebNnKkNXi2Uz golang.org/x/crypto v0.7.0/go.mod h1:pYwdfH91IfpZVANVyUOhSIPZaFoJGxTFbZhFTx+dXZU= golang.org/x/crypto v0.23.0 h1:dIJU/v2J8Mdglj/8rJ6UUOM3Zc9zLZxVZwwxMooUSAI= golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8= -golang.org/x/exp v0.0.0-20231110203233-9a3e6036ecaa h1:FRnLl4eNAQl8hwxVVC17teOw8kdjVDVAiFMtgUdTSRQ= -golang.org/x/exp v0.0.0-20231110203233-9a3e6036ecaa/go.mod h1:zk2irFbV9DP96SEBUUAy67IdHUaZuSnrz1n472HUCLE= golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4= golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs= golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= diff --git a/internal/certs/certs.go b/internal/certs/certs.go index dcbf7ef5..ab31d8b1 100644 --- a/internal/certs/certs.go +++ b/internal/certs/certs.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH diff --git a/internal/certs/certs_test.go b/internal/certs/certs_test.go index e9a36cd0..e2f1af51 100644 --- a/internal/certs/certs_test.go +++ b/internal/certs/certs_test.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH diff --git a/internal/filter/filter.go b/internal/filter/filter.go index bdc6afb3..daccdb0b 100644 --- a/internal/filter/filter.go +++ b/internal/filter/filter.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH diff --git a/internal/filter/filter_test.go b/internal/filter/filter_test.go index 14bcc59c..bc344406 100644 --- a/internal/filter/filter_test.go +++ b/internal/filter/filter_test.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH diff --git a/internal/misc/doc.go b/internal/misc/doc.go index 1fec00ae..1101add9 100644 --- a/internal/misc/doc.go +++ b/internal/misc/doc.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH diff --git a/internal/misc/mime.go b/internal/misc/mime.go index 0e699a38..acc1ba3e 100644 --- a/internal/misc/mime.go +++ b/internal/misc/mime.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH diff --git a/internal/misc/mime_test.go b/internal/misc/mime_test.go index bd5a31c2..19f94756 100644 --- a/internal/misc/mime_test.go +++ b/internal/misc/mime_test.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH diff --git a/pkg/models/models.go b/pkg/models/models.go index 520cd9c3..b743d183 100644 --- a/pkg/models/models.go +++ b/pkg/models/models.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH diff --git a/pkg/models/models_test.go b/pkg/models/models_test.go index a40100f1..777a4281 100644 --- a/pkg/models/models_test.go +++ b/pkg/models/models_test.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH diff --git a/pkg/options/log.go b/pkg/options/log.go index 226072ea..bd18c658 100644 --- a/pkg/options/log.go +++ b/pkg/options/log.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH @@ -9,9 +9,8 @@ package options import ( + "log/slog" "strings" - - "golang.org/x/exp/slog" ) // LogLevel implements a helper type to be used in configurations. diff --git a/pkg/options/log_test.go b/pkg/options/log_test.go index 2272f0f1..a99c598b 100644 --- a/pkg/options/log_test.go +++ b/pkg/options/log_test.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH @@ -9,9 +9,8 @@ package options import ( + "log/slog" "testing" - - "golang.org/x/exp/slog" ) func TestMarshalFlag(t *testing.T) { diff --git a/pkg/options/options.go b/pkg/options/options.go index 961b4b45..c0ad2bc1 100644 --- a/pkg/options/options.go +++ b/pkg/options/options.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH @@ -12,13 +12,14 @@ package options import ( "fmt" "log" + "log/slog" "os" + "github.com/csaf-poc/csaf_distribution/v3/util" + "github.com/BurntSushi/toml" "github.com/jessevdk/go-flags" "github.com/mitchellh/go-homedir" - - "github.com/csaf-poc/csaf_distribution/v3/util" ) // Parser helps parsing command line arguments and loading @@ -147,3 +148,13 @@ func ErrorCheck(err error) { log.Fatalf("error: %v\n", err) } } + +// ErrorCheckStructured checks if err is not nil and terminates the program if +// so. This is similar to [ErrorCheck], but uses [slog] instead of the +// non-structured Go logging. +func ErrorCheckStructured(err error) { + if err != nil { + slog.Error("Error while executing program", "err", err) + os.Exit(1) + } +} diff --git a/pkg/options/options_test.go b/pkg/options/options_test.go index 3feba934..9aab23b6 100644 --- a/pkg/options/options_test.go +++ b/pkg/options/options_test.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH diff --git a/util/client.go b/util/client.go index 239f8ecf..5a11c7b3 100644 --- a/util/client.go +++ b/util/client.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH diff --git a/util/csv.go b/util/csv.go index aee0e6de..d84644c8 100644 --- a/util/csv.go +++ b/util/csv.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH diff --git a/util/doc.go b/util/doc.go index df6809ef..2caf4e39 100644 --- a/util/doc.go +++ b/util/doc.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH diff --git a/util/file.go b/util/file.go index bff7110d..85bda0a0 100644 --- a/util/file.go +++ b/util/file.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH diff --git a/util/file_test.go b/util/file_test.go index aafce6d9..3f648b8b 100644 --- a/util/file_test.go +++ b/util/file_test.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH diff --git a/util/hash.go b/util/hash.go index 84cfa08b..b5dcaa9e 100644 --- a/util/hash.go +++ b/util/hash.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH diff --git a/util/json.go b/util/json.go index 851974b4..f66ab866 100644 --- a/util/json.go +++ b/util/json.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2021 German Federal Office for Information Security (BSI) // Software-Engineering: 2021 Intevation GmbH diff --git a/util/set.go b/util/set.go index 0df693dd..61eb14b0 100644 --- a/util/set.go +++ b/util/set.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2023 German Federal Office for Information Security (BSI) // Software-Engineering: 2023 Intevation GmbH diff --git a/util/url.go b/util/url.go index fb454f42..f59c078b 100644 --- a/util/url.go +++ b/util/url.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH diff --git a/util/version.go b/util/version.go index 45d60b8e..010f8cd4 100644 --- a/util/version.go +++ b/util/version.go @@ -1,7 +1,7 @@ -// This file is Free Software under the MIT License -// without warranty, see README.md and LICENSES/MIT.txt for details. +// This file is Free Software under the Apache-2.0 License +// without warranty, see README.md and LICENSES/Apache-2.0.txt for details. // -// SPDX-License-Identifier: MIT +// SPDX-License-Identifier: Apache-2.0 // // SPDX-FileCopyrightText: 2022 German Federal Office for Information Security (BSI) // Software-Engineering: 2022 Intevation GmbH