Update Clang toolchain from 18.0.0 to 18.1.8

[alexcrichton]

This is done in the interest of assisting #12075 and #11626. Currently the Rust toolchain cannot be updated because the latest nightly uses LLVM 18.1.7 and coverage information breaks. This breakage is because LLVM 18.1.7 records coverage information with version "9" but LLVM 18.0.0 recorded coverage information with version "8". This means that the recordings created by Rust binaries use version "9" which are unreadable by the processing that OSS-Fuzz does with the 18.0.0-based toolchain using version "8".

This commit updates the Clang toolchain to the latest 18.x.x release to get the two in sync so the same coverage recording version is used.

[alexcrichton]

@maflcko you mentioned in #11626 that before doing this all existing projects should be un-pinned from their @sha256... pins. Is that required to bump Clang? I would have expected the other way around where some new projects might need pinning as a result of this.

Also, do you know of a way to more easily enumerate the projects which break as a result of this upate? I probably can't feasibly build everything locally. If CI takes care of this though I can also just wait for that.

[alexcrichton]

Also, for reference, I've confirmed that by layering #12075 on top of this Rust projects no longer have any warnings in coverage builds and coverage looks like it might work.

[alexcrichton]

Also, following up from your comment here you mentioned that pinned projects might break since they're using clang 15. I think though that the decoding of coverage data supports older versions, just not newer versions, so my assumption would be that LLVM 18 tooling would be able to decode clang 15-generated coverage information. I don't have data to back up this assumption, however.

[jonathanmetzman]

/gcbrun trial_build.py all

[maflcko]

Is that required to bump Clang?

Yes, because the coverage container uses the current llvm to parse the coverage profile (regardless of what the project uses), but if the profile was generated with llvm-15 (pinned projects) it will fail.

It should be possible to observe this in the trial build.

[maflcko]

LLVM 18 tooling would be able to decode clang 15-generated coverage information

In theory, yes, earlier coverage profiles can be read. However, the raw profile version is a separate versioning, and a breaking change every time, as far as I understood it.

[maflcko]

unrelated nit: Could add a comment to check if CMAKE_VERSION is up-to-date? Otherwise, a future bump could cause issues, because cmake may not properly detect the newer llvm version. Most recently this happened for 32-bit builds.

Generally, I'd say the two should be bumped at the same time to avoid build issues in projects down the line.

[maflcko]

Resolved as part of #12083

[alexcrichton]

Thanks! I'll admit I wasn't sure what CMAKE_VERSION was referring to here so assistance is appreciated!

[maflcko]

For reference, the trial build result is https://github.com/google/oss-fuzz/pull/12077/checks?check_run_id=26334505585:

Failed builds: 112/1020:

{'ampproject': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-1baa1318-ff44-4132-98cc-8636a204c4db.txt', 'arrow': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-3cd88fc7-1175-4836-abca-486ee84a9558.txt', 'askama': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-d253f22f-f9b3-4b97-a5d8-55703f14ab52.txt', 'bignum-fuzzer': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-4721c69c-ceee-4fe3-822a-0c134d330d89.txt', 'bincode': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-a01f2477-8613-453b-9cce-19249d85ee68.txt', 'boost': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-cbf8f798-0949-4d20-8509-545e02bdbc7f.txt', 'boost-json': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-501c833f-7ab7-4edb-80a9-75fb8705fbac.txt', 'bson-rust': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-7e4990ef-32b0-4955-845c-8ef847116c49.txt', 'cloud-hypervisor': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-c0262abb-a0a5-4892-b1ba-4af67a8cd709.txt', 'cras': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-fd8d1f6b-bd96-4183-b137-48ffa61a8523.txt', 'crosvm': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-72f0d425-b0a7-47e8-a80e-81f9a8079ab3.txt', 'cryptofuzz': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-bf9ffa46-3e56-4db6-bd99-fc285007719d.txt', 'evo-inflector': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-fa1fd607-ee8d-4f08-bdac-d017e34778dd.txt', 'file': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-bd4566a2-1c2f-4446-a413-dbdd850f32ef.txt', 'flate2-rs': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-1aa8cc08-8d87-4b93-a2fa-d2f75937ae74.txt', 'fontations': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-70a9b0d4-878a-4299-97bd-4e910142ce55.txt', 'freeimage': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-ec7541b9-1ab2-4a36-9269-933f7c02e3ad.txt', 'gdbm': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-fd5d270f-12c4-437b-ac69-60c2d4b7b0e8.txt', 'gimli': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-5e6fa704-a6f2-4636-8a9b-c2023f8e2587.txt', 'gitoxide': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-555e1cc1-665b-42f8-a8d8-1f6f427824e2.txt', 'gnutls': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-13de2bcf-833e-469e-b298-6e87b8c4a989.txt', 'grpc-swift': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-fad133dd-71fc-4480-ae42-9c56e3179f3a.txt', 'hadoop': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-eba8387e-7055-461d-b2c1-613058a65abb.txt', 'hdf5': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-c3b8b921-b012-4d4f-b839-f8fba62f9f53.txt', 'httparse': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-6ec5fb70-e3a0-4460-abd1-83d89837fe3b.txt', 'hyperium': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-2f76969d-5e15-4b7b-99f0-6260bf51cb4b.txt', 'image-png': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-58cf0b4e-c1ab-4606-98ce-1c330114e97a.txt', 'immer': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-34f03349-8057-44c0-8971-a4f07bd2d7c1.txt', 'istio-ztunnel': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-9b6dc683-44a8-4c4e-8630-41957973c489.txt', 'itext7': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-580f58bf-4fa0-461d-b20c-0575343063d9.txt', 'itoa': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-1b5aced2-f923-40bc-a5c6-fb8135f9225e.txt', 'jackson-datatype-joda': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-c1f83a3e-1a84-42e1-b22c-e362746153a7.txt', 'javapoet': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-5de23b48-6518-4681-8955-ab5d14e580a8.txt', 'joni': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-3d565a0a-8c72-4cab-b842-a5445dcde364.txt', 'json-flattener': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-cd28232a-3f79-4fcb-a78b-43a24800fa4b.txt', 'json5format': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-748d2fbe-c271-4d09-b87a-d6f7474df528.txt', 'jsonp-api': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-9789e85a-46c2-439a-b99b-5d1100a94def.txt', 'kie-soup': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-20c9f4d6-a398-4afb-bc94-9705d14dd63a.txt', 'kimageformats': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-cc4fa8ee-e32b-4879-932b-508dea94ac2b.txt', 'knot-dns': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-8551c34b-1e1c-47a9-98d1-447b81d90951.txt', 'lark-parser': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-c6c39671-6283-422b-8576-ccbbcbe2b1bf.txt', 'leveldb': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-b4cb6eb4-1590-4f54-b4db-bbd6e591a4d4.txt', 'libavif': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-598380b0-134e-4f9d-b66e-c247316069f8.txt', 'libecc': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-3e4c485a-a6c9-4e99-a265-86951837fa6a.txt', 'librawspeed': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-ca27ef49-7f48-4d5c-bede-01232c1dda7e.txt', 'libressl': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-777bf521-bdf0-4680-8c53-9a3ba644d549.txt', 'librsvg': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-bdb5ab1b-7427-429c-83fd-2eb380f378e6.txt', 'libxml2': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-0b30f500-3355-4067-840f-bec8fc381d2e.txt', 'linkerd2-proxy': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-b720b476-4021-40d6-a462-7c2c7527ad11.txt', 'lua': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-ca500d39-e7d6-4fdc-b24e-0d507b9c8674.txt', 'mariadb-connector-j': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-bca5521d-831a-48c1-952f-e5e4883599ce.txt', 'metadata-extractor': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-8a76192b-ad97-4fb1-81d5-7b027b10b3aa.txt', 'migtd': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-40da0044-a6f9-42a9-a2b5-be0685426836.txt', 'miniz_oxide': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-cb3a88d1-14dc-4b00-8033-b16f55e35a7c.txt', 'monero': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-af36f764-dd36-4d7d-93d6-a15a637aa03a.txt', 'mp4parse-rust': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-5a2a13e3-b3be-4c69-9663-fa149e6f18ac.txt', 'mp4san': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-c62e80c9-d052-45d2-bb19-1c1511a38337.txt', 'muduo': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-42aad0e3-89e3-455f-a840-9851bab2f247.txt', 'naga': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-1b87a731-fed1-433a-99c0-39ae5ce56558.txt', 'nettle': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-743c7a22-6e77-464a-9d46-e10796508baf.txt', 'nom': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-381abbab-bf3e-4827-b1fc-a2c0395883ea.txt', 'opencv': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-1b108b51-d7d7-4da7-a99d-5df6f45c38b1.txt', 'opendal': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-642558ee-5cff-4849-8c39-6c4e1f96117e.txt', 'opensk': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-da6ae1b2-605f-427f-9e9a-f688c9113284.txt', 'openssh': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-3cd27d79-c163-425e-8852-79fc497ba7d2.txt', 'openweave': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-f8dc91ac-849f-46ec-99f6-a14b9ece42d9.txt', 'pcl': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-52606f6d-fc5a-4366-9f51-508debd5bcb7.txt', 'pest': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-2656e7d8-a76c-4411-adf0-2548aabe9ba9.txt', 'poco': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-76820bcc-7ea2-431e-8e4a-de23d60dfaec.txt', 'powerdns': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-3ec7d313-8c0d-4217-9712-817226c361c0.txt', 'prost': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-c7ed8777-670b-4fbc-b924-e422aa7a9e10.txt', 'qcms': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-170f86ed-9b76-4193-8d42-982452bbf5cf.txt', 'quick-xml': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-bb1e70d0-dbd1-4521-abbd-b52297aa881b.txt', 'redis-rs': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-22925525-4ab2-42bc-9f42-c77b1e40cfd8.txt', 'rhai': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-106f713b-c4e7-403c-a9cc-b2dbd84be3c8.txt', 'rnp': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-0da297c3-2094-431d-a262-4ce850f84289.txt', 'ron': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-aceeb418-d00f-4e4d-a8f5-8fdd626e8abd.txt', 'rust-brotli': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-53ec08a9-5380-43ed-a091-79a6eaa014f4.txt', 'rust-lexical': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-bdaa2b41-5b69-43d1-acdd-572f7ce82449.txt', 'rust-regex': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-93c802c8-90c9-4409-8e77-9d06d3376403.txt', 'rustls': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-47ecca05-126c-4234-9fb5-4097f445f688.txt', 'ryu': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-8ccb9eb7-c3d8-4e53-a656-32d1b2667632.txt', 'samba': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-4e589b67-73cf-4b5a-98ea-727dd150624e.txt', 'serde-yaml': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-7168989b-60b9-4fa5-89bd-7d1504bd7404.txt', 'serde_json': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-00bca1ea-cfad-40f6-83b2-112cf3bfabd6.txt', 'serde_urlencoded': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-6b8eb144-5780-4c04-9cc8-f6e539ba60ea.txt', 'servo': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-f6d60b1d-69ae-4ec8-8f7a-4fb183f51bc3.txt', 'solidity': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-b5f0092f-007f-49f7-bf4a-8d175378ba05.txt', 'spdm-rs': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-6518cf50-0693-457d-8f1e-d3fb3204cc1c.txt', 'starlark-rust': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-86e76fe9-e3db-446c-9cbd-4134cb809167.txt', 'suricata': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-396e8533-2023-43b9-80f2-3400b004f0a0.txt', 'swift-nio': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-fbacfa05-fd73-472c-b6ad-5e3ea94c2f1a.txt', 'swift-protobuf': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-c4edcdc1-3149-416a-af78-4819c06fd9bd.txt', 'tarantool': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-d70cee67-ef64-40a0-b236-a9d3c829e9cd.txt', 'td-shim': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-1775692b-3cae-4444-bd65-9ead786e858d.txt', 'textwrap': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-2092668b-8db4-47c2-bcac-4f53030db4e7.txt', 'tinyusb': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-6378828a-8776-478b-b951-caea354fc2cb.txt', 'tokio': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-9e01625c-cae1-4a07-8b20-067f6133aa84.txt', 'toml_edit': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-a608df9f-94d4-421a-af18-fcf9731ed8ca.txt', 'trust-dns': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-f8903117-4781-4e66-8419-80a5b95f741c.txt', 'tungstenite-rs': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-84e455af-8234-4216-b532-bff2ce0b2340.txt', 'unicode-rs': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-29bf191d-b243-4b92-9318-a84014a2cc22.txt', 'vtpm-td': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-961c540a-1a05-4fb0-9c05-c50b23a44453.txt', 'vulnerable-project': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-83ae22b8-3b3f-45dc-983c-4bdfcf7f0f57.txt', 'wasmer': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-207cbf6c-8ed7-4f77-a02f-c5230ff68b5e.txt', 'wolfssl': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-2a62acf6-6984-4e07-9a0a-66b13eb41cd4.txt', 'xnu': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-b23fb7b7-bf12-4a25-9b16-12eeb39c791b.txt', 'zeek': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-9a53b4e9-e765-4adf-9aec-e62e4366eab6.txt', 'zip-rs': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-4b3b82c0-f587-4dc6-a8ab-648cf184b187.txt', 'hive': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-185afefa-89c9-4e9c-80fa-777057a8475f.txt', 'apache-commons-imaging': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-6fcc07c5-a33a-4277-a45a-bcd088665460.txt', 'envoy': 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-b01a6fe2-1743-4a2f-8bf0-273d44169f88.txt' }

[alexcrichton]

Thanks for the links! I'll start reviewing those hopefully soon.

Also I was having a tough time understanding what you were referring to before about LLVM 15 breaking. I thought that because the current coverage container was using LLVM 18 that updating it to LLVM 18.1 wouldn't be an issue because it would already be broken with LLVM 15 coverage information. Upon checking though it looks like the version in the LLVM source for profiling was "8" both with LLVM 15 and LLVM 18.0.0. The first change happened with LLVM 18.1 so that also makes more sense to me.

I'll try to dig in further to the failure logs and see what the impact of this is. Also I'll note I'm happy to rebase around other changes, so please don't block on me for anything.

[maflcko]

The failures should all be related to the raw coverage profile version in some way or another. I don't see another way other than to atomically and globally bump the coverage version for all projects and all languages. But that requires the projects to be un-pinned, and a rust-nightly bump to be combined into this pull.

My recommendation would be to change #12075 to nightly-2024-02-12 for now, then wait for it to land and then bump 2024-02-12 to the current date as part of this pull request.

[alexcrichton]

Ok I've gone through many of the failures and I'm sort of quite new to updating the toolchain here so I wanted to ask a few questions. I've tried to bucket all the various failure logs into a few categories:

• Some builds failed with network failures, many java-based builds fell into this category
• Rust-based builds mostly failed - as expected they all have coverage version mismatches because Update default Rust to nightly-2024-02-12  #12075 wasn't included in the test run. Once Update default Rust to nightly-2024-02-12  #12075 merges I'll apply the recommended suggestion of adding a Rust update to this PR as well.
• Swift-based builds all looked to fail - I think this means that the Swift compiler needs to be updated to LLVM 18.1+ as well
• Some arm64-related errors happened but I couldn't make heads or tails of them e.g. here and here
• Some projects failed because of what I believe are changes in Clang itself emitting new warnings or similar
  • freeimage linker failure due to DWARF
  • leveldb (error: no type named 'remove_all' in namespace 'std::filesystem')
  • librawspeed
  • envoy
  • samba
  • muduo
  • wolfssl
• Some builds failed in ways that seemed unique
  • file failed because a mv command was too big
  • I can't seem to fully load the lua log file so I can't review
  • openssh seemed to crash in a test of some kind
• Two projects failed with coverage mismatches in a way I didn't understand. These weren't pinned to older containers but they also both have a custom corpus, so I don't know if that factors in here
  • hdf5
  • vulnerable-project
• As expected all containers with @sha256:... pins all failed due to profiling version mismatches.

My main question is how to handle most of these. Two action items for this PR are to update Rust in this PR and update Swift as well. Everything else though I'm less certain about. For example resolving new Clang errors will require source changes. I tested a few of the @sha256:... pinned builds and I presume they're pinned because they succeeded with Clang 15 and failed when Clang was updated to 18, and I can at least confirm they're still failing with Clang 18.1.7 as well. Should I pin all the new failures to the Clang 18 builder so the fuzzers at least still build even if coverage information is broken?

There's still other failures I don't fully understand which I'm not sure if y'all would recognize or not

[alexcrichton]

Oh and one final category of failures I forgot to mention are those that failed to build but also failed to build according to their latest status, so I ignored a few builds like that.

[maflcko]

Some arm64-related errors happened but I couldn't make heads or tails of them

They are expected, I think, and can be ignored for now, because the infra check does not spin up arm64 machines.

• I can't seem to fully load the lua log file so I can't review

You can use curl 'https://oss-fuzz-gcb-logs.storage.googleapis.com/log-ca500d39-e7d6-4fdc-b24e-0d507b9c8674.txt' | tail -111 to see the tail of the (large) log only. It is the arm failure.

Two projects failed with coverage mismatches in a way I didn't understand. These weren't pinned to older containers but they also both have a custom corpus, so I don't know if that factors in here

You will have to rebase or merge with master before the trial build. Otherwise the changes (2c03690) aren't picked up.

[maflcko]

If more than one project is affected by a build warning, you can soften it. For example:

diff --git a/infra/base-images/base-clang/Dockerfile b/infra/base-images/base-clang/Dockerfile
index f61b85443..c82ed1008 100644
--- a/infra/base-images/base-clang/Dockerfile
+++ b/infra/base-images/base-clang/Dockerfile
@@ -58,9 +58,9 @@ ENV CCC "clang++"
 # The implicit-function-declaration and implicit-int errors are downgraded to a
 # warning, to allow compiling legacy code.
 # See https://releases.llvm.org/16.0.0/tools/clang/docs/ReleaseNotes.html#potentially-breaking-changes
-# Same for deprecated-declarations, int-conversion,
+# Same for vla-cxx-extension, deprecated-declarations, int-conversion,
 # incompatible-function-pointer-types, enum-constexpr-conversion
 
-ENV CFLAGS "-O1 -fno-omit-frame-pointer -gline-tables-only -Wno-error=enum-constexpr-conversion -Wno-error=incompatible-function-pointer-types -Wno-error=int-conversion -Wno-error=deprecated-declarations -Wno-error=implicit-function-declaration -Wno-error=implicit-int -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION"
+ENV CFLAGS "-O1 -fno-omit-frame-pointer -gline-tables-only -Wno-error=enum-constexpr-conversion -Wno-error=incompatible-function-pointer-types -Wno-error=int-conversion -Wno-error=deprecated-declarations -Wno-error=vla-cxx-extension -Wno-error=implicit-function-declaration -Wno-error=implicit-int -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION"
 ENV CXXFLAGS_EXTRA "-stdlib=libc++"
 ENV CXXFLAGS "$CFLAGS $CXXFLAGS_EXTRA"

[alexcrichton]

Swift-based builds all looked to fail - I think this means that the Swift compiler needs to be updated to LLVM 18.1+ as well

Looks like Swift 5.8.1 is currently used which uses LLVM 13.0.0. The latest release of Swift is 5.10.1 which comes with LLVM 15.0.0. It looks like Swift 6 is in development but I wasn't able to find a binary to download and see if it's at the right version.

Given that I don't think there's an easy fix for Swift for now.

[maflcko]

Suggested change

	-Wno-error=format-truncation \
	-Wno-error=thread-safety-reference-return \
	-Wno-error=nan-infinity-disabled \

nit: I'd say, if there is only a single project needing the suppressions, it should go into the project file. Especially, those warnings may point to real errors (as opposed to just warn about compile extensions, deprecations, or implicit behavior).

[maflcko]

I think you forgot to add the three -Wno-error= to the three project's build.sh in the latest push?

[alexcrichton]

Oh sorry I misinterpreted this as leaving the flags out entirely. I'll try to work through some C++-specific failures and add per-image flags.

[jonathanmetzman]

/gcbrun trial_build.py all

[alexcrichton]

From the trial build I've categorized the failures into:

Miscellaneous failures

• rust-lexical - failed to compile with latest rustc, perhaps an LTO bug?
• vulnerable-project - invalid instrumentation profile data, unclear

Various failures in C++ projects

• envoy (use-after-poison)
• leveldb (no type named 'remove_all' in namespace 'std::filesystem')
• libphonenumber (-Wreorder-ctor)

Projects that fail to compile with the latest Rust compiler

• istio-ztunnel
• suricata
• wasmer

Transient failures - OOM, out of disk, network issues, etc

• bs4
• django
• freeradius
• icu
• lark-parser
• libpsl
• libreoffice
• libtasn1
• mpv
• openweave

Swift failures - swift older LLVM causes coverage mismatch

• grpc-swift
• swift-nio
• swift-protobuf

ARM64 failures

• lua
• libxml2
• tinyusb

Java related failures having to do with maven and network issues

• apache-commons-lang
• dropwizard
• htmlunit
• jsign
• json-simple
• spatial4j
• slf4j-api
• tablesaw
• twelve-monkeys

Failures due to `@sha256` pinning, all failed due to coverage version mismatch

• nettle
• tarantool
• ampproject
• knot-dns
• xnu
• arrow
• gnutls
• boost
• powerdns
• rnp
• poco
• solidity
• kimageformats
• opencv
• boost-json
• pcl
• cryptofuzz
• immer
• monero
• zeek
• libecc
• libressl
• bignum-fuzzer

Given this categorization the open questions I would have are:

• Should I remove all @sha256 pins of containers? Some will start failing to build but they're all guaranteed to have bad coverage information. Or should I add @sha256 pins for builds that are broken by this update?
• Is it ok to break coverage for Swift-based fuzzing?
• Is it ok to break 3-4 C/C++ projects that have a unique new error each from Clang that's causing a failure? This would remove -Wno-error=format-truncation from this PR for example and would leave libphonenumber broken. (this question is less relevant if @sha256 pins are added)
• Is it ok to break a few projects that don't work with the latest Rust compiler?

[maflcko]

Should I remove all @sha256 pins of containers? Some will start failing to build but they're all guaranteed to have bad coverage information. Or should I add @sha256 pins for builds that are broken by this update?

I am working on unpinning them, but it will take some time.

See https://github.com/google/oss-fuzz/pulls?q=is%3Aopen+is%3Apr+author%3Amaflcko+%22Use+latest+builder%22 for the current progress.

[maflcko]

But yeah, I'd say to remove the pin of all projects here. This will fix a few projects, like #12128 (comment). A few will remain broken, but those can be handled later/separately.

Also, make sure to rebase again to pick up d63f82f.

[alexcrichton]

Sounds good, I've rebased, removed the one-off warning flag allowances, and removed @sha256:... pins

[alexcrichton]

Only envoy/samba needed new flags, looks like the other projects have updated in the meantime and no longer need a fix

[fanquake]

nit: there's only a couple commits difference (llvm/llvm-project@llvmorg-18.1.7...llvmorg-18.1.8), but this could be 18.1.8 (which should also be the last 18.x tag).

[jonathanmetzman]

Do we need another trial build here?

[maflcko]

• arrow fixed in arrow: Use latest builder #12179

[maflcko]

Did you fix librawspeed?

librawspeed. Needs a project-specific wno-error (https://oss-fuzz-gcb-logs.storage.googleapis.com/log-1ddeeed2-5e7c-4047-9412-2bc62a172313.txt)

[maflcko]

Also, I am not sure what to do about the remaining build failures, such as xnu.

xnu. See googleprojectzero/SockFuzzer#16

Fixing the bug would be ideal, but I am not sure how relevant the project is, given that no reply has been received upstream and the project itself has been stale for more than a year?

Given that this pull request here will unlock coverage runs for quite a few rust projects, I'd say pinning xnu again would be acceptable (leading to coverage failures there instead). However, I am not an OSS-Fuzz maintainer, so I don't have a say here.

[alexcrichton]

For librawspeed I'm now getting mkdir: cannot create directory 'build': File exists trying to build it locally so I'm not sure what happened to the original error. At some point the error also went away, so I'm not really sure what's going on. I may be mixing everything up locally and not using the right builds of the base containers perhaps? I thought I did everything right but I'm not 100% sure.

For xnu/others/etc, one thing I'd like to do is to clarify what the expectations are here. I don't personally have the time to keep going through all the projects and update their own build scripts and such. If that's expected of updates like this I think that's reasonable but I'll probably close this PR and shelve it for someone else to tackle later. I'm happy to rebase but I know very little about C/C++ fuzzing/updates/infrastructure here so I don't know how to approach these failures and what's expected of me vs what's expected of projects themselves.

[jonathanmetzman]

Also, I am not sure what to do about the remaining build failures, such as xnu.

xnu. See googleprojectzero/SockFuzzer#16

Fixing the bug would be ideal, but I am not sure how relevant the project is, given that no reply has been received upstream and the project itself has been stale for more than a year?

Given that this pull request here will unlock coverage runs for quite a few rust projects, I'd say pinning xnu again would be acceptable (leading to coverage failures there instead). However, I am not an OSS-Fuzz maintainer, so I don't have a say here.

I'm OK pinning it.

[jonathanmetzman]

Also, I am not sure what to do about the remaining build failures, such as xnu.

xnu. See googleprojectzero/SockFuzzer#16

Fixing the bug would be ideal, but I am not sure how relevant the project is, given that no reply has been received upstream and the project itself has been stale for more than a year?

Given that this pull request here will unlock coverage runs for quite a few rust projects, I'd say pinning xnu again would be acceptable (leading to coverage failures there instead). However, I am not an OSS-Fuzz maintainer, so I don't have a say here.

I'm OK pinning it.

[alexcrichton]

My conclusion is that I don't have the time/energy to keep up with all the changes needed for this, so I'm going to close this to make room for if anyone else is interested in driving the update.

[maflcko]

Thanks for the work so far. I can pick it up later, if no one beats me to it.