scalafmt failing

[SethTisue]

after #1677

-# June 3, 2023
-nightly=2.13.12-bin-1a2373b
+# July 3, 2023
+nightly=2.13.12-bin-7c63166

Bisecting shows the problem to have started with scala/scala#10392 ("IndexedSeq.head throws NoSuchElementException")

https://scala-ci.typesafe.com/job/scala-2.13.x-jdk17-integrate-community-build/1239/artifact/logs/scalafmt-build.log

[scalafmt] [error] Failed tests:
[scalafmt] [error] 	org.scalafmt.FidelityTest
[scalafmt] [error] 	org.scalafmt.FormatTests

[scalafmt] Caused by: java.util.NoSuchElementException: last of empty IndexedSeq

[SethTisue]

@som-snytt not obvious me to what the cause might be. I'll keep digging.

[som-snytt]

[warn] 1 deprecation (since 18); re-run with -deprecation for details

I think this is our first time seeing Java since.

Nice comment is cold comfort:

  * NOTE(olafurpg). The pattern match in this file has gotten out of hand. It's
  * difficult even for myself to keep track of what's going on

It turns out that Scalameta has (where IndexedSeqOptimized is a dummy trait for 2.13)

@data class Tokens private (
    private val tokens: Array[Token],
    private val start: Int,
    private val end: Int
) extends immutable.IndexedSeq[Token] with IndexedSeqOptimized[Token] {
  def apply(idx: Int): Token = tokens(start + idx)
  def length: Int = end - start

head is defined as apply(0) (the old implementation) and last is inherited, which used to be apply(length-1) but now checks isEmpty first. Since start is typically an index into the large tokens array, apply(-1) does not throw but generally supplies a scala.meta.tokens.Token$Space.

Probably that is just a bug in Scalameta, and the additional bug in Scalafmt is not to assume non-empty tokens (see added filter):

defnBeforeTemplate(leftOwner).filter(_.tokens.nonEmpty).map { x =>
  val policyEnd = Policy.End.On(x.tokens.last)
  delayedBreakPolicy(policyEnd)(forceNewlineBeforeExtends)
}

or what have you.

For a while, I assumed the problem would be brittle inheritance in IndexedSeq, which may still be an issue. Not sure if the following comment just means compatibility between 2.12 and 2.13, etc.

  /* Both head and headOption need to be implemented here due to
   * binary incompatibility caused by
   * https://github.com/scala/scala/commit/b20dd00b11f06c14c823d277cdfb58043a2586fc
   */
  override def head: Token = apply(0)

[SethTisue]

Jenkins is currently chewing on the sbt 1.9.1->2 upgrade. after that, #1681 should get us to where we would be able to pull in a Scalameta fix

fyi @kitbellew

[SethTisue]

@lrytz does this give you cold feet about scala/scala#10392 ?

while you ponder that, I'll try making Som's suggested changes in scalacommunitybuild forks of scalameta and scalafmt.

[som-snytt]

@SethTisue revealing bugs in library code is a good thing. I'm using "good thing" in its generic, non-trademarked sense.

[SethTisue]

ah, @som-snytt is way ahead of me... I'm just now seeing scalameta/scalameta#3235 and scalameta/scalafmt#3581. I'll give those a try

[SethTisue]

yup that did it 🎉 , thx @som-snytt

[alexander-klimov]

What do I need to do to get around this with Scala 3.3.3 and SBT 1.10 ?
I'm getting this exact error when using Scalafmt 🤔