-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix double-execution of repo-scanning functions #92
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -59,11 +59,12 @@ object RepoSnapshot { | |
def log(message: String)(implicit repo: Repo): Unit = logger.info(s"${repo.full_name} - $message") | ||
def logAround[T](desc: String)(thunk: => Future[T])(implicit repo: Repo): Future[T] = { | ||
val start = System.currentTimeMillis() | ||
thunk.onComplete { attempt => | ||
val fut = thunk // evaluate thunk, evaluate only once! | ||
fut.onComplete { attempt => | ||
val elapsedMs = System.currentTimeMillis() - start | ||
log(s"'$desc' $elapsedMs ms : success=${attempt.isSuccess}") | ||
} | ||
thunk | ||
fut | ||
Comment on lines
-62
to
+67
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is the key part of the fix - we just evaluate There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 👏 |
||
} | ||
|
||
def isMergedToMain(pr: PullRequest)(implicit repo: Repo): Boolean = | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
package lib | ||
|
||
import org.scalatest.flatspec.AnyFlatSpec | ||
import org.scalatest.matchers.should.Matchers | ||
|
||
import java.util.concurrent.Executors | ||
import java.util.concurrent.atomic.{AtomicLong, LongAdder} | ||
import scala.concurrent.duration._ | ||
import scala.concurrent.{Await, ExecutionContext, Future} | ||
|
||
class DogpileSpec extends AnyFlatSpec with Matchers { | ||
it should "not concurrently execute the side-effecting function" in { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I added this test while trying to debug this issue. It never reproduced the problem - because the problem wasn't in |
||
|
||
val executionCount = new LongAdder() | ||
val runningCounter = new AtomicLong(0) | ||
val clashCounter = new LongAdder() | ||
|
||
val dogpile = new Dogpile[String]({ | ||
val numRunning = runningCounter.incrementAndGet() | ||
executionCount.increment() | ||
if (numRunning > 1) { | ||
clashCounter.increment() | ||
} | ||
Thread.sleep(10) | ||
runningCounter.decrementAndGet() | ||
Future.successful("OK") | ||
}) | ||
|
||
implicit val ec = ExecutionContext.fromExecutor(Executors.newFixedThreadPool(4)) | ||
|
||
val numExecutions = 20 | ||
val allF = Future.traverse(1 to numExecutions)(_ => dogpile.doAtLeastOneMore()) | ||
Await.ready(allF, 15.seconds) | ||
|
||
executionCount.intValue() should be <= numExecutions | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Actually, in this test, what I would hope is that the execution count is more like 2, rather than 20, but due to the behaviour of |
||
runningCounter.get() shouldBe 0 | ||
clashCounter.longValue() shouldBe 0 | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is concerning to me, but doesn't seem to be currently causing severe problems: the docs for
AtomicReference. updateAndGet()
say:I only realised this while trying to hunt down the cause for double-executions just now, and thing is that the function we're using here is not side-effect-free - it's very very side-effecty, labelling repos, creating GitHub comments, etc.
This code should be updated in another PR to be truly safe, and run the side-effecting code with the desired semantics, which are closer to 'throttle-last'.