Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Benchmarks. Change output format and modes #5217

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Benchmarks. Change output format and modes #5217

wants to merge 4 commits into from
+206 −36

Conversation

igordmn
Copy link
Collaborator

@igordmn igordmn commented Jan 27, 2025 •
edited
Loading

  • Replace total CPU time by CPU/GPU/TOTAL average frame time
  • Because Mode.CPU now measures GPU, rename it to SIMPLE
  • Because the difference between FRAMES and SIMPLE is only in vsync, rename it to VSYNC_EMULATION
  • Remove Mode.FRAMES_GPU, as it affects only output
  • Add Compose/Kotlin version, frame/warmup count
  • Add writing to CSV on Desktop
    image

Example of output into console:

Running emulating 120 FPS

# AnimatedVisibility
Version: 1.7.1 (Kotlin 2.1.0)
1000 frames (warmup 100)

CPU average frame time: 268.626us
GPU average frame time: 1.383332ms
TOTAL average frame time: 1.651958ms

Worst p1 CPU (ms): 2.017518ms
Worst p2 CPU (ms): 1.566880ms
Worst p5 CPU (ms): 1.188572ms
Worst p10 CPU (ms): 982.504us
Worst p25 CPU (ms): 753.478us
Worst p50 CPU (ms): 592.363us

Worst p1 GPU (ms): 5.945218ms
Worst p10 GPU (ms): 2.379730ms
Worst p50 GPU (ms): 1.909719ms

Missed frames (no buffering):
    - count: 3
    - ratio: 0.003     
                
Missed frames (double buffering):
    - count: 3
    - ratio: 0.003     
                
Results saved to D:\Work\compose-multiplatform\benchmarks\multiplatform\benchmarks\build\benchmarks\AnimatedVisibility.csv

# LazyGrid
...

@igordmn igordmn changed the title Benchmarks. Change output format Benchmarks. Change output format and modes Jan 27, 2025
@igordmn igordmn requested a review from pjBooms January 27, 2025 03:34
@igordmn igordmn force-pushed the igor.demin/benchmarks-improvements branch 2 times, most recently from 08339a5 to f22c2e1 Compare January 27, 2025 03:45
@igordmn igordmn force-pushed the igor.demin/benchmarks-improvements branch from f22c2e1 to 769a2df Compare January 27, 2025 03:46
pjBooms

This comment was marked as resolved.

Sign in to view

pjBooms
pjBooms suggested changes Feb 10, 2025
View reviewed changes
benchmarks/multiplatform/benchmarks/src/commonMain/kotlin/Benchmarks.kt Outdated
val percentileCPUAverage: List<BenchmarkPercentileAverage>,
val percentileGPUAverage: List<BenchmarkPercentileAverage>,
val noBufferingMissedFrames: MissedFrames,
val doubleBufferingMissedFrames: MissedFrames
) {
fun prettyPrint() {
if (Args.isModeEnabled(Mode.CPU)) {
println("$frameCount frames CPU render time: $renderTime")
println("Version: " + Args.versionInfo)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not print a version if it is not set

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

benchmarks/multiplatform/benchmarks/src/commonMain/kotlin/Benchmarks.kt Outdated
listOf(0.01, 0.02, 0.05, 0.1, 0.25, 0.5).map { percentile ->
conditions,
averageFrameInfo,
listOf(0.01, 0.02, 0.05, 0.1, 0.25, 0.5).map { percentile ->
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extra space

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

benchmarks/multiplatform/benchmarks/src/commonMain/kotlin/MeasureComposable.kt Outdated
@@ -8,6 +8,9 @@ import kotlin.time.Duration
import kotlin.time.Duration.Companion.nanoseconds
import kotlin.time.ExperimentalTime
import kotlinx.coroutines.*
import org.jetbrains.skia.Color
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it from previous PR?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

benchmarks/multiplatform/benchmarks/src/iosMain/kotlin/main.ios.kt Outdated
@@ -9,3 +9,8 @@ fun main(args : List<String>) {
Args.parseArgs(args.toTypedArray())
runBlocking { runBenchmarks(graphicsContext = graphicsContext()) }
}

actual fun saveBenchmarksOnDisk(name: String, stats: BenchmarkStats) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it second actual implementation for iOS?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed. Messed up with interactive rebase, and didn't check compilation on macOs 🫠

benchmarks/multiplatform/benchmarks/src/macosMain/kotlin/main.macos.kt Outdated
@@ -8,3 +8,7 @@ fun main(args : Array<String>) {
Args.parseArgs(args)
runBlocking { runBenchmarks(graphicsContext = graphicsContext()) }
}

actual fun saveBenchmarksOnDisk(name: String, stats: BenchmarkStats) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it second actual implementation for macOS?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed

benchmarks/multiplatform/benchmarks/src/commonMain/kotlin/Benchmarks.kt Outdated
}

expect fun saveBenchmarksOnDisk(name: String, stats: BenchmarkStats)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename to saveBenchmarkStatsOnDisk?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

benchmarks/multiplatform/benchmarks/src/desktopMain/kotlin/Benchmarks.desktop.kt Outdated
import java.time.LocalDateTime
import java.time.format.DateTimeFormatter

actual fun saveBenchmarksOnDisk(name: String, stats: BenchmarkStats) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have not you tried kotlinx.io to make it common?

Copy link
Collaborator Author

@igordmn igordmn Feb 14, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implemented

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also checked that iOS/wasm work as before.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked native macOs - it saves files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pjBooms pjBooms pjBooms requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@igordmn @pjBooms