Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use premultiplied colours in rendering #6456

Open
wants to merge 19 commits into
base: master
Choose a base branch
from
Open

Use premultiplied colours in rendering #6456

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
0c0d607
Disable source alpha multiplication
frenzibyte Dec 13, 2024
8237df8
Update raw-caching glyph store to output premultiplied
frenzibyte Dec 13, 2024
bf0d5d4
Update texture atlas smoothing to output premultiplied
frenzibyte Dec 13, 2024
9a6149c
Update smooth path texture generation to output premultiplied
frenzibyte Dec 13, 2024
130cf69
Introduce `PremultipliedImage` and update texture loading processes
frenzibyte Dec 14, 2024
89d6c25
Update iOS texture loading process to load without conversion
frenzibyte Dec 14, 2024
0db36b7
Introduce `PremultipliedColour` and use in vertex specifications
frenzibyte Dec 14, 2024
21f500d
Intrdouce `UniformColour` for premultiplied colours in uniforms
frenzibyte Dec 14, 2024
5c9e39d
Update renderer clear info to use `PremultipliedColour`
frenzibyte Dec 13, 2024
f8f3b31
Update all shaders to interpret colours as premultiplied
frenzibyte Dec 13, 2024
6f10e23
Update android texture loader store
frenzibyte Dec 14, 2024
8a4abb3
Fix `SRGBColour` not clamping alpha on multiplication
frenzibyte Dec 18, 2024
1ff2b65
Add test scene for blending correctness
frenzibyte Dec 19, 2024
dc00efe
Fix mixture blending not blending alpha
frenzibyte Dec 19, 2024
7be757a
Fix last test case
frenzibyte Dec 19, 2024
39c888b
Decompose `PremultipliedColour` structure into RGBA fields
frenzibyte Dec 22, 2024
df53238
Add indexer for image pixels that accepts `PremultipliedColour`s
frenzibyte Dec 22, 2024
ccd0269
Mimic image API in `PremultipliedAlpha` and remove access to underlyi…
frenzibyte Dec 22, 2024
59493e3
Revert "Fix `SRGBColour` not clamping alpha on multiplication"
frenzibyte Dec 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions osu.Framework.Android/Graphics/Textures/AndroidTextureLoaderStore.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
using osu.Framework.Graphics.Textures;
using osu.Framework.IO.Stores;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.PixelFormats;

namespace osu.Framework.Android.Graphics.Textures
{
Expand All @@ -17,7 +18,7 @@ public AndroidTextureLoaderStore(IResourceStore<byte[]> store)
{
}

protected override Image<TPixel> ImageFromStream<TPixel>(Stream stream)
protected override PremultipliedImage ImageFromStream(Stream stream)
{
using (var bitmap = BitmapFactory.DecodeStream(stream))
{
Expand All @@ -34,7 +35,7 @@ protected override Image<TPixel> ImageFromStream<TPixel>(Stream stream)
}

bitmap.Recycle();
return Image.LoadPixelData<TPixel>(result, bitmap.Width, bitmap.Height);
return PremultipliedImage.FromPremultiplied(Image.LoadPixelData<Rgba32>(result, bitmap.Width, bitmap.Height));
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to https://developer.android.com/reference/android/graphics/BitmapFactory.Options#inPremultiplied which states that it is enabled by default, this line should be correct (cc @Susko3 if you can confirm this).

}
}
}
Expand Down
2 changes: 1 addition & 1 deletion osu.Framework.Benchmarks/BenchmarkFontLoading.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ private void runFor(GlyphStore store)

var icon = (IconUsage)propValue;
using (var upload = store.Get(icon.Icon.ToString()))
Trace.Assert(upload.Data != null);
Trace.Assert(upload.PremultipliedData != null);

if (remainingCount-- == 0)
return;
Expand Down
Binary file added osu.Framework.Tests/Resources/Textures/figma-blending-additive.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added osu.Framework.Tests/Resources/Textures/figma-blending-mixture.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
342 changes: 342 additions & 0 deletions osu.Framework.Tests/Visual/Drawables/TestSceneBlendingCorrectness.cs
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion osu.Framework.Tests/Visual/Drawables/TestSceneComplexBlending.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@ public TestSceneComplexBlending()
colourEquation.Current.Value = foregroundContainer.Blending.RGBEquation;
alphaEquation.Current.Value = foregroundContainer.Blending.AlphaEquation;

blendingSrcDropdown.Current.Value = BlendingType.SrcAlpha;
blendingSrcDropdown.Current.Value = BlendingType.One;
blendingDestDropdown.Current.Value = BlendingType.OneMinusSrcAlpha;
blendingAlphaSrcDropdown.Current.Value = BlendingType.One;
blendingAlphaDestDropdown.Current.Value = BlendingType.One;
Expand Down
8 changes: 5 additions & 3 deletions osu.Framework.Tests/Visual/Graphics/TestSceneShaderStorageBufferObject.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,14 @@
using NUnit.Framework;
using osu.Framework.Allocation;
using osu.Framework.Graphics;
using osu.Framework.Graphics.Colour;
using osu.Framework.Graphics.Primitives;
using osu.Framework.Graphics.Rendering;
using osu.Framework.Graphics.Rendering.Vertices;
using osu.Framework.Graphics.Shaders;
using osu.Framework.Graphics.Shaders.Types;
using osuTK;
using osuTK.Graphics;
using osuTK.Graphics.ES30;

namespace osu.Framework.Tests.Visual.Graphics
Expand Down Expand Up @@ -116,7 +118,7 @@ protected override void Draw(IRenderer renderer)
var rng = new Random(1337);

for (int i = 0; i < colourBuffer.Size; i++)
colourBuffer[i] = new ColourData { Colour = new Vector4(rng.NextSingle(), rng.NextSingle(), rng.NextSingle(), 1) };
colourBuffer[i] = new ColourData { Colour = PremultipliedColour.FromStraight(new Color4(rng.NextSingle(), rng.NextSingle(), rng.NextSingle(), 1)) };
}

// Bind the custom shader and SSBO.
Expand Down Expand Up @@ -212,7 +214,7 @@ protected override void Draw(IRenderer renderer)

for (int i = 0; i < areas.Count; i++)
{
int colourIndex = colourBuffer.Push(new ColourData { Colour = new Vector4(rng.NextSingle(), rng.NextSingle(), rng.NextSingle(), 1) });
int colourIndex = colourBuffer.Push(new ColourData { Colour = PremultipliedColour.FromStraight(new Color4(rng.NextSingle(), rng.NextSingle(), rng.NextSingle(), 1)) });

// Bind the SSBO. This may change between iterations if a buffer transition happens via the above push.
shader.BindUniformBlock("g_ColourBuffer", colourBuffer.CurrentBuffer);
Expand Down Expand Up @@ -260,7 +262,7 @@ protected override void Dispose(bool isDisposing)
[StructLayout(LayoutKind.Sequential, Pack = 1)]
private record struct ColourData
{
public UniformVector4 Colour;
public UniformColour Colour;
}

[StructLayout(LayoutKind.Sequential)]
Expand Down
7 changes: 3 additions & 4 deletions osu.Framework.Tests/Visual/Input/TestSceneInputResampler.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

using System;
using osu.Framework.Allocation;
using osu.Framework.Extensions.Color4Extensions;
using osu.Framework.Graphics;
using osu.Framework.Graphics.Lines;
using osu.Framework.Graphics.Rendering;
Expand All @@ -15,8 +16,6 @@
using osu.Framework.Testing;
using osuTK;
using osuTK.Graphics;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.PixelFormats;

namespace osu.Framework.Tests.Visual.Input
{
Expand All @@ -33,12 +32,12 @@ private void load(IRenderer renderer)
{
const int width = 2;
Texture gradientTexture = renderer.CreateTexture(width, 1, true);
var image = new Image<Rgba32>(width, 1);
var image = new PremultipliedImage(width, 1);

for (int i = 0; i < width; ++i)
{
byte brightnessByte = (byte)((float)i / (width - 1) * 255);
image[i, 0] = new Rgba32(brightnessByte, brightnessByte, brightnessByte);
image[i, 0] = new Color4(brightnessByte, brightnessByte, brightnessByte, 255).ToPremultiplied();
}

gradientTexture.SetData(new TextureUpload(image));
Expand Down
2 changes: 1 addition & 1 deletion osu.Framework.Tests/Visual/Performance/TestSceneTextureUploadPerformance.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ private class ReusableTextureUpload : ITextureUpload
{
private readonly ITextureUpload upload;

public ReadOnlySpan<Rgba32> Data => upload.Data;
public ReadOnlySpan<Rgba32> PremultipliedData => upload.PremultipliedData;

public int Level => upload.Level;

Expand Down
2 changes: 1 addition & 1 deletion osu.Framework.Tests/Visual/Platform/TestSceneClipboard.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ public void TestImage()
clipboardImage = image!.Clone();

var texture = renderer.CreateTexture(image.Width, image.Height);
texture.SetData(new TextureUpload(image));
texture.SetData(new TextureUpload(PremultipliedImage.FromStraight(image)));

Child = new Sprite
{
Expand Down
2 changes: 1 addition & 1 deletion osu.Framework.Tests/Visual/Sprites/TestSceneScreenshot.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ private void takeScreenshot()
var image = t.GetResultSafely();

var tex = renderer.CreateTexture(image.Width, image.Height);
tex.SetData(new TextureUpload(image));
tex.SetData(new TextureUpload(PremultipliedImage.FromStraight(image)));

display.Texture = tex;
background.Show();
Expand Down
17 changes: 8 additions & 9 deletions osu.Framework.Tests/Visual/UserInterface/TestSceneCircularBlob.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
// See the LICENCE file in the repository root for full licence text.

using osu.Framework.Allocation;
using osu.Framework.Extensions.Color4Extensions;
using osu.Framework.Graphics;
using osu.Framework.Graphics.Colour;
using osu.Framework.Graphics.Containers;
Expand All @@ -11,8 +12,6 @@
using osu.Framework.Graphics.UserInterface;
using osuTK;
using osuTK.Graphics;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.PixelFormats;

namespace osu.Framework.Tests.Visual.UserInterface
{
Expand All @@ -32,31 +31,31 @@ private void load()
{
const int width = 128;

var image = new Image<Rgba32>(width, 1);
var image = new PremultipliedImage(width, 1);

gradientTextureHorizontal = renderer.CreateTexture(width, 1, true);

for (int i = 0; i < width; ++i)
{
float brightness = (float)i / (width - 1);
image[i, 0] = new Rgba32((byte)(128 + (1 - brightness) * 127), (byte)(128 + brightness * 127), 128, 255);
image[i, 0] = new Color4((byte)(128 + (1 - brightness) * 127), (byte)(128 + brightness * 127), 128, 255).ToPremultiplied();
}

gradientTextureHorizontal.SetData(new TextureUpload(image));

image = new Image<Rgba32>(width, 1);
image = new PremultipliedImage(width, 1);

gradientTextureVertical = renderer.CreateTexture(1, width, true);

for (int i = 0; i < width; ++i)
{
float brightness = (float)i / (width - 1);
image[i, 0] = new Rgba32((byte)(128 + (1 - brightness) * 127), (byte)(128 + brightness * 127), 128, 255);
image[i, 0] = new Color4((byte)(128 + (1 - brightness) * 127), (byte)(128 + brightness * 127), 128, 255).ToPremultiplied();
}

gradientTextureVertical.SetData(new TextureUpload(image));

image = new Image<Rgba32>(width, width);
image = new PremultipliedImage(width, width);

gradientTextureBoth = renderer.CreateTexture(width, width, true);

Expand All @@ -66,11 +65,11 @@ private void load()
{
float brightness = (float)i / (width - 1);
float brightness2 = (float)j / (width - 1);
image[i, j] = new Rgba32(
image[i, j] = new Color4(
(byte)(128 + (1 + brightness - brightness2) / 2 * 127),
(byte)(128 + (1 + brightness2 - brightness) / 2 * 127),
(byte)(128 + (brightness + brightness2) / 2 * 127),
255);
255).ToPremultiplied();
}
}

Expand Down
17 changes: 8 additions & 9 deletions osu.Framework.Tests/Visual/UserInterface/TestSceneCircularProgress.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
#nullable disable

using osu.Framework.Allocation;
using osu.Framework.Extensions.Color4Extensions;
using osu.Framework.Graphics;
using osu.Framework.Graphics.Colour;
using osu.Framework.Graphics.Containers;
Expand All @@ -13,8 +14,6 @@
using osu.Framework.Graphics.UserInterface;
using osuTK;
using osuTK.Graphics;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.PixelFormats;

namespace osu.Framework.Tests.Visual.UserInterface
{
Expand All @@ -38,31 +37,31 @@ private void load()
{
const int width = 128;

var image = new Image<Rgba32>(width, 1);
var image = new PremultipliedImage(width, 1);

gradientTextureHorizontal = renderer.CreateTexture(width, 1, true);

for (int i = 0; i < width; ++i)
{
float brightness = (float)i / (width - 1);
image[i, 0] = new Rgba32((byte)(128 + (1 - brightness) * 127), (byte)(128 + brightness * 127), 128, 255);
image[i, 0] = new Color4((byte)(128 + (1 - brightness) * 127), (byte)(128 + brightness * 127), 128, 255).ToPremultiplied();
}

gradientTextureHorizontal.SetData(new TextureUpload(image));

image = new Image<Rgba32>(width, 1);
image = new PremultipliedImage(width, 1);

gradientTextureVertical = renderer.CreateTexture(1, width, true);

for (int i = 0; i < width; ++i)
{
float brightness = (float)i / (width - 1);
image[i, 0] = new Rgba32((byte)(128 + (1 - brightness) * 127), (byte)(128 + brightness * 127), 128, 255);
image[i, 0] = new Color4((byte)(128 + (1 - brightness) * 127), (byte)(128 + brightness * 127), 128, 255).ToPremultiplied();
}

gradientTextureVertical.SetData(new TextureUpload(image));

image = new Image<Rgba32>(width, width);
image = new PremultipliedImage(width, width);

gradientTextureBoth = renderer.CreateTexture(width, width, true);

Expand All @@ -72,11 +71,11 @@ private void load()
{
float brightness = (float)i / (width - 1);
float brightness2 = (float)j / (width - 1);
image[i, j] = new Rgba32(
image[i, j] = new Color4(
(byte)(128 + (1 + brightness - brightness2) / 2 * 127),
(byte)(128 + (1 + brightness2 - brightness) / 2 * 127),
(byte)(128 + (brightness + brightness2) / 2 * 127),
255);
255).ToPremultiplied();
}
}

Expand Down
7 changes: 3 additions & 4 deletions osu.Framework.Tests/Visual/UserInterface/TestSceneDrawablePath.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
using System.Collections.Generic;
using NUnit.Framework;
using osu.Framework.Allocation;
using osu.Framework.Extensions.Color4Extensions;
using osu.Framework.Graphics;
using osu.Framework.Graphics.Containers;
using osu.Framework.Graphics.Lines;
Expand All @@ -15,8 +16,6 @@
using osu.Framework.Utils;
using osuTK;
using osuTK.Graphics;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.PixelFormats;

namespace osu.Framework.Tests.Visual.UserInterface
{
Expand All @@ -29,12 +28,12 @@ public partial class TestSceneDrawablePath : FrameworkTestScene
[BackgroundDependencyLoader]
private void load(IRenderer renderer)
{
var image = new Image<Rgba32>(texture_width, 1);
var image = new PremultipliedImage(texture_width, 1);

for (int i = 0; i < texture_width; ++i)
{
byte brightnessByte = (byte)((float)i / (texture_width - 1) * 255);
image[i, 0] = new Rgba32(255, 255, 255, brightnessByte);
image[i, 0] = new Color4(255, 255, 255, brightnessByte).ToPremultiplied();
}

gradientTexture = renderer.CreateTexture(texture_width, 1, true);
Expand Down
9 changes: 6 additions & 3 deletions osu.Framework.iOS/Graphics/Textures/IOSTextureLoaderStore.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
using osu.Framework.Graphics.Textures;
using osu.Framework.IO.Stores;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.PixelFormats;
using UIKit;

namespace osu.Framework.iOS.Graphics.Textures
Expand All @@ -19,7 +20,7 @@ public IOSTextureLoaderStore(IResourceStore<byte[]> store)
{
}

protected override Image<TPixel> ImageFromStream<TPixel>(Stream stream)
protected override PremultipliedImage ImageFromStream(Stream stream)
{
using (var nativeData = NSData.FromStream(stream))
{
Expand All @@ -39,9 +40,11 @@ protected override Image<TPixel> ImageFromStream<TPixel>(Stream stream)
using (CGBitmapContext textureContext = new CGBitmapContext(data, width, height, 8, width * 4, CGColorSpace.CreateDeviceRGB(), CGImageAlphaInfo.PremultipliedLast))
textureContext.DrawImage(new CGRect(0, 0, width, height), uiImage.CGImage);

var image = Image.LoadPixelData<TPixel>(data, width, height);
var image = Image.LoadPixelData<Rgba32>(data, width, height);

return image;
// UIImage/CGBitmapContext already produce images in premultiplied form,
// so we can directly wrap the image without performing any conversion.
return PremultipliedImage.FromPremultiplied(image);
}
}
}
Expand Down
8 changes: 8 additions & 0 deletions osu.Framework/Extensions/Color4Extensions/Color4Extensions.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
using osuTK.Graphics;
using System;
using System.Globalization;
using osu.Framework.Graphics.Colour;

namespace osu.Framework.Extensions.Color4Extensions
{
Expand Down Expand Up @@ -288,5 +289,12 @@ public static (float h, float s, float v) ToHSV(this Color4 colour)

return (h, s, v);
}

/// <summary>
/// Converts a <see cref="Color4"/> to premultiplied alpha and returns a resultant <see cref="PremultipliedColour"/>.
/// The input colour is assumed to be in straight-alpha form.
/// </summary>
/// <param name="colour">The colour as straight-alpha.</param>
public static PremultipliedColour ToPremultiplied(this Color4 colour) => PremultipliedColour.FromStraight(colour);
}
}
Loading
Loading