Work towards part of https://github.com/flutter/flutter/issues/138798
Allow updating single glyphs in the glyph atlas, without replacing the entire bitmap. Required to efficiently append/update to large atlases.
Uses the utility added in https://github.com/flutter/engine/pull/51361
I counted the removal of 58 static casts. There was one addition made to the original utility however. Vulkan HPP was promoting all enums to its own mask type. This in itself is problematic but we got away with it because there was no one else doing this kind of promotion. Till we added our own utility. To avoid polluting the namespace with methods that may cause ambiguity, enums that are masks must explicitly be marked as maskable with `IMPELLER_ENUM_IS_MASK` in the `impeller` namespace.
No change in functionality.
Reland of: https://github.com/flutter/engine/pull/50139
Metal does not seem to like it when we collect 50+ command buffers at once. Adjust the aiks context logic to regularly flush the cmd buffers.
Reverts flutter/engine#50139
Initiated by: jonahwilliams
This change reverts the following previous change:
Original Description:
The Impeller Vulkan backend benefits from batching submission to the vk graphics queue. Managing this automatically is non-trivial and adds surprising/fragile thread based behavior, see: https://github.com/flutter/engine/pull/49870
Instead, introduce an impeller::CommandQueue object that command buffers must be submitted to in lieu of CommandBuffer->Submit, which has been made private.
TLDR
old
```c++
buffer->Submit();
```
new
```c++
context.GetQueue()->Submit({buffer});
```
The Metal and GLES implementations internally just call the private CommandBuffer->Submit, though there may be future opportunities to simplify here. The Vulkan implementation is where the meat is.
Aiks takes advantage of this by storing all command buffers on the aiks context while rendering a frame, and then performing one submit in aiks_context render. I don't think this will introduce any thread safety problems, as we don't guarantee much about aiks context - nor do we use it in a multithreaded context as far as I know.
Other tasks such as image upload still just directly submit their command buffers via the queue.
Fixes https://github.com/flutter/flutter/issues/141123
The Impeller Vulkan backend benefits from batching submission to the vk graphics queue. Managing this automatically is non-trivial and adds surprising/fragile thread based behavior, see: https://github.com/flutter/engine/pull/49870
Instead, introduce an impeller::CommandQueue object that command buffers must be submitted to in lieu of CommandBuffer->Submit, which has been made private.
TLDR
old
```c++
buffer->Submit();
```
new
```c++
context.GetQueue()->Submit({buffer});
```
The Metal and GLES implementations internally just call the private CommandBuffer->Submit, though there may be future opportunities to simplify here. The Vulkan implementation is where the meat is.
Aiks takes advantage of this by storing all command buffers on the aiks context while rendering a frame, and then performing one submit in aiks_context render. I don't think this will introduce any thread safety problems, as we don't guarantee much about aiks context - nor do we use it in a multithreaded context as far as I know.
Other tasks such as image upload still just directly submit their command buffers via the queue.
Fixes https://github.com/flutter/flutter/issues/141123
Generated by https://github.com/flutter/engine/pull/48903 (`dart ./tools/header_guard_check/bin/main.dart --fix`).
As discussed with @cbracken and @jmagman, the guards are not technically needed on the Mac/iOS code, but they (a) do not hurt and (b) still provide value if for some reason `#include` is used instead of `#import` (though I suspect we could try to add that to the tool in the future as well).
The backend specific sampler libraries hold a strong reference to the native sampler objects and never clear this cache. As a result of this, we don't theoretically need rendering commands to increment a shared_ptr ref count - instead the sampler library can provide the Sampler object as a const ref and guarantee that it continues to be valid.
This allows us to reduce the amount of refcount ops for commands that use samplers.
Additionally, the sampler library uses nullptr as a sentinel for failing to construct a sampler object. Since sampler already has an isValid member that is checked - we can replace this with a specific invalid object subtype.
The original change was reverted as it broke the GPU tracer reset logic. The previous logic assumed that the first started command buffer would also be the first submitted command buffer. With the direct encoding, this is no longer the case - so we shift the logic so that we reset query pools once at startup and then after the queries are finished being recorded.
This might actually be better in general since we should be doing less work in the frame workload.
Reverts flutter/engine#49780
Initiated by: jonahwilliams
This change reverts the following previous change:
Original Description:
Part of https://github.com/flutter/flutter/issues/140804
Rather than using impeller::Command, the impeller::RenderPass records most state directly into the Vulkan command buffer. This should remove allocation/free overhead of the intermediary structures and make further improvements to the backend even easier. This required a number of other changes to the renderer:
1. The render pass holds a strong ptr to the context. This helps avoid locking continually while encoding, which is quite slow.
2. barriers need to be encoded on the _producing_ side, and not the consuming side. This is because we'll actually run the consuming code before the producing code. i.e. we transition to shader read at the end of a render pass instead of when binding.
3. I've updated the binding code to also provide the descriptor type so that we don't need to look it up from the desc. set.
4. I added a test render pass class that records commands.
Part of https://github.com/flutter/flutter/issues/140804
Rather than using impeller::Command, the impeller::RenderPass records most state directly into the Vulkan command buffer. This should remove allocation/free overhead of the intermediary structures and make further improvements to the backend even easier. This required a number of other changes to the renderer:
1. The render pass holds a strong ptr to the context. This helps avoid locking continually while encoding, which is quite slow.
2. barriers need to be encoded on the _producing_ side, and not the consuming side. This is because we'll actually run the consuming code before the producing code. i.e. we transition to shader read at the end of a render pass instead of when binding.
3. I've updated the binding code to also provide the descriptor type so that we don't need to look it up from the desc. set.
4. I added a test render pass class that records commands.
The Flutter GPU tests were broken, and it turns out that I had never set them up to run on CI. This fixes the HostBuffer and gets the test suite running on CI.
Part of https://github.com/flutter/flutter/issues/140804
Migrate the rest of the commands in impeller to use the new API. Hide RenderPass::AddCommand. On subsequent changes I will be able to begin making some of these methods virtual so we can add more direct pass through. Though the vulkan backend will be blocked on changes to descriptor sets: https://github.com/flutter/engine/pull/49686
Reland of https://github.com/flutter/engine/pull/49505
---
part of https://github.com/flutter/flutter/issues/140804
We can't use the existing host buffer abstraction as that requires us to collect all allocations up front. By itself, this isn't sufficient for #140804 , because we'll need a way to mark ranges as dirty and/or flush if we don't have host coherent memory. But by itself this change should be beneficial as we'll create fewer device buffers and should do less allocation in general.
The size of the device buffers is 1024 Kb, somewhat arbitrarily chosen.
Reverts flutter/engine#49505
Initiated by: jonahwilliams
This change reverts the following previous change:
Original Description:
part of https://github.com/flutter/flutter/issues/140804
We can't use the existing host buffer abstraction as that requires us to collect all allocations up front. By itself, this isn't sufficient for #140804 , because we'll need a way to mark ranges as dirty and/or flush if we don't have host coherent memory. But by itself this change should be beneficial as we'll create fewer device buffers and should do less allocation in general.
The size of the device buffers is 1024 Kb, somewhat arbitrarily chosen.
part of https://github.com/flutter/flutter/issues/140804
We can't use the existing host buffer abstraction as that requires us to collect all allocations up front. By itself, this isn't sufficient for #140804 , because we'll need a way to mark ranges as dirty and/or flush if we don't have host coherent memory. But by itself this change should be beneficial as we'll create fewer device buffers and should do less allocation in general.
The size of the device buffers is 1024 Kb, somewhat arbitrarily chosen.
Make the wrapped HostBuffer wrapper track/look up emplacements using a
fake byte offset.
This is a trick to keep Flutter GPU working after
https://github.com/flutter/engine/pull/49505 lands. I'll likely swing
around and change how `BufferView` works later on. We can simplify a lot
by making Flutter GPU `BufferView`s just take `DeviceBuffer` handles.
* Switch from legacy uniform semantics to uniform structs.
* Completely separate shader bundle from runtime stage.
* Packing multiple backends per shader.
* Pack struct and member fields into the shader bundle flatbuffer.
* Bind uniforms with correct metadata for GLES.
* Add uniform struct size and member offset reflection.
This is part of the work towards supporting OpenGLES and Vulkan for runtime stage shaders.
Removes some redundant work we had around SkSL. Now only bundles the shaders we actually ask for from the command line.
@bdero, we should figure out if this is the right approach for flutter_gpu.
With this change, the IPLR format goes from having a root table of shader related information to a root table of shader information per `sksl`, `metal`, `opengles`, and `vulkan` platforms.
This may end up allowing us to revert https://github.com/flutter/engine/pull/47278, but I'm not sure I understand all the implications of that at this point.
I have run some but not all tests locally.
* Add `--shader-bundle` mode to impellerc that takes a simple JSON spec and produces a single flatbuffer with a pack of named shaders.
* Added "single invocation" mode to the impellerc GN templates.
* Record vertex attribute reflection information.
* Light refactoring of the compiler frontend to make compiler invocations easier to follow.
Example shader bundle spec (json form of the yaml spec as shown in the [Flutter GPU](https://docs.google.com/document/d/1Sh1BAC5c_kkuMVreo7ymBzPoMzb7lamZRPsI7GBXv5M/edit?resourcekey=0-5w8u2V-LS41tCHeoE8bDTQ#heading=h.a3gmnzue7wgq) doc):
```json
{
"UnlitFragment": {
"type": "fragment",
"file": "shaders/flutter_gpu_unlit.frag"
},
"UnlitVertex": {
"type": "vertex",
"file": "shaders/flutter_gpu_unlit.vert"
},
"TextureFragment": {
"type": "fragment",
"file": "shaders/flutter_gpu_texture.frag"
},
"TextureVertex": {
"type": "vertex",
"file": "shaders/flutter_gpu_texture.vert"
}
}
```
Example impellerc invocation:
```bash
impellerc \
--include=~/projects/flutter/engine/src/flutter/impeller/compiler/shader_lib \
--runtime-stage-metal \
--sl=assets/TestLibrary.shaderbundle \
--shader-bundle='{"UnlitFragment": {"type": "fragment", "file": "shaders/flutter_gpu_unlit.frag"}, "UnlitVertex": {"type": "vertex", "file": "shaders/flutter_gpu_unlit.vert"}, "TextureFragment": {"type": "fragment", "file": "shaders/flutter_gpu_texture.frag"}, "TextureVertex": {"type": "vertex", "file": "shaders/flutter_gpu_texture.vert"}}'
```
Runtime usage:
```dart
/// Add a render pass encoder to the command buffer so that we can start
/// encoding commands.
final encoder = commandBuffer.createRenderPass(renderTarget);
/// Load a shader bundle asset.
final library =
gpu.ShaderLibrary.fromAsset('assets/TestLibrary.shaderbundle')!;
/// Create a RenderPipeline using shaders from the asset.
final vertex = library['TextureVertex']!;
final fragment = library['TextureFragment']!;
final pipeline = gpu.gpuContext.createRenderPipeline(vertex, fragment);
encoder.bindPipeline(pipeline);
```
https://github.com/flutter/engine/assets/919017/6f3e9a59-d180-4ba6-b14c-fa6d7056965c
Places the binding data in a vector, since they key was only meaningful on metal but not used anywhere. I don't think that we need to specificially handle the case where our own contents bind the same contents multiple times, but interested to discuss if folks disagree.
Placing the vertex buffer on the binding object meant that we were actually paying 2x the size cost for it (one for vertex bindings, one for fragment bindings). By moving this object onto command itself, we reduce the size and avoid spliting up the command state in a weird way.
Also updates most of the contents to prefer moving the VertexBuffer.
First triangle, in the framework! 🎉
Adds shader libraries, pipelines, command buffers, render passes, etc.
* Light pipelines/shader objects. No optimization yet, pipeline warming
to come.
* "Dynamic" command style. Don't re-send bindings if you don't need to.
Essentially: https://github.com/flutter/flutter/issues/133179
* No need to explicitly encode passes.
* Minimal descriptor usage.
* Nothing is async, except for the optional command buffer completion
callback.
It took a bunch of experimenting to get here, but I think things are
starting to look pretty neat. :)
Todo:
* Land the shader bundle format/remove the testing hacks & fixtures that
piggyback off of the runtime effect system.
* Add remaining calls for blend config, clearing bindings, etc.
* Inconsistent error handling patterns that need cleanup.
* Maybe: Surface exceptions for validation errors.
* Handle the texture usage bitmask more elegantly.
This patch does the following:
- Updates `flutter_tester` to set up an Impeller rendering context and surface if `--enable-impeller` is set to true, using the Vulkan backend with Swiftshader.
- Updates `run_tests.py` to run all tests except the smoke test (that one really has no rendering impact whatsoever) with and without `--enable-impeller`.
- Updates a few tests to work that were trivial:
- A couple tests needed updated goldens for very minor rendering differences. Filed https://github.com/flutter/flutter/issues/135684 to track using Skia gold for this instead.
- Disabled SKP screenshotting if Impeller is enabled, and updated the test checking that to verify an error is thrown if an SKP is requested.
- The Dart GPU based test now asserts that the gpu context is available if Impeller is enabled, and does not deadlock if run in a single threaded mode.
- We were missing some trace events around `Canvas::SaveLayer` for Impeller as compared to Skia.
- A couple other tests had strict checks about exception messages that are slightly different between Skia and Impeller.
- I've filed bugs for other tests that may require a little more work, and skipped them for now. For FragmentProgram on Vulkan I reused an existing bug.
This is part of my attempt to address https://github.com/flutter/flutter/issues/135693, although @chinmaygarde and I had slightly different ideas about how to do this.
The goals here are:
- Run the Dart unit tests we already have with Impeller enabled.
- Enable running more of the framework tests (including gold tests) with Impeller enabled.
- Run all of these tests via public `dart:ui` API rather than mucking around in C++ internals in the engine.
Resolves https://github.com/flutter/flutter/issues/132516.
Add `impeller::HostBuffer` wrapper to Flutter GPU.
* Allows for lazy batch uploads of sparse host data to the GPU.
* Handles platform alignment requirements.
* API returns buffer view handles that will be fed to commands.
Move the GpuContext to its new home. I added a `flutter_tester` test that just verifies an exception is gracefully thrown when Impeller isn't available.
In a later patch, I'll land a way to eagerly supply the Impeller context on the cpp side to enable testing through the playground (as mentioned in https://github.com/flutter/flutter/issues/127712).
Part of https://github.com/flutter/flutter/issues/131346
Stubs a minimal test of the FFI utilities that `dart:ui` uses, but using
public symbols exported from the engine library. If this goes well, I'll
move the stuff from `dart:ui` into here and begin landing parts of the
API with test coverage.
