DEV/flutter_flutter
1
0
Fork 0
mirror of https://github.com/flutter/flutter.git synced 2026-02-20 02:29:02 +08:00
Code Issues Packages Projects Releases Wiki Activity

Add set semantics enabled API and wire iOS a11y bridge (#161265)

Browse Source 
fixes https://github.com/flutter/flutter/issues/158399

old pr https://github.com/flutter/engine/pull/56691

previously the only correct way to enable semantics is that ios
embedding receive signal from native OS, it call SetSemanticsEnabled to
shell and then to dart to enable semantics tree generation.

If for some reason framework decide to enable semantics first, e.g.
through SemanticsBinding.instance.ensureSemantics(typically due to
integration test or ci that wants to test semantics), the update will be
dropped in shell. Even if it later on receive signal from native OS to
turn on semantics, it can't construct the complete accessibility tree in
embedding because the updatesemantics sends diff update and previous
updates are gone forever. It will end up in a broken state.

This pr changes so that the only source of truth will be in the
framework side. When framework starts generating the the semantics tree,
it will call SetSemanticsTreeEnabled through dart:ui, and the embedding
needs to prepare itself to accept semantics update after receiving the
message.

This however require some refactoring on iOS embedding because it will
only create a11y bridge when receiving OS notification when assitive
technologies turns on.

This requires three phase transition

add an empty dart:ui API setSemanticsTreeEnabled
makes framework calls the empty API.
merge this pr with actual implementation of setSemanticsTreeEnabled

I will do the android part in a separate pr


## Pre-launch Checklist

- [ ] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [ ] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [ ] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [ ] I signed the [CLA].
- [ ] I listed at least one issue that this PR fixes in the description
above.
- [ ] I updated/added relevant documentation (doc comments with `///`).
- [ ] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [ ] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [ ] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview
[Tree Hygiene]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
[test-exempt]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes
[Discord]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md
[Data Driven Fixes]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
This commit is contained in:
chunhtai 2025-03-24 13:11:13 -07:00 committed by GitHub
parent 45d86cf670
commit 26037dff87
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
34 changed files with 557 additions and 72 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
dev/integration_tests/ios_host_app/flutterapp/lib/main
View File

@ -32,6 +32,7 @@ const BasicMessageChannel<String> _kReloadChannel =
void main() {
// Ensures bindings are initialized before doing anything.
WidgetsFlutterBinding.ensureInitialized();
ui.PlatformDispatcher.instance.setSemanticsTreeEnabled(true);
// Start listening immediately for messages from the iOS side. ObjC calls
// will be made to let us know when we should be changing the app state.
_kReloadChannel.setMessageHandler(run);

1
engine/src/flutter/ci/licenses_golden/excluded_files
View File

@ -283,6 +283,7 @@
../../../flutter/runtime/fixtures
../../../flutter/runtime/no_dart_plugin_registrant_unittests.cc
../../../flutter/runtime/platform_isolate_manager_unittests.cc
../../../flutter/runtime/runtime_controller_unittests.cc
../../../flutter/runtime/type_conversions_unittests.cc
../../../flutter/shell/common/animator_unittests.cc
../../../flutter/shell/common/base64_unittests.cc

2
engine/src/flutter/ci/licenses_golden/licenses_flutter
View File

@ -52880,6 +52880,7 @@ ORIGIN: ../../../flutter/shell/platform/darwin/ios/platform_message_handler_ios.
ORIGIN: ../../../flutter/shell/platform/darwin/ios/platform_message_handler_ios_test.mm + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/shell/platform/darwin/ios/platform_view_ios.h + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/shell/platform/darwin/ios/platform_view_ios.mm + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/shell/platform/darwin/ios/platform_view_ios_test.mm + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/shell/platform/darwin/ios/rendering_api_selection.h + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/shell/platform/darwin/ios/rendering_api_selection.mm + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/shell/platform/darwin/macos/framework/Headers/FlutterAppDelegate.h + ../../../flutter/LICENSE
@ -55876,6 +55877,7 @@ FILE: ../../../flutter/shell/platform/darwin/ios/platform_message_handler_ios.mm
FILE: ../../../flutter/shell/platform/darwin/ios/platform_message_handler_ios_test.mm
FILE: ../../../flutter/shell/platform/darwin/ios/platform_view_ios.h
FILE: ../../../flutter/shell/platform/darwin/ios/platform_view_ios.mm
FILE: ../../../flutter/shell/platform/darwin/ios/platform_view_ios_test.mm
FILE: ../../../flutter/shell/platform/darwin/ios/rendering_api_selection.h
FILE: ../../../flutter/shell/platform/darwin/ios/rendering_api_selection.mm
FILE: ../../../flutter/shell/platform/darwin/macos/framework/Headers/FlutterAppDelegate.h

1
engine/src/flutter/lib/ui/dart_ui.cc
View File

@ -99,6 +99,7 @@ typedef CanvasPath Path;
V(PlatformConfigurationNativeApi::UpdateSemantics) \
V(PlatformConfigurationNativeApi::SetNeedsReportTimings) \
V(PlatformConfigurationNativeApi::SetIsolateDebugName) \
V(PlatformConfigurationNativeApi::SetSemanticsTreeEnabled) \
V(PlatformConfigurationNativeApi::RequestDartPerformanceMode) \
V(PlatformConfigurationNativeApi::GetPersistentIsolateData) \
V(PlatformConfigurationNativeApi::ComputePlatformResolvedLocale) \

20
engine/src/flutter/lib/ui/platform_dispatcher.dart
View File

@ -719,6 +719,26 @@ class PlatformDispatcher {
@Native<Void Function(Int64)>(symbol: 'PlatformConfigurationNativeApi::RegisterBackgroundIsolate')
external static void __registerBackgroundIsolate(int rootIsolateId);
/// Informs the engine whether the framework is generating a semantics tree.
///
/// Only framework knows when semantics tree should be generated. It uses this
/// method to notify the engine whether the framework will generate a semantics tree.
///
/// In the case where platforms want to enable semantics, e.g. when
/// assistive technologies are enabled, it notifies framework through
/// [onSemanticsEnabledChanged].
///
/// After this has been set to true, platforms are expected to prepare for accepting
/// semantics update sent via [FlutterView.updateSemantics]. When this is set to false, platforms
/// may dispose any resources associated with processing semantics as no further
/// semantics updates will be sent via [FlutterView.updateSemantics].
///
/// One must call this method with true before sending update through [updateSemantics].
void setSemanticsTreeEnabled(bool enabled) => _setSemanticsTreeEnabled(enabled);
@Native<Void Function(Bool)>(symbol: 'PlatformConfigurationNativeApi::SetSemanticsTreeEnabled')
external static void _setSemanticsTreeEnabled(bool update);
/// Deprecated. Migrate to [ChannelBuffers.setListener] instead.
///
/// Called whenever this platform dispatcher receives a message from a

5
engine/src/flutter/lib/ui/window.dart
View File

@ -390,9 +390,8 @@ class FlutterView {
/// Change the retained semantics data about this [FlutterView].
///
/// If [PlatformDispatcher.semanticsEnabled] is true, the user has requested that this function
/// be called whenever the semantic content of this [FlutterView]
/// changes.
/// [PlatformDispatcher.setSemanticsTreeEnabled] must be called with true
/// before sending update through this method.
///
/// This function disposes the given update, which means the semantics update
/// cannot be used further.

8
engine/src/flutter/lib/ui/window/platform_configuration.cc
View File

@ -669,6 +669,14 @@ void PlatformConfigurationNativeApi::UpdateSemantics(int64_t view_id,
view_id, update);
}
void PlatformConfigurationNativeApi::SetSemanticsTreeEnabled(bool enabled) {
UIDartState::ThrowIfUIOperationsProhibited();
UIDartState::Current()
->platform_configuration()
->client()
->SetSemanticsTreeEnabled(enabled);
}
Dart_Handle PlatformConfigurationNativeApi::ComputePlatformResolvedLocale(
Dart_Handle supportedLocalesHandle) {
UIDartState::ThrowIfUIOperationsProhibited();

9
engine/src/flutter/lib/ui/window/platform_configuration.h
View File

@ -97,6 +97,13 @@ class PlatformConfigurationClient {
///
virtual void UpdateSemantics(int64_t viewId, SemanticsUpdate* update) = 0;
//--------------------------------------------------------------------------
/// @brief Notifies whether Framework starts generating semantics tree.
///
/// @param[in] enabled True if Framework starts generating semantics tree.
///
virtual void SetSemanticsTreeEnabled(bool enabled) = 0;
//--------------------------------------------------------------------------
/// @brief When the Flutter application has a message to send to the
/// underlying platform, the message needs to be forwarded to
@ -625,6 +632,8 @@ class PlatformConfigurationNativeApi {
static void UpdateSemantics(int64_t viewId, SemanticsUpdate* update);
static void SetSemanticsTreeEnabled(bool enabled);
static void SetNeedsReportTimings(bool value);
static Dart_Handle GetPersistentIsolateData();

2
engine/src/flutter/lib/web_ui/lib/platform_dispatcher.dart
View File

@ -85,6 +85,8 @@ abstract class PlatformDispatcher {
void scheduleWarmUpFrame({required VoidCallback beginFrame, required VoidCallback drawFrame});
void setSemanticsTreeEnabled(bool enabled) {}
AccessibilityFeatures get accessibilityFeatures;
VoidCallback? get onAccessibilityFeaturesChanged;

9
engine/src/flutter/lib/web_ui/lib/src/engine/platform_dispatcher.dart
View File

@ -700,6 +700,15 @@ class EnginePlatformDispatcher extends ui.PlatformDispatcher {
FrameService.instance.scheduleWarmUpFrame(beginFrame: beginFrame, drawFrame: drawFrame);
}
@override
void setSemanticsTreeEnabled(bool enabled) {
if (!enabled) {
for (final EngineFlutterView view in views) {
view.semantics.reset();
}
}
}
/// Updates the application's rendering on the GPU with the newly provided
/// [Scene]. This function must be called within the scope of the
/// [onBeginFrame] or [onDrawFrame] callbacks being invoked. If this function

2
engine/src/flutter/runtime/BUILD.gn
View File

@ -140,6 +140,7 @@ if (enable_unittests) {
"dart_service_isolate_unittests.cc",
"dart_vm_unittests.cc",
"platform_isolate_manager_unittests.cc",
"runtime_controller_unittests.cc",
"type_conversions_unittests.cc",
]
@ -153,6 +154,7 @@ if (enable_unittests) {
"//flutter/common",
"//flutter/fml",
"//flutter/lib/snapshot",
"//flutter/shell/common:shell_test_fixture_sources",
"//flutter/skia",
"//flutter/testing",
"//flutter/testing:dart",

1
engine/src/flutter/runtime/dart_isolate_unittests.cc
View File

@ -712,6 +712,7 @@ class FakePlatformConfigurationClient : public PlatformConfigurationClient {
double width,
double height) override {}
void UpdateSemantics(int64_t view_id, SemanticsUpdate* update) override {}
void SetSemanticsTreeEnabled(bool enabled) override {}
void HandlePlatformMessage(
std::unique_ptr<PlatformMessage> message) override {}
FontCollection& GetFontCollection() override {

101
engine/src/flutter/runtime/fixtures/runtime_test.dart
View File

@ -6,6 +6,8 @@
import 'dart:async';
import 'dart:isolate';
import 'dart:typed_data';
import 'dart:ui';
import 'split_lib_test.dart' deferred as splitlib;
@ -219,3 +221,102 @@ Function createEntryPointForPlatIsoSendAndRecvTest() {
void mainForPlatformIsolatesThrowError() {
throw AssertionError('Error from platform isolate');
}
@pragma('vm:entry-point')
void sendSemanticsUpdate() {
final SemanticsUpdateBuilder builder = SemanticsUpdateBuilder();
const String identifier = 'identifier';
const String label = 'label';
final List<StringAttribute> labelAttributes = <StringAttribute>[
SpellOutStringAttribute(range: const TextRange(start: 1, end: 2)),
];
const String value = 'value';
final List<StringAttribute> valueAttributes = <StringAttribute>[
SpellOutStringAttribute(range: const TextRange(start: 2, end: 3)),
];
const String increasedValue = 'increasedValue';
final List<StringAttribute> increasedValueAttributes = <StringAttribute>[
SpellOutStringAttribute(range: const TextRange(start: 4, end: 5)),
];
const String decreasedValue = 'decreasedValue';
final List<StringAttribute> decreasedValueAttributes = <StringAttribute>[
SpellOutStringAttribute(range: const TextRange(start: 5, end: 6)),
];
const String hint = 'hint';
final List<StringAttribute> hintAttributes = <StringAttribute>[
LocaleStringAttribute(
locale: const Locale('en', 'MX'),
range: const TextRange(start: 0, end: 1),
),
];
const String tooltip = 'tooltip';
final Float64List transform = Float64List(16);
final Int32List childrenInTraversalOrder = Int32List(0);
final Int32List childrenInHitTestOrder = Int32List(0);
final Int32List additionalActions = Int32List(0);
transform[0] = 1;
transform[1] = 0;
transform[2] = 0;
transform[3] = 0;
transform[4] = 0;
transform[5] = 1;
transform[6] = 0;
transform[7] = 0;
transform[8] = 0;
transform[9] = 0;
transform[10] = 1;
transform[11] = 0;
transform[12] = 0;
transform[13] = 0;
transform[14] = 0;
transform[15] = 0;
builder.updateNode(
id: 0,
flags: 0,
actions: 0,
maxValueLength: 0,
currentValueLength: 0,
textSelectionBase: -1,
textSelectionExtent: -1,
platformViewId: -1,
scrollChildren: 0,
scrollIndex: 0,
scrollPosition: 0,
scrollExtentMax: 0,
scrollExtentMin: 0,
rect: const Rect.fromLTRB(0, 0, 10, 10),
elevation: 0,
thickness: 0,
identifier: identifier,
label: label,
labelAttributes: labelAttributes,
value: value,
valueAttributes: valueAttributes,
increasedValue: increasedValue,
increasedValueAttributes: increasedValueAttributes,
decreasedValue: decreasedValue,
decreasedValueAttributes: decreasedValueAttributes,
hint: hint,
hintAttributes: hintAttributes,
tooltip: tooltip,
textDirection: TextDirection.ltr,
transform: transform,
childrenInTraversalOrder: childrenInTraversalOrder,
childrenInHitTestOrder: childrenInHitTestOrder,
additionalActions: additionalActions,
controlsNodes: null,
);
_semanticsUpdate(builder.build());
}
@pragma('vm:external-name', 'SemanticsUpdate')
external void _semanticsUpdate(SemanticsUpdate update);

10
engine/src/flutter/runtime/runtime_controller.cc
View File

@ -450,10 +450,12 @@ void RuntimeController::CheckIfAllViewsRendered() {
// |PlatformConfigurationClient|
void RuntimeController::UpdateSemantics(int64_t view_id,
SemanticsUpdate* update) {
if (platform_data_.semantics_enabled) {
client_.UpdateSemantics(view_id, update->takeNodes(),
update->takeActions());
}
client_.UpdateSemantics(view_id, update->takeNodes(), update->takeActions());
}
// |PlatformConfigurationClient|
void RuntimeController::SetSemanticsTreeEnabled(bool enabled) {
client_.SetSemanticsTreeEnabled(enabled);
}
// |PlatformConfigurationClient|

9
engine/src/flutter/runtime/runtime_controller.h
View File

@ -643,6 +643,12 @@ class RuntimeController : public PlatformConfigurationClient,
// |PlatformConfigurationClient|
std::shared_ptr<const fml::Mapping> GetPersistentIsolateData() override;
// |PlatformConfigurationClient|
void UpdateSemantics(int64_t view_id, SemanticsUpdate* update) override;
// |PlatformConfigurationClient|
void SetSemanticsTreeEnabled(bool enabled) override;
const fml::WeakPtr<IOManager>& GetIOManager() const {
return context_.io_manager;
}
@ -768,9 +774,6 @@ class RuntimeController : public PlatformConfigurationClient,
double width,
double height) override;
// |PlatformConfigurationClient|
void UpdateSemantics(int64_t view_id, SemanticsUpdate* update) override;
// |PlatformConfigurationClient|
void HandlePlatformMessage(std::unique_ptr<PlatformMessage> message) override;

149
engine/src/flutter/runtime/runtime_controller_unittests.cc Normal file
View File

@ -0,0 +1,149 @@
// Copyright 2013 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
#include "flutter/runtime/runtime_controller.h"
#include "flutter/runtime/runtime_delegate.h"
#include "flutter/lib/ui/semantics/semantics_update.h"
#include "flutter/shell/common/shell_test.h"
#include "flutter/testing/testing.h"
namespace flutter::testing {
// For namespacing when running tests.
using RuntimeControllerTest = ShellTest;
class MockRuntimeDelegate : public RuntimeDelegate {
public:
FontCollection font;
std::vector<SemanticsNodeUpdates> updates;
std::vector<CustomAccessibilityActionUpdates> actions;
std::string DefaultRouteName() override { return ""; }
void ScheduleFrame(bool regenerate_layer_trees = true) override {}
void OnAllViewsRendered() override {}
void Render(int64_t view_id,
std::unique_ptr<flutter::LayerTree> layer_tree,
float device_pixel_ratio) override {}
void UpdateSemantics(int64_t view_id,
SemanticsNodeUpdates update,
CustomAccessibilityActionUpdates actions) override {
this->updates.push_back(update);
this->actions.push_back(actions);
}
void SetSemanticsTreeEnabled(bool enabled) override {}
void HandlePlatformMessage(
std::unique_ptr<PlatformMessage> message) override {}
FontCollection& GetFontCollection() override { return font; }
std::shared_ptr<AssetManager> GetAssetManager() override { return nullptr; }
void OnRootIsolateCreated() override {};
void UpdateIsolateDescription(const std::string isolate_name,
int64_t isolate_port) override {};
void SetNeedsReportTimings(bool value) override {};
std::unique_ptr<std::vector<std::string>> ComputePlatformResolvedLocale(
const std::vector<std::string>& supported_locale_data) override {
return nullptr;
}
void RequestDartDeferredLibrary(intptr_t loading_unit_id) override {}
void RequestViewFocusChange(const ViewFocusChangeRequest& request) override {}
std::weak_ptr<PlatformMessageHandler> GetPlatformMessageHandler()
const override {
return {};
}
void SendChannelUpdate(std::string name, bool listening) override {}
double GetScaledFontSize(double unscaled_font_size,
int configuration_id) const override {
return 0.0;
}
};
class RuntimeControllerTester {
public:
explicit RuntimeControllerTester(UIDartState::Context& context)
: context_(context),
runtime_controller_(delegate_,
nullptr,
{},
{},
{},
{},
{},
nullptr,
context_) {}
void CanUpdateSemanticsWhenSetSemanticsTreeEnabled(SemanticsUpdate* update) {
ASSERT_TRUE(delegate_.updates.empty());
ASSERT_TRUE(delegate_.actions.empty());
runtime_controller_.SetSemanticsTreeEnabled(true);
runtime_controller_.UpdateSemantics(0, update);
ASSERT_FALSE(delegate_.updates.empty());
ASSERT_FALSE(delegate_.actions.empty());
}
private:
MockRuntimeDelegate delegate_;
UIDartState::Context& context_;
RuntimeController runtime_controller_;
};
TEST_F(RuntimeControllerTest, CanUpdateSemanticsWhenSetSemanticsTreeEnabled) {
fml::AutoResetWaitableEvent message_latch;
// The code in this test is mostly setup code to get a SemanticsUpdate object.
// The real test is in RuntimeControllerTester::CanUpdateSemantics.
TaskRunners task_runners("test", // label
GetCurrentTaskRunner(), // platform
CreateNewThread(), // raster
CreateNewThread(), // ui
CreateNewThread() // io
);
UIDartState::Context context(task_runners);
auto tester = std::make_shared<RuntimeControllerTester>(context);
auto native_semantics_update = [tester,
&message_latch](Dart_NativeArguments args) {
auto handle = Dart_GetNativeArgument(args, 0);
intptr_t peer = 0;
Dart_Handle result = Dart_GetNativeInstanceField(
handle, tonic::DartWrappable::kPeerIndex, &peer);
ASSERT_FALSE(Dart_IsError(result));
SemanticsUpdate* update = reinterpret_cast<SemanticsUpdate*>(peer);
tester->CanUpdateSemanticsWhenSetSemanticsTreeEnabled(update);
message_latch.Signal();
};
Settings settings = CreateSettingsForFixture();
AddNativeCallback("SemanticsUpdate",
CREATE_NATIVE_ENTRY(native_semantics_update));
std::unique_ptr<Shell> shell = CreateShell(settings, task_runners);
ASSERT_TRUE(shell->IsSetup());
auto configuration = RunConfiguration::InferFromSettings(settings);
configuration.SetEntrypoint("sendSemanticsUpdate");
shell->RunEngine(std::move(configuration), [](auto result) {
ASSERT_EQ(result, Engine::RunStatus::Success);
});
message_latch.Wait();
DestroyShell(std::move(shell), task_runners);
}
} // namespace flutter::testing

2
engine/src/flutter/runtime/runtime_delegate.h
View File

@ -36,6 +36,8 @@ class RuntimeDelegate {
SemanticsNodeUpdates update,
CustomAccessibilityActionUpdates actions) = 0;
virtual void SetSemanticsTreeEnabled(bool enabled) = 0;
virtual void HandlePlatformMessage(
std::unique_ptr<PlatformMessage> message) = 0;

4
engine/src/flutter/shell/common/engine.cc
View File

@ -503,6 +503,10 @@ void Engine::UpdateSemantics(int64_t view_id,
std::move(actions));
}
void Engine::SetSemanticsTreeEnabled(bool enabled) {
delegate_.OnEngineSetSemanticsTreeEnabled(enabled);
}
void Engine::HandlePlatformMessage(std::unique_ptr<PlatformMessage> message) {
if (message->channel() == kAssetChannel) {
HandleAssetPlatformMessage(std::move(message));

17
engine/src/flutter/shell/common/engine.h
View File

@ -161,6 +161,20 @@ class Engine final : public RuntimeDelegate, PointerDataDispatcher::Delegate {
SemanticsNodeUpdates updates,
CustomAccessibilityActionUpdates actions) = 0;
//--------------------------------------------------------------------------
/// @brief When the Framework starts or stops generating semantics
/// tree,
/// this new information needs to be conveyed to the underlying
/// platform so that they can prepare to accept semantics
/// update. The engine delegates this task to the shell via this
/// call.
///
/// @see `OnEngineUpdateSemantics`
///
/// @param[in] enabled whether Framework starts generating semantics tree.
///
virtual void OnEngineSetSemanticsTreeEnabled(bool enabled) = 0;
//--------------------------------------------------------------------------
/// @brief When the Flutter application has a message to send to the
/// underlying platform, the message needs to be forwarded to
@ -1016,6 +1030,9 @@ class Engine final : public RuntimeDelegate, PointerDataDispatcher::Delegate {
SemanticsNodeUpdates update,
CustomAccessibilityActionUpdates actions) override;
// |RuntimeDelegate|
void SetSemanticsTreeEnabled(bool enabled) override;
// |RuntimeDelegate|
void HandlePlatformMessage(std::unique_ptr<PlatformMessage> message) override;

1
engine/src/flutter/shell/common/engine_animator_unittests.cc
View File

@ -56,6 +56,7 @@ class MockDelegate : public Engine::Delegate {
OnEngineUpdateSemantics,
(int64_t, SemanticsNodeUpdates, CustomAccessibilityActionUpdates),
(override));
MOCK_METHOD(void, OnEngineSetSemanticsTreeEnabled, (bool), (override));
MOCK_METHOD(void,
OnEngineHandlePlatformMessage,
(std::unique_ptr<PlatformMessage>),

2
engine/src/flutter/shell/common/engine_unittests.cc
View File

@ -64,6 +64,7 @@ class MockDelegate : public Engine::Delegate {
OnEngineUpdateSemantics,
(int64_t, SemanticsNodeUpdates, CustomAccessibilityActionUpdates),
(override));
MOCK_METHOD(void, OnEngineSetSemanticsTreeEnabled, (bool), (override));
MOCK_METHOD(void,
OnEngineHandlePlatformMessage,
(std::unique_ptr<PlatformMessage>),
@ -115,6 +116,7 @@ class MockRuntimeDelegate : public RuntimeDelegate {
UpdateSemantics,
(int64_t, SemanticsNodeUpdates, CustomAccessibilityActionUpdates),
(override));
MOCK_METHOD(void, SetSemanticsTreeEnabled, (bool), (override));
MOCK_METHOD(void,
HandlePlatformMessage,
(std::unique_ptr<PlatformMessage>),

4
engine/src/flutter/shell/common/platform_view.cc
View File

@ -130,6 +130,10 @@ void PlatformView::UpdateSemantics(
// NOLINTNEXTLINE(performance-unnecessary-value-param)
CustomAccessibilityActionUpdates actions) {}
void PlatformView::SetSemanticsTreeEnabled(
bool enabled // NOLINT(performance-unnecessary-value-param)
) {}
void PlatformView::SendChannelUpdate(const std::string& name, bool listening) {}
void PlatformView::HandlePlatformMessage(

9
engine/src/flutter/shell/common/platform_view.h
View File

@ -514,6 +514,15 @@ class PlatformView {
SemanticsNodeUpdates updates,
CustomAccessibilityActionUpdates actions);
//----------------------------------------------------------------------------
/// @brief Used by the framework to tell the embedder to prepare or clear
/// resoruce for accepting semantics tree.
///
/// @param[in] enabled whether framework starts or stops sending semantics
/// updates
///
virtual void SetSemanticsTreeEnabled(bool enabled);
//----------------------------------------------------------------------------
/// @brief Used by the framework to tell the embedder that it has
/// registered a listener on a given channel.

14
engine/src/flutter/shell/common/shell.cc
View File

@ -1334,6 +1334,20 @@ void Shell::OnEngineUpdateSemantics(int64_t view_id,
});
}
// |Engine::Delegate|
void Shell::OnEngineSetSemanticsTreeEnabled(bool enabled) {
FML_DCHECK(is_set_up_);
FML_DCHECK(task_runners_.GetUITaskRunner()->RunsTasksOnCurrentThread());
task_runners_.GetPlatformTaskRunner()->RunNowOrPostTask(
task_runners_.GetPlatformTaskRunner(),
[view = platform_view_->GetWeakPtr(), enabled] {
if (view) {
view->SetSemanticsTreeEnabled(enabled);
}
});
}
// |Engine::Delegate|
void Shell::OnEngineHandlePlatformMessage(
std::unique_ptr<PlatformMessage> message) {

3
engine/src/flutter/shell/common/shell.h
View File

@ -671,6 +671,9 @@ class Shell final : public PlatformView::Delegate,
SemanticsNodeUpdates update,
CustomAccessibilityActionUpdates actions) override;
// |Engine::Delegate|
void OnEngineSetSemanticsTreeEnabled(bool enabled) override;
// |Engine::Delegate|
void OnEngineHandlePlatformMessage(
std::unique_ptr<PlatformMessage> message) override;

1
engine/src/flutter/shell/platform/darwin/ios/BUILD.gn
View File

@ -242,6 +242,7 @@ shared_library("ios_test_flutter") {
"ios_context_noop_unittests.mm",
"ios_surface_noop_unittests.mm",
"platform_message_handler_ios_test.mm",
"platform_view_ios_test.mm",
]
deps = [
":flutter_framework",

28
engine/src/flutter/shell/platform/darwin/ios/platform_view_ios.h
View File

@ -87,6 +87,9 @@ class PlatformViewIOS final : public PlatformView {
// |PlatformView|
void SetSemanticsEnabled(bool enabled) override;
// |PlatformView|
void SetSemanticsTreeEnabled(bool enabled) override;
// |PlatformView|
void HandlePlatformMessage(std::unique_ptr<flutter::PlatformMessage> message) override;
@ -128,6 +131,11 @@ class PlatformViewIOS final : public PlatformView {
return platform_message_handler_;
}
/**
* Gets the accessibility bridge created in this platform view.
*/
AccessibilityBridge* GetAccessibilityBridge() { return accessibility_bridge_.get(); }
private:
/// Smart pointer for use with objective-c observers.
/// This guarantees we remove the observer.
@ -143,24 +151,6 @@ class PlatformViewIOS final : public PlatformView {
id<NSObject> observer_ = nil;
};
/// Wrapper that guarantees we communicate clearing Accessibility
/// information to Dart.
class AccessibilityBridgeManager {
public:
explicit AccessibilityBridgeManager(const std::function<void(bool)>& set_semantics_enabled);
AccessibilityBridgeManager(const std::function<void(bool)>& set_semantics_enabled,
AccessibilityBridge* bridge);
explicit operator bool() const noexcept { return static_cast<bool>(accessibility_bridge_); }
AccessibilityBridge* get() const noexcept { return accessibility_bridge_.get(); }
void Set(std::unique_ptr<AccessibilityBridge> bridge);
void Clear();
private:
FML_DISALLOW_COPY_AND_ASSIGN(AccessibilityBridgeManager);
std::unique_ptr<AccessibilityBridge> accessibility_bridge_;
std::function<void(bool)> set_semantics_enabled_;
};
__weak FlutterViewController* owner_controller_;
// Since the `ios_surface_` is created on the platform thread but
// used on the raster thread we need to protect it with a mutex.
@ -168,7 +158,7 @@ class PlatformViewIOS final : public PlatformView {
std::unique_ptr<IOSSurface> ios_surface_;
std::shared_ptr<IOSContext> ios_context_;
__weak FlutterPlatformViewsController* platform_views_controller_;
AccessibilityBridgeManager accessibility_bridge_;
std::unique_ptr<AccessibilityBridge> accessibility_bridge_;
ScopedObserver dealloc_view_controller_observer_;
std::vector<std::string> platform_resolved_locale_;
std::shared_ptr<PlatformMessageHandlerIos> platform_message_handler_;

63
engine/src/flutter/shell/platform/darwin/ios/platform_view_ios.mm
View File

@ -18,29 +18,6 @@ FLUTTER_ASSERT_ARC
namespace flutter {
PlatformViewIOS::AccessibilityBridgeManager::AccessibilityBridgeManager(
const std::function<void(bool)>& set_semantics_enabled)
: AccessibilityBridgeManager(set_semantics_enabled, nullptr) {}
PlatformViewIOS::AccessibilityBridgeManager::AccessibilityBridgeManager(
const std::function<void(bool)>& set_semantics_enabled,
AccessibilityBridge* bridge)
: accessibility_bridge_(bridge), set_semantics_enabled_(set_semantics_enabled) {
if (bridge) {
set_semantics_enabled_(true);
}
}
void PlatformViewIOS::AccessibilityBridgeManager::Set(std::unique_ptr<AccessibilityBridge> bridge) {
accessibility_bridge_ = std::move(bridge);
set_semantics_enabled_(true);
}
void PlatformViewIOS::AccessibilityBridgeManager::Clear() {
set_semantics_enabled_(false);
accessibility_bridge_.reset();
}
PlatformViewIOS::PlatformViewIOS(PlatformView::Delegate& delegate,
const std::shared_ptr<IOSContext>& context,
__weak FlutterPlatformViewsController* platform_views_controller,
@ -48,7 +25,6 @@ PlatformViewIOS::PlatformViewIOS(PlatformView::Delegate& delegate,
: PlatformView(delegate, task_runners),
ios_context_(context),
platform_views_controller_(platform_views_controller),
accessibility_bridge_([this](bool enabled) { PlatformView::SetSemanticsEnabled(enabled); }),
platform_message_handler_(
new PlatformMessageHandlerIos(task_runners.GetPlatformTaskRunner())) {}
@ -86,7 +62,7 @@ void PlatformViewIOS::SetOwnerViewController(__weak FlutterViewController* owner
if (ios_surface_ || !owner_controller) {
NotifyDestroyed();
ios_surface_.reset();
accessibility_bridge_.Clear();
accessibility_bridge_.reset();
}
owner_controller_ = owner_controller;
@ -98,7 +74,7 @@ void PlatformViewIOS::SetOwnerViewController(__weak FlutterViewController* owner
queue:[NSOperationQueue mainQueue]
usingBlock:^(NSNotification* note) {
// Implicit copy of 'this' is fine.
accessibility_bridge_.Clear();
accessibility_bridge_.reset();
owner_controller_ = nil;
}]);
@ -121,8 +97,8 @@ void PlatformViewIOS::attachView() {
FML_DCHECK(ios_surface_ != nullptr);
if (accessibility_bridge_) {
accessibility_bridge_.Set(std::make_unique<AccessibilityBridge>(
owner_controller_, this, owner_controller_.platformViewsController));
accessibility_bridge_ = std::make_unique<AccessibilityBridge>(
owner_controller_, this, owner_controller_.platformViewsController);
}
}
@ -161,22 +137,10 @@ std::shared_ptr<impeller::Context> PlatformViewIOS::GetImpellerContext() const {
// |PlatformView|
void PlatformViewIOS::SetSemanticsEnabled(bool enabled) {
if (!owner_controller_) {
FML_LOG(WARNING) << "Could not set semantics to enabled, this "
"PlatformViewIOS has no ViewController.";
return;
}
if (enabled && !accessibility_bridge_) {
accessibility_bridge_.Set(std::make_unique<AccessibilityBridge>(
owner_controller_, this, owner_controller_.platformViewsController));
} else if (!enabled && accessibility_bridge_) {
accessibility_bridge_.Clear();
} else {
PlatformView::SetSemanticsEnabled(enabled);
}
PlatformView::SetSemanticsEnabled(enabled);
}
// |shell:PlatformView|
// |PlatformView|
void PlatformViewIOS::SetAccessibilityFeatures(int32_t flags) {
PlatformView::SetAccessibilityFeatures(flags);
}
@ -186,6 +150,7 @@ void PlatformViewIOS::UpdateSemantics(int64_t view_id,
flutter::SemanticsNodeUpdates update,
flutter::CustomAccessibilityActionUpdates actions) {
FML_DCHECK(owner_controller_);
FML_DCHECK(accessibility_bridge_);
if (accessibility_bridge_) {
accessibility_bridge_.get()->UpdateSemantics(std::move(update), actions);
[[NSNotificationCenter defaultCenter] postNotificationName:FlutterSemanticsUpdateNotification
@ -193,6 +158,20 @@ void PlatformViewIOS::UpdateSemantics(int64_t view_id,
}
}
// |PlatformView|
void PlatformViewIOS::SetSemanticsTreeEnabled(bool enabled) {
FML_DCHECK(owner_controller_);
if (enabled) {
if (accessibility_bridge_) {
return;
}
accessibility_bridge_ =
std::make_unique<AccessibilityBridge>(owner_controller_, this, platform_views_controller_);
} else {
accessibility_bridge_.reset();
}
}
// |PlatformView|
std::unique_ptr<VsyncWaiter> PlatformViewIOS::CreateVSyncWaiter() {
return std::make_unique<VsyncWaiterIOS>(task_runners_);

105
engine/src/flutter/shell/platform/darwin/ios/platform_view_ios_test.mm Normal file
View File

@ -0,0 +1,105 @@
// Copyright 2013 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
#import <OCMock/OCMock.h>
#import <XCTest/XCTest.h>
#import "flutter/fml/thread.h"
#import "flutter/shell/platform/darwin/common/framework/Headers/FlutterMacros.h"
#import "flutter/shell/platform/darwin/ios/platform_view_ios.h"
FLUTTER_ASSERT_ARC
namespace flutter {
namespace {
class MockDelegate : public PlatformView::Delegate {
public:
void OnPlatformViewCreated(std::unique_ptr<Surface> surface) override {}
void OnPlatformViewDestroyed() override {}
void OnPlatformViewScheduleFrame() override {}
void OnPlatformViewAddView(int64_t view_id,
const ViewportMetrics& viewport_metrics,
AddViewCallback callback) override {}
void OnPlatformViewRemoveView(int64_t view_id, RemoveViewCallback callback) override {}
void OnPlatformViewSetNextFrameCallback(const fml::closure& closure) override {}
void OnPlatformViewSetViewportMetrics(int64_t view_id, const ViewportMetrics& metrics) override {}
const flutter::Settings& OnPlatformViewGetSettings() const override { return settings_; }
void OnPlatformViewDispatchPlatformMessage(std::unique_ptr<PlatformMessage> message) override {}
void OnPlatformViewDispatchPointerDataPacket(std::unique_ptr<PointerDataPacket> packet) override {
}
void OnPlatformViewSendViewFocusEvent(const ViewFocusEvent& event) override {}
void OnPlatformViewDispatchSemanticsAction(int64_t view_id,
int32_t node_id,
SemanticsAction action,
fml::MallocMapping args) override {}
void OnPlatformViewSetSemanticsEnabled(bool enabled) override {}
void OnPlatformViewSetAccessibilityFeatures(int32_t flags) override {}
void OnPlatformViewRegisterTexture(std::shared_ptr<Texture> texture) override {}
void OnPlatformViewUnregisterTexture(int64_t texture_id) override {}
void OnPlatformViewMarkTextureFrameAvailable(int64_t texture_id) override {}
void LoadDartDeferredLibrary(intptr_t loading_unit_id,
std::unique_ptr<const fml::Mapping> snapshot_data,
std::unique_ptr<const fml::Mapping> snapshot_instructions) override {
}
void LoadDartDeferredLibraryError(intptr_t loading_unit_id,
const std::string error_message,
bool transient) override {}
void UpdateAssetResolverByType(std::unique_ptr<flutter::AssetResolver> updated_asset_resolver,
flutter::AssetResolver::AssetResolverType type) override {}
flutter::Settings settings_;
};
} // namespace
} // namespace flutter
@interface PlatformViewIOSTest : XCTestCase
@end
@implementation PlatformViewIOSTest
- (void)testSetSemanticsTreeEnabled {
flutter::MockDelegate mock_delegate;
auto thread = std::make_unique<fml::Thread>("PlatformViewIOSTest");
auto thread_task_runner = thread->GetTaskRunner();
flutter::TaskRunners runners(/*label=*/self.name.UTF8String,
/*platform=*/thread_task_runner,
/*raster=*/thread_task_runner,
/*ui=*/thread_task_runner,
/*io=*/thread_task_runner);
id messenger = OCMProtocolMock(@protocol(FlutterBinaryMessenger));
id engine = OCMClassMock([FlutterEngine class]);
id flutterViewController = OCMClassMock([FlutterViewController class]);
OCMStub([flutterViewController isViewLoaded]).andReturn(NO);
OCMStub([flutterViewController engine]).andReturn(engine);
OCMStub([engine binaryMessenger]).andReturn(messenger);
auto platform_view = std::make_unique<flutter::PlatformViewIOS>(
/*delegate=*/mock_delegate,
/*rendering_api=*/flutter::IOSRenderingAPI::kMetal,
/*platform_views_controller=*/nil,
/*task_runners=*/runners,
/*worker_task_runner=*/nil,
/*is_gpu_disabled_sync_switch=*/std::make_shared<fml::SyncSwitch>());
fml::AutoResetWaitableEvent latch;
thread_task_runner->PostTask([&] {
platform_view->SetOwnerViewController(flutterViewController);
XCTAssertFalse(platform_view->GetAccessibilityBridge());
platform_view->SetSemanticsTreeEnabled(true);
XCTAssertTrue(platform_view->GetAccessibilityBridge());
platform_view->SetSemanticsTreeEnabled(false);
XCTAssertFalse(platform_view->GetAccessibilityBridge());
latch.Signal();
});
latch.Wait();
[engine stopMocking];
}
@end

3
engine/src/flutter/shell/platform/embedder/fixtures/main.dart
View File

@ -310,6 +310,7 @@ Future<void> a11y_main() async {
)
..updateCustomAction(id: 21, label: 'Archive', hint: 'archive message');
PlatformDispatcher.instance.setSemanticsTreeEnabled(true);
PlatformDispatcher.instance.views.first.updateSemantics(builder.build());
signalNativeTest();
@ -397,6 +398,7 @@ Future<void> a11y_string_attributes() async {
controlsNodes: null,
);
PlatformDispatcher.instance.setSemanticsTreeEnabled(true);
PlatformDispatcher.instance.views.first.updateSemantics(builder.build());
signalNativeTest();
}
@ -1689,6 +1691,7 @@ Future<void> a11y_main_multi_view() async {
);
}
PlatformDispatcher.instance.setSemanticsTreeEnabled(true);
for (final view in PlatformDispatcher.instance.views) {
view.updateSemantics(createForView(view).build());
}

1
engine/src/flutter/shell/platform/windows/fixtures/main.dart
View File

@ -471,6 +471,7 @@ Future<void> sendSemanticsTreeInfo() async {
return builder.build();
}
ui.PlatformDispatcher.instance.setSemanticsTreeEnabled(true);
view1.updateSemantics(createSemanticsUpdate(view1.viewId + 1));
view2.updateSemantics(createSemanticsUpdate(view2.viewId + 1));
signal();

2
engine/src/flutter/testing/ios_scenario_app/lib/src/locale_initialization.dart
View File

@ -79,7 +79,7 @@ class LocaleInitialization extends Scenario {
);
final SemanticsUpdate semanticsUpdate = semanticsUpdateBuilder.build();
view.platformDispatcher.setSemanticsTreeEnabled(true);
view.updateSemantics(semanticsUpdate);
}

5
packages/flutter/lib/src/semantics/binding.dart
View File

@ -28,6 +28,7 @@ mixin SemanticsBinding on BindingBase {
..onSemanticsActionEvent = _handleSemanticsActionEvent
..onAccessibilityFeaturesChanged = handleAccessibilityFeaturesChanged;
_handleSemanticsEnabledChanged();
addSemanticsEnabledListener(_handleFrameworkSemanticsEnabledChanged);
}
/// The current [SemanticsBinding], if one has been created.
@ -154,6 +155,10 @@ mixin SemanticsBinding on BindingBase {
performSemanticsAction(decodedAction);
}
void _handleFrameworkSemanticsEnabledChanged() {
platformDispatcher.setSemanticsTreeEnabled(semanticsEnabled);
}
/// Called whenever the platform requests an action to be performed on a
/// [SemanticsNode].
///

35
packages/flutter/test/semantics/semantics_binding_set_semantics_tree_enabled_test.dart Normal file
View File

@ -0,0 +1,35 @@
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
import 'dart:ui';
import 'package:flutter_test/flutter_test.dart';
void main() {
test('SemanticsHandle ensureSemantics calls setSemanticsTreeEnabled', () async {
final SemanticsTestBinding binding = SemanticsTestBinding();
expect(binding.platformDispatcher.semanticsTreeEnabled, isFalse);
final SemanticsHandle handle = binding.ensureSemantics();
expect(binding.platformDispatcher.semanticsTreeEnabled, isTrue);
handle.dispose();
expect(binding.platformDispatcher.semanticsTreeEnabled, isFalse);
});
}
class SemanticsTestBinding extends AutomatedTestWidgetsFlutterBinding {
@override
TestPlatformDispatcherSpy get platformDispatcher => _platformDispatcherSpy;
static final TestPlatformDispatcherSpy _platformDispatcherSpy = TestPlatformDispatcherSpy(
platformDispatcher: PlatformDispatcher.instance,
);
}
class TestPlatformDispatcherSpy extends TestPlatformDispatcher {
TestPlatformDispatcherSpy({required super.platformDispatcher});
bool semanticsTreeEnabled = false;
@override
void setSemanticsTreeEnabled(bool enabled) {
semanticsTreeEnabled = enabled;
}
}