Michael Goderbauer 6f09064e78
Stand-alone widget tree with multiple render trees to enable multi-view rendering (#125003)
This change enables Flutter to generate multiple Scenes to be rendered into separate FlutterViews from a single widget tree. Each Scene is described by a separate render tree, which are all associated with the single widget tree.

This PR implements the framework-side mechanisms to describe the content to be rendered into multiple views. Separate engine-side changes are necessary to provide these views to the framework and to draw the framework-generated Scene into them.

## Summary of changes

The details of this change are described in [flutter.dev/go/multiple-views](https://flutter.dev/go/multiple-views). Below is a high-level summary organized by layers.

### Rendering layer changes

* The `RendererBinding` no longer owns a single `renderView`. In fact, it doesn't OWN any `RenderView`s at all anymore. Instead, it offers an API (`addRenderView`/`removeRenderView`) to add and remove `RenderView`s that then will be MANAGED by the binding. The `RenderView` itself is now owned by a higher-level abstraction (e.g. the `RawView` Element of the widgets layer, see below), who is also in charge of adding it to the binding. When added, the binding will interact with the `RenderView` to produce a frame (e.g. by calling `compositeFrame` on it) and to perform hit tests for incoming pointer events. Multiple `RenderView`s can be added to the binding (typically one per `FlutterView`) to produce multiple Scenes.
* Instead of owning a single `pipelineOwner`, the `RendererBinding` now owns the root of the `PipelineOwner` tree (exposed as `rootPipelineOwner` on the binding). Each `PipelineOwner` in that tree (except for the root) typically manages its own render tree typically rooted in one of the `RenderView`s mentioned in the previous bullet. During frame production, the binding will instruct each `PipelineOwner` of that tree to flush layout, paint, semantics etc. A higher-level abstraction (e.g. the widgets layer, see below) is in charge of adding `PipelineOwner`s to this tree.
* Backwards compatibility: The old `renderView` and `pipelineOwner` properties of the `RendererBinding` are retained, but marked as deprecated. Care has been taken to keep their original behavior for the deprecation period, i.e. if you just call `runApp`, the render tree bootstrapped by this call is rooted in the deprecated `RendererBinding.renderView` and managed by the deprecated `RendererBinding.pipelineOwner`.

### Widgets layer changes

* The `WidgetsBinding` no longer attaches the widget tree to an existing render tree. Instead, it bootstraps a stand-alone widget tree that is not backed by a render tree. For this, `RenderObjectToWidgetAdapter` has been replaced by `RootWidget`.
* Multiple render trees can be bootstrapped and attached to the widget tree with the help of the `View` widget, which internally is backed by a `RawView` widget. Configured with a `FlutterView` to render into, the `RawView` creates a new `PipelineOwner` and a new `RenderView` for the new render tree. It adds the new `RenderView` to the `RendererBinding` and its `PipelineOwner` to the pipeline owner tree.
* The `View` widget can only appear in certain well-defined locations in the widget tree since it bootstraps a new render tree and does not insert a `RenderObject` into an ancestor. However, almost all Elements expect that their children insert `RenderObject`s, otherwise they will not function properly. To produce a good error message when the `View` widget is used in an illegal location, the `debugMustInsertRenderObjectIntoSlot` method has been added to Element, where a child can ask whether a given slot must insert a RenderObject into its ancestor or not. In practice, the `View` widget can be used as a child of the `RootWidget`, inside the `view` slot of the `ViewAnchor` (see below) and inside a `ViewCollection` (see below). In those locations, the `View` widget may be wrapped in other non-RenderObjectWidgets (e.g. InheritedWidgets).
* The new `ViewAnchor` can be used to create a side-view inside a parent `View`. The `child` of the `ViewAnchor` widget renders into the parent `View` as usual, but the `view` slot can take on another `View` widget, which has access to all inherited widgets above the `ViewAnchor`. Metaphorically speaking, the view is anchored to the location of the `ViewAnchor` in the widget tree.
* The new `ViewCollection` widget allows for multiple sibling views as it takes a list of `View`s as children. It can be used in all the places that accept a `View` widget.

## Google3

As of July 5, 2023 this change passed a TAP global presubmit (TGP) in google3: tap/OCL:544707016:BASE:545809771:1688597935864:e43dd651

## Note to reviewers

This change is big (sorry). I suggest focusing the initial review on the changes inside of `packages/flutter` first. The majority of the changes describe above are implemented in (listed in suggested review order):

* `rendering/binding.dart`
* `widgets/binding.dart`
* `widgets/view.dart`
* `widgets/framework.dart`

All other changes included in the PR are basically the fallout of what's implemented in those files. Also note that a lot of the lines added in this PR are documentation and tests.

I am also very happy to walk reviewers through the code in person or via video call, if that is helpful.

I appreciate any feedback.

## Feedback to address before submitting ("TODO")
2023-07-17 16:14:08 +00:00

447 lines
18 KiB
Dart

// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
import 'dart:io' show Platform;
import 'dart:ui' as ui show FlutterView, Scene, SceneBuilder, SemanticsUpdate;
import 'package:flutter/foundation.dart';
import 'package:flutter/services.dart';
import 'binding.dart';
import 'box.dart';
import 'debug.dart';
import 'layer.dart';
import 'object.dart';
/// The layout constraints for the root render object.
@immutable
class ViewConfiguration {
/// Creates a view configuration.
///
/// By default, the view has zero [size] and a [devicePixelRatio] of 1.0.
const ViewConfiguration({
this.size = Size.zero,
this.devicePixelRatio = 1.0,
});
/// The size of the output surface.
final Size size;
/// The pixel density of the output surface.
final double devicePixelRatio;
/// Creates a transformation matrix that applies the [devicePixelRatio].
///
/// The matrix translates points from the local coordinate system of the
/// app (in logical pixels) to the global coordinate system of the
/// [FlutterView] (in physical pixels).
Matrix4 toMatrix() {
return Matrix4.diagonal3Values(devicePixelRatio, devicePixelRatio, 1.0);
}
@override
bool operator ==(Object other) {
if (other.runtimeType != runtimeType) {
return false;
}
return other is ViewConfiguration
&& other.size == size
&& other.devicePixelRatio == devicePixelRatio;
}
@override
int get hashCode => Object.hash(size, devicePixelRatio);
@override
String toString() => '$size at ${debugFormatDouble(devicePixelRatio)}x';
}
/// The root of the render tree.
///
/// The view represents the total output surface of the render tree and handles
/// bootstrapping the rendering pipeline. The view has a unique child
/// [RenderBox], which is required to fill the entire output surface.
class RenderView extends RenderObject with RenderObjectWithChildMixin<RenderBox> {
/// Creates the root of the render tree.
///
/// Typically created by the binding (e.g., [RendererBinding]).
///
/// Providing a [configuration] is optional, but a configuration must be set
/// before calling [prepareInitialFrame]. This decouples creating the
/// [RenderView] object from configuring it. Typically, the object is created
/// by the [View] widget and configured by the [RendererBinding] when the
/// [RenderView] is registered with it by the [View] widget.
RenderView({
RenderBox? child,
ViewConfiguration? configuration,
required ui.FlutterView view,
}) : _configuration = configuration,
_view = view {
this.child = child;
}
/// The current layout size of the view.
Size get size => _size;
Size _size = Size.zero;
/// The constraints used for the root layout.
///
/// Typically, this configuration is set by the [RendererBinding], when the
/// [RenderView] is registered with it. It will also update the configuration
/// if necessary. Therefore, if used in conjunction with the [RendererBinding]
/// this property must not be set manually as the [RendererBinding] will just
/// override it.
///
/// For tests that want to change the size of the view, set
/// [TestFlutterView.physicalSize] on the appropriate [TestFlutterView]
/// (typically [WidgetTester.view]) instead of setting a configuration
/// directly on the [RenderView].
ViewConfiguration get configuration => _configuration!;
ViewConfiguration? _configuration;
set configuration(ViewConfiguration value) {
if (_configuration == value) {
return;
}
final ViewConfiguration? oldConfiguration = _configuration;
_configuration = value;
if (_rootTransform == null) {
// [prepareInitialFrame] has not been called yet, nothing to do for now.
return;
}
if (oldConfiguration?.toMatrix() != configuration.toMatrix()) {
replaceRootLayer(_updateMatricesAndCreateNewRootLayer());
}
assert(_rootTransform != null);
markNeedsLayout();
}
/// Whether a [configuration] has been set.
bool get hasConfiguration => _configuration != null;
/// The [FlutterView] into which this [RenderView] will render.
ui.FlutterView get flutterView => _view;
final ui.FlutterView _view;
/// Whether Flutter should automatically compute the desired system UI.
///
/// When this setting is enabled, Flutter will hit-test the layer tree at the
/// top and bottom of the screen on each frame looking for an
/// [AnnotatedRegionLayer] with an instance of a [SystemUiOverlayStyle]. The
/// hit-test result from the top of the screen provides the status bar settings
/// and the hit-test result from the bottom of the screen provides the system
/// nav bar settings.
///
/// If there is no [AnnotatedRegionLayer] on the bottom, the hit-test result
/// from the top provides the system nav bar settings. If there is no
/// [AnnotatedRegionLayer] on the top, the hit-test result from the bottom
/// provides the system status bar settings.
///
/// Setting this to false does not cause previous automatic adjustments to be
/// reset, nor does setting it to true cause the app to update immediately.
///
/// If you want to imperatively set the system ui style instead, it is
/// recommended that [automaticSystemUiAdjustment] is set to false.
///
/// See also:
///
/// * [AnnotatedRegion], for placing [SystemUiOverlayStyle] in the layer tree.
/// * [SystemChrome.setSystemUIOverlayStyle], for imperatively setting the system ui style.
bool automaticSystemUiAdjustment = true;
/// Bootstrap the rendering pipeline by preparing the first frame.
///
/// This should only be called once, and must be called before changing
/// [configuration]. It is typically called immediately after calling the
/// constructor.
///
/// This does not actually schedule the first frame. Call
/// [PipelineOwner.requestVisualUpdate] on [owner] to do that.
void prepareInitialFrame() {
assert(owner != null);
assert(_rootTransform == null);
scheduleInitialLayout();
scheduleInitialPaint(_updateMatricesAndCreateNewRootLayer());
assert(_rootTransform != null);
}
Matrix4? _rootTransform;
TransformLayer _updateMatricesAndCreateNewRootLayer() {
_rootTransform = configuration.toMatrix();
final TransformLayer rootLayer = TransformLayer(transform: _rootTransform);
rootLayer.attach(this);
assert(_rootTransform != null);
return rootLayer;
}
// We never call layout() on this class, so this should never get
// checked. (This class is laid out using scheduleInitialLayout().)
@override
void debugAssertDoesMeetConstraints() { assert(false); }
@override
void performResize() {
assert(false);
}
@override
void performLayout() {
assert(_rootTransform != null);
_size = configuration.size;
assert(_size.isFinite);
if (child != null) {
child!.layout(BoxConstraints.tight(_size));
}
}
/// Determines the set of render objects located at the given position.
///
/// Returns true if the given point is contained in this render object or one
/// of its descendants. Adds any render objects that contain the point to the
/// given hit test result.
///
/// The [position] argument is in the coordinate system of the render view,
/// which is to say, in logical pixels. This is not necessarily the same
/// coordinate system as that expected by the root [Layer], which will
/// normally be in physical (device) pixels.
bool hitTest(HitTestResult result, { required Offset position }) {
if (child != null) {
child!.hitTest(BoxHitTestResult.wrap(result), position: position);
}
result.add(HitTestEntry(this));
return true;
}
@override
bool get isRepaintBoundary => true;
@override
void paint(PaintingContext context, Offset offset) {
if (child != null) {
context.paintChild(child!, offset);
}
assert(() {
final List<DebugPaintCallback> localCallbacks = _debugPaintCallbacks.toList();
for (final DebugPaintCallback paintCallback in localCallbacks) {
if (_debugPaintCallbacks.contains(paintCallback)) {
paintCallback(context, offset, this);
}
}
return true;
}());
}
@override
void applyPaintTransform(RenderBox child, Matrix4 transform) {
assert(_rootTransform != null);
transform.multiply(_rootTransform!);
super.applyPaintTransform(child, transform);
}
/// Uploads the composited layer tree to the engine.
///
/// Actually causes the output of the rendering pipeline to appear on screen.
void compositeFrame() {
if (!kReleaseMode) {
FlutterTimeline.startSync('COMPOSITING');
}
try {
final ui.SceneBuilder builder = ui.SceneBuilder();
final ui.Scene scene = layer!.buildScene(builder);
if (automaticSystemUiAdjustment) {
_updateSystemChrome();
}
_view.render(scene);
scene.dispose();
assert(() {
if (debugRepaintRainbowEnabled || debugRepaintTextRainbowEnabled) {
debugCurrentRepaintColor = debugCurrentRepaintColor.withHue((debugCurrentRepaintColor.hue + 2.0) % 360.0);
}
return true;
}());
} finally {
if (!kReleaseMode) {
FlutterTimeline.finishSync();
}
}
}
/// Sends the provided [SemanticsUpdate] to the [FlutterView] associated with
/// this [RenderView].
///
/// A [SemanticsUpdate] is produced by a [SemanticsOwner] during the
/// [EnginePhase.flushSemantics] phase.
void updateSemantics(ui.SemanticsUpdate update) {
_view.updateSemantics(update);
}
void _updateSystemChrome() {
// Take overlay style from the place where a system status bar and system
// navigation bar are placed to update system style overlay.
// The center of the system navigation bar and the center of the status bar
// are used to get SystemUiOverlayStyle's to update system overlay appearance.
//
// Horizontal center of the screen
// V
// ++++++++++++++++++++++++++
// | |
// | System status bar | <- Vertical center of the status bar
// | |
// ++++++++++++++++++++++++++
// | |
// | Content |
// ~ ~
// | |
// ++++++++++++++++++++++++++
// | |
// | System navigation bar | <- Vertical center of the navigation bar
// | |
// ++++++++++++++++++++++++++ <- bounds.bottom
final Rect bounds = paintBounds;
// Center of the status bar
final Offset top = Offset(
// Horizontal center of the screen
bounds.center.dx,
// The vertical center of the system status bar. The system status bar
// height is kept as top window padding.
_view.padding.top / 2.0,
);
// Center of the navigation bar
final Offset bottom = Offset(
// Horizontal center of the screen
bounds.center.dx,
// Vertical center of the system navigation bar. The system navigation bar
// height is kept as bottom window padding. The "1" needs to be subtracted
// from the bottom because available pixels are in (0..bottom) range.
// I.e. for a device with 1920 height, bound.bottom is 1920, but the most
// bottom drawn pixel is at 1919 position.
bounds.bottom - 1.0 - _view.padding.bottom / 2.0,
);
final SystemUiOverlayStyle? upperOverlayStyle = layer!.find<SystemUiOverlayStyle>(top);
// Only android has a customizable system navigation bar.
SystemUiOverlayStyle? lowerOverlayStyle;
switch (defaultTargetPlatform) {
case TargetPlatform.android:
lowerOverlayStyle = layer!.find<SystemUiOverlayStyle>(bottom);
case TargetPlatform.fuchsia:
case TargetPlatform.iOS:
case TargetPlatform.linux:
case TargetPlatform.macOS:
case TargetPlatform.windows:
break;
}
// If there are no overlay style in the UI don't bother updating.
if (upperOverlayStyle == null && lowerOverlayStyle == null) {
return;
}
// If both are not null, the upper provides the status bar properties and the lower provides
// the system navigation bar properties. This is done for advanced use cases where a widget
// on the top (for instance an app bar) will create an annotated region to set the status bar
// style and another widget on the bottom will create an annotated region to set the system
// navigation bar style.
if (upperOverlayStyle != null && lowerOverlayStyle != null) {
final SystemUiOverlayStyle overlayStyle = SystemUiOverlayStyle(
statusBarBrightness: upperOverlayStyle.statusBarBrightness,
statusBarIconBrightness: upperOverlayStyle.statusBarIconBrightness,
statusBarColor: upperOverlayStyle.statusBarColor,
systemStatusBarContrastEnforced: upperOverlayStyle.systemStatusBarContrastEnforced,
systemNavigationBarColor: lowerOverlayStyle.systemNavigationBarColor,
systemNavigationBarDividerColor: lowerOverlayStyle.systemNavigationBarDividerColor,
systemNavigationBarIconBrightness: lowerOverlayStyle.systemNavigationBarIconBrightness,
systemNavigationBarContrastEnforced: lowerOverlayStyle.systemNavigationBarContrastEnforced,
);
SystemChrome.setSystemUIOverlayStyle(overlayStyle);
return;
}
// If only one of the upper or the lower overlay style is not null, it provides all properties.
// This is done for developer convenience as it allows setting both status bar style and
// navigation bar style using only one annotated region layer (for instance the one
// automatically created by an [AppBar]).
final bool isAndroid = defaultTargetPlatform == TargetPlatform.android;
final SystemUiOverlayStyle definedOverlayStyle = (upperOverlayStyle ?? lowerOverlayStyle)!;
final SystemUiOverlayStyle overlayStyle = SystemUiOverlayStyle(
statusBarBrightness: definedOverlayStyle.statusBarBrightness,
statusBarIconBrightness: definedOverlayStyle.statusBarIconBrightness,
statusBarColor: definedOverlayStyle.statusBarColor,
systemStatusBarContrastEnforced: definedOverlayStyle.systemStatusBarContrastEnforced,
systemNavigationBarColor: isAndroid ? definedOverlayStyle.systemNavigationBarColor : null,
systemNavigationBarDividerColor: isAndroid ? definedOverlayStyle.systemNavigationBarDividerColor : null,
systemNavigationBarIconBrightness: isAndroid ? definedOverlayStyle.systemNavigationBarIconBrightness : null,
systemNavigationBarContrastEnforced: isAndroid ? definedOverlayStyle.systemNavigationBarContrastEnforced : null,
);
SystemChrome.setSystemUIOverlayStyle(overlayStyle);
}
@override
Rect get paintBounds => Offset.zero & (size * configuration.devicePixelRatio);
@override
Rect get semanticBounds {
assert(_rootTransform != null);
return MatrixUtils.transformRect(_rootTransform!, Offset.zero & size);
}
@override
void debugFillProperties(DiagnosticPropertiesBuilder properties) {
// call to ${super.debugFillProperties(description)} is omitted because the
// root superclasses don't include any interesting information for this
// class
assert(() {
properties.add(DiagnosticsNode.message('debug mode enabled - ${kIsWeb ? 'Web' : Platform.operatingSystem}'));
return true;
}());
properties.add(DiagnosticsProperty<Size>('view size', _view.physicalSize, tooltip: 'in physical pixels'));
properties.add(DoubleProperty('device pixel ratio', _view.devicePixelRatio, tooltip: 'physical pixels per logical pixel'));
properties.add(DiagnosticsProperty<ViewConfiguration>('configuration', configuration, tooltip: 'in logical pixels'));
if (_view.platformDispatcher.semanticsEnabled) {
properties.add(DiagnosticsNode.message('semantics enabled'));
}
}
static final List<DebugPaintCallback> _debugPaintCallbacks = <DebugPaintCallback>[];
/// Registers a [DebugPaintCallback] that is called every time a [RenderView]
/// repaints in debug mode.
///
/// The callback may paint a debug overlay on top of the content of the
/// [RenderView] provided to the callback. Callbacks are invoked in the
/// order they were registered in.
///
/// Neither registering a callback nor the continued presence of a callback
/// changes how often [RenderView]s are repainted. It is up to the owner of
/// the callback to call [markNeedsPaint] on any [RenderView] for which it
/// wants to update the painted overlay.
///
/// Does nothing in release mode.
static void debugAddPaintCallback(DebugPaintCallback callback) {
assert(() {
_debugPaintCallbacks.add(callback);
return true;
}());
}
/// Removes a callback registered with [debugAddPaintCallback].
///
/// It does not schedule a frame to repaint the [RenderView]s without the
/// overlay painted by the removed callback. It is up to the owner of the
/// callback to call [markNeedsPaint] on the relevant [RenderView]s to
/// repaint them without the overlay.
///
/// Does nothing in release mode.
static void debugRemovePaintCallback(DebugPaintCallback callback) {
assert(() {
_debugPaintCallbacks.remove(callback);
return true;
}());
}
}
/// A callback for painting a debug overlay on top of the provided [RenderView].
///
/// Used by [RenderView.debugAddPaintCallback] and
/// [RenderView.debugRemovePaintCallback].
typedef DebugPaintCallback = void Function(PaintingContext context, Offset offset, RenderView renderView);