webrtc_m130/call/adaptation/resource_adaptation_module_interface.h

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

87 lines
3.6 KiB
C
Raw Normal View History

/*
* Copyright 2019 The WebRTC Project Authors. All rights reserved.
*
* Use of this source code is governed by a BSD-style license
* that can be found in the LICENSE file in the root of the source
* tree. An additional intellectual property rights grant can be found
* in the file PATENTS. All contributing project authors may
* be found in the AUTHORS file in the root of the source tree.
*/
#ifndef CALL_ADAPTATION_RESOURCE_ADAPTATION_MODULE_INTERFACE_H_
#define CALL_ADAPTATION_RESOURCE_ADAPTATION_MODULE_INTERFACE_H_
Introduce ResourceAdaptationModuleListener and VideoSourceRestrictions. The VideoSourceRestrictions describe the maximum pixels per frame and max frame rate of a video source. This CL makes the ResourceAdaptationModuleInterface responsible for the reconfiguration of video sources. The VideoSourceRestrictions is the output of an adaptation module, and the ResourceAdaptationModuleListener handles the callback for when the source restrictions change. The OveruseFrameDetectorResourceAdaptationModule is updated to output its changes using these interfaces, and VideoStreamEncoder - now a listener - is made responsible for triggering the reconfiguring the video source. Performing the reconfiguration still requires interacting with the VideoSourceProxy - it is still partially responsible for keeping track of rtc::VideoSinkWants settings and performing AddOrUpdateSink(). For now this may look a bit weird: the VideoSourceProxy tells the VideoStreamEncoder about the new restrictions, and then the VideoStreamEncoder tells the VideoSourceProxy to apply these restrictions on the source/sink. This exercises the listener though, and unblocks the next CL. The next CL should move all "configuring the source" logic to the VideoStreamEncoder instead, so that the only information that is tracked by OveruseFrameDetectorResourceAdaptationModule is what it actually outputs to the listener. See the next CL (https://webrtc-review.googlesource.com/c/src/+/162802) where a VideoSourceController is introduced, to be owned by the VideoStreamEncoder rather than the adaptation module. Bug: webrtc:11222 Change-Id: I450ce74f51d96c4b98009a06134db671893d8fdc Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162522 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Commit-Queue: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#30227}
2020-01-10 15:44:01 +01:00
#include <limits>
#include <utility>
#include "absl/types/optional.h"
namespace webrtc {
Introduce ResourceAdaptationModuleListener and VideoSourceRestrictions. The VideoSourceRestrictions describe the maximum pixels per frame and max frame rate of a video source. This CL makes the ResourceAdaptationModuleInterface responsible for the reconfiguration of video sources. The VideoSourceRestrictions is the output of an adaptation module, and the ResourceAdaptationModuleListener handles the callback for when the source restrictions change. The OveruseFrameDetectorResourceAdaptationModule is updated to output its changes using these interfaces, and VideoStreamEncoder - now a listener - is made responsible for triggering the reconfiguring the video source. Performing the reconfiguration still requires interacting with the VideoSourceProxy - it is still partially responsible for keeping track of rtc::VideoSinkWants settings and performing AddOrUpdateSink(). For now this may look a bit weird: the VideoSourceProxy tells the VideoStreamEncoder about the new restrictions, and then the VideoStreamEncoder tells the VideoSourceProxy to apply these restrictions on the source/sink. This exercises the listener though, and unblocks the next CL. The next CL should move all "configuring the source" logic to the VideoStreamEncoder instead, so that the only information that is tracked by OveruseFrameDetectorResourceAdaptationModule is what it actually outputs to the listener. See the next CL (https://webrtc-review.googlesource.com/c/src/+/162802) where a VideoSourceController is introduced, to be owned by the VideoStreamEncoder rather than the adaptation module. Bug: webrtc:11222 Change-Id: I450ce74f51d96c4b98009a06134db671893d8fdc Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162522 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Commit-Queue: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#30227}
2020-01-10 15:44:01 +01:00
// Describes optional restrictions to the resolution and frame rate of a video
// source.
class VideoSourceRestrictions {
public:
// All values must be positive or nullopt.
// TODO(hbos): Support expressing "disable this stream"?
VideoSourceRestrictions(absl::optional<size_t> max_pixels_per_frame,
absl::optional<size_t> target_pixels_per_frame,
absl::optional<double> max_frame_rate);
const absl::optional<size_t>& max_pixels_per_frame() const;
const absl::optional<size_t>& target_pixels_per_frame() const;
const absl::optional<double>& max_frame_rate() const;
private:
// These map to rtc::VideoSinkWants's |max_pixel_count| and
// |target_pixel_count|.
// TODO(hbos): It's not clear what "target" means; either make it well-defined
// or remove it in favor of only using |max_pixels_per_frame_|.
absl::optional<size_t> max_pixels_per_frame_;
absl::optional<size_t> target_pixels_per_frame_;
absl::optional<double> max_frame_rate_;
};
// The listener is responsible for carrying out the reconfiguration of the video
// source such that the VideoSourceRestrictions are fulfilled.
class ResourceAdaptationModuleListener {
public:
virtual ~ResourceAdaptationModuleListener();
// TODO(hbos): When we support the muli-stream use case, the arguments need to
// specify which video stream's source needs to be reconfigured.
virtual void OnVideoSourceRestrictionsUpdated(
VideoSourceRestrictions restrictions) = 0;
};
// Responsible for reconfiguring encoded streams based on resource consumption,
// such as scaling down resolution or frame rate when CPU is overused. This
// interface is meant to be injectable into VideoStreamEncoder.
//
// [UNDER CONSTRUCTION] This interface is work-in-progress. In the future it
// needs to be able to handle all the necessary input and output for resource
// adaptation decision making.
//
// TODO(https://crbug.com/webrtc/11222): Make this interface feature-complete so
// that a module (such as OveruseFrameDetectorResourceAdaptationModule) is fully
// operational through this abstract interface.
class ResourceAdaptationModuleInterface {
public:
virtual ~ResourceAdaptationModuleInterface();
// TODO(hbos): When input/output of the module is adequetly handled by this
// interface, these methods need to say which stream to start/stop, enabling
// multi-stream aware implementations of ResourceAdaptationModuleInterface. We
// don't want to do this before we have the right interfaces (e.g. if we pass
// in a VideoStreamEncoder here directly then have a dependency on a different
// build target). For the multi-stream use case we may consider making
// ResourceAdaptationModuleInterface reference counted.
Introduce ResourceAdaptationModuleListener and VideoSourceRestrictions. The VideoSourceRestrictions describe the maximum pixels per frame and max frame rate of a video source. This CL makes the ResourceAdaptationModuleInterface responsible for the reconfiguration of video sources. The VideoSourceRestrictions is the output of an adaptation module, and the ResourceAdaptationModuleListener handles the callback for when the source restrictions change. The OveruseFrameDetectorResourceAdaptationModule is updated to output its changes using these interfaces, and VideoStreamEncoder - now a listener - is made responsible for triggering the reconfiguring the video source. Performing the reconfiguration still requires interacting with the VideoSourceProxy - it is still partially responsible for keeping track of rtc::VideoSinkWants settings and performing AddOrUpdateSink(). For now this may look a bit weird: the VideoSourceProxy tells the VideoStreamEncoder about the new restrictions, and then the VideoStreamEncoder tells the VideoSourceProxy to apply these restrictions on the source/sink. This exercises the listener though, and unblocks the next CL. The next CL should move all "configuring the source" logic to the VideoStreamEncoder instead, so that the only information that is tracked by OveruseFrameDetectorResourceAdaptationModule is what it actually outputs to the listener. See the next CL (https://webrtc-review.googlesource.com/c/src/+/162802) where a VideoSourceController is introduced, to be owned by the VideoStreamEncoder rather than the adaptation module. Bug: webrtc:11222 Change-Id: I450ce74f51d96c4b98009a06134db671893d8fdc Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162522 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Commit-Queue: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#30227}
2020-01-10 15:44:01 +01:00
virtual void StartCheckForOveruse(
ResourceAdaptationModuleListener* adaptation_listener) = 0;
virtual void StopCheckForOveruse() = 0;
};
} // namespace webrtc
#endif // CALL_ADAPTATION_RESOURCE_ADAPTATION_MODULE_INTERFACE_H_