webrtc_m130/call/adaptation/resource_adaptation_processor_interface.h

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

91 lines
4.0 KiB
C
Raw Normal View History

/*
* Copyright 2020 The WebRTC Project Authors. All rights reserved.
*
* Use of this source code is governed by a BSD-style license
* that can be found in the LICENSE file in the root of the source
* tree. An additional intellectual property rights grant can be found
* in the file PATENTS. All contributing project authors may
* be found in the AUTHORS file in the root of the source tree.
*/
#ifndef CALL_ADAPTATION_RESOURCE_ADAPTATION_PROCESSOR_INTERFACE_H_
#define CALL_ADAPTATION_RESOURCE_ADAPTATION_PROCESSOR_INTERFACE_H_
#include "absl/types/optional.h"
#include "api/rtp_parameters.h"
[Adaptation] Make Resources reference counted and add more DCHECKs. In a future CL, adaptation processing and stream encoder resource management will happen on different task queues. When this is the case, asynchronous tasks will be posted in both directions and some resources will have internal states used on multiple threads. This CL makes the Resource class reference counted in order to support posting tasks to a different threads without risk of use-after-free when a posted task is executed with a delay. This is preferred over WeakPtr strategies because WeakPtrs are single-threaded and preferred over raw pointer usage because the reference counted approach enables more compile-time and run-time assurance. This is also "future proof"; when resources can be injected through public APIs, ownership needs to be shared between libwebrtc and the application (e.g. Chrome). To reduce the risk of making mistakes in the future CL, sequence checkers and task queue DCHECKs are added as well as other DCHECKs to make sure things have been cleaned up before destruction, e.g: - Processor gets a sequence checker. It is entirely single-threaded. - Processor must not have any attached listeners or resources on destruction. - Resources must not have any listeners on destruction. - The Manager, EncodeUsageResource and QualityScalerResource DCHECKs they are running on the encoder queue. - TODOs are added illustrating where we want to add PostTasks in the future CL. Lastly, upon VideoStreamEncoder::Stop() we delete the ResourceAdaptationProcessor. Because the Processor is already used in posted tasks, some if statements are added to ensure the Processor is not used after destruction. Bug: webrtc:11542, webrtc:11520 Change-Id: Ibaa8a61d86d87a71f477d1075a117c28d9d2d285 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174760 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31217}
2020-05-11 16:29:22 +02:00
#include "api/scoped_refptr.h"
#include "api/video/video_adaptation_counters.h"
#include "api/video/video_frame.h"
#include "call/adaptation/encoder_settings.h"
#include "call/adaptation/resource.h"
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
#include "call/adaptation/video_source_restrictions.h"
[Adaptation] Make Resources reference counted and add more DCHECKs. In a future CL, adaptation processing and stream encoder resource management will happen on different task queues. When this is the case, asynchronous tasks will be posted in both directions and some resources will have internal states used on multiple threads. This CL makes the Resource class reference counted in order to support posting tasks to a different threads without risk of use-after-free when a posted task is executed with a delay. This is preferred over WeakPtr strategies because WeakPtrs are single-threaded and preferred over raw pointer usage because the reference counted approach enables more compile-time and run-time assurance. This is also "future proof"; when resources can be injected through public APIs, ownership needs to be shared between libwebrtc and the application (e.g. Chrome). To reduce the risk of making mistakes in the future CL, sequence checkers and task queue DCHECKs are added as well as other DCHECKs to make sure things have been cleaned up before destruction, e.g: - Processor gets a sequence checker. It is entirely single-threaded. - Processor must not have any attached listeners or resources on destruction. - Resources must not have any listeners on destruction. - The Manager, EncodeUsageResource and QualityScalerResource DCHECKs they are running on the encoder queue. - TODOs are added illustrating where we want to add PostTasks in the future CL. Lastly, upon VideoStreamEncoder::Stop() we delete the ResourceAdaptationProcessor. Because the Processor is already used in posted tasks, some if statements are added to ensure the Processor is not used after destruction. Bug: webrtc:11542, webrtc:11520 Change-Id: Ibaa8a61d86d87a71f477d1075a117c28d9d2d285 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174760 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31217}
2020-05-11 16:29:22 +02:00
#include "rtc_base/task_queue.h"
Introduce ResourceAdaptationModuleListener and VideoSourceRestrictions. The VideoSourceRestrictions describe the maximum pixels per frame and max frame rate of a video source. This CL makes the ResourceAdaptationModuleInterface responsible for the reconfiguration of video sources. The VideoSourceRestrictions is the output of an adaptation module, and the ResourceAdaptationModuleListener handles the callback for when the source restrictions change. The OveruseFrameDetectorResourceAdaptationModule is updated to output its changes using these interfaces, and VideoStreamEncoder - now a listener - is made responsible for triggering the reconfiguring the video source. Performing the reconfiguration still requires interacting with the VideoSourceProxy - it is still partially responsible for keeping track of rtc::VideoSinkWants settings and performing AddOrUpdateSink(). For now this may look a bit weird: the VideoSourceProxy tells the VideoStreamEncoder about the new restrictions, and then the VideoStreamEncoder tells the VideoSourceProxy to apply these restrictions on the source/sink. This exercises the listener though, and unblocks the next CL. The next CL should move all "configuring the source" logic to the VideoStreamEncoder instead, so that the only information that is tracked by OveruseFrameDetectorResourceAdaptationModule is what it actually outputs to the listener. See the next CL (https://webrtc-review.googlesource.com/c/src/+/162802) where a VideoSourceController is introduced, to be owned by the VideoStreamEncoder rather than the adaptation module. Bug: webrtc:11222 Change-Id: I450ce74f51d96c4b98009a06134db671893d8fdc Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162522 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Commit-Queue: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#30227}
2020-01-10 15:44:01 +01:00
namespace webrtc {
Introduce ResourceAdaptationModuleListener and VideoSourceRestrictions. The VideoSourceRestrictions describe the maximum pixels per frame and max frame rate of a video source. This CL makes the ResourceAdaptationModuleInterface responsible for the reconfiguration of video sources. The VideoSourceRestrictions is the output of an adaptation module, and the ResourceAdaptationModuleListener handles the callback for when the source restrictions change. The OveruseFrameDetectorResourceAdaptationModule is updated to output its changes using these interfaces, and VideoStreamEncoder - now a listener - is made responsible for triggering the reconfiguring the video source. Performing the reconfiguration still requires interacting with the VideoSourceProxy - it is still partially responsible for keeping track of rtc::VideoSinkWants settings and performing AddOrUpdateSink(). For now this may look a bit weird: the VideoSourceProxy tells the VideoStreamEncoder about the new restrictions, and then the VideoStreamEncoder tells the VideoSourceProxy to apply these restrictions on the source/sink. This exercises the listener though, and unblocks the next CL. The next CL should move all "configuring the source" logic to the VideoStreamEncoder instead, so that the only information that is tracked by OveruseFrameDetectorResourceAdaptationModule is what it actually outputs to the listener. See the next CL (https://webrtc-review.googlesource.com/c/src/+/162802) where a VideoSourceController is introduced, to be owned by the VideoStreamEncoder rather than the adaptation module. Bug: webrtc:11222 Change-Id: I450ce74f51d96c4b98009a06134db671893d8fdc Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162522 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Commit-Queue: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#30227}
2020-01-10 15:44:01 +01:00
// The listener is responsible for carrying out the reconfiguration of the video
// source such that the VideoSourceRestrictions are fulfilled.
class ResourceAdaptationProcessorListener {
Introduce ResourceAdaptationModuleListener and VideoSourceRestrictions. The VideoSourceRestrictions describe the maximum pixels per frame and max frame rate of a video source. This CL makes the ResourceAdaptationModuleInterface responsible for the reconfiguration of video sources. The VideoSourceRestrictions is the output of an adaptation module, and the ResourceAdaptationModuleListener handles the callback for when the source restrictions change. The OveruseFrameDetectorResourceAdaptationModule is updated to output its changes using these interfaces, and VideoStreamEncoder - now a listener - is made responsible for triggering the reconfiguring the video source. Performing the reconfiguration still requires interacting with the VideoSourceProxy - it is still partially responsible for keeping track of rtc::VideoSinkWants settings and performing AddOrUpdateSink(). For now this may look a bit weird: the VideoSourceProxy tells the VideoStreamEncoder about the new restrictions, and then the VideoStreamEncoder tells the VideoSourceProxy to apply these restrictions on the source/sink. This exercises the listener though, and unblocks the next CL. The next CL should move all "configuring the source" logic to the VideoStreamEncoder instead, so that the only information that is tracked by OveruseFrameDetectorResourceAdaptationModule is what it actually outputs to the listener. See the next CL (https://webrtc-review.googlesource.com/c/src/+/162802) where a VideoSourceController is introduced, to be owned by the VideoStreamEncoder rather than the adaptation module. Bug: webrtc:11222 Change-Id: I450ce74f51d96c4b98009a06134db671893d8fdc Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162522 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Commit-Queue: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#30227}
2020-01-10 15:44:01 +01:00
public:
virtual ~ResourceAdaptationProcessorListener();
Introduce ResourceAdaptationModuleListener and VideoSourceRestrictions. The VideoSourceRestrictions describe the maximum pixels per frame and max frame rate of a video source. This CL makes the ResourceAdaptationModuleInterface responsible for the reconfiguration of video sources. The VideoSourceRestrictions is the output of an adaptation module, and the ResourceAdaptationModuleListener handles the callback for when the source restrictions change. The OveruseFrameDetectorResourceAdaptationModule is updated to output its changes using these interfaces, and VideoStreamEncoder - now a listener - is made responsible for triggering the reconfiguring the video source. Performing the reconfiguration still requires interacting with the VideoSourceProxy - it is still partially responsible for keeping track of rtc::VideoSinkWants settings and performing AddOrUpdateSink(). For now this may look a bit weird: the VideoSourceProxy tells the VideoStreamEncoder about the new restrictions, and then the VideoStreamEncoder tells the VideoSourceProxy to apply these restrictions on the source/sink. This exercises the listener though, and unblocks the next CL. The next CL should move all "configuring the source" logic to the VideoStreamEncoder instead, so that the only information that is tracked by OveruseFrameDetectorResourceAdaptationModule is what it actually outputs to the listener. See the next CL (https://webrtc-review.googlesource.com/c/src/+/162802) where a VideoSourceController is introduced, to be owned by the VideoStreamEncoder rather than the adaptation module. Bug: webrtc:11222 Change-Id: I450ce74f51d96c4b98009a06134db671893d8fdc Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162522 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Commit-Queue: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#30227}
2020-01-10 15:44:01 +01:00
// The |restrictions| are filtered by degradation preference but not the
// |adaptation_counters|, which are currently only reported for legacy stats
// calculation purposes.
Introduce ResourceAdaptationModuleListener and VideoSourceRestrictions. The VideoSourceRestrictions describe the maximum pixels per frame and max frame rate of a video source. This CL makes the ResourceAdaptationModuleInterface responsible for the reconfiguration of video sources. The VideoSourceRestrictions is the output of an adaptation module, and the ResourceAdaptationModuleListener handles the callback for when the source restrictions change. The OveruseFrameDetectorResourceAdaptationModule is updated to output its changes using these interfaces, and VideoStreamEncoder - now a listener - is made responsible for triggering the reconfiguring the video source. Performing the reconfiguration still requires interacting with the VideoSourceProxy - it is still partially responsible for keeping track of rtc::VideoSinkWants settings and performing AddOrUpdateSink(). For now this may look a bit weird: the VideoSourceProxy tells the VideoStreamEncoder about the new restrictions, and then the VideoStreamEncoder tells the VideoSourceProxy to apply these restrictions on the source/sink. This exercises the listener though, and unblocks the next CL. The next CL should move all "configuring the source" logic to the VideoStreamEncoder instead, so that the only information that is tracked by OveruseFrameDetectorResourceAdaptationModule is what it actually outputs to the listener. See the next CL (https://webrtc-review.googlesource.com/c/src/+/162802) where a VideoSourceController is introduced, to be owned by the VideoStreamEncoder rather than the adaptation module. Bug: webrtc:11222 Change-Id: I450ce74f51d96c4b98009a06134db671893d8fdc Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162522 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Commit-Queue: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#30227}
2020-01-10 15:44:01 +01:00
virtual void OnVideoSourceRestrictionsUpdated(
VideoSourceRestrictions restrictions,
const VideoAdaptationCounters& adaptation_counters,
[Adaptation] Make Resources reference counted and add more DCHECKs. In a future CL, adaptation processing and stream encoder resource management will happen on different task queues. When this is the case, asynchronous tasks will be posted in both directions and some resources will have internal states used on multiple threads. This CL makes the Resource class reference counted in order to support posting tasks to a different threads without risk of use-after-free when a posted task is executed with a delay. This is preferred over WeakPtr strategies because WeakPtrs are single-threaded and preferred over raw pointer usage because the reference counted approach enables more compile-time and run-time assurance. This is also "future proof"; when resources can be injected through public APIs, ownership needs to be shared between libwebrtc and the application (e.g. Chrome). To reduce the risk of making mistakes in the future CL, sequence checkers and task queue DCHECKs are added as well as other DCHECKs to make sure things have been cleaned up before destruction, e.g: - Processor gets a sequence checker. It is entirely single-threaded. - Processor must not have any attached listeners or resources on destruction. - Resources must not have any listeners on destruction. - The Manager, EncodeUsageResource and QualityScalerResource DCHECKs they are running on the encoder queue. - TODOs are added illustrating where we want to add PostTasks in the future CL. Lastly, upon VideoStreamEncoder::Stop() we delete the ResourceAdaptationProcessor. Because the Processor is already used in posted tasks, some if statements are added to ensure the Processor is not used after destruction. Bug: webrtc:11542, webrtc:11520 Change-Id: Ibaa8a61d86d87a71f477d1075a117c28d9d2d285 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174760 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31217}
2020-05-11 16:29:22 +02:00
rtc::scoped_refptr<Resource> reason) = 0;
Introduce ResourceAdaptationModuleListener and VideoSourceRestrictions. The VideoSourceRestrictions describe the maximum pixels per frame and max frame rate of a video source. This CL makes the ResourceAdaptationModuleInterface responsible for the reconfiguration of video sources. The VideoSourceRestrictions is the output of an adaptation module, and the ResourceAdaptationModuleListener handles the callback for when the source restrictions change. The OveruseFrameDetectorResourceAdaptationModule is updated to output its changes using these interfaces, and VideoStreamEncoder - now a listener - is made responsible for triggering the reconfiguring the video source. Performing the reconfiguration still requires interacting with the VideoSourceProxy - it is still partially responsible for keeping track of rtc::VideoSinkWants settings and performing AddOrUpdateSink(). For now this may look a bit weird: the VideoSourceProxy tells the VideoStreamEncoder about the new restrictions, and then the VideoStreamEncoder tells the VideoSourceProxy to apply these restrictions on the source/sink. This exercises the listener though, and unblocks the next CL. The next CL should move all "configuring the source" logic to the VideoStreamEncoder instead, so that the only information that is tracked by OveruseFrameDetectorResourceAdaptationModule is what it actually outputs to the listener. See the next CL (https://webrtc-review.googlesource.com/c/src/+/162802) where a VideoSourceController is introduced, to be owned by the VideoStreamEncoder rather than the adaptation module. Bug: webrtc:11222 Change-Id: I450ce74f51d96c4b98009a06134db671893d8fdc Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162522 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Commit-Queue: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#30227}
2020-01-10 15:44:01 +01:00
};
[Adaptation] Make Resources reference counted and add more DCHECKs. In a future CL, adaptation processing and stream encoder resource management will happen on different task queues. When this is the case, asynchronous tasks will be posted in both directions and some resources will have internal states used on multiple threads. This CL makes the Resource class reference counted in order to support posting tasks to a different threads without risk of use-after-free when a posted task is executed with a delay. This is preferred over WeakPtr strategies because WeakPtrs are single-threaded and preferred over raw pointer usage because the reference counted approach enables more compile-time and run-time assurance. This is also "future proof"; when resources can be injected through public APIs, ownership needs to be shared between libwebrtc and the application (e.g. Chrome). To reduce the risk of making mistakes in the future CL, sequence checkers and task queue DCHECKs are added as well as other DCHECKs to make sure things have been cleaned up before destruction, e.g: - Processor gets a sequence checker. It is entirely single-threaded. - Processor must not have any attached listeners or resources on destruction. - Resources must not have any listeners on destruction. - The Manager, EncodeUsageResource and QualityScalerResource DCHECKs they are running on the encoder queue. - TODOs are added illustrating where we want to add PostTasks in the future CL. Lastly, upon VideoStreamEncoder::Stop() we delete the ResourceAdaptationProcessor. Because the Processor is already used in posted tasks, some if statements are added to ensure the Processor is not used after destruction. Bug: webrtc:11542, webrtc:11520 Change-Id: Ibaa8a61d86d87a71f477d1075a117c28d9d2d285 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174760 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31217}
2020-05-11 16:29:22 +02:00
// The Resource Adaptation Processor is responsible for reacting to resource
// usage measurements (e.g. overusing or underusing CPU). When a resource is
// overused the Processor is responsible for performing mitigations in order to
// consume less resources.
class ResourceAdaptationProcessorInterface {
public:
virtual ~ResourceAdaptationProcessorInterface();
[Adaptation] Make Resources reference counted and add more DCHECKs. In a future CL, adaptation processing and stream encoder resource management will happen on different task queues. When this is the case, asynchronous tasks will be posted in both directions and some resources will have internal states used on multiple threads. This CL makes the Resource class reference counted in order to support posting tasks to a different threads without risk of use-after-free when a posted task is executed with a delay. This is preferred over WeakPtr strategies because WeakPtrs are single-threaded and preferred over raw pointer usage because the reference counted approach enables more compile-time and run-time assurance. This is also "future proof"; when resources can be injected through public APIs, ownership needs to be shared between libwebrtc and the application (e.g. Chrome). To reduce the risk of making mistakes in the future CL, sequence checkers and task queue DCHECKs are added as well as other DCHECKs to make sure things have been cleaned up before destruction, e.g: - Processor gets a sequence checker. It is entirely single-threaded. - Processor must not have any attached listeners or resources on destruction. - Resources must not have any listeners on destruction. - The Manager, EncodeUsageResource and QualityScalerResource DCHECKs they are running on the encoder queue. - TODOs are added illustrating where we want to add PostTasks in the future CL. Lastly, upon VideoStreamEncoder::Stop() we delete the ResourceAdaptationProcessor. Because the Processor is already used in posted tasks, some if statements are added to ensure the Processor is not used after destruction. Bug: webrtc:11542, webrtc:11520 Change-Id: Ibaa8a61d86d87a71f477d1075a117c28d9d2d285 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174760 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31217}
2020-05-11 16:29:22 +02:00
virtual void InitializeOnResourceAdaptationQueue() = 0;
virtual DegradationPreference degradation_preference() const = 0;
// Reinterprets "balanced + screenshare" as "maintain-resolution".
// TODO(hbos): Don't do this. This is not what "balanced" means. If the
// application wants to maintain resolution it should set that degradation
// preference rather than depend on non-standard behaviors.
virtual DegradationPreference effective_degradation_preference() const = 0;
// Starts or stops listening to resources, effectively enabling or disabling
// processing.
// TODO(https://crbug.com/webrtc/11172): Automatically register and unregister
// with AddResource() and RemoveResource() instead. When the processor is
// multi-stream aware, stream-specific resouces will get added and removed
// over time.
virtual void StartResourceAdaptation() = 0;
virtual void StopResourceAdaptation() = 0;
virtual void AddAdaptationListener(
ResourceAdaptationProcessorListener* adaptation_listener) = 0;
[Adaptation] Make Resources reference counted and add more DCHECKs. In a future CL, adaptation processing and stream encoder resource management will happen on different task queues. When this is the case, asynchronous tasks will be posted in both directions and some resources will have internal states used on multiple threads. This CL makes the Resource class reference counted in order to support posting tasks to a different threads without risk of use-after-free when a posted task is executed with a delay. This is preferred over WeakPtr strategies because WeakPtrs are single-threaded and preferred over raw pointer usage because the reference counted approach enables more compile-time and run-time assurance. This is also "future proof"; when resources can be injected through public APIs, ownership needs to be shared between libwebrtc and the application (e.g. Chrome). To reduce the risk of making mistakes in the future CL, sequence checkers and task queue DCHECKs are added as well as other DCHECKs to make sure things have been cleaned up before destruction, e.g: - Processor gets a sequence checker. It is entirely single-threaded. - Processor must not have any attached listeners or resources on destruction. - Resources must not have any listeners on destruction. - The Manager, EncodeUsageResource and QualityScalerResource DCHECKs they are running on the encoder queue. - TODOs are added illustrating where we want to add PostTasks in the future CL. Lastly, upon VideoStreamEncoder::Stop() we delete the ResourceAdaptationProcessor. Because the Processor is already used in posted tasks, some if statements are added to ensure the Processor is not used after destruction. Bug: webrtc:11542, webrtc:11520 Change-Id: Ibaa8a61d86d87a71f477d1075a117c28d9d2d285 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174760 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31217}
2020-05-11 16:29:22 +02:00
virtual void RemoveAdaptationListener(
ResourceAdaptationProcessorListener* adaptation_listener) = 0;
virtual void AddResource(rtc::scoped_refptr<Resource> resource) = 0;
virtual void RemoveResource(rtc::scoped_refptr<Resource> resource) = 0;
virtual void SetDegradationPreference(
DegradationPreference degradation_preference) = 0;
virtual void SetIsScreenshare(bool is_screenshare) = 0;
virtual void ResetVideoSourceRestrictions() = 0;
// May trigger one or more adaptations. It is meant to reduce resolution -
// useful if a frame was dropped due to its size - however, the implementation
// may not guarantee this (see resource_adaptation_processor.h).
// TODO(hbos): This is only part of the interface for backwards-compatiblity
// reasons. Can we replace this by something which actually satisfies the
// resolution constraints or get rid of it altogether?
virtual void TriggerAdaptationDueToFrameDroppedDueToSize(
[Adaptation] Make Resources reference counted and add more DCHECKs. In a future CL, adaptation processing and stream encoder resource management will happen on different task queues. When this is the case, asynchronous tasks will be posted in both directions and some resources will have internal states used on multiple threads. This CL makes the Resource class reference counted in order to support posting tasks to a different threads without risk of use-after-free when a posted task is executed with a delay. This is preferred over WeakPtr strategies because WeakPtrs are single-threaded and preferred over raw pointer usage because the reference counted approach enables more compile-time and run-time assurance. This is also "future proof"; when resources can be injected through public APIs, ownership needs to be shared between libwebrtc and the application (e.g. Chrome). To reduce the risk of making mistakes in the future CL, sequence checkers and task queue DCHECKs are added as well as other DCHECKs to make sure things have been cleaned up before destruction, e.g: - Processor gets a sequence checker. It is entirely single-threaded. - Processor must not have any attached listeners or resources on destruction. - Resources must not have any listeners on destruction. - The Manager, EncodeUsageResource and QualityScalerResource DCHECKs they are running on the encoder queue. - TODOs are added illustrating where we want to add PostTasks in the future CL. Lastly, upon VideoStreamEncoder::Stop() we delete the ResourceAdaptationProcessor. Because the Processor is already used in posted tasks, some if statements are added to ensure the Processor is not used after destruction. Bug: webrtc:11542, webrtc:11520 Change-Id: Ibaa8a61d86d87a71f477d1075a117c28d9d2d285 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174760 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31217}
2020-05-11 16:29:22 +02:00
rtc::scoped_refptr<Resource> reason_resource) = 0;
};
} // namespace webrtc
#endif // CALL_ADAPTATION_RESOURCE_ADAPTATION_PROCESSOR_INTERFACE_H_