admin/webrtc_m130
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
webrtc_m130/call/adaptation/resource_adaptation_processor.cc

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

393 lines
16 KiB
C++
Raw Normal View History

[Adaptation] Introducing call/adaptation/ResourceAdaptationProcessor. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing This gets to the heart of unblocking call-level adaptation, largely made possible due to the previous CLs in the chain. The parts of the code that are responsible for responding to resource usage signals, obtaining adaptations and applying them are moved to ResourceAdaptationProcessor in call/adaptation/. The parts of the code that are responsible for managing VideoStreamEncoder-specific resources stay inside the VideoStreamEncoderResourceManager class in video/adaptation/. After this CL lands it should soon be possible to move the Processor over to a separate task queue and let the Manager stay on the encoder queue if PostTasks are added for communication between the two objects. Bug: webrtc:11172 Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31105}
2020-04-17 18:36:19 +02:00
/*
* Copyright 2020 The WebRTC Project Authors. All rights reserved.
*
* Use of this source code is governed by a BSD-style license
* that can be found in the LICENSE file in the root of the source
* tree. An additional intellectual property rights grant can be found
* in the file PATENTS. All contributing project authors may
* be found in the AUTHORS file in the root of the source tree.
*/
#include "call/adaptation/resource_adaptation_processor.h"
[Adaptation] Make Resources reference counted and add more DCHECKs. In a future CL, adaptation processing and stream encoder resource management will happen on different task queues. When this is the case, asynchronous tasks will be posted in both directions and some resources will have internal states used on multiple threads. This CL makes the Resource class reference counted in order to support posting tasks to a different threads without risk of use-after-free when a posted task is executed with a delay. This is preferred over WeakPtr strategies because WeakPtrs are single-threaded and preferred over raw pointer usage because the reference counted approach enables more compile-time and run-time assurance. This is also "future proof"; when resources can be injected through public APIs, ownership needs to be shared between libwebrtc and the application (e.g. Chrome). To reduce the risk of making mistakes in the future CL, sequence checkers and task queue DCHECKs are added as well as other DCHECKs to make sure things have been cleaned up before destruction, e.g: - Processor gets a sequence checker. It is entirely single-threaded. - Processor must not have any attached listeners or resources on destruction. - Resources must not have any listeners on destruction. - The Manager, EncodeUsageResource and QualityScalerResource DCHECKs they are running on the encoder queue. - TODOs are added illustrating where we want to add PostTasks in the future CL. Lastly, upon VideoStreamEncoder::Stop() we delete the ResourceAdaptationProcessor. Because the Processor is already used in posted tasks, some if statements are added to ensure the Processor is not used after destruction. Bug: webrtc:11542, webrtc:11520 Change-Id: Ibaa8a61d86d87a71f477d1075a117c28d9d2d285 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174760 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31217}
2020-05-11 16:29:22 +02:00
#include <algorithm>
[Adaptation] Add more ResourceAdaptationProcessor logging. This should help debugging when adaptation is or is not happening unexpectedly. Log spam is prevented by not logging if the same result happened to the same resource already and we haven't adapted since then. Bug: webrtc:11616 Change-Id: Ia6c5cc35061d252f1c66f2f2bf3b94d2485498d6 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176221 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Harald Alvestrand <hta@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31378}
2020-05-28 16:22:42 +02:00
#include <string>
[Adaptation] Introducing call/adaptation/ResourceAdaptationProcessor. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing This gets to the heart of unblocking call-level adaptation, largely made possible due to the previous CLs in the chain. The parts of the code that are responsible for responding to resource usage signals, obtaining adaptations and applying them are moved to ResourceAdaptationProcessor in call/adaptation/. The parts of the code that are responsible for managing VideoStreamEncoder-specific resources stay inside the VideoStreamEncoderResourceManager class in video/adaptation/. After this CL lands it should soon be possible to move the Processor over to a separate task queue and let the Manager stay on the encoder queue if PostTasks are added for communication between the two objects. Bug: webrtc:11172 Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31105}
2020-04-17 18:36:19 +02:00
#include <utility>
#include "absl/algorithm/container.h"
[Adaptation] VideoStreamAdapter broadcasts adaptations This moves this responsibility from the ResourceAdaptaitonProcessor to the VideoStreamAdapter. A new interface for listening to adaptation changes was added, and the ResourceAdaptationProcessor now listens on the VideoStreamAdapter for those changes. This means that going forward, 1. We can do adaptations outside of resource limitations, like setting prior adaptations on a resource like initial frame dropper is designed to. 2. Adaptations can be on a different queue than the ResourceAdaptaitonProcessor's queue since updates are pushed through the listener. Bug: webrtc:11700 Change-Id: I6de0dec748dba095e702f0b9893c5afa50b51aa9 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176859 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31615}
2020-07-01 11:47:46 +02:00
#include "api/video/video_adaptation_counters.h"
#include "call/adaptation/video_stream_adapter.h"
[Adaptation] Add more ResourceAdaptationProcessor logging. This should help debugging when adaptation is or is not happening unexpectedly. Log spam is prevented by not logging if the same result happened to the same resource already and we haven't adapted since then. Bug: webrtc:11616 Change-Id: Ia6c5cc35061d252f1c66f2f2bf3b94d2485498d6 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176221 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Harald Alvestrand <hta@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31378}
2020-05-28 16:22:42 +02:00
#include "rtc_base/logging.h"
[Adaptation] Remove Resource::UsageState() and ClearUsageState(). ResourceListener::OnResourceUsageStateMeasured() now takes ResourceUsageState as argument, making Resource::UsageState() superfluous. With the existing "fire-and-forget" behavior of always clearing usage state on reacting to a signal, there is no longer a need to call ClearUsageState() so this too is removed. (We may want to have a callback in the future to hint to the Resource that it is a good idea to clear internal measurement samples, i.e. because the load of the system is about to change, but we can revisit that when we need it.) Moving the usage state to the callback has the benefit of getting rid of the assumption that UsageState() has to return the same value every time it is called in the same task. This CL is also the final nail in the coffin for Resource needing to know about the adaptation task queue: ResourceAdaptationProcessor's ResourceListener now takes care of posting to the adaptation task queue. To support this, the processor's SequenceChecker is replaced by a TaskQueueBase pointer. Bug: webrtc:11525, webrtc:11618 Change-Id: I2277e71cc3759c85b62465020935603f03792c94 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176376 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31416}
2020-06-03 09:21:34 +02:00
#include "rtc_base/ref_counted_object.h"
[Adaptation] Add more ResourceAdaptationProcessor logging. This should help debugging when adaptation is or is not happening unexpectedly. Log spam is prevented by not logging if the same result happened to the same resource already and we haven't adapted since then. Bug: webrtc:11616 Change-Id: Ia6c5cc35061d252f1c66f2f2bf3b94d2485498d6 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176221 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Harald Alvestrand <hta@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31378}
2020-05-28 16:22:42 +02:00
#include "rtc_base/strings/string_builder.h"
[Adaptation] Make ResourceAdaptationProcessorInterface resources thread-safe This is one less dependency on the task queue, and will make things like removing resources and cleanup much easier in the future. Bug: webrtc:11754 Change-Id: I732f1935d1b58ffe09ca2a2bf59beebc1930214d Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178869 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31686}
2020-07-09 13:35:33 +02:00
#include "rtc_base/synchronization/sequence_checker.h"
[Adaptation] Remove Resource::UsageState() and ClearUsageState(). ResourceListener::OnResourceUsageStateMeasured() now takes ResourceUsageState as argument, making Resource::UsageState() superfluous. With the existing "fire-and-forget" behavior of always clearing usage state on reacting to a signal, there is no longer a need to call ClearUsageState() so this too is removed. (We may want to have a callback in the future to hint to the Resource that it is a good idea to clear internal measurement samples, i.e. because the load of the system is about to change, but we can revisit that when we need it.) Moving the usage state to the callback has the benefit of getting rid of the assumption that UsageState() has to return the same value every time it is called in the same task. This CL is also the final nail in the coffin for Resource needing to know about the adaptation task queue: ResourceAdaptationProcessor's ResourceListener now takes care of posting to the adaptation task queue. To support this, the processor's SequenceChecker is replaced by a TaskQueueBase pointer. Bug: webrtc:11525, webrtc:11618 Change-Id: I2277e71cc3759c85b62465020935603f03792c94 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176376 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31416}
2020-06-03 09:21:34 +02:00
#include "rtc_base/task_utils/to_queued_task.h"
[Adaptation] Introducing call/adaptation/ResourceAdaptationProcessor. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing This gets to the heart of unblocking call-level adaptation, largely made possible due to the previous CLs in the chain. The parts of the code that are responsible for responding to resource usage signals, obtaining adaptations and applying them are moved to ResourceAdaptationProcessor in call/adaptation/. The parts of the code that are responsible for managing VideoStreamEncoder-specific resources stay inside the VideoStreamEncoderResourceManager class in video/adaptation/. After this CL lands it should soon be possible to move the Processor over to a separate task queue and let the Manager stay on the encoder queue if PostTasks are added for communication between the two objects. Bug: webrtc:11172 Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31105}
2020-04-17 18:36:19 +02:00
namespace webrtc {
[Adaptation] Remove Resource::UsageState() and ClearUsageState(). ResourceListener::OnResourceUsageStateMeasured() now takes ResourceUsageState as argument, making Resource::UsageState() superfluous. With the existing "fire-and-forget" behavior of always clearing usage state on reacting to a signal, there is no longer a need to call ClearUsageState() so this too is removed. (We may want to have a callback in the future to hint to the Resource that it is a good idea to clear internal measurement samples, i.e. because the load of the system is about to change, but we can revisit that when we need it.) Moving the usage state to the callback has the benefit of getting rid of the assumption that UsageState() has to return the same value every time it is called in the same task. This CL is also the final nail in the coffin for Resource needing to know about the adaptation task queue: ResourceAdaptationProcessor's ResourceListener now takes care of posting to the adaptation task queue. To support this, the processor's SequenceChecker is replaced by a TaskQueueBase pointer. Bug: webrtc:11525, webrtc:11618 Change-Id: I2277e71cc3759c85b62465020935603f03792c94 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176376 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31416}
2020-06-03 09:21:34 +02:00
ResourceAdaptationProcessor::ResourceListenerDelegate::ResourceListenerDelegate(
ResourceAdaptationProcessor* processor)
[Adaptation] Remove resource adaptation queue Resource adaptation needs refactoring for async adaptations. For now the resource adaptation processor can work on the encoder thread, until it is refactored to support async adaptation. Bug: webrtc:11867 Change-Id: I9c46da356db19c0fd52748c999ccb216f2ca923b Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182040 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31991}
2020-08-25 13:12:12 +02:00
: task_queue_(nullptr), processor_(processor) {}
void ResourceAdaptationProcessor::ResourceListenerDelegate::SetTaskQueue(
TaskQueueBase* task_queue) {
RTC_DCHECK(!task_queue_);
RTC_DCHECK(task_queue);
task_queue_ = task_queue;
RTC_DCHECK_RUN_ON(task_queue_);
[Adaptation] Remove Resource::UsageState() and ClearUsageState(). ResourceListener::OnResourceUsageStateMeasured() now takes ResourceUsageState as argument, making Resource::UsageState() superfluous. With the existing "fire-and-forget" behavior of always clearing usage state on reacting to a signal, there is no longer a need to call ClearUsageState() so this too is removed. (We may want to have a callback in the future to hint to the Resource that it is a good idea to clear internal measurement samples, i.e. because the load of the system is about to change, but we can revisit that when we need it.) Moving the usage state to the callback has the benefit of getting rid of the assumption that UsageState() has to return the same value every time it is called in the same task. This CL is also the final nail in the coffin for Resource needing to know about the adaptation task queue: ResourceAdaptationProcessor's ResourceListener now takes care of posting to the adaptation task queue. To support this, the processor's SequenceChecker is replaced by a TaskQueueBase pointer. Bug: webrtc:11525, webrtc:11618 Change-Id: I2277e71cc3759c85b62465020935603f03792c94 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176376 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31416}
2020-06-03 09:21:34 +02:00
}
void ResourceAdaptationProcessor::ResourceListenerDelegate::
OnProcessorDestroyed() {
[Adaptation] Remove resource adaptation queue Resource adaptation needs refactoring for async adaptations. For now the resource adaptation processor can work on the encoder thread, until it is refactored to support async adaptation. Bug: webrtc:11867 Change-Id: I9c46da356db19c0fd52748c999ccb216f2ca923b Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182040 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31991}
2020-08-25 13:12:12 +02:00
RTC_DCHECK_RUN_ON(task_queue_);
[Adaptation] Remove Resource::UsageState() and ClearUsageState(). ResourceListener::OnResourceUsageStateMeasured() now takes ResourceUsageState as argument, making Resource::UsageState() superfluous. With the existing "fire-and-forget" behavior of always clearing usage state on reacting to a signal, there is no longer a need to call ClearUsageState() so this too is removed. (We may want to have a callback in the future to hint to the Resource that it is a good idea to clear internal measurement samples, i.e. because the load of the system is about to change, but we can revisit that when we need it.) Moving the usage state to the callback has the benefit of getting rid of the assumption that UsageState() has to return the same value every time it is called in the same task. This CL is also the final nail in the coffin for Resource needing to know about the adaptation task queue: ResourceAdaptationProcessor's ResourceListener now takes care of posting to the adaptation task queue. To support this, the processor's SequenceChecker is replaced by a TaskQueueBase pointer. Bug: webrtc:11525, webrtc:11618 Change-Id: I2277e71cc3759c85b62465020935603f03792c94 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176376 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31416}
2020-06-03 09:21:34 +02:00
processor_ = nullptr;
}
void ResourceAdaptationProcessor::ResourceListenerDelegate::
OnResourceUsageStateMeasured(rtc::scoped_refptr<Resource> resource,
ResourceUsageState usage_state) {
[Adaptation] Remove resource adaptation queue Resource adaptation needs refactoring for async adaptations. For now the resource adaptation processor can work on the encoder thread, until it is refactored to support async adaptation. Bug: webrtc:11867 Change-Id: I9c46da356db19c0fd52748c999ccb216f2ca923b Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182040 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31991}
2020-08-25 13:12:12 +02:00
if (!task_queue_->IsCurrent()) {
task_queue_->PostTask(ToQueuedTask(
[Adaptation] Remove Resource::UsageState() and ClearUsageState(). ResourceListener::OnResourceUsageStateMeasured() now takes ResourceUsageState as argument, making Resource::UsageState() superfluous. With the existing "fire-and-forget" behavior of always clearing usage state on reacting to a signal, there is no longer a need to call ClearUsageState() so this too is removed. (We may want to have a callback in the future to hint to the Resource that it is a good idea to clear internal measurement samples, i.e. because the load of the system is about to change, but we can revisit that when we need it.) Moving the usage state to the callback has the benefit of getting rid of the assumption that UsageState() has to return the same value every time it is called in the same task. This CL is also the final nail in the coffin for Resource needing to know about the adaptation task queue: ResourceAdaptationProcessor's ResourceListener now takes care of posting to the adaptation task queue. To support this, the processor's SequenceChecker is replaced by a TaskQueueBase pointer. Bug: webrtc:11525, webrtc:11618 Change-Id: I2277e71cc3759c85b62465020935603f03792c94 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176376 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31416}
2020-06-03 09:21:34 +02:00
[this_ref = rtc::scoped_refptr<ResourceListenerDelegate>(this),
resource, usage_state] {
this_ref->OnResourceUsageStateMeasured(resource, usage_state);
}));
return;
}
[Adaptation] Remove resource adaptation queue Resource adaptation needs refactoring for async adaptations. For now the resource adaptation processor can work on the encoder thread, until it is refactored to support async adaptation. Bug: webrtc:11867 Change-Id: I9c46da356db19c0fd52748c999ccb216f2ca923b Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182040 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31991}
2020-08-25 13:12:12 +02:00
RTC_DCHECK_RUN_ON(task_queue_);
[Adaptation] Remove Resource::UsageState() and ClearUsageState(). ResourceListener::OnResourceUsageStateMeasured() now takes ResourceUsageState as argument, making Resource::UsageState() superfluous. With the existing "fire-and-forget" behavior of always clearing usage state on reacting to a signal, there is no longer a need to call ClearUsageState() so this too is removed. (We may want to have a callback in the future to hint to the Resource that it is a good idea to clear internal measurement samples, i.e. because the load of the system is about to change, but we can revisit that when we need it.) Moving the usage state to the callback has the benefit of getting rid of the assumption that UsageState() has to return the same value every time it is called in the same task. This CL is also the final nail in the coffin for Resource needing to know about the adaptation task queue: ResourceAdaptationProcessor's ResourceListener now takes care of posting to the adaptation task queue. To support this, the processor's SequenceChecker is replaced by a TaskQueueBase pointer. Bug: webrtc:11525, webrtc:11618 Change-Id: I2277e71cc3759c85b62465020935603f03792c94 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176376 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31416}
2020-06-03 09:21:34 +02:00
if (processor_) {
processor_->OnResourceUsageStateMeasured(resource, usage_state);
}
}
[Adaptation] Add more ResourceAdaptationProcessor logging. This should help debugging when adaptation is or is not happening unexpectedly. Log spam is prevented by not logging if the same result happened to the same resource already and we haven't adapted since then. Bug: webrtc:11616 Change-Id: Ia6c5cc35061d252f1c66f2f2bf3b94d2485498d6 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176221 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Harald Alvestrand <hta@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31378}
2020-05-28 16:22:42 +02:00
ResourceAdaptationProcessor::MitigationResultAndLogMessage::
MitigationResultAndLogMessage()
: result(MitigationResult::kAdaptationApplied), message() {}
ResourceAdaptationProcessor::MitigationResultAndLogMessage::
MitigationResultAndLogMessage(MitigationResult result, std::string message)
: result(result), message(std::move(message)) {}
[Adaptation] Introducing call/adaptation/ResourceAdaptationProcessor. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing This gets to the heart of unblocking call-level adaptation, largely made possible due to the previous CLs in the chain. The parts of the code that are responsible for responding to resource usage signals, obtaining adaptations and applying them are moved to ResourceAdaptationProcessor in call/adaptation/. The parts of the code that are responsible for managing VideoStreamEncoder-specific resources stay inside the VideoStreamEncoderResourceManager class in video/adaptation/. After this CL lands it should soon be possible to move the Processor over to a separate task queue and let the Manager stay on the encoder queue if PostTasks are added for communication between the two objects. Bug: webrtc:11172 Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31105}
2020-04-17 18:36:19 +02:00
ResourceAdaptationProcessor::ResourceAdaptationProcessor(
[Adaptation] VideoStreamAdapter broadcasts adaptations This moves this responsibility from the ResourceAdaptaitonProcessor to the VideoStreamAdapter. A new interface for listening to adaptation changes was added, and the ResourceAdaptationProcessor now listens on the VideoStreamAdapter for those changes. This means that going forward, 1. We can do adaptations outside of resource limitations, like setting prior adaptations on a resource like initial frame dropper is designed to. 2. Adaptations can be on a different queue than the ResourceAdaptaitonProcessor's queue since updates are pushed through the listener. Bug: webrtc:11700 Change-Id: I6de0dec748dba095e702f0b9893c5afa50b51aa9 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176859 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31615}
2020-07-01 11:47:46 +02:00
VideoStreamAdapter* stream_adapter)
[Adaptation] Remove resource adaptation queue Resource adaptation needs refactoring for async adaptations. For now the resource adaptation processor can work on the encoder thread, until it is refactored to support async adaptation. Bug: webrtc:11867 Change-Id: I9c46da356db19c0fd52748c999ccb216f2ca923b Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182040 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31991}
2020-08-25 13:12:12 +02:00
: task_queue_(nullptr),
[Adaptation] Remove Resource::UsageState() and ClearUsageState(). ResourceListener::OnResourceUsageStateMeasured() now takes ResourceUsageState as argument, making Resource::UsageState() superfluous. With the existing "fire-and-forget" behavior of always clearing usage state on reacting to a signal, there is no longer a need to call ClearUsageState() so this too is removed. (We may want to have a callback in the future to hint to the Resource that it is a good idea to clear internal measurement samples, i.e. because the load of the system is about to change, but we can revisit that when we need it.) Moving the usage state to the callback has the benefit of getting rid of the assumption that UsageState() has to return the same value every time it is called in the same task. This CL is also the final nail in the coffin for Resource needing to know about the adaptation task queue: ResourceAdaptationProcessor's ResourceListener now takes care of posting to the adaptation task queue. To support this, the processor's SequenceChecker is replaced by a TaskQueueBase pointer. Bug: webrtc:11525, webrtc:11618 Change-Id: I2277e71cc3759c85b62465020935603f03792c94 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176376 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31416}
2020-06-03 09:21:34 +02:00
resource_listener_delegate_(
new rtc::RefCountedObject<ResourceListenerDelegate>(this)),
[Adaptation] Introducing call/adaptation/ResourceAdaptationProcessor. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing This gets to the heart of unblocking call-level adaptation, largely made possible due to the previous CLs in the chain. The parts of the code that are responsible for responding to resource usage signals, obtaining adaptations and applying them are moved to ResourceAdaptationProcessor in call/adaptation/. The parts of the code that are responsible for managing VideoStreamEncoder-specific resources stay inside the VideoStreamEncoderResourceManager class in video/adaptation/. After this CL lands it should soon be possible to move the Processor over to a separate task queue and let the Manager stay on the encoder queue if PostTasks are added for communication between the two objects. Bug: webrtc:11172 Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31105}
2020-04-17 18:36:19 +02:00
resources_(),
[Adaptation] VideoStreamAdapter broadcasts adaptations This moves this responsibility from the ResourceAdaptaitonProcessor to the VideoStreamAdapter. A new interface for listening to adaptation changes was added, and the ResourceAdaptationProcessor now listens on the VideoStreamAdapter for those changes. This means that going forward, 1. We can do adaptations outside of resource limitations, like setting prior adaptations on a resource like initial frame dropper is designed to. 2. Adaptations can be on a different queue than the ResourceAdaptaitonProcessor's queue since updates are pushed through the listener. Bug: webrtc:11700 Change-Id: I6de0dec748dba095e702f0b9893c5afa50b51aa9 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176859 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31615}
2020-07-01 11:47:46 +02:00
stream_adapter_(stream_adapter),
[Adaptation] Add OnAdaptationApplied(), remove ResourceListenerResponse. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing The ResourceListenerResponse was used to make the QualityScaler not clear QP samples and instead increase its frequency of checking for QP under certain circumstances, see enum description: https://webrtc.googlesource.com/src.git/+/c70b1028d47c1aee4892545190cd66e97d09cd55/call/adaptation/resource.h#33 Because the QualityScaler depends on whether and how adaptation happened it should listen to adaptation happening. This CL moves the logic that was previously in VideoStreamAdapter closer to the QualityScaler: QualityScalerResource::OnAdaptationApplied(). This would allow the VideoStreamAdapter to operate on a separate task queue in the future, with no dependencies on any stream-specific resources that might operate on other task queues. Bug: webrtc:11172, webrtc:11521 Change-Id: I07971a8a5fab5715f4ccb7d2c63f1b92bd47170f Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173090 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31143}
2020-04-28 12:24:33 +02:00
last_reported_source_restrictions_(),
[Adaptation] Remove processing_in_progress_ from ResourceAdaptationProcessor This flag was used to protect an AdaptationListener from adapting again, but since AdaptationListener was removed this guard is no longer needed. Bug: webrtc:11834 Change-Id: If79e960f57aaf56ce2849329ff6737414fc60e9a Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181068 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Commit-Queue: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31905}
2020-08-11 09:14:46 +02:00
previous_mitigation_results_() {
[Adaptation] VideoStreamAdapter broadcasts adaptations This moves this responsibility from the ResourceAdaptaitonProcessor to the VideoStreamAdapter. A new interface for listening to adaptation changes was added, and the ResourceAdaptationProcessor now listens on the VideoStreamAdapter for those changes. This means that going forward, 1. We can do adaptations outside of resource limitations, like setting prior adaptations on a resource like initial frame dropper is designed to. 2. Adaptations can be on a different queue than the ResourceAdaptaitonProcessor's queue since updates are pushed through the listener. Bug: webrtc:11700 Change-Id: I6de0dec748dba095e702f0b9893c5afa50b51aa9 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176859 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31615}
2020-07-01 11:47:46 +02:00
RTC_DCHECK(stream_adapter_);
}
[Adaptation] Make Resources reference counted and add more DCHECKs. In a future CL, adaptation processing and stream encoder resource management will happen on different task queues. When this is the case, asynchronous tasks will be posted in both directions and some resources will have internal states used on multiple threads. This CL makes the Resource class reference counted in order to support posting tasks to a different threads without risk of use-after-free when a posted task is executed with a delay. This is preferred over WeakPtr strategies because WeakPtrs are single-threaded and preferred over raw pointer usage because the reference counted approach enables more compile-time and run-time assurance. This is also "future proof"; when resources can be injected through public APIs, ownership needs to be shared between libwebrtc and the application (e.g. Chrome). To reduce the risk of making mistakes in the future CL, sequence checkers and task queue DCHECKs are added as well as other DCHECKs to make sure things have been cleaned up before destruction, e.g: - Processor gets a sequence checker. It is entirely single-threaded. - Processor must not have any attached listeners or resources on destruction. - Resources must not have any listeners on destruction. - The Manager, EncodeUsageResource and QualityScalerResource DCHECKs they are running on the encoder queue. - TODOs are added illustrating where we want to add PostTasks in the future CL. Lastly, upon VideoStreamEncoder::Stop() we delete the ResourceAdaptationProcessor. Because the Processor is already used in posted tasks, some if statements are added to ensure the Processor is not used after destruction. Bug: webrtc:11542, webrtc:11520 Change-Id: Ibaa8a61d86d87a71f477d1075a117c28d9d2d285 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174760 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31217}
2020-05-11 16:29:22 +02:00
ResourceAdaptationProcessor::~ResourceAdaptationProcessor() {
[Adaptation] Remove resource adaptation queue Resource adaptation needs refactoring for async adaptations. For now the resource adaptation processor can work on the encoder thread, until it is refactored to support async adaptation. Bug: webrtc:11867 Change-Id: I9c46da356db19c0fd52748c999ccb216f2ca923b Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182040 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31991}
2020-08-25 13:12:12 +02:00
RTC_DCHECK_RUN_ON(task_queue_);
[Adaptation] Make Resources reference counted and add more DCHECKs. In a future CL, adaptation processing and stream encoder resource management will happen on different task queues. When this is the case, asynchronous tasks will be posted in both directions and some resources will have internal states used on multiple threads. This CL makes the Resource class reference counted in order to support posting tasks to a different threads without risk of use-after-free when a posted task is executed with a delay. This is preferred over WeakPtr strategies because WeakPtrs are single-threaded and preferred over raw pointer usage because the reference counted approach enables more compile-time and run-time assurance. This is also "future proof"; when resources can be injected through public APIs, ownership needs to be shared between libwebrtc and the application (e.g. Chrome). To reduce the risk of making mistakes in the future CL, sequence checkers and task queue DCHECKs are added as well as other DCHECKs to make sure things have been cleaned up before destruction, e.g: - Processor gets a sequence checker. It is entirely single-threaded. - Processor must not have any attached listeners or resources on destruction. - Resources must not have any listeners on destruction. - The Manager, EncodeUsageResource and QualityScalerResource DCHECKs they are running on the encoder queue. - TODOs are added illustrating where we want to add PostTasks in the future CL. Lastly, upon VideoStreamEncoder::Stop() we delete the ResourceAdaptationProcessor. Because the Processor is already used in posted tasks, some if statements are added to ensure the Processor is not used after destruction. Bug: webrtc:11542, webrtc:11520 Change-Id: Ibaa8a61d86d87a71f477d1075a117c28d9d2d285 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174760 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31217}
2020-05-11 16:29:22 +02:00
RTC_DCHECK(resources_.empty())
<< "There are resource(s) attached to a ResourceAdaptationProcessor "
<< "being destroyed.";
[Adaptation] VideoStreamAdapter broadcasts adaptations This moves this responsibility from the ResourceAdaptaitonProcessor to the VideoStreamAdapter. A new interface for listening to adaptation changes was added, and the ResourceAdaptationProcessor now listens on the VideoStreamAdapter for those changes. This means that going forward, 1. We can do adaptations outside of resource limitations, like setting prior adaptations on a resource like initial frame dropper is designed to. 2. Adaptations can be on a different queue than the ResourceAdaptaitonProcessor's queue since updates are pushed through the listener. Bug: webrtc:11700 Change-Id: I6de0dec748dba095e702f0b9893c5afa50b51aa9 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176859 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31615}
2020-07-01 11:47:46 +02:00
stream_adapter_->RemoveRestrictionsListener(this);
[Adaptation] Remove Resource::UsageState() and ClearUsageState(). ResourceListener::OnResourceUsageStateMeasured() now takes ResourceUsageState as argument, making Resource::UsageState() superfluous. With the existing "fire-and-forget" behavior of always clearing usage state on reacting to a signal, there is no longer a need to call ClearUsageState() so this too is removed. (We may want to have a callback in the future to hint to the Resource that it is a good idea to clear internal measurement samples, i.e. because the load of the system is about to change, but we can revisit that when we need it.) Moving the usage state to the callback has the benefit of getting rid of the assumption that UsageState() has to return the same value every time it is called in the same task. This CL is also the final nail in the coffin for Resource needing to know about the adaptation task queue: ResourceAdaptationProcessor's ResourceListener now takes care of posting to the adaptation task queue. To support this, the processor's SequenceChecker is replaced by a TaskQueueBase pointer. Bug: webrtc:11525, webrtc:11618 Change-Id: I2277e71cc3759c85b62465020935603f03792c94 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176376 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31416}
2020-06-03 09:21:34 +02:00
resource_listener_delegate_->OnProcessorDestroyed();
[Adaptation] Make Resources reference counted and add more DCHECKs. In a future CL, adaptation processing and stream encoder resource management will happen on different task queues. When this is the case, asynchronous tasks will be posted in both directions and some resources will have internal states used on multiple threads. This CL makes the Resource class reference counted in order to support posting tasks to a different threads without risk of use-after-free when a posted task is executed with a delay. This is preferred over WeakPtr strategies because WeakPtrs are single-threaded and preferred over raw pointer usage because the reference counted approach enables more compile-time and run-time assurance. This is also "future proof"; when resources can be injected through public APIs, ownership needs to be shared between libwebrtc and the application (e.g. Chrome). To reduce the risk of making mistakes in the future CL, sequence checkers and task queue DCHECKs are added as well as other DCHECKs to make sure things have been cleaned up before destruction, e.g: - Processor gets a sequence checker. It is entirely single-threaded. - Processor must not have any attached listeners or resources on destruction. - Resources must not have any listeners on destruction. - The Manager, EncodeUsageResource and QualityScalerResource DCHECKs they are running on the encoder queue. - TODOs are added illustrating where we want to add PostTasks in the future CL. Lastly, upon VideoStreamEncoder::Stop() we delete the ResourceAdaptationProcessor. Because the Processor is already used in posted tasks, some if statements are added to ensure the Processor is not used after destruction. Bug: webrtc:11542, webrtc:11520 Change-Id: Ibaa8a61d86d87a71f477d1075a117c28d9d2d285 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174760 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31217}
2020-05-11 16:29:22 +02:00
}
[Adaptation] Introducing call/adaptation/ResourceAdaptationProcessor. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing This gets to the heart of unblocking call-level adaptation, largely made possible due to the previous CLs in the chain. The parts of the code that are responsible for responding to resource usage signals, obtaining adaptations and applying them are moved to ResourceAdaptationProcessor in call/adaptation/. The parts of the code that are responsible for managing VideoStreamEncoder-specific resources stay inside the VideoStreamEncoderResourceManager class in video/adaptation/. After this CL lands it should soon be possible to move the Processor over to a separate task queue and let the Manager stay on the encoder queue if PostTasks are added for communication between the two objects. Bug: webrtc:11172 Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31105}
2020-04-17 18:36:19 +02:00
[Adaptation] Remove resource adaptation queue Resource adaptation needs refactoring for async adaptations. For now the resource adaptation processor can work on the encoder thread, until it is refactored to support async adaptation. Bug: webrtc:11867 Change-Id: I9c46da356db19c0fd52748c999ccb216f2ca923b Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182040 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31991}
2020-08-25 13:12:12 +02:00
void ResourceAdaptationProcessor::SetTaskQueue(TaskQueueBase* task_queue) {
RTC_DCHECK(!task_queue_);
RTC_DCHECK(task_queue);
task_queue_ = task_queue;
resource_listener_delegate_->SetTaskQueue(task_queue);
RTC_DCHECK_RUN_ON(task_queue_);
// Now that we have the queue we can attach as adaptation listener.
[Adaptation] VideoStreamAdapter broadcasts adaptations This moves this responsibility from the ResourceAdaptaitonProcessor to the VideoStreamAdapter. A new interface for listening to adaptation changes was added, and the ResourceAdaptationProcessor now listens on the VideoStreamAdapter for those changes. This means that going forward, 1. We can do adaptations outside of resource limitations, like setting prior adaptations on a resource like initial frame dropper is designed to. 2. Adaptations can be on a different queue than the ResourceAdaptaitonProcessor's queue since updates are pushed through the listener. Bug: webrtc:11700 Change-Id: I6de0dec748dba095e702f0b9893c5afa50b51aa9 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176859 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31615}
2020-07-01 11:47:46 +02:00
stream_adapter_->AddRestrictionsListener(this);
[Adaptation] Make Resources reference counted and add more DCHECKs. In a future CL, adaptation processing and stream encoder resource management will happen on different task queues. When this is the case, asynchronous tasks will be posted in both directions and some resources will have internal states used on multiple threads. This CL makes the Resource class reference counted in order to support posting tasks to a different threads without risk of use-after-free when a posted task is executed with a delay. This is preferred over WeakPtr strategies because WeakPtrs are single-threaded and preferred over raw pointer usage because the reference counted approach enables more compile-time and run-time assurance. This is also "future proof"; when resources can be injected through public APIs, ownership needs to be shared between libwebrtc and the application (e.g. Chrome). To reduce the risk of making mistakes in the future CL, sequence checkers and task queue DCHECKs are added as well as other DCHECKs to make sure things have been cleaned up before destruction, e.g: - Processor gets a sequence checker. It is entirely single-threaded. - Processor must not have any attached listeners or resources on destruction. - Resources must not have any listeners on destruction. - The Manager, EncodeUsageResource and QualityScalerResource DCHECKs they are running on the encoder queue. - TODOs are added illustrating where we want to add PostTasks in the future CL. Lastly, upon VideoStreamEncoder::Stop() we delete the ResourceAdaptationProcessor. Because the Processor is already used in posted tasks, some if statements are added to ensure the Processor is not used after destruction. Bug: webrtc:11542, webrtc:11520 Change-Id: Ibaa8a61d86d87a71f477d1075a117c28d9d2d285 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174760 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31217}
2020-05-11 16:29:22 +02:00
}
[Adaptation] Introducing call/adaptation/ResourceAdaptationProcessor. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing This gets to the heart of unblocking call-level adaptation, largely made possible due to the previous CLs in the chain. The parts of the code that are responsible for responding to resource usage signals, obtaining adaptations and applying them are moved to ResourceAdaptationProcessor in call/adaptation/. The parts of the code that are responsible for managing VideoStreamEncoder-specific resources stay inside the VideoStreamEncoderResourceManager class in video/adaptation/. After this CL lands it should soon be possible to move the Processor over to a separate task queue and let the Manager stay on the encoder queue if PostTasks are added for communication between the two objects. Bug: webrtc:11172 Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31105}
2020-04-17 18:36:19 +02:00
[Adaptation] VideoStreamAdapter broadcasts adaptations This moves this responsibility from the ResourceAdaptaitonProcessor to the VideoStreamAdapter. A new interface for listening to adaptation changes was added, and the ResourceAdaptationProcessor now listens on the VideoStreamAdapter for those changes. This means that going forward, 1. We can do adaptations outside of resource limitations, like setting prior adaptations on a resource like initial frame dropper is designed to. 2. Adaptations can be on a different queue than the ResourceAdaptaitonProcessor's queue since updates are pushed through the listener. Bug: webrtc:11700 Change-Id: I6de0dec748dba095e702f0b9893c5afa50b51aa9 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176859 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31615}
2020-07-01 11:47:46 +02:00
void ResourceAdaptationProcessor::AddResourceLimitationsListener(
ResourceLimitationsListener* limitations_listener) {
[Adaptation] Remove resource adaptation queue Resource adaptation needs refactoring for async adaptations. For now the resource adaptation processor can work on the encoder thread, until it is refactored to support async adaptation. Bug: webrtc:11867 Change-Id: I9c46da356db19c0fd52748c999ccb216f2ca923b Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182040 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31991}
2020-08-25 13:12:12 +02:00
RTC_DCHECK_RUN_ON(task_queue_);
[Adaptation] VideoStreamAdapter broadcasts adaptations This moves this responsibility from the ResourceAdaptaitonProcessor to the VideoStreamAdapter. A new interface for listening to adaptation changes was added, and the ResourceAdaptationProcessor now listens on the VideoStreamAdapter for those changes. This means that going forward, 1. We can do adaptations outside of resource limitations, like setting prior adaptations on a resource like initial frame dropper is designed to. 2. Adaptations can be on a different queue than the ResourceAdaptaitonProcessor's queue since updates are pushed through the listener. Bug: webrtc:11700 Change-Id: I6de0dec748dba095e702f0b9893c5afa50b51aa9 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176859 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31615}
2020-07-01 11:47:46 +02:00
RTC_DCHECK(std::find(resource_limitations_listeners_.begin(),
resource_limitations_listeners_.end(),
limitations_listener) ==
resource_limitations_listeners_.end());
resource_limitations_listeners_.push_back(limitations_listener);
[Adaptation] Introducing call/adaptation/ResourceAdaptationProcessor. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing This gets to the heart of unblocking call-level adaptation, largely made possible due to the previous CLs in the chain. The parts of the code that are responsible for responding to resource usage signals, obtaining adaptations and applying them are moved to ResourceAdaptationProcessor in call/adaptation/. The parts of the code that are responsible for managing VideoStreamEncoder-specific resources stay inside the VideoStreamEncoderResourceManager class in video/adaptation/. After this CL lands it should soon be possible to move the Processor over to a separate task queue and let the Manager stay on the encoder queue if PostTasks are added for communication between the two objects. Bug: webrtc:11172 Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31105}
2020-04-17 18:36:19 +02:00
}
[Adaptation] Remove resource adaptation queue Resource adaptation needs refactoring for async adaptations. For now the resource adaptation processor can work on the encoder thread, until it is refactored to support async adaptation. Bug: webrtc:11867 Change-Id: I9c46da356db19c0fd52748c999ccb216f2ca923b Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182040 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31991}
2020-08-25 13:12:12 +02:00
[Adaptation] VideoStreamAdapter broadcasts adaptations This moves this responsibility from the ResourceAdaptaitonProcessor to the VideoStreamAdapter. A new interface for listening to adaptation changes was added, and the ResourceAdaptationProcessor now listens on the VideoStreamAdapter for those changes. This means that going forward, 1. We can do adaptations outside of resource limitations, like setting prior adaptations on a resource like initial frame dropper is designed to. 2. Adaptations can be on a different queue than the ResourceAdaptaitonProcessor's queue since updates are pushed through the listener. Bug: webrtc:11700 Change-Id: I6de0dec748dba095e702f0b9893c5afa50b51aa9 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176859 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31615}
2020-07-01 11:47:46 +02:00
void ResourceAdaptationProcessor::RemoveResourceLimitationsListener(
ResourceLimitationsListener* limitations_listener) {
[Adaptation] Remove resource adaptation queue Resource adaptation needs refactoring for async adaptations. For now the resource adaptation processor can work on the encoder thread, until it is refactored to support async adaptation. Bug: webrtc:11867 Change-Id: I9c46da356db19c0fd52748c999ccb216f2ca923b Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182040 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31991}
2020-08-25 13:12:12 +02:00
RTC_DCHECK_RUN_ON(task_queue_);
[Adaptation] VideoStreamAdapter broadcasts adaptations This moves this responsibility from the ResourceAdaptaitonProcessor to the VideoStreamAdapter. A new interface for listening to adaptation changes was added, and the ResourceAdaptationProcessor now listens on the VideoStreamAdapter for those changes. This means that going forward, 1. We can do adaptations outside of resource limitations, like setting prior adaptations on a resource like initial frame dropper is designed to. 2. Adaptations can be on a different queue than the ResourceAdaptaitonProcessor's queue since updates are pushed through the listener. Bug: webrtc:11700 Change-Id: I6de0dec748dba095e702f0b9893c5afa50b51aa9 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176859 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31615}
2020-07-01 11:47:46 +02:00
auto it =
std::find(resource_limitations_listeners_.begin(),
resource_limitations_listeners_.end(), limitations_listener);
RTC_DCHECK(it != resource_limitations_listeners_.end());
resource_limitations_listeners_.erase(it);
[Adaptation] Make Resources reference counted and add more DCHECKs. In a future CL, adaptation processing and stream encoder resource management will happen on different task queues. When this is the case, asynchronous tasks will be posted in both directions and some resources will have internal states used on multiple threads. This CL makes the Resource class reference counted in order to support posting tasks to a different threads without risk of use-after-free when a posted task is executed with a delay. This is preferred over WeakPtr strategies because WeakPtrs are single-threaded and preferred over raw pointer usage because the reference counted approach enables more compile-time and run-time assurance. This is also "future proof"; when resources can be injected through public APIs, ownership needs to be shared between libwebrtc and the application (e.g. Chrome). To reduce the risk of making mistakes in the future CL, sequence checkers and task queue DCHECKs are added as well as other DCHECKs to make sure things have been cleaned up before destruction, e.g: - Processor gets a sequence checker. It is entirely single-threaded. - Processor must not have any attached listeners or resources on destruction. - Resources must not have any listeners on destruction. - The Manager, EncodeUsageResource and QualityScalerResource DCHECKs they are running on the encoder queue. - TODOs are added illustrating where we want to add PostTasks in the future CL. Lastly, upon VideoStreamEncoder::Stop() we delete the ResourceAdaptationProcessor. Because the Processor is already used in posted tasks, some if statements are added to ensure the Processor is not used after destruction. Bug: webrtc:11542, webrtc:11520 Change-Id: Ibaa8a61d86d87a71f477d1075a117c28d9d2d285 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174760 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31217}
2020-05-11 16:29:22 +02:00
}
void ResourceAdaptationProcessor::AddResource(
rtc::scoped_refptr<Resource> resource) {
[Adaptation] Implement remove for most limited resources. When a resource is removed from the ResourceAdaptationProcessor, and the resource is the most limited resource, we reset the current adaptation to the next most limited resource level. In the case that this resource is the only adapted resource then we reset all adaptations. Bug: webrtc:11636 Change-Id: I29acc5a3934f42f00db79b3bb2171156b1313937 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176406 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31520}
2020-06-12 16:00:03 +02:00
RTC_DCHECK(resource);
[Adaptation] Make ResourceAdaptationProcessorInterface resources thread-safe This is one less dependency on the task queue, and will make things like removing resources and cleanup much easier in the future. Bug: webrtc:11754 Change-Id: I732f1935d1b58ffe09ca2a2bf59beebc1930214d Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178869 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31686}
2020-07-09 13:35:33 +02:00
{
MutexLock crit(&resources_lock_);
RTC_DCHECK(absl::c_find(resources_, resource) == resources_.end())
<< "Resource \"" << resource->Name() << "\" was already registered.";
resources_.push_back(resource);
}
[Adaptation] Implement remove for most limited resources. When a resource is removed from the ResourceAdaptationProcessor, and the resource is the most limited resource, we reset the current adaptation to the next most limited resource level. In the case that this resource is the only adapted resource then we reset all adaptations. Bug: webrtc:11636 Change-Id: I29acc5a3934f42f00db79b3bb2171156b1313937 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176406 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31520}
2020-06-12 16:00:03 +02:00
resource->SetResourceListener(resource_listener_delegate_);
Reland "[Adaptation] Remove QualityScalerResource when disabled." This is a reland of ba8abbb630cdd9d05e22c830d0845e920762850d This can be relanded as the queuing issues that were causing a crash in the WebRTC roll in Chromium have been resolved. I have added the Chromium failing targets to the CQ for this commit and they have succeeded. Original change's description: > [Adaptation] Remove QualityScalerResource when disabled. > > Bug: webrtc:11843 > Change-Id: I2d3e40356c266f189db0242f3c7590e6d83e4456 > Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181369 > Commit-Queue: Evan Shrubsole <eshr@google.com> > Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> > Cr-Commit-Position: refs/heads/master@{#31924} Bug: webrtc:11843 Change-Id: I228331293060ef996f1dd7f8e18d52b0818f526b Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182080 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31996}
2020-08-25 13:12:28 +02:00
RTC_LOG(INFO) << "Registered resource \"" << resource->Name() << "\".";
[Adaptation] Introducing call/adaptation/ResourceAdaptationProcessor. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing This gets to the heart of unblocking call-level adaptation, largely made possible due to the previous CLs in the chain. The parts of the code that are responsible for responding to resource usage signals, obtaining adaptations and applying them are moved to ResourceAdaptationProcessor in call/adaptation/. The parts of the code that are responsible for managing VideoStreamEncoder-specific resources stay inside the VideoStreamEncoderResourceManager class in video/adaptation/. After this CL lands it should soon be possible to move the Processor over to a separate task queue and let the Manager stay on the encoder queue if PostTasks are added for communication between the two objects. Bug: webrtc:11172 Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31105}
2020-04-17 18:36:19 +02:00
}
[Adaptation] Adding adaptation resources from Call. This CL adds AddAdaptationResource to Call and AddAdaptationResource/GetAdaptationResources method to relevant VideoSendStream and VideoStreamEncoder interfaces and implementations. Unittests are added to ensure that resources can be added to the Call both before and after the creation of a VideoSendStream and that the resources always gets added to the streams. In a follow-up CL, we will continue to plumb the resources all the way to PeerConnectionInterface, and an integration test will then be added to ensure that injected resources are capable of triggering adaptation. Bug: webrtc:11525 Change-Id: I499e9c23c3e359df943414d420b2e0ce2e9b2d56 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/177002 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Stefan Holmer <stefan@webrtc.org> Commit-Queue: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31499}
2020-06-11 12:07:14 +02:00
std::vector<rtc::scoped_refptr<Resource>>
ResourceAdaptationProcessor::GetResources() const {
[Adaptation] Make ResourceAdaptationProcessorInterface resources thread-safe This is one less dependency on the task queue, and will make things like removing resources and cleanup much easier in the future. Bug: webrtc:11754 Change-Id: I732f1935d1b58ffe09ca2a2bf59beebc1930214d Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178869 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31686}
2020-07-09 13:35:33 +02:00
MutexLock crit(&resources_lock_);
[Adaptation] Adding adaptation resources from Call. This CL adds AddAdaptationResource to Call and AddAdaptationResource/GetAdaptationResources method to relevant VideoSendStream and VideoStreamEncoder interfaces and implementations. Unittests are added to ensure that resources can be added to the Call both before and after the creation of a VideoSendStream and that the resources always gets added to the streams. In a follow-up CL, we will continue to plumb the resources all the way to PeerConnectionInterface, and an integration test will then be added to ensure that injected resources are capable of triggering adaptation. Bug: webrtc:11525 Change-Id: I499e9c23c3e359df943414d420b2e0ce2e9b2d56 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/177002 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Stefan Holmer <stefan@webrtc.org> Commit-Queue: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31499}
2020-06-11 12:07:14 +02:00
return resources_;
}
[Adaptation] Make Resources reference counted and add more DCHECKs. In a future CL, adaptation processing and stream encoder resource management will happen on different task queues. When this is the case, asynchronous tasks will be posted in both directions and some resources will have internal states used on multiple threads. This CL makes the Resource class reference counted in order to support posting tasks to a different threads without risk of use-after-free when a posted task is executed with a delay. This is preferred over WeakPtr strategies because WeakPtrs are single-threaded and preferred over raw pointer usage because the reference counted approach enables more compile-time and run-time assurance. This is also "future proof"; when resources can be injected through public APIs, ownership needs to be shared between libwebrtc and the application (e.g. Chrome). To reduce the risk of making mistakes in the future CL, sequence checkers and task queue DCHECKs are added as well as other DCHECKs to make sure things have been cleaned up before destruction, e.g: - Processor gets a sequence checker. It is entirely single-threaded. - Processor must not have any attached listeners or resources on destruction. - Resources must not have any listeners on destruction. - The Manager, EncodeUsageResource and QualityScalerResource DCHECKs they are running on the encoder queue. - TODOs are added illustrating where we want to add PostTasks in the future CL. Lastly, upon VideoStreamEncoder::Stop() we delete the ResourceAdaptationProcessor. Because the Processor is already used in posted tasks, some if statements are added to ensure the Processor is not used after destruction. Bug: webrtc:11542, webrtc:11520 Change-Id: Ibaa8a61d86d87a71f477d1075a117c28d9d2d285 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174760 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31217}
2020-05-11 16:29:22 +02:00
void ResourceAdaptationProcessor::RemoveResource(
rtc::scoped_refptr<Resource> resource) {
[Adaptation] Implement remove for most limited resources. When a resource is removed from the ResourceAdaptationProcessor, and the resource is the most limited resource, we reset the current adaptation to the next most limited resource level. In the case that this resource is the only adapted resource then we reset all adaptations. Bug: webrtc:11636 Change-Id: I29acc5a3934f42f00db79b3bb2171156b1313937 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176406 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31520}
2020-06-12 16:00:03 +02:00
RTC_DCHECK(resource);
RTC_LOG(INFO) << "Removing resource \"" << resource->Name() << "\".";
[Adaptation] Make ResourceAdaptationProcessorInterface resources thread-safe This is one less dependency on the task queue, and will make things like removing resources and cleanup much easier in the future. Bug: webrtc:11754 Change-Id: I732f1935d1b58ffe09ca2a2bf59beebc1930214d Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178869 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31686}
2020-07-09 13:35:33 +02:00
resource->SetResourceListener(nullptr);
{
MutexLock crit(&resources_lock_);
auto it = absl::c_find(resources_, resource);
RTC_DCHECK(it != resources_.end()) << "Resource \"" << resource->Name()
<< "\" was not a registered resource.";
resources_.erase(it);
}
RemoveLimitationsImposedByResource(std::move(resource));
}
void ResourceAdaptationProcessor::RemoveLimitationsImposedByResource(
rtc::scoped_refptr<Resource> resource) {
[Adaptation] Remove resource adaptation queue Resource adaptation needs refactoring for async adaptations. For now the resource adaptation processor can work on the encoder thread, until it is refactored to support async adaptation. Bug: webrtc:11867 Change-Id: I9c46da356db19c0fd52748c999ccb216f2ca923b Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182040 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31991}
2020-08-25 13:12:12 +02:00
if (!task_queue_->IsCurrent()) {
task_queue_->PostTask(ToQueuedTask(
[Adaptation] Make ResourceAdaptationProcessorInterface resources thread-safe This is one less dependency on the task queue, and will make things like removing resources and cleanup much easier in the future. Bug: webrtc:11754 Change-Id: I732f1935d1b58ffe09ca2a2bf59beebc1930214d Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178869 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31686}
2020-07-09 13:35:33 +02:00
[this, resource]() { RemoveLimitationsImposedByResource(resource); }));
return;
}
[Adaptation] Remove resource adaptation queue Resource adaptation needs refactoring for async adaptations. For now the resource adaptation processor can work on the encoder thread, until it is refactored to support async adaptation. Bug: webrtc:11867 Change-Id: I9c46da356db19c0fd52748c999ccb216f2ca923b Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182040 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31991}
2020-08-25 13:12:12 +02:00
RTC_DCHECK_RUN_ON(task_queue_);
[Adaptation] Implement remove for most limited resources. When a resource is removed from the ResourceAdaptationProcessor, and the resource is the most limited resource, we reset the current adaptation to the next most limited resource level. In the case that this resource is the only adapted resource then we reset all adaptations. Bug: webrtc:11636 Change-Id: I29acc5a3934f42f00db79b3bb2171156b1313937 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176406 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31520}
2020-06-12 16:00:03 +02:00
auto resource_adaptation_limits =
adaptation_limits_by_resources_.find(resource);
if (resource_adaptation_limits != adaptation_limits_by_resources_.end()) {
VideoStreamAdapter::RestrictionsWithCounters adaptation_limits =
resource_adaptation_limits->second;
adaptation_limits_by_resources_.erase(resource_adaptation_limits);
[Adaptation] Make ResourceAdaptationProcessorInterface resources thread-safe This is one less dependency on the task queue, and will make things like removing resources and cleanup much easier in the future. Bug: webrtc:11754 Change-Id: I732f1935d1b58ffe09ca2a2bf59beebc1930214d Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178869 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31686}
2020-07-09 13:35:33 +02:00
if (adaptation_limits_by_resources_.empty()) {
// Only the resource being removed was adapted so clear restrictions.
stream_adapter_->ClearRestrictions();
return;
}
VideoStreamAdapter::RestrictionsWithCounters most_limited =
FindMostLimitedResources().second;
if (adaptation_limits.counters.Total() <= most_limited.counters.Total()) {
// The removed limitations were less limited than the most limited
// resource. Don't change the current restrictions.
return;
}
// Apply the new most limited resource as the next restrictions.
Adaptation adapt_to = stream_adapter_->GetAdaptationTo(
most_limited.counters, most_limited.restrictions);
RTC_DCHECK_EQ(adapt_to.status(), Adaptation::Status::kValid);
stream_adapter_->ApplyAdaptation(adapt_to, nullptr);
RTC_LOG(INFO) << "Most limited resource removed. Restoring restrictions to "
"next most limited restrictions: "
<< most_limited.restrictions.ToString() << " with counters "
<< most_limited.counters.ToString();
[Adaptation] Implement remove for most limited resources. When a resource is removed from the ResourceAdaptationProcessor, and the resource is the most limited resource, we reset the current adaptation to the next most limited resource level. In the case that this resource is the only adapted resource then we reset all adaptations. Bug: webrtc:11636 Change-Id: I29acc5a3934f42f00db79b3bb2171156b1313937 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176406 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31520}
2020-06-12 16:00:03 +02:00
}
[Adaptation] Make Resources reference counted and add more DCHECKs. In a future CL, adaptation processing and stream encoder resource management will happen on different task queues. When this is the case, asynchronous tasks will be posted in both directions and some resources will have internal states used on multiple threads. This CL makes the Resource class reference counted in order to support posting tasks to a different threads without risk of use-after-free when a posted task is executed with a delay. This is preferred over WeakPtr strategies because WeakPtrs are single-threaded and preferred over raw pointer usage because the reference counted approach enables more compile-time and run-time assurance. This is also "future proof"; when resources can be injected through public APIs, ownership needs to be shared between libwebrtc and the application (e.g. Chrome). To reduce the risk of making mistakes in the future CL, sequence checkers and task queue DCHECKs are added as well as other DCHECKs to make sure things have been cleaned up before destruction, e.g: - Processor gets a sequence checker. It is entirely single-threaded. - Processor must not have any attached listeners or resources on destruction. - Resources must not have any listeners on destruction. - The Manager, EncodeUsageResource and QualityScalerResource DCHECKs they are running on the encoder queue. - TODOs are added illustrating where we want to add PostTasks in the future CL. Lastly, upon VideoStreamEncoder::Stop() we delete the ResourceAdaptationProcessor. Because the Processor is already used in posted tasks, some if statements are added to ensure the Processor is not used after destruction. Bug: webrtc:11542, webrtc:11520 Change-Id: Ibaa8a61d86d87a71f477d1075a117c28d9d2d285 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174760 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31217}
2020-05-11 16:29:22 +02:00
}
[Adaptation] Add OnAdaptationApplied(), remove ResourceListenerResponse. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing The ResourceListenerResponse was used to make the QualityScaler not clear QP samples and instead increase its frequency of checking for QP under certain circumstances, see enum description: https://webrtc.googlesource.com/src.git/+/c70b1028d47c1aee4892545190cd66e97d09cd55/call/adaptation/resource.h#33 Because the QualityScaler depends on whether and how adaptation happened it should listen to adaptation happening. This CL moves the logic that was previously in VideoStreamAdapter closer to the QualityScaler: QualityScalerResource::OnAdaptationApplied(). This would allow the VideoStreamAdapter to operate on a separate task queue in the future, with no dependencies on any stream-specific resources that might operate on other task queues. Bug: webrtc:11172, webrtc:11521 Change-Id: I07971a8a5fab5715f4ccb7d2c63f1b92bd47170f Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173090 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31143}
2020-04-28 12:24:33 +02:00
void ResourceAdaptationProcessor::OnResourceUsageStateMeasured(
[Adaptation] Remove Resource::UsageState() and ClearUsageState(). ResourceListener::OnResourceUsageStateMeasured() now takes ResourceUsageState as argument, making Resource::UsageState() superfluous. With the existing "fire-and-forget" behavior of always clearing usage state on reacting to a signal, there is no longer a need to call ClearUsageState() so this too is removed. (We may want to have a callback in the future to hint to the Resource that it is a good idea to clear internal measurement samples, i.e. because the load of the system is about to change, but we can revisit that when we need it.) Moving the usage state to the callback has the benefit of getting rid of the assumption that UsageState() has to return the same value every time it is called in the same task. This CL is also the final nail in the coffin for Resource needing to know about the adaptation task queue: ResourceAdaptationProcessor's ResourceListener now takes care of posting to the adaptation task queue. To support this, the processor's SequenceChecker is replaced by a TaskQueueBase pointer. Bug: webrtc:11525, webrtc:11618 Change-Id: I2277e71cc3759c85b62465020935603f03792c94 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176376 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31416}
2020-06-03 09:21:34 +02:00
rtc::scoped_refptr<Resource> resource,
ResourceUsageState usage_state) {
[Adaptation] Remove resource adaptation queue Resource adaptation needs refactoring for async adaptations. For now the resource adaptation processor can work on the encoder thread, until it is refactored to support async adaptation. Bug: webrtc:11867 Change-Id: I9c46da356db19c0fd52748c999ccb216f2ca923b Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182040 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31991}
2020-08-25 13:12:12 +02:00
RTC_DCHECK_RUN_ON(task_queue_);
[Adaptation] Implement remove for most limited resources. When a resource is removed from the ResourceAdaptationProcessor, and the resource is the most limited resource, we reset the current adaptation to the next most limited resource level. In the case that this resource is the only adapted resource then we reset all adaptations. Bug: webrtc:11636 Change-Id: I29acc5a3934f42f00db79b3bb2171156b1313937 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176406 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31520}
2020-06-12 16:00:03 +02:00
RTC_DCHECK(resource);
// |resource| could have been removed after signalling.
[Adaptation] Make ResourceAdaptationProcessorInterface resources thread-safe This is one less dependency on the task queue, and will make things like removing resources and cleanup much easier in the future. Bug: webrtc:11754 Change-Id: I732f1935d1b58ffe09ca2a2bf59beebc1930214d Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178869 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31686}
2020-07-09 13:35:33 +02:00
{
MutexLock crit(&resources_lock_);
if (absl::c_find(resources_, resource) == resources_.end()) {
RTC_LOG(INFO) << "Ignoring signal from removed resource \""
<< resource->Name() << "\".";
return;
}
[Adaptation] Implement remove for most limited resources. When a resource is removed from the ResourceAdaptationProcessor, and the resource is the most limited resource, we reset the current adaptation to the next most limited resource level. In the case that this resource is the only adapted resource then we reset all adaptations. Bug: webrtc:11636 Change-Id: I29acc5a3934f42f00db79b3bb2171156b1313937 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176406 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31520}
2020-06-12 16:00:03 +02:00
}
[Adaptation] Add more ResourceAdaptationProcessor logging. This should help debugging when adaptation is or is not happening unexpectedly. Log spam is prevented by not logging if the same result happened to the same resource already and we haven't adapted since then. Bug: webrtc:11616 Change-Id: Ia6c5cc35061d252f1c66f2f2bf3b94d2485498d6 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176221 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Harald Alvestrand <hta@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31378}
2020-05-28 16:22:42 +02:00
MitigationResultAndLogMessage result_and_message;
switch (usage_state) {
[Adaptation] Introducing call/adaptation/ResourceAdaptationProcessor. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing This gets to the heart of unblocking call-level adaptation, largely made possible due to the previous CLs in the chain. The parts of the code that are responsible for responding to resource usage signals, obtaining adaptations and applying them are moved to ResourceAdaptationProcessor in call/adaptation/. The parts of the code that are responsible for managing VideoStreamEncoder-specific resources stay inside the VideoStreamEncoderResourceManager class in video/adaptation/. After this CL lands it should soon be possible to move the Processor over to a separate task queue and let the Manager stay on the encoder queue if PostTasks are added for communication between the two objects. Bug: webrtc:11172 Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31105}
2020-04-17 18:36:19 +02:00
case ResourceUsageState::kOveruse:
[Adaptation] Add more ResourceAdaptationProcessor logging. This should help debugging when adaptation is or is not happening unexpectedly. Log spam is prevented by not logging if the same result happened to the same resource already and we haven't adapted since then. Bug: webrtc:11616 Change-Id: Ia6c5cc35061d252f1c66f2f2bf3b94d2485498d6 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176221 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Harald Alvestrand <hta@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31378}
2020-05-28 16:22:42 +02:00
result_and_message = OnResourceOveruse(resource);
[Adaptation] Add OnAdaptationApplied(), remove ResourceListenerResponse. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing The ResourceListenerResponse was used to make the QualityScaler not clear QP samples and instead increase its frequency of checking for QP under certain circumstances, see enum description: https://webrtc.googlesource.com/src.git/+/c70b1028d47c1aee4892545190cd66e97d09cd55/call/adaptation/resource.h#33 Because the QualityScaler depends on whether and how adaptation happened it should listen to adaptation happening. This CL moves the logic that was previously in VideoStreamAdapter closer to the QualityScaler: QualityScalerResource::OnAdaptationApplied(). This would allow the VideoStreamAdapter to operate on a separate task queue in the future, with no dependencies on any stream-specific resources that might operate on other task queues. Bug: webrtc:11172, webrtc:11521 Change-Id: I07971a8a5fab5715f4ccb7d2c63f1b92bd47170f Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173090 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31143}
2020-04-28 12:24:33 +02:00
break;
[Adaptation] Introducing call/adaptation/ResourceAdaptationProcessor. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing This gets to the heart of unblocking call-level adaptation, largely made possible due to the previous CLs in the chain. The parts of the code that are responsible for responding to resource usage signals, obtaining adaptations and applying them are moved to ResourceAdaptationProcessor in call/adaptation/. The parts of the code that are responsible for managing VideoStreamEncoder-specific resources stay inside the VideoStreamEncoderResourceManager class in video/adaptation/. After this CL lands it should soon be possible to move the Processor over to a separate task queue and let the Manager stay on the encoder queue if PostTasks are added for communication between the two objects. Bug: webrtc:11172 Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31105}
2020-04-17 18:36:19 +02:00
case ResourceUsageState::kUnderuse:
[Adaptation] Add more ResourceAdaptationProcessor logging. This should help debugging when adaptation is or is not happening unexpectedly. Log spam is prevented by not logging if the same result happened to the same resource already and we haven't adapted since then. Bug: webrtc:11616 Change-Id: Ia6c5cc35061d252f1c66f2f2bf3b94d2485498d6 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176221 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Harald Alvestrand <hta@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31378}
2020-05-28 16:22:42 +02:00
result_and_message = OnResourceUnderuse(resource);
[Adaptation] Add OnAdaptationApplied(), remove ResourceListenerResponse. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing The ResourceListenerResponse was used to make the QualityScaler not clear QP samples and instead increase its frequency of checking for QP under certain circumstances, see enum description: https://webrtc.googlesource.com/src.git/+/c70b1028d47c1aee4892545190cd66e97d09cd55/call/adaptation/resource.h#33 Because the QualityScaler depends on whether and how adaptation happened it should listen to adaptation happening. This CL moves the logic that was previously in VideoStreamAdapter closer to the QualityScaler: QualityScalerResource::OnAdaptationApplied(). This would allow the VideoStreamAdapter to operate on a separate task queue in the future, with no dependencies on any stream-specific resources that might operate on other task queues. Bug: webrtc:11172, webrtc:11521 Change-Id: I07971a8a5fab5715f4ccb7d2c63f1b92bd47170f Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173090 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31143}
2020-04-28 12:24:33 +02:00
break;
[Adaptation] Introducing call/adaptation/ResourceAdaptationProcessor. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing This gets to the heart of unblocking call-level adaptation, largely made possible due to the previous CLs in the chain. The parts of the code that are responsible for responding to resource usage signals, obtaining adaptations and applying them are moved to ResourceAdaptationProcessor in call/adaptation/. The parts of the code that are responsible for managing VideoStreamEncoder-specific resources stay inside the VideoStreamEncoderResourceManager class in video/adaptation/. After this CL lands it should soon be possible to move the Processor over to a separate task queue and let the Manager stay on the encoder queue if PostTasks are added for communication between the two objects. Bug: webrtc:11172 Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31105}
2020-04-17 18:36:19 +02:00
}
[Adaptation] Add more ResourceAdaptationProcessor logging. This should help debugging when adaptation is or is not happening unexpectedly. Log spam is prevented by not logging if the same result happened to the same resource already and we haven't adapted since then. Bug: webrtc:11616 Change-Id: Ia6c5cc35061d252f1c66f2f2bf3b94d2485498d6 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176221 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Harald Alvestrand <hta@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31378}
2020-05-28 16:22:42 +02:00
// Maybe log the result of the operation.
auto it = previous_mitigation_results_.find(resource.get());
if (it != previous_mitigation_results_.end() &&
it->second == result_and_message.result) {
// This resource has previously reported the same result and we haven't
// successfully adapted since - don't log to avoid spam.
return;
}
[Adaptation] Prep Resource for api/ move. Introduce VSE-Resource. This CL is in preparation for moving Resource to the api/ folder. It does not move it, but makes it such that the moving CL can be a pure move. In order to do this, we must stop depending on rtc_base/rtc::TaskQueue in favor of api/webrtc::TaskQueueBase. There are also other rtc_base/ dependencies that we do not want to expose to the api/ folder, like critical sections and thread annotations which are not publically exposed. To get around this, we make Resource an abstract interface and move all of the base class functionality into a new non-api/ class: VideoStreamEncoderResource. The Resource now has Register/UnregisterAdaptationTaskQueue() methods. By explicitly unregistering, we can ensure validity of the pointer even if the Resource outlives the PeerConnection. While public interface methods are only to be called on the adaptation task queue, posting to the task queue happens off-queue, so a |lock_| is introduced to guard it. Bug: webrtc:11525 Change-Id: I50b3a30960cdec9032016c779b47001c01dad32f Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176320 Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Commit-Queue: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31402}
2020-06-01 17:59:05 +02:00
RTC_LOG(INFO) << "Resource \"" << resource->Name() << "\" signalled "
[Adaptation] Add more ResourceAdaptationProcessor logging. This should help debugging when adaptation is or is not happening unexpectedly. Log spam is prevented by not logging if the same result happened to the same resource already and we haven't adapted since then. Bug: webrtc:11616 Change-Id: Ia6c5cc35061d252f1c66f2f2bf3b94d2485498d6 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176221 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Harald Alvestrand <hta@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31378}
2020-05-28 16:22:42 +02:00
<< ResourceUsageStateToString(usage_state) << ". "
<< result_and_message.message;
if (result_and_message.result == MitigationResult::kAdaptationApplied) {
previous_mitigation_results_.clear();
} else {
previous_mitigation_results_.insert(
std::make_pair(resource.get(), result_and_message.result));
}
[Adaptation] Introducing call/adaptation/ResourceAdaptationProcessor. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing This gets to the heart of unblocking call-level adaptation, largely made possible due to the previous CLs in the chain. The parts of the code that are responsible for responding to resource usage signals, obtaining adaptations and applying them are moved to ResourceAdaptationProcessor in call/adaptation/. The parts of the code that are responsible for managing VideoStreamEncoder-specific resources stay inside the VideoStreamEncoderResourceManager class in video/adaptation/. After this CL lands it should soon be possible to move the Processor over to a separate task queue and let the Manager stay on the encoder queue if PostTasks are added for communication between the two objects. Bug: webrtc:11172 Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31105}
2020-04-17 18:36:19 +02:00
}
[Adaptation] Add more ResourceAdaptationProcessor logging. This should help debugging when adaptation is or is not happening unexpectedly. Log spam is prevented by not logging if the same result happened to the same resource already and we haven't adapted since then. Bug: webrtc:11616 Change-Id: Ia6c5cc35061d252f1c66f2f2bf3b94d2485498d6 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176221 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Harald Alvestrand <hta@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31378}
2020-05-28 16:22:42 +02:00
ResourceAdaptationProcessor::MitigationResultAndLogMessage
ResourceAdaptationProcessor::OnResourceUnderuse(
[Adaptation] Make Resources reference counted and add more DCHECKs. In a future CL, adaptation processing and stream encoder resource management will happen on different task queues. When this is the case, asynchronous tasks will be posted in both directions and some resources will have internal states used on multiple threads. This CL makes the Resource class reference counted in order to support posting tasks to a different threads without risk of use-after-free when a posted task is executed with a delay. This is preferred over WeakPtr strategies because WeakPtrs are single-threaded and preferred over raw pointer usage because the reference counted approach enables more compile-time and run-time assurance. This is also "future proof"; when resources can be injected through public APIs, ownership needs to be shared between libwebrtc and the application (e.g. Chrome). To reduce the risk of making mistakes in the future CL, sequence checkers and task queue DCHECKs are added as well as other DCHECKs to make sure things have been cleaned up before destruction, e.g: - Processor gets a sequence checker. It is entirely single-threaded. - Processor must not have any attached listeners or resources on destruction. - Resources must not have any listeners on destruction. - The Manager, EncodeUsageResource and QualityScalerResource DCHECKs they are running on the encoder queue. - TODOs are added illustrating where we want to add PostTasks in the future CL. Lastly, upon VideoStreamEncoder::Stop() we delete the ResourceAdaptationProcessor. Because the Processor is already used in posted tasks, some if statements are added to ensure the Processor is not used after destruction. Bug: webrtc:11542, webrtc:11520 Change-Id: Ibaa8a61d86d87a71f477d1075a117c28d9d2d285 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174760 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31217}
2020-05-11 16:29:22 +02:00
rtc::scoped_refptr<Resource> reason_resource) {
[Adaptation] Remove resource adaptation queue Resource adaptation needs refactoring for async adaptations. For now the resource adaptation processor can work on the encoder thread, until it is refactored to support async adaptation. Bug: webrtc:11867 Change-Id: I9c46da356db19c0fd52748c999ccb216f2ca923b Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182040 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31991}
2020-08-25 13:12:12 +02:00
RTC_DCHECK_RUN_ON(task_queue_);
[Adaptation] Introducing call/adaptation/ResourceAdaptationProcessor. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing This gets to the heart of unblocking call-level adaptation, largely made possible due to the previous CLs in the chain. The parts of the code that are responsible for responding to resource usage signals, obtaining adaptations and applying them are moved to ResourceAdaptationProcessor in call/adaptation/. The parts of the code that are responsible for managing VideoStreamEncoder-specific resources stay inside the VideoStreamEncoderResourceManager class in video/adaptation/. After this CL lands it should soon be possible to move the Processor over to a separate task queue and let the Manager stay on the encoder queue if PostTasks are added for communication between the two objects. Bug: webrtc:11172 Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31105}
2020-04-17 18:36:19 +02:00
// How can this stream be adapted up?
[Adaptation] Apply AdaptationConstraints to all resources Some restrictions previously were preventing adaptation up caused by the quality resource. However, it makes sense to use the same restrictions in the case of other resources. This CL removes now unneeded wire-up of reason/resource causing adaptation. Bug: webrtc:11771 Change-Id: Iec301a59d2a41d32d23b6be340f3b5637d697e52 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181580 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31926}
2020-08-13 11:35:24 +02:00
Adaptation adaptation = stream_adapter_->GetAdaptationUp();
[Adaptation] Add OnAdaptationApplied(), remove ResourceListenerResponse. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing The ResourceListenerResponse was used to make the QualityScaler not clear QP samples and instead increase its frequency of checking for QP under certain circumstances, see enum description: https://webrtc.googlesource.com/src.git/+/c70b1028d47c1aee4892545190cd66e97d09cd55/call/adaptation/resource.h#33 Because the QualityScaler depends on whether and how adaptation happened it should listen to adaptation happening. This CL moves the logic that was previously in VideoStreamAdapter closer to the QualityScaler: QualityScalerResource::OnAdaptationApplied(). This would allow the VideoStreamAdapter to operate on a separate task queue in the future, with no dependencies on any stream-specific resources that might operate on other task queues. Bug: webrtc:11172, webrtc:11521 Change-Id: I07971a8a5fab5715f4ccb7d2c63f1b92bd47170f Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173090 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31143}
2020-04-28 12:24:33 +02:00
if (adaptation.status() != Adaptation::Status::kValid) {
[Adaptation] Add more ResourceAdaptationProcessor logging. This should help debugging when adaptation is or is not happening unexpectedly. Log spam is prevented by not logging if the same result happened to the same resource already and we haven't adapted since then. Bug: webrtc:11616 Change-Id: Ia6c5cc35061d252f1c66f2f2bf3b94d2485498d6 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176221 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Harald Alvestrand <hta@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31378}
2020-05-28 16:22:42 +02:00
rtc::StringBuilder message;
message << "Not adapting up because VideoStreamAdapter returned "
<< Adaptation::StatusToString(adaptation.status());
return MitigationResultAndLogMessage(MitigationResult::kRejectedByAdapter,
message.Release());
[Adaptation] Add OnAdaptationApplied(), remove ResourceListenerResponse. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing The ResourceListenerResponse was used to make the QualityScaler not clear QP samples and instead increase its frequency of checking for QP under certain circumstances, see enum description: https://webrtc.googlesource.com/src.git/+/c70b1028d47c1aee4892545190cd66e97d09cd55/call/adaptation/resource.h#33 Because the QualityScaler depends on whether and how adaptation happened it should listen to adaptation happening. This CL moves the logic that was previously in VideoStreamAdapter closer to the QualityScaler: QualityScalerResource::OnAdaptationApplied(). This would allow the VideoStreamAdapter to operate on a separate task queue in the future, with no dependencies on any stream-specific resources that might operate on other task queues. Bug: webrtc:11172, webrtc:11521 Change-Id: I07971a8a5fab5715f4ccb7d2c63f1b92bd47170f Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173090 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31143}
2020-04-28 12:24:33 +02:00
}
[Adaptation] Move AdaptationConstraints to VideoStreamAdapter Bug: webrtc:11754 Change-Id: Ic51059834ea61c08e977a3a6b5bba48abc86f873 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178906 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31688}
2020-07-09 13:51:39 +02:00
// Check that resource is most limited.
Only allow most limited resource to trigger adapt up A more detailed explaination is in the bug, but this changes the way that adaptation happens when multiple resources are limited. Only the one that is most limited can trigger an adaptation up. If multiple resources are most limited both need to underuse to adapt up. Some of the changes in this patch to make it all work: * VideoStreamEncoder unittests that did not reflect this new behaviour have been changed. * PeekNextRestrictions returns the adaptation counters as well as the restrictions. * Adaptation statstics have changed so that when adapting up all resources are tagged as triggering the adaptation. Additionally the statistics for the current adaptation is now the total number of adaptations per reason, rather then the number of adaptations due to that reason. * PreventAdaptUpDueToActiveCounts is removed as most limited resource is a strong implementation of that. Bug: webrtc:11553 Change-Id: If1545a201c8e019598edf82657a1befde8b05268 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176128 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31497}
2020-06-11 10:45:29 +02:00
std::vector<rtc::scoped_refptr<Resource>> most_limited_resources;
[Adaptation] Implement remove for most limited resources. When a resource is removed from the ResourceAdaptationProcessor, and the resource is the most limited resource, we reset the current adaptation to the next most limited resource level. In the case that this resource is the only adapted resource then we reset all adaptations. Bug: webrtc:11636 Change-Id: I29acc5a3934f42f00db79b3bb2171156b1313937 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176406 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31520}
2020-06-12 16:00:03 +02:00
VideoStreamAdapter::RestrictionsWithCounters most_limited_restrictions;
Only allow most limited resource to trigger adapt up A more detailed explaination is in the bug, but this changes the way that adaptation happens when multiple resources are limited. Only the one that is most limited can trigger an adaptation up. If multiple resources are most limited both need to underuse to adapt up. Some of the changes in this patch to make it all work: * VideoStreamEncoder unittests that did not reflect this new behaviour have been changed. * PeekNextRestrictions returns the adaptation counters as well as the restrictions. * Adaptation statstics have changed so that when adapting up all resources are tagged as triggering the adaptation. Additionally the statistics for the current adaptation is now the total number of adaptations per reason, rather then the number of adaptations due to that reason. * PreventAdaptUpDueToActiveCounts is removed as most limited resource is a strong implementation of that. Bug: webrtc:11553 Change-Id: If1545a201c8e019598edf82657a1befde8b05268 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176128 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31497}
2020-06-11 10:45:29 +02:00
std::tie(most_limited_resources, most_limited_restrictions) =
FindMostLimitedResources();
// If the most restricted resource is less limited than current restrictions
// then proceed with adapting up.
[Adaptation] Implement remove for most limited resources. When a resource is removed from the ResourceAdaptationProcessor, and the resource is the most limited resource, we reset the current adaptation to the next most limited resource level. In the case that this resource is the only adapted resource then we reset all adaptations. Bug: webrtc:11636 Change-Id: I29acc5a3934f42f00db79b3bb2171156b1313937 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176406 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31520}
2020-06-12 16:00:03 +02:00
if (!most_limited_resources.empty() &&
[Adaptation] Adaptation holds restrictions This refactoring enables us to do multi-step adaptations instead of the normal single step adaptations. This work is required pre-requisite work to remove the DropDueToSize methods from the ResourceAdaptationProcessorInterface. This work also gives a path to fixing webrtc:11719, since double adaptations can be done in a controlled manner with this API. Bug: webrtc:11700,webrtc:11719 Change-Id: Ic7d27a49c82d0cf7ebaf79d7bdf0bc6f382770ed Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178200 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31631}
2020-07-03 15:16:43 +02:00
most_limited_restrictions.counters.Total() >=
[Adaptation] Implement remove for most limited resources. When a resource is removed from the ResourceAdaptationProcessor, and the resource is the most limited resource, we reset the current adaptation to the next most limited resource level. In the case that this resource is the only adapted resource then we reset all adaptations. Bug: webrtc:11636 Change-Id: I29acc5a3934f42f00db79b3bb2171156b1313937 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176406 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31520}
2020-06-12 16:00:03 +02:00
stream_adapter_->adaptation_counters().Total()) {
Only allow most limited resource to trigger adapt up A more detailed explaination is in the bug, but this changes the way that adaptation happens when multiple resources are limited. Only the one that is most limited can trigger an adaptation up. If multiple resources are most limited both need to underuse to adapt up. Some of the changes in this patch to make it all work: * VideoStreamEncoder unittests that did not reflect this new behaviour have been changed. * PeekNextRestrictions returns the adaptation counters as well as the restrictions. * Adaptation statstics have changed so that when adapting up all resources are tagged as triggering the adaptation. Additionally the statistics for the current adaptation is now the total number of adaptations per reason, rather then the number of adaptations due to that reason. * PreventAdaptUpDueToActiveCounts is removed as most limited resource is a strong implementation of that. Bug: webrtc:11553 Change-Id: If1545a201c8e019598edf82657a1befde8b05268 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176128 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31497}
2020-06-11 10:45:29 +02:00
// If |reason_resource| is not one of the most limiting resources then abort
// adaptation.
if (absl::c_find(most_limited_resources, reason_resource) ==
most_limited_resources.end()) {
rtc::StringBuilder message;
message << "Resource \"" << reason_resource->Name()
<< "\" was not the most limited resource.";
return MitigationResultAndLogMessage(
MitigationResult::kNotMostLimitedResource, message.Release());
}
if (most_limited_resources.size() > 1) {
// If there are multiple most limited resources, all must signal underuse
// before the adaptation is applied.
[Adaptation] Adaptation holds restrictions This refactoring enables us to do multi-step adaptations instead of the normal single step adaptations. This work is required pre-requisite work to remove the DropDueToSize methods from the ResourceAdaptationProcessorInterface. This work also gives a path to fixing webrtc:11719, since double adaptations can be done in a controlled manner with this API. Bug: webrtc:11700,webrtc:11719 Change-Id: Ic7d27a49c82d0cf7ebaf79d7bdf0bc6f382770ed Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178200 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31631}
2020-07-03 15:16:43 +02:00
UpdateResourceLimitations(reason_resource, adaptation.restrictions(),
adaptation.counters());
Only allow most limited resource to trigger adapt up A more detailed explaination is in the bug, but this changes the way that adaptation happens when multiple resources are limited. Only the one that is most limited can trigger an adaptation up. If multiple resources are most limited both need to underuse to adapt up. Some of the changes in this patch to make it all work: * VideoStreamEncoder unittests that did not reflect this new behaviour have been changed. * PeekNextRestrictions returns the adaptation counters as well as the restrictions. * Adaptation statstics have changed so that when adapting up all resources are tagged as triggering the adaptation. Additionally the statistics for the current adaptation is now the total number of adaptations per reason, rather then the number of adaptations due to that reason. * PreventAdaptUpDueToActiveCounts is removed as most limited resource is a strong implementation of that. Bug: webrtc:11553 Change-Id: If1545a201c8e019598edf82657a1befde8b05268 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176128 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31497}
2020-06-11 10:45:29 +02:00
rtc::StringBuilder message;
message << "Resource \"" << reason_resource->Name()
<< "\" was not the only most limited resource.";
return MitigationResultAndLogMessage(
MitigationResult::kSharedMostLimitedResource, message.Release());
}
}
[Adaptation] Introducing call/adaptation/ResourceAdaptationProcessor. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing This gets to the heart of unblocking call-level adaptation, largely made possible due to the previous CLs in the chain. The parts of the code that are responsible for responding to resource usage signals, obtaining adaptations and applying them are moved to ResourceAdaptationProcessor in call/adaptation/. The parts of the code that are responsible for managing VideoStreamEncoder-specific resources stay inside the VideoStreamEncoderResourceManager class in video/adaptation/. After this CL lands it should soon be possible to move the Processor over to a separate task queue and let the Manager stay on the encoder queue if PostTasks are added for communication between the two objects. Bug: webrtc:11172 Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31105}
2020-04-17 18:36:19 +02:00
// Apply adaptation.
[Adaptation] VideoStreamAdapter broadcasts adaptations This moves this responsibility from the ResourceAdaptaitonProcessor to the VideoStreamAdapter. A new interface for listening to adaptation changes was added, and the ResourceAdaptationProcessor now listens on the VideoStreamAdapter for those changes. This means that going forward, 1. We can do adaptations outside of resource limitations, like setting prior adaptations on a resource like initial frame dropper is designed to. 2. Adaptations can be on a different queue than the ResourceAdaptaitonProcessor's queue since updates are pushed through the listener. Bug: webrtc:11700 Change-Id: I6de0dec748dba095e702f0b9893c5afa50b51aa9 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176859 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31615}
2020-07-01 11:47:46 +02:00
stream_adapter_->ApplyAdaptation(adaptation, reason_resource);
[Adaptation] Add more ResourceAdaptationProcessor logging. This should help debugging when adaptation is or is not happening unexpectedly. Log spam is prevented by not logging if the same result happened to the same resource already and we haven't adapted since then. Bug: webrtc:11616 Change-Id: Ia6c5cc35061d252f1c66f2f2bf3b94d2485498d6 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176221 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Harald Alvestrand <hta@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31378}
2020-05-28 16:22:42 +02:00
rtc::StringBuilder message;
message << "Adapted up successfully. Unfiltered adaptations: "
<< stream_adapter_->adaptation_counters().ToString();
return MitigationResultAndLogMessage(MitigationResult::kAdaptationApplied,
message.Release());
[Adaptation] Introducing call/adaptation/ResourceAdaptationProcessor. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing This gets to the heart of unblocking call-level adaptation, largely made possible due to the previous CLs in the chain. The parts of the code that are responsible for responding to resource usage signals, obtaining adaptations and applying them are moved to ResourceAdaptationProcessor in call/adaptation/. The parts of the code that are responsible for managing VideoStreamEncoder-specific resources stay inside the VideoStreamEncoderResourceManager class in video/adaptation/. After this CL lands it should soon be possible to move the Processor over to a separate task queue and let the Manager stay on the encoder queue if PostTasks are added for communication between the two objects. Bug: webrtc:11172 Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31105}
2020-04-17 18:36:19 +02:00
}
[Adaptation] Add more ResourceAdaptationProcessor logging. This should help debugging when adaptation is or is not happening unexpectedly. Log spam is prevented by not logging if the same result happened to the same resource already and we haven't adapted since then. Bug: webrtc:11616 Change-Id: Ia6c5cc35061d252f1c66f2f2bf3b94d2485498d6 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176221 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Harald Alvestrand <hta@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31378}
2020-05-28 16:22:42 +02:00
ResourceAdaptationProcessor::MitigationResultAndLogMessage
ResourceAdaptationProcessor::OnResourceOveruse(
[Adaptation] Make Resources reference counted and add more DCHECKs. In a future CL, adaptation processing and stream encoder resource management will happen on different task queues. When this is the case, asynchronous tasks will be posted in both directions and some resources will have internal states used on multiple threads. This CL makes the Resource class reference counted in order to support posting tasks to a different threads without risk of use-after-free when a posted task is executed with a delay. This is preferred over WeakPtr strategies because WeakPtrs are single-threaded and preferred over raw pointer usage because the reference counted approach enables more compile-time and run-time assurance. This is also "future proof"; when resources can be injected through public APIs, ownership needs to be shared between libwebrtc and the application (e.g. Chrome). To reduce the risk of making mistakes in the future CL, sequence checkers and task queue DCHECKs are added as well as other DCHECKs to make sure things have been cleaned up before destruction, e.g: - Processor gets a sequence checker. It is entirely single-threaded. - Processor must not have any attached listeners or resources on destruction. - Resources must not have any listeners on destruction. - The Manager, EncodeUsageResource and QualityScalerResource DCHECKs they are running on the encoder queue. - TODOs are added illustrating where we want to add PostTasks in the future CL. Lastly, upon VideoStreamEncoder::Stop() we delete the ResourceAdaptationProcessor. Because the Processor is already used in posted tasks, some if statements are added to ensure the Processor is not used after destruction. Bug: webrtc:11542, webrtc:11520 Change-Id: Ibaa8a61d86d87a71f477d1075a117c28d9d2d285 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174760 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31217}
2020-05-11 16:29:22 +02:00
rtc::scoped_refptr<Resource> reason_resource) {
[Adaptation] Remove resource adaptation queue Resource adaptation needs refactoring for async adaptations. For now the resource adaptation processor can work on the encoder thread, until it is refactored to support async adaptation. Bug: webrtc:11867 Change-Id: I9c46da356db19c0fd52748c999ccb216f2ca923b Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182040 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31991}
2020-08-25 13:12:12 +02:00
RTC_DCHECK_RUN_ON(task_queue_);
[Adaptation] Introducing call/adaptation/ResourceAdaptationProcessor. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing This gets to the heart of unblocking call-level adaptation, largely made possible due to the previous CLs in the chain. The parts of the code that are responsible for responding to resource usage signals, obtaining adaptations and applying them are moved to ResourceAdaptationProcessor in call/adaptation/. The parts of the code that are responsible for managing VideoStreamEncoder-specific resources stay inside the VideoStreamEncoderResourceManager class in video/adaptation/. After this CL lands it should soon be possible to move the Processor over to a separate task queue and let the Manager stay on the encoder queue if PostTasks are added for communication between the two objects. Bug: webrtc:11172 Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31105}
2020-04-17 18:36:19 +02:00
// How can this stream be adapted up?
Adaptation adaptation = stream_adapter_->GetAdaptationDown();
[Adaptation] Make resource most limited if kLimitReached hit This occurs when a resource causes an adaptation down but the current adaptations can not be adapted any more. Any further adaptation will result in the status kLimitReached, and so any resource that adapts down should also be most limited. Bug: webrtc:11695 Change-Id: Idfdf23f482b1b4a132cec49a9be76adc0aec4361 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181586 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31933}
2020-08-14 10:08:54 +02:00
if (adaptation.status() == Adaptation::Status::kLimitReached) {
// Add resource as most limited.
VideoStreamAdapter::RestrictionsWithCounters restrictions;
std::tie(std::ignore, restrictions) = FindMostLimitedResources();
UpdateResourceLimitations(reason_resource, restrictions.restrictions,
restrictions.counters);
}
[Adaptation] Add OnAdaptationApplied(), remove ResourceListenerResponse. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing The ResourceListenerResponse was used to make the QualityScaler not clear QP samples and instead increase its frequency of checking for QP under certain circumstances, see enum description: https://webrtc.googlesource.com/src.git/+/c70b1028d47c1aee4892545190cd66e97d09cd55/call/adaptation/resource.h#33 Because the QualityScaler depends on whether and how adaptation happened it should listen to adaptation happening. This CL moves the logic that was previously in VideoStreamAdapter closer to the QualityScaler: QualityScalerResource::OnAdaptationApplied(). This would allow the VideoStreamAdapter to operate on a separate task queue in the future, with no dependencies on any stream-specific resources that might operate on other task queues. Bug: webrtc:11172, webrtc:11521 Change-Id: I07971a8a5fab5715f4ccb7d2c63f1b92bd47170f Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173090 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31143}
2020-04-28 12:24:33 +02:00
if (adaptation.status() != Adaptation::Status::kValid) {
[Adaptation] Add more ResourceAdaptationProcessor logging. This should help debugging when adaptation is or is not happening unexpectedly. Log spam is prevented by not logging if the same result happened to the same resource already and we haven't adapted since then. Bug: webrtc:11616 Change-Id: Ia6c5cc35061d252f1c66f2f2bf3b94d2485498d6 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176221 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Harald Alvestrand <hta@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31378}
2020-05-28 16:22:42 +02:00
rtc::StringBuilder message;
message << "Not adapting down because VideoStreamAdapter returned "
<< Adaptation::StatusToString(adaptation.status());
return MitigationResultAndLogMessage(MitigationResult::kRejectedByAdapter,
message.Release());
[Adaptation] Add OnAdaptationApplied(), remove ResourceListenerResponse. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing The ResourceListenerResponse was used to make the QualityScaler not clear QP samples and instead increase its frequency of checking for QP under certain circumstances, see enum description: https://webrtc.googlesource.com/src.git/+/c70b1028d47c1aee4892545190cd66e97d09cd55/call/adaptation/resource.h#33 Because the QualityScaler depends on whether and how adaptation happened it should listen to adaptation happening. This CL moves the logic that was previously in VideoStreamAdapter closer to the QualityScaler: QualityScalerResource::OnAdaptationApplied(). This would allow the VideoStreamAdapter to operate on a separate task queue in the future, with no dependencies on any stream-specific resources that might operate on other task queues. Bug: webrtc:11172, webrtc:11521 Change-Id: I07971a8a5fab5715f4ccb7d2c63f1b92bd47170f Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173090 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31143}
2020-04-28 12:24:33 +02:00
}
[Adaptation] Introducing call/adaptation/ResourceAdaptationProcessor. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing This gets to the heart of unblocking call-level adaptation, largely made possible due to the previous CLs in the chain. The parts of the code that are responsible for responding to resource usage signals, obtaining adaptations and applying them are moved to ResourceAdaptationProcessor in call/adaptation/. The parts of the code that are responsible for managing VideoStreamEncoder-specific resources stay inside the VideoStreamEncoderResourceManager class in video/adaptation/. After this CL lands it should soon be possible to move the Processor over to a separate task queue and let the Manager stay on the encoder queue if PostTasks are added for communication between the two objects. Bug: webrtc:11172 Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31105}
2020-04-17 18:36:19 +02:00
// Apply adaptation.
[Adaptation] Adaptation holds restrictions This refactoring enables us to do multi-step adaptations instead of the normal single step adaptations. This work is required pre-requisite work to remove the DropDueToSize methods from the ResourceAdaptationProcessorInterface. This work also gives a path to fixing webrtc:11719, since double adaptations can be done in a controlled manner with this API. Bug: webrtc:11700,webrtc:11719 Change-Id: Ic7d27a49c82d0cf7ebaf79d7bdf0bc6f382770ed Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178200 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31631}
2020-07-03 15:16:43 +02:00
UpdateResourceLimitations(reason_resource, adaptation.restrictions(),
adaptation.counters());
[Adaptation] Move AdaptationListeners to VideoStreamAdapter This is a step needed for multi-stream and new mitigations. It also cleans up needing to signal adaptation changes in mutiple places from ResourceAdaptationProcessor. R=hbos@webrtc.org Bug: webrtc:11754 Change-Id: Ib185dc9f66fbb4a087eb9e970c68c3f47eafb17f Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178874 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31684}
2020-07-09 11:47:10 +02:00
stream_adapter_->ApplyAdaptation(adaptation, reason_resource);
[Adaptation] Add more ResourceAdaptationProcessor logging. This should help debugging when adaptation is or is not happening unexpectedly. Log spam is prevented by not logging if the same result happened to the same resource already and we haven't adapted since then. Bug: webrtc:11616 Change-Id: Ia6c5cc35061d252f1c66f2f2bf3b94d2485498d6 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176221 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Harald Alvestrand <hta@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31378}
2020-05-28 16:22:42 +02:00
rtc::StringBuilder message;
message << "Adapted down successfully. Unfiltered adaptations: "
<< stream_adapter_->adaptation_counters().ToString();
return MitigationResultAndLogMessage(MitigationResult::kAdaptationApplied,
message.Release());
[Adaptation] Introducing call/adaptation/ResourceAdaptationProcessor. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing This gets to the heart of unblocking call-level adaptation, largely made possible due to the previous CLs in the chain. The parts of the code that are responsible for responding to resource usage signals, obtaining adaptations and applying them are moved to ResourceAdaptationProcessor in call/adaptation/. The parts of the code that are responsible for managing VideoStreamEncoder-specific resources stay inside the VideoStreamEncoderResourceManager class in video/adaptation/. After this CL lands it should soon be possible to move the Processor over to a separate task queue and let the Manager stay on the encoder queue if PostTasks are added for communication between the two objects. Bug: webrtc:11172 Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31105}
2020-04-17 18:36:19 +02:00
}
[Adaptation] Implement remove for most limited resources. When a resource is removed from the ResourceAdaptationProcessor, and the resource is the most limited resource, we reset the current adaptation to the next most limited resource level. In the case that this resource is the only adapted resource then we reset all adaptations. Bug: webrtc:11636 Change-Id: I29acc5a3934f42f00db79b3bb2171156b1313937 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176406 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31520}
2020-06-12 16:00:03 +02:00
std::pair<std::vector<rtc::scoped_refptr<Resource>>,
VideoStreamAdapter::RestrictionsWithCounters>
Only allow most limited resource to trigger adapt up A more detailed explaination is in the bug, but this changes the way that adaptation happens when multiple resources are limited. Only the one that is most limited can trigger an adaptation up. If multiple resources are most limited both need to underuse to adapt up. Some of the changes in this patch to make it all work: * VideoStreamEncoder unittests that did not reflect this new behaviour have been changed. * PeekNextRestrictions returns the adaptation counters as well as the restrictions. * Adaptation statstics have changed so that when adapting up all resources are tagged as triggering the adaptation. Additionally the statistics for the current adaptation is now the total number of adaptations per reason, rather then the number of adaptations due to that reason. * PreventAdaptUpDueToActiveCounts is removed as most limited resource is a strong implementation of that. Bug: webrtc:11553 Change-Id: If1545a201c8e019598edf82657a1befde8b05268 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176128 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31497}
2020-06-11 10:45:29 +02:00
ResourceAdaptationProcessor::FindMostLimitedResources() const {
std::vector<rtc::scoped_refptr<Resource>> most_limited_resources;
[Adaptation] Implement remove for most limited resources. When a resource is removed from the ResourceAdaptationProcessor, and the resource is the most limited resource, we reset the current adaptation to the next most limited resource level. In the case that this resource is the only adapted resource then we reset all adaptations. Bug: webrtc:11636 Change-Id: I29acc5a3934f42f00db79b3bb2171156b1313937 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176406 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31520}
2020-06-12 16:00:03 +02:00
VideoStreamAdapter::RestrictionsWithCounters most_limited_restrictions{
VideoSourceRestrictions(), VideoAdaptationCounters()};
Only allow most limited resource to trigger adapt up A more detailed explaination is in the bug, but this changes the way that adaptation happens when multiple resources are limited. Only the one that is most limited can trigger an adaptation up. If multiple resources are most limited both need to underuse to adapt up. Some of the changes in this patch to make it all work: * VideoStreamEncoder unittests that did not reflect this new behaviour have been changed. * PeekNextRestrictions returns the adaptation counters as well as the restrictions. * Adaptation statstics have changed so that when adapting up all resources are tagged as triggering the adaptation. Additionally the statistics for the current adaptation is now the total number of adaptations per reason, rather then the number of adaptations due to that reason. * PreventAdaptUpDueToActiveCounts is removed as most limited resource is a strong implementation of that. Bug: webrtc:11553 Change-Id: If1545a201c8e019598edf82657a1befde8b05268 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176128 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31497}
2020-06-11 10:45:29 +02:00
for (const auto& resource_and_adaptation_limit_ :
adaptation_limits_by_resources_) {
[Adaptation] Implement remove for most limited resources. When a resource is removed from the ResourceAdaptationProcessor, and the resource is the most limited resource, we reset the current adaptation to the next most limited resource level. In the case that this resource is the only adapted resource then we reset all adaptations. Bug: webrtc:11636 Change-Id: I29acc5a3934f42f00db79b3bb2171156b1313937 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176406 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31520}
2020-06-12 16:00:03 +02:00
const auto& restrictions_with_counters =
Only allow most limited resource to trigger adapt up A more detailed explaination is in the bug, but this changes the way that adaptation happens when multiple resources are limited. Only the one that is most limited can trigger an adaptation up. If multiple resources are most limited both need to underuse to adapt up. Some of the changes in this patch to make it all work: * VideoStreamEncoder unittests that did not reflect this new behaviour have been changed. * PeekNextRestrictions returns the adaptation counters as well as the restrictions. * Adaptation statstics have changed so that when adapting up all resources are tagged as triggering the adaptation. Additionally the statistics for the current adaptation is now the total number of adaptations per reason, rather then the number of adaptations due to that reason. * PreventAdaptUpDueToActiveCounts is removed as most limited resource is a strong implementation of that. Bug: webrtc:11553 Change-Id: If1545a201c8e019598edf82657a1befde8b05268 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176128 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31497}
2020-06-11 10:45:29 +02:00
resource_and_adaptation_limit_.second;
[Adaptation] Adaptation holds restrictions This refactoring enables us to do multi-step adaptations instead of the normal single step adaptations. This work is required pre-requisite work to remove the DropDueToSize methods from the ResourceAdaptationProcessorInterface. This work also gives a path to fixing webrtc:11719, since double adaptations can be done in a controlled manner with this API. Bug: webrtc:11700,webrtc:11719 Change-Id: Ic7d27a49c82d0cf7ebaf79d7bdf0bc6f382770ed Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178200 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31631}
2020-07-03 15:16:43 +02:00
if (restrictions_with_counters.counters.Total() >
most_limited_restrictions.counters.Total()) {
[Adaptation] Implement remove for most limited resources. When a resource is removed from the ResourceAdaptationProcessor, and the resource is the most limited resource, we reset the current adaptation to the next most limited resource level. In the case that this resource is the only adapted resource then we reset all adaptations. Bug: webrtc:11636 Change-Id: I29acc5a3934f42f00db79b3bb2171156b1313937 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176406 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31520}
2020-06-12 16:00:03 +02:00
most_limited_restrictions = restrictions_with_counters;
Only allow most limited resource to trigger adapt up A more detailed explaination is in the bug, but this changes the way that adaptation happens when multiple resources are limited. Only the one that is most limited can trigger an adaptation up. If multiple resources are most limited both need to underuse to adapt up. Some of the changes in this patch to make it all work: * VideoStreamEncoder unittests that did not reflect this new behaviour have been changed. * PeekNextRestrictions returns the adaptation counters as well as the restrictions. * Adaptation statstics have changed so that when adapting up all resources are tagged as triggering the adaptation. Additionally the statistics for the current adaptation is now the total number of adaptations per reason, rather then the number of adaptations due to that reason. * PreventAdaptUpDueToActiveCounts is removed as most limited resource is a strong implementation of that. Bug: webrtc:11553 Change-Id: If1545a201c8e019598edf82657a1befde8b05268 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176128 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31497}
2020-06-11 10:45:29 +02:00
most_limited_resources.clear();
most_limited_resources.push_back(resource_and_adaptation_limit_.first);
[Adaptation] Adaptation holds restrictions This refactoring enables us to do multi-step adaptations instead of the normal single step adaptations. This work is required pre-requisite work to remove the DropDueToSize methods from the ResourceAdaptationProcessorInterface. This work also gives a path to fixing webrtc:11719, since double adaptations can be done in a controlled manner with this API. Bug: webrtc:11700,webrtc:11719 Change-Id: Ic7d27a49c82d0cf7ebaf79d7bdf0bc6f382770ed Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178200 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31631}
2020-07-03 15:16:43 +02:00
} else if (most_limited_restrictions.counters ==
restrictions_with_counters.counters) {
Only allow most limited resource to trigger adapt up A more detailed explaination is in the bug, but this changes the way that adaptation happens when multiple resources are limited. Only the one that is most limited can trigger an adaptation up. If multiple resources are most limited both need to underuse to adapt up. Some of the changes in this patch to make it all work: * VideoStreamEncoder unittests that did not reflect this new behaviour have been changed. * PeekNextRestrictions returns the adaptation counters as well as the restrictions. * Adaptation statstics have changed so that when adapting up all resources are tagged as triggering the adaptation. Additionally the statistics for the current adaptation is now the total number of adaptations per reason, rather then the number of adaptations due to that reason. * PreventAdaptUpDueToActiveCounts is removed as most limited resource is a strong implementation of that. Bug: webrtc:11553 Change-Id: If1545a201c8e019598edf82657a1befde8b05268 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176128 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31497}
2020-06-11 10:45:29 +02:00
most_limited_resources.push_back(resource_and_adaptation_limit_.first);
}
[Adaptation] Adapt up requires previous underuse The problem is that a resource that signals Underuse would be able to trigger an adapt up when it was never limited in the past. This means that an underused resource would be able to negate the adaptations made for an overused one. For example, consider a fast CPU on a bad link. The QP for the image is high but the CPU is underused. Without requiring previous underuse, everytime the QP would signal overuse and trigger an adpatation down, the CPU would signal underuse and trigger an adaptation up. This works today as we want by using the active counts in the VideoStreamEncoderResourceManager. This change makes it a normal behaviour independant of active counts. The problem with active counts is that is only works with 2 resources. When resources are injectable it no longer works as expected. Bug: webrtc:11522, webrtc:11523 Change-Id: I140636ce206d74e00a6b6f8558162bb8afffda1c Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174482 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31181}
2020-05-07 13:45:14 +02:00
}
Only allow most limited resource to trigger adapt up A more detailed explaination is in the bug, but this changes the way that adaptation happens when multiple resources are limited. Only the one that is most limited can trigger an adaptation up. If multiple resources are most limited both need to underuse to adapt up. Some of the changes in this patch to make it all work: * VideoStreamEncoder unittests that did not reflect this new behaviour have been changed. * PeekNextRestrictions returns the adaptation counters as well as the restrictions. * Adaptation statstics have changed so that when adapting up all resources are tagged as triggering the adaptation. Additionally the statistics for the current adaptation is now the total number of adaptations per reason, rather then the number of adaptations due to that reason. * PreventAdaptUpDueToActiveCounts is removed as most limited resource is a strong implementation of that. Bug: webrtc:11553 Change-Id: If1545a201c8e019598edf82657a1befde8b05268 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176128 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31497}
2020-06-11 10:45:29 +02:00
return std::make_pair(std::move(most_limited_resources),
most_limited_restrictions);
[Adaptation] Adapt up requires previous underuse The problem is that a resource that signals Underuse would be able to trigger an adapt up when it was never limited in the past. This means that an underused resource would be able to negate the adaptations made for an overused one. For example, consider a fast CPU on a bad link. The QP for the image is high but the CPU is underused. Without requiring previous underuse, everytime the QP would signal overuse and trigger an adpatation down, the CPU would signal underuse and trigger an adaptation up. This works today as we want by using the active counts in the VideoStreamEncoderResourceManager. This change makes it a normal behaviour independant of active counts. The problem with active counts is that is only works with 2 resources. When resources are injectable it no longer works as expected. Bug: webrtc:11522, webrtc:11523 Change-Id: I140636ce206d74e00a6b6f8558162bb8afffda1c Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174482 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31181}
2020-05-07 13:45:14 +02:00
}
Only allow most limited resource to trigger adapt up A more detailed explaination is in the bug, but this changes the way that adaptation happens when multiple resources are limited. Only the one that is most limited can trigger an adaptation up. If multiple resources are most limited both need to underuse to adapt up. Some of the changes in this patch to make it all work: * VideoStreamEncoder unittests that did not reflect this new behaviour have been changed. * PeekNextRestrictions returns the adaptation counters as well as the restrictions. * Adaptation statstics have changed so that when adapting up all resources are tagged as triggering the adaptation. Additionally the statistics for the current adaptation is now the total number of adaptations per reason, rather then the number of adaptations due to that reason. * PreventAdaptUpDueToActiveCounts is removed as most limited resource is a strong implementation of that. Bug: webrtc:11553 Change-Id: If1545a201c8e019598edf82657a1befde8b05268 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176128 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31497}
2020-06-11 10:45:29 +02:00
void ResourceAdaptationProcessor::UpdateResourceLimitations(
rtc::scoped_refptr<Resource> reason_resource,
[Adaptation] Adaptation holds restrictions This refactoring enables us to do multi-step adaptations instead of the normal single step adaptations. This work is required pre-requisite work to remove the DropDueToSize methods from the ResourceAdaptationProcessorInterface. This work also gives a path to fixing webrtc:11719, since double adaptations can be done in a controlled manner with this API. Bug: webrtc:11700,webrtc:11719 Change-Id: Ic7d27a49c82d0cf7ebaf79d7bdf0bc6f382770ed Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178200 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31631}
2020-07-03 15:16:43 +02:00
const VideoSourceRestrictions& restrictions,
const VideoAdaptationCounters& counters) {
[Adaptation] VideoStreamAdapter broadcasts adaptations This moves this responsibility from the ResourceAdaptaitonProcessor to the VideoStreamAdapter. A new interface for listening to adaptation changes was added, and the ResourceAdaptationProcessor now listens on the VideoStreamAdapter for those changes. This means that going forward, 1. We can do adaptations outside of resource limitations, like setting prior adaptations on a resource like initial frame dropper is designed to. 2. Adaptations can be on a different queue than the ResourceAdaptaitonProcessor's queue since updates are pushed through the listener. Bug: webrtc:11700 Change-Id: I6de0dec748dba095e702f0b9893c5afa50b51aa9 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176859 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31615}
2020-07-01 11:47:46 +02:00
auto& adaptation_limits = adaptation_limits_by_resources_[reason_resource];
[Adaptation] Adaptation holds restrictions This refactoring enables us to do multi-step adaptations instead of the normal single step adaptations. This work is required pre-requisite work to remove the DropDueToSize methods from the ResourceAdaptationProcessorInterface. This work also gives a path to fixing webrtc:11719, since double adaptations can be done in a controlled manner with this API. Bug: webrtc:11700,webrtc:11719 Change-Id: Ic7d27a49c82d0cf7ebaf79d7bdf0bc6f382770ed Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178200 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31631}
2020-07-03 15:16:43 +02:00
if (adaptation_limits.restrictions == restrictions &&
adaptation_limits.counters == counters) {
[Adaptation] VideoStreamAdapter broadcasts adaptations This moves this responsibility from the ResourceAdaptaitonProcessor to the VideoStreamAdapter. A new interface for listening to adaptation changes was added, and the ResourceAdaptationProcessor now listens on the VideoStreamAdapter for those changes. This means that going forward, 1. We can do adaptations outside of resource limitations, like setting prior adaptations on a resource like initial frame dropper is designed to. 2. Adaptations can be on a different queue than the ResourceAdaptaitonProcessor's queue since updates are pushed through the listener. Bug: webrtc:11700 Change-Id: I6de0dec748dba095e702f0b9893c5afa50b51aa9 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176859 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31615}
2020-07-01 11:47:46 +02:00
return;
}
[Adaptation] Adaptation holds restrictions This refactoring enables us to do multi-step adaptations instead of the normal single step adaptations. This work is required pre-requisite work to remove the DropDueToSize methods from the ResourceAdaptationProcessorInterface. This work also gives a path to fixing webrtc:11719, since double adaptations can be done in a controlled manner with this API. Bug: webrtc:11700,webrtc:11719 Change-Id: Ic7d27a49c82d0cf7ebaf79d7bdf0bc6f382770ed Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178200 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31631}
2020-07-03 15:16:43 +02:00
adaptation_limits = {restrictions, counters};
[Adaptation] Implement remove for most limited resources. When a resource is removed from the ResourceAdaptationProcessor, and the resource is the most limited resource, we reset the current adaptation to the next most limited resource level. In the case that this resource is the only adapted resource then we reset all adaptations. Bug: webrtc:11636 Change-Id: I29acc5a3934f42f00db79b3bb2171156b1313937 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176406 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31520}
2020-06-12 16:00:03 +02:00
std::map<rtc::scoped_refptr<Resource>, VideoAdaptationCounters> limitations;
for (const auto& p : adaptation_limits_by_resources_) {
[Adaptation] Adaptation holds restrictions This refactoring enables us to do multi-step adaptations instead of the normal single step adaptations. This work is required pre-requisite work to remove the DropDueToSize methods from the ResourceAdaptationProcessorInterface. This work also gives a path to fixing webrtc:11719, since double adaptations can be done in a controlled manner with this API. Bug: webrtc:11700,webrtc:11719 Change-Id: Ic7d27a49c82d0cf7ebaf79d7bdf0bc6f382770ed Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178200 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31631}
2020-07-03 15:16:43 +02:00
limitations.insert(std::make_pair(p.first, p.second.counters));
[Adaptation] Implement remove for most limited resources. When a resource is removed from the ResourceAdaptationProcessor, and the resource is the most limited resource, we reset the current adaptation to the next most limited resource level. In the case that this resource is the only adapted resource then we reset all adaptations. Bug: webrtc:11636 Change-Id: I29acc5a3934f42f00db79b3bb2171156b1313937 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176406 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31520}
2020-06-12 16:00:03 +02:00
}
[Adaptation] VideoStreamAdapter broadcasts adaptations This moves this responsibility from the ResourceAdaptaitonProcessor to the VideoStreamAdapter. A new interface for listening to adaptation changes was added, and the ResourceAdaptationProcessor now listens on the VideoStreamAdapter for those changes. This means that going forward, 1. We can do adaptations outside of resource limitations, like setting prior adaptations on a resource like initial frame dropper is designed to. 2. Adaptations can be on a different queue than the ResourceAdaptaitonProcessor's queue since updates are pushed through the listener. Bug: webrtc:11700 Change-Id: I6de0dec748dba095e702f0b9893c5afa50b51aa9 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176859 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31615}
2020-07-01 11:47:46 +02:00
for (auto limitations_listener : resource_limitations_listeners_) {
limitations_listener->OnResourceLimitationChanged(reason_resource,
limitations);
[Adaptation] Implement remove for most limited resources. When a resource is removed from the ResourceAdaptationProcessor, and the resource is the most limited resource, we reset the current adaptation to the next most limited resource level. In the case that this resource is the only adapted resource then we reset all adaptations. Bug: webrtc:11636 Change-Id: I29acc5a3934f42f00db79b3bb2171156b1313937 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176406 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31520}
2020-06-12 16:00:03 +02:00
}
}
[Adaptation] VideoStreamAdapter broadcasts adaptations This moves this responsibility from the ResourceAdaptaitonProcessor to the VideoStreamAdapter. A new interface for listening to adaptation changes was added, and the ResourceAdaptationProcessor now listens on the VideoStreamAdapter for those changes. This means that going forward, 1. We can do adaptations outside of resource limitations, like setting prior adaptations on a resource like initial frame dropper is designed to. 2. Adaptations can be on a different queue than the ResourceAdaptaitonProcessor's queue since updates are pushed through the listener. Bug: webrtc:11700 Change-Id: I6de0dec748dba095e702f0b9893c5afa50b51aa9 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176859 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31615}
2020-07-01 11:47:46 +02:00
void ResourceAdaptationProcessor::OnVideoSourceRestrictionsUpdated(
VideoSourceRestrictions restrictions,
const VideoAdaptationCounters& adaptation_counters,
rtc::scoped_refptr<Resource> reason,
const VideoSourceRestrictions& unfiltered_restrictions) {
[Adaptation] Remove resource adaptation queue Resource adaptation needs refactoring for async adaptations. For now the resource adaptation processor can work on the encoder thread, until it is refactored to support async adaptation. Bug: webrtc:11867 Change-Id: I9c46da356db19c0fd52748c999ccb216f2ca923b Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182040 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31991}
2020-08-25 13:12:12 +02:00
RTC_DCHECK_RUN_ON(task_queue_);
[Adaptation] VideoStreamAdapter broadcasts adaptations This moves this responsibility from the ResourceAdaptaitonProcessor to the VideoStreamAdapter. A new interface for listening to adaptation changes was added, and the ResourceAdaptationProcessor now listens on the VideoStreamAdapter for those changes. This means that going forward, 1. We can do adaptations outside of resource limitations, like setting prior adaptations on a resource like initial frame dropper is designed to. 2. Adaptations can be on a different queue than the ResourceAdaptaitonProcessor's queue since updates are pushed through the listener. Bug: webrtc:11700 Change-Id: I6de0dec748dba095e702f0b9893c5afa50b51aa9 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176859 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31615}
2020-07-01 11:47:46 +02:00
if (reason) {
[Adaptation] Adaptation holds restrictions This refactoring enables us to do multi-step adaptations instead of the normal single step adaptations. This work is required pre-requisite work to remove the DropDueToSize methods from the ResourceAdaptationProcessorInterface. This work also gives a path to fixing webrtc:11719, since double adaptations can be done in a controlled manner with this API. Bug: webrtc:11700,webrtc:11719 Change-Id: Ic7d27a49c82d0cf7ebaf79d7bdf0bc6f382770ed Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/178200 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31631}
2020-07-03 15:16:43 +02:00
UpdateResourceLimitations(reason, unfiltered_restrictions,
adaptation_counters);
[Adaptation] VideoStreamAdapter broadcasts adaptations This moves this responsibility from the ResourceAdaptaitonProcessor to the VideoStreamAdapter. A new interface for listening to adaptation changes was added, and the ResourceAdaptationProcessor now listens on the VideoStreamAdapter for those changes. This means that going forward, 1. We can do adaptations outside of resource limitations, like setting prior adaptations on a resource like initial frame dropper is designed to. 2. Adaptations can be on a different queue than the ResourceAdaptaitonProcessor's queue since updates are pushed through the listener. Bug: webrtc:11700 Change-Id: I6de0dec748dba095e702f0b9893c5afa50b51aa9 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176859 Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#31615}
2020-07-01 11:47:46 +02:00
} else if (adaptation_counters.Total() == 0) {
// Adaptations are cleared.
adaptation_limits_by_resources_.clear();
previous_mitigation_results_.clear();
for (auto limitations_listener : resource_limitations_listeners_) {
limitations_listener->OnResourceLimitationChanged(nullptr, {});
}
Only allow most limited resource to trigger adapt up A more detailed explaination is in the bug, but this changes the way that adaptation happens when multiple resources are limited. Only the one that is most limited can trigger an adaptation up. If multiple resources are most limited both need to underuse to adapt up. Some of the changes in this patch to make it all work: * VideoStreamEncoder unittests that did not reflect this new behaviour have been changed. * PeekNextRestrictions returns the adaptation counters as well as the restrictions. * Adaptation statstics have changed so that when adapting up all resources are tagged as triggering the adaptation. Additionally the statistics for the current adaptation is now the total number of adaptations per reason, rather then the number of adaptations due to that reason. * PreventAdaptUpDueToActiveCounts is removed as most limited resource is a strong implementation of that. Bug: webrtc:11553 Change-Id: If1545a201c8e019598edf82657a1befde8b05268 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176128 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31497}
2020-06-11 10:45:29 +02:00
}
[Adaptation] Adapt up requires previous underuse The problem is that a resource that signals Underuse would be able to trigger an adapt up when it was never limited in the past. This means that an underused resource would be able to negate the adaptations made for an overused one. For example, consider a fast CPU on a bad link. The QP for the image is high but the CPU is underused. Without requiring previous underuse, everytime the QP would signal overuse and trigger an adpatation down, the CPU would signal underuse and trigger an adaptation up. This works today as we want by using the active counts in the VideoStreamEncoderResourceManager. This change makes it a normal behaviour independant of active counts. The problem with active counts is that is only works with 2 resources. When resources are injectable it no longer works as expected. Bug: webrtc:11522, webrtc:11523 Change-Id: I140636ce206d74e00a6b6f8558162bb8afffda1c Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174482 Commit-Queue: Evan Shrubsole <eshr@google.com> Reviewed-by: Henrik Boström <hbos@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31181}
2020-05-07 13:45:14 +02:00
}
[Adaptation] Introducing call/adaptation/ResourceAdaptationProcessor. This CL is part of the Call-Level Adaptation Processing design doc: https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing This gets to the heart of unblocking call-level adaptation, largely made possible due to the previous CLs in the chain. The parts of the code that are responsible for responding to resource usage signals, obtaining adaptations and applying them are moved to ResourceAdaptationProcessor in call/adaptation/. The parts of the code that are responsible for managing VideoStreamEncoder-specific resources stay inside the VideoStreamEncoderResourceManager class in video/adaptation/. After this CL lands it should soon be possible to move the Processor over to a separate task queue and let the Manager stay on the encoder queue if PostTasks are added for communication between the two objects. Bug: webrtc:11172 Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Cr-Commit-Position: refs/heads/master@{#31105}
2020-04-17 18:36:19 +02:00
} // namespace webrtc
Reference in New Issue Copy Permalink