webrtc_m130/video/video_source_sink_controller.cc

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

195 lines
6.0 KiB
C++
Raw Normal View History

VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
/*
* Copyright 2020 The WebRTC Project Authors. All rights reserved.
*
* Use of this source code is governed by a BSD-style license
* that can be found in the LICENSE file in the root of the source
* tree. An additional intellectual property rights grant can be found
* in the file PATENTS. All contributing project authors may
* be found in the AUTHORS file in the root of the source tree.
*/
#include "video/video_source_sink_controller.h"
#include <algorithm>
#include <limits>
#include <utility>
#include "rtc_base/logging.h"
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
#include "rtc_base/numerics/safe_conversions.h"
#include "rtc_base/strings/string_builder.h"
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
namespace webrtc {
namespace {
std::string WantsToString(const rtc::VideoSinkWants& wants) {
rtc::StringBuilder ss;
ss << "max_fps=" << wants.max_framerate_fps
<< " max_pixel_count=" << wants.max_pixel_count << " target_pixel_count="
<< (wants.target_pixel_count.has_value()
? std::to_string(wants.target_pixel_count.value())
: "null")
<< " resolutions={";
for (size_t i = 0; i < wants.resolutions.size(); ++i) {
if (i != 0)
ss << ",";
ss << wants.resolutions[i].width << "x" << wants.resolutions[i].height;
}
ss << "}";
return ss.Release();
}
} // namespace
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
VideoSourceSinkController::VideoSourceSinkController(
rtc::VideoSinkInterface<VideoFrame>* sink,
rtc::VideoSourceInterface<VideoFrame>* source)
: sink_(sink), source_(source) {
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
RTC_DCHECK(sink_);
}
Remove use of asyncinvoker from WebRtcVideoSendStream. This turned out to be a bit complicated, mostly related to the tests, but here's what's changed: * No AsyncInvoker (and avoid ClearInternal) in WebRtcVideoSendStream (WVSS) * The reason it was there is due to a "design leak" from VideoSourceSinkController/VideoStreamEncoder where the former uses locks in all methods and is unaware of a threading model. That design affected downstream objects, pushed the need for an async hop into WVSS and added a lock. A suggestion was made to address this in a follow-up change, here: https://webrtc-review.googlesource.com/c/src/+/165684 * All methods in VideoSourceSinkController are now called on a known and checked sequence and this CL removes the lock. This also makes checking state consistent (i.e. calling a getter twice in a row on the same sequence, will always return the same value, avoiding race with other threads). * Handling of reporting state changes from the encoder queue to the VSSC, is done by VideoStreamEncoder. * VideoSendStreamImpl is still instantiated on the incorrect thread [1] but has two initialization steps [2]. The second one already runs on the right thread. Addressing that TODO [1] is something we should do but it has side effects to consider. For the purposes of this CL the steps relating to the encoder (setting the sink pointer) have been moved to [2]. [1] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;l=94 [2] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;drc=f4a9991cce74a37d006438ec0e366313ed33162e;l=115 Bug: webrtc:11222, webrtc:11908 Change-Id: Ie46d46e3a52bbe225951b4bd580ecb8cc9cad873 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184508 Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Tommi <tommi@webrtc.org> Cr-Commit-Position: refs/heads/master@{#32150}
2020-09-21 14:31:23 +02:00
VideoSourceSinkController::~VideoSourceSinkController() {
RTC_DCHECK_RUN_ON(&sequence_checker_);
}
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
void VideoSourceSinkController::SetSource(
rtc::VideoSourceInterface<VideoFrame>* source) {
Remove use of asyncinvoker from WebRtcVideoSendStream. This turned out to be a bit complicated, mostly related to the tests, but here's what's changed: * No AsyncInvoker (and avoid ClearInternal) in WebRtcVideoSendStream (WVSS) * The reason it was there is due to a "design leak" from VideoSourceSinkController/VideoStreamEncoder where the former uses locks in all methods and is unaware of a threading model. That design affected downstream objects, pushed the need for an async hop into WVSS and added a lock. A suggestion was made to address this in a follow-up change, here: https://webrtc-review.googlesource.com/c/src/+/165684 * All methods in VideoSourceSinkController are now called on a known and checked sequence and this CL removes the lock. This also makes checking state consistent (i.e. calling a getter twice in a row on the same sequence, will always return the same value, avoiding race with other threads). * Handling of reporting state changes from the encoder queue to the VSSC, is done by VideoStreamEncoder. * VideoSendStreamImpl is still instantiated on the incorrect thread [1] but has two initialization steps [2]. The second one already runs on the right thread. Addressing that TODO [1] is something we should do but it has side effects to consider. For the purposes of this CL the steps relating to the encoder (setting the sink pointer) have been moved to [2]. [1] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;l=94 [2] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;drc=f4a9991cce74a37d006438ec0e366313ed33162e;l=115 Bug: webrtc:11222, webrtc:11908 Change-Id: Ie46d46e3a52bbe225951b4bd580ecb8cc9cad873 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184508 Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Tommi <tommi@webrtc.org> Cr-Commit-Position: refs/heads/master@{#32150}
2020-09-21 14:31:23 +02:00
RTC_DCHECK_RUN_ON(&sequence_checker_);
rtc::VideoSourceInterface<VideoFrame>* old_source = source_;
source_ = source;
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
if (old_source != source && old_source)
old_source->RemoveSink(sink_);
Remove use of asyncinvoker from WebRtcVideoSendStream. This turned out to be a bit complicated, mostly related to the tests, but here's what's changed: * No AsyncInvoker (and avoid ClearInternal) in WebRtcVideoSendStream (WVSS) * The reason it was there is due to a "design leak" from VideoSourceSinkController/VideoStreamEncoder where the former uses locks in all methods and is unaware of a threading model. That design affected downstream objects, pushed the need for an async hop into WVSS and added a lock. A suggestion was made to address this in a follow-up change, here: https://webrtc-review.googlesource.com/c/src/+/165684 * All methods in VideoSourceSinkController are now called on a known and checked sequence and this CL removes the lock. This also makes checking state consistent (i.e. calling a getter twice in a row on the same sequence, will always return the same value, avoiding race with other threads). * Handling of reporting state changes from the encoder queue to the VSSC, is done by VideoStreamEncoder. * VideoSendStreamImpl is still instantiated on the incorrect thread [1] but has two initialization steps [2]. The second one already runs on the right thread. Addressing that TODO [1] is something we should do but it has side effects to consider. For the purposes of this CL the steps relating to the encoder (setting the sink pointer) have been moved to [2]. [1] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;l=94 [2] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;drc=f4a9991cce74a37d006438ec0e366313ed33162e;l=115 Bug: webrtc:11222, webrtc:11908 Change-Id: Ie46d46e3a52bbe225951b4bd580ecb8cc9cad873 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184508 Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Tommi <tommi@webrtc.org> Cr-Commit-Position: refs/heads/master@{#32150}
2020-09-21 14:31:23 +02:00
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
if (!source)
return;
Remove use of asyncinvoker from WebRtcVideoSendStream. This turned out to be a bit complicated, mostly related to the tests, but here's what's changed: * No AsyncInvoker (and avoid ClearInternal) in WebRtcVideoSendStream (WVSS) * The reason it was there is due to a "design leak" from VideoSourceSinkController/VideoStreamEncoder where the former uses locks in all methods and is unaware of a threading model. That design affected downstream objects, pushed the need for an async hop into WVSS and added a lock. A suggestion was made to address this in a follow-up change, here: https://webrtc-review.googlesource.com/c/src/+/165684 * All methods in VideoSourceSinkController are now called on a known and checked sequence and this CL removes the lock. This also makes checking state consistent (i.e. calling a getter twice in a row on the same sequence, will always return the same value, avoiding race with other threads). * Handling of reporting state changes from the encoder queue to the VSSC, is done by VideoStreamEncoder. * VideoSendStreamImpl is still instantiated on the incorrect thread [1] but has two initialization steps [2]. The second one already runs on the right thread. Addressing that TODO [1] is something we should do but it has side effects to consider. For the purposes of this CL the steps relating to the encoder (setting the sink pointer) have been moved to [2]. [1] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;l=94 [2] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;drc=f4a9991cce74a37d006438ec0e366313ed33162e;l=115 Bug: webrtc:11222, webrtc:11908 Change-Id: Ie46d46e3a52bbe225951b4bd580ecb8cc9cad873 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184508 Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Tommi <tommi@webrtc.org> Cr-Commit-Position: refs/heads/master@{#32150}
2020-09-21 14:31:23 +02:00
source->AddOrUpdateSink(sink_, CurrentSettingsToSinkWants());
}
bool VideoSourceSinkController::HasSource() const {
RTC_DCHECK_RUN_ON(&sequence_checker_);
return source_ != nullptr;
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
}
void VideoSourceSinkController::RequestRefreshFrame() {
RTC_DCHECK_RUN_ON(&sequence_checker_);
if (source_)
source_->RequestRefreshFrame();
}
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
void VideoSourceSinkController::PushSourceSinkSettings() {
Remove use of asyncinvoker from WebRtcVideoSendStream. This turned out to be a bit complicated, mostly related to the tests, but here's what's changed: * No AsyncInvoker (and avoid ClearInternal) in WebRtcVideoSendStream (WVSS) * The reason it was there is due to a "design leak" from VideoSourceSinkController/VideoStreamEncoder where the former uses locks in all methods and is unaware of a threading model. That design affected downstream objects, pushed the need for an async hop into WVSS and added a lock. A suggestion was made to address this in a follow-up change, here: https://webrtc-review.googlesource.com/c/src/+/165684 * All methods in VideoSourceSinkController are now called on a known and checked sequence and this CL removes the lock. This also makes checking state consistent (i.e. calling a getter twice in a row on the same sequence, will always return the same value, avoiding race with other threads). * Handling of reporting state changes from the encoder queue to the VSSC, is done by VideoStreamEncoder. * VideoSendStreamImpl is still instantiated on the incorrect thread [1] but has two initialization steps [2]. The second one already runs on the right thread. Addressing that TODO [1] is something we should do but it has side effects to consider. For the purposes of this CL the steps relating to the encoder (setting the sink pointer) have been moved to [2]. [1] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;l=94 [2] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;drc=f4a9991cce74a37d006438ec0e366313ed33162e;l=115 Bug: webrtc:11222, webrtc:11908 Change-Id: Ie46d46e3a52bbe225951b4bd580ecb8cc9cad873 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184508 Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Tommi <tommi@webrtc.org> Cr-Commit-Position: refs/heads/master@{#32150}
2020-09-21 14:31:23 +02:00
RTC_DCHECK_RUN_ON(&sequence_checker_);
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
if (!source_)
return;
rtc::VideoSinkWants wants = CurrentSettingsToSinkWants();
RTC_LOG(LS_INFO) << "Pushing SourceSink restrictions: "
<< WantsToString(wants);
source_->AddOrUpdateSink(sink_, wants);
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
}
VideoSourceRestrictions VideoSourceSinkController::restrictions() const {
Remove use of asyncinvoker from WebRtcVideoSendStream. This turned out to be a bit complicated, mostly related to the tests, but here's what's changed: * No AsyncInvoker (and avoid ClearInternal) in WebRtcVideoSendStream (WVSS) * The reason it was there is due to a "design leak" from VideoSourceSinkController/VideoStreamEncoder where the former uses locks in all methods and is unaware of a threading model. That design affected downstream objects, pushed the need for an async hop into WVSS and added a lock. A suggestion was made to address this in a follow-up change, here: https://webrtc-review.googlesource.com/c/src/+/165684 * All methods in VideoSourceSinkController are now called on a known and checked sequence and this CL removes the lock. This also makes checking state consistent (i.e. calling a getter twice in a row on the same sequence, will always return the same value, avoiding race with other threads). * Handling of reporting state changes from the encoder queue to the VSSC, is done by VideoStreamEncoder. * VideoSendStreamImpl is still instantiated on the incorrect thread [1] but has two initialization steps [2]. The second one already runs on the right thread. Addressing that TODO [1] is something we should do but it has side effects to consider. For the purposes of this CL the steps relating to the encoder (setting the sink pointer) have been moved to [2]. [1] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;l=94 [2] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;drc=f4a9991cce74a37d006438ec0e366313ed33162e;l=115 Bug: webrtc:11222, webrtc:11908 Change-Id: Ie46d46e3a52bbe225951b4bd580ecb8cc9cad873 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184508 Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Tommi <tommi@webrtc.org> Cr-Commit-Position: refs/heads/master@{#32150}
2020-09-21 14:31:23 +02:00
RTC_DCHECK_RUN_ON(&sequence_checker_);
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
return restrictions_;
}
absl::optional<size_t> VideoSourceSinkController::pixels_per_frame_upper_limit()
const {
Remove use of asyncinvoker from WebRtcVideoSendStream. This turned out to be a bit complicated, mostly related to the tests, but here's what's changed: * No AsyncInvoker (and avoid ClearInternal) in WebRtcVideoSendStream (WVSS) * The reason it was there is due to a "design leak" from VideoSourceSinkController/VideoStreamEncoder where the former uses locks in all methods and is unaware of a threading model. That design affected downstream objects, pushed the need for an async hop into WVSS and added a lock. A suggestion was made to address this in a follow-up change, here: https://webrtc-review.googlesource.com/c/src/+/165684 * All methods in VideoSourceSinkController are now called on a known and checked sequence and this CL removes the lock. This also makes checking state consistent (i.e. calling a getter twice in a row on the same sequence, will always return the same value, avoiding race with other threads). * Handling of reporting state changes from the encoder queue to the VSSC, is done by VideoStreamEncoder. * VideoSendStreamImpl is still instantiated on the incorrect thread [1] but has two initialization steps [2]. The second one already runs on the right thread. Addressing that TODO [1] is something we should do but it has side effects to consider. For the purposes of this CL the steps relating to the encoder (setting the sink pointer) have been moved to [2]. [1] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;l=94 [2] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;drc=f4a9991cce74a37d006438ec0e366313ed33162e;l=115 Bug: webrtc:11222, webrtc:11908 Change-Id: Ie46d46e3a52bbe225951b4bd580ecb8cc9cad873 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184508 Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Tommi <tommi@webrtc.org> Cr-Commit-Position: refs/heads/master@{#32150}
2020-09-21 14:31:23 +02:00
RTC_DCHECK_RUN_ON(&sequence_checker_);
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
return pixels_per_frame_upper_limit_;
}
absl::optional<double> VideoSourceSinkController::frame_rate_upper_limit()
const {
Remove use of asyncinvoker from WebRtcVideoSendStream. This turned out to be a bit complicated, mostly related to the tests, but here's what's changed: * No AsyncInvoker (and avoid ClearInternal) in WebRtcVideoSendStream (WVSS) * The reason it was there is due to a "design leak" from VideoSourceSinkController/VideoStreamEncoder where the former uses locks in all methods and is unaware of a threading model. That design affected downstream objects, pushed the need for an async hop into WVSS and added a lock. A suggestion was made to address this in a follow-up change, here: https://webrtc-review.googlesource.com/c/src/+/165684 * All methods in VideoSourceSinkController are now called on a known and checked sequence and this CL removes the lock. This also makes checking state consistent (i.e. calling a getter twice in a row on the same sequence, will always return the same value, avoiding race with other threads). * Handling of reporting state changes from the encoder queue to the VSSC, is done by VideoStreamEncoder. * VideoSendStreamImpl is still instantiated on the incorrect thread [1] but has two initialization steps [2]. The second one already runs on the right thread. Addressing that TODO [1] is something we should do but it has side effects to consider. For the purposes of this CL the steps relating to the encoder (setting the sink pointer) have been moved to [2]. [1] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;l=94 [2] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;drc=f4a9991cce74a37d006438ec0e366313ed33162e;l=115 Bug: webrtc:11222, webrtc:11908 Change-Id: Ie46d46e3a52bbe225951b4bd580ecb8cc9cad873 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184508 Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Tommi <tommi@webrtc.org> Cr-Commit-Position: refs/heads/master@{#32150}
2020-09-21 14:31:23 +02:00
RTC_DCHECK_RUN_ON(&sequence_checker_);
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
return frame_rate_upper_limit_;
}
bool VideoSourceSinkController::rotation_applied() const {
Remove use of asyncinvoker from WebRtcVideoSendStream. This turned out to be a bit complicated, mostly related to the tests, but here's what's changed: * No AsyncInvoker (and avoid ClearInternal) in WebRtcVideoSendStream (WVSS) * The reason it was there is due to a "design leak" from VideoSourceSinkController/VideoStreamEncoder where the former uses locks in all methods and is unaware of a threading model. That design affected downstream objects, pushed the need for an async hop into WVSS and added a lock. A suggestion was made to address this in a follow-up change, here: https://webrtc-review.googlesource.com/c/src/+/165684 * All methods in VideoSourceSinkController are now called on a known and checked sequence and this CL removes the lock. This also makes checking state consistent (i.e. calling a getter twice in a row on the same sequence, will always return the same value, avoiding race with other threads). * Handling of reporting state changes from the encoder queue to the VSSC, is done by VideoStreamEncoder. * VideoSendStreamImpl is still instantiated on the incorrect thread [1] but has two initialization steps [2]. The second one already runs on the right thread. Addressing that TODO [1] is something we should do but it has side effects to consider. For the purposes of this CL the steps relating to the encoder (setting the sink pointer) have been moved to [2]. [1] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;l=94 [2] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;drc=f4a9991cce74a37d006438ec0e366313ed33162e;l=115 Bug: webrtc:11222, webrtc:11908 Change-Id: Ie46d46e3a52bbe225951b4bd580ecb8cc9cad873 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184508 Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Tommi <tommi@webrtc.org> Cr-Commit-Position: refs/heads/master@{#32150}
2020-09-21 14:31:23 +02:00
RTC_DCHECK_RUN_ON(&sequence_checker_);
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
return rotation_applied_;
}
int VideoSourceSinkController::resolution_alignment() const {
Remove use of asyncinvoker from WebRtcVideoSendStream. This turned out to be a bit complicated, mostly related to the tests, but here's what's changed: * No AsyncInvoker (and avoid ClearInternal) in WebRtcVideoSendStream (WVSS) * The reason it was there is due to a "design leak" from VideoSourceSinkController/VideoStreamEncoder where the former uses locks in all methods and is unaware of a threading model. That design affected downstream objects, pushed the need for an async hop into WVSS and added a lock. A suggestion was made to address this in a follow-up change, here: https://webrtc-review.googlesource.com/c/src/+/165684 * All methods in VideoSourceSinkController are now called on a known and checked sequence and this CL removes the lock. This also makes checking state consistent (i.e. calling a getter twice in a row on the same sequence, will always return the same value, avoiding race with other threads). * Handling of reporting state changes from the encoder queue to the VSSC, is done by VideoStreamEncoder. * VideoSendStreamImpl is still instantiated on the incorrect thread [1] but has two initialization steps [2]. The second one already runs on the right thread. Addressing that TODO [1] is something we should do but it has side effects to consider. For the purposes of this CL the steps relating to the encoder (setting the sink pointer) have been moved to [2]. [1] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;l=94 [2] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;drc=f4a9991cce74a37d006438ec0e366313ed33162e;l=115 Bug: webrtc:11222, webrtc:11908 Change-Id: Ie46d46e3a52bbe225951b4bd580ecb8cc9cad873 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184508 Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Tommi <tommi@webrtc.org> Cr-Commit-Position: refs/heads/master@{#32150}
2020-09-21 14:31:23 +02:00
RTC_DCHECK_RUN_ON(&sequence_checker_);
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
return resolution_alignment_;
}
const std::vector<rtc::VideoSinkWants::FrameSize>&
VideoSourceSinkController::resolutions() const {
RTC_DCHECK_RUN_ON(&sequence_checker_);
return resolutions_;
}
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
void VideoSourceSinkController::SetRestrictions(
VideoSourceRestrictions restrictions) {
Remove use of asyncinvoker from WebRtcVideoSendStream. This turned out to be a bit complicated, mostly related to the tests, but here's what's changed: * No AsyncInvoker (and avoid ClearInternal) in WebRtcVideoSendStream (WVSS) * The reason it was there is due to a "design leak" from VideoSourceSinkController/VideoStreamEncoder where the former uses locks in all methods and is unaware of a threading model. That design affected downstream objects, pushed the need for an async hop into WVSS and added a lock. A suggestion was made to address this in a follow-up change, here: https://webrtc-review.googlesource.com/c/src/+/165684 * All methods in VideoSourceSinkController are now called on a known and checked sequence and this CL removes the lock. This also makes checking state consistent (i.e. calling a getter twice in a row on the same sequence, will always return the same value, avoiding race with other threads). * Handling of reporting state changes from the encoder queue to the VSSC, is done by VideoStreamEncoder. * VideoSendStreamImpl is still instantiated on the incorrect thread [1] but has two initialization steps [2]. The second one already runs on the right thread. Addressing that TODO [1] is something we should do but it has side effects to consider. For the purposes of this CL the steps relating to the encoder (setting the sink pointer) have been moved to [2]. [1] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;l=94 [2] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;drc=f4a9991cce74a37d006438ec0e366313ed33162e;l=115 Bug: webrtc:11222, webrtc:11908 Change-Id: Ie46d46e3a52bbe225951b4bd580ecb8cc9cad873 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184508 Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Tommi <tommi@webrtc.org> Cr-Commit-Position: refs/heads/master@{#32150}
2020-09-21 14:31:23 +02:00
RTC_DCHECK_RUN_ON(&sequence_checker_);
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
restrictions_ = std::move(restrictions);
}
void VideoSourceSinkController::SetPixelsPerFrameUpperLimit(
absl::optional<size_t> pixels_per_frame_upper_limit) {
Remove use of asyncinvoker from WebRtcVideoSendStream. This turned out to be a bit complicated, mostly related to the tests, but here's what's changed: * No AsyncInvoker (and avoid ClearInternal) in WebRtcVideoSendStream (WVSS) * The reason it was there is due to a "design leak" from VideoSourceSinkController/VideoStreamEncoder where the former uses locks in all methods and is unaware of a threading model. That design affected downstream objects, pushed the need for an async hop into WVSS and added a lock. A suggestion was made to address this in a follow-up change, here: https://webrtc-review.googlesource.com/c/src/+/165684 * All methods in VideoSourceSinkController are now called on a known and checked sequence and this CL removes the lock. This also makes checking state consistent (i.e. calling a getter twice in a row on the same sequence, will always return the same value, avoiding race with other threads). * Handling of reporting state changes from the encoder queue to the VSSC, is done by VideoStreamEncoder. * VideoSendStreamImpl is still instantiated on the incorrect thread [1] but has two initialization steps [2]. The second one already runs on the right thread. Addressing that TODO [1] is something we should do but it has side effects to consider. For the purposes of this CL the steps relating to the encoder (setting the sink pointer) have been moved to [2]. [1] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;l=94 [2] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;drc=f4a9991cce74a37d006438ec0e366313ed33162e;l=115 Bug: webrtc:11222, webrtc:11908 Change-Id: Ie46d46e3a52bbe225951b4bd580ecb8cc9cad873 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184508 Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Tommi <tommi@webrtc.org> Cr-Commit-Position: refs/heads/master@{#32150}
2020-09-21 14:31:23 +02:00
RTC_DCHECK_RUN_ON(&sequence_checker_);
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
pixels_per_frame_upper_limit_ = std::move(pixels_per_frame_upper_limit);
}
void VideoSourceSinkController::SetFrameRateUpperLimit(
absl::optional<double> frame_rate_upper_limit) {
Remove use of asyncinvoker from WebRtcVideoSendStream. This turned out to be a bit complicated, mostly related to the tests, but here's what's changed: * No AsyncInvoker (and avoid ClearInternal) in WebRtcVideoSendStream (WVSS) * The reason it was there is due to a "design leak" from VideoSourceSinkController/VideoStreamEncoder where the former uses locks in all methods and is unaware of a threading model. That design affected downstream objects, pushed the need for an async hop into WVSS and added a lock. A suggestion was made to address this in a follow-up change, here: https://webrtc-review.googlesource.com/c/src/+/165684 * All methods in VideoSourceSinkController are now called on a known and checked sequence and this CL removes the lock. This also makes checking state consistent (i.e. calling a getter twice in a row on the same sequence, will always return the same value, avoiding race with other threads). * Handling of reporting state changes from the encoder queue to the VSSC, is done by VideoStreamEncoder. * VideoSendStreamImpl is still instantiated on the incorrect thread [1] but has two initialization steps [2]. The second one already runs on the right thread. Addressing that TODO [1] is something we should do but it has side effects to consider. For the purposes of this CL the steps relating to the encoder (setting the sink pointer) have been moved to [2]. [1] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;l=94 [2] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;drc=f4a9991cce74a37d006438ec0e366313ed33162e;l=115 Bug: webrtc:11222, webrtc:11908 Change-Id: Ie46d46e3a52bbe225951b4bd580ecb8cc9cad873 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184508 Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Tommi <tommi@webrtc.org> Cr-Commit-Position: refs/heads/master@{#32150}
2020-09-21 14:31:23 +02:00
RTC_DCHECK_RUN_ON(&sequence_checker_);
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
frame_rate_upper_limit_ = std::move(frame_rate_upper_limit);
}
void VideoSourceSinkController::SetRotationApplied(bool rotation_applied) {
Remove use of asyncinvoker from WebRtcVideoSendStream. This turned out to be a bit complicated, mostly related to the tests, but here's what's changed: * No AsyncInvoker (and avoid ClearInternal) in WebRtcVideoSendStream (WVSS) * The reason it was there is due to a "design leak" from VideoSourceSinkController/VideoStreamEncoder where the former uses locks in all methods and is unaware of a threading model. That design affected downstream objects, pushed the need for an async hop into WVSS and added a lock. A suggestion was made to address this in a follow-up change, here: https://webrtc-review.googlesource.com/c/src/+/165684 * All methods in VideoSourceSinkController are now called on a known and checked sequence and this CL removes the lock. This also makes checking state consistent (i.e. calling a getter twice in a row on the same sequence, will always return the same value, avoiding race with other threads). * Handling of reporting state changes from the encoder queue to the VSSC, is done by VideoStreamEncoder. * VideoSendStreamImpl is still instantiated on the incorrect thread [1] but has two initialization steps [2]. The second one already runs on the right thread. Addressing that TODO [1] is something we should do but it has side effects to consider. For the purposes of this CL the steps relating to the encoder (setting the sink pointer) have been moved to [2]. [1] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;l=94 [2] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;drc=f4a9991cce74a37d006438ec0e366313ed33162e;l=115 Bug: webrtc:11222, webrtc:11908 Change-Id: Ie46d46e3a52bbe225951b4bd580ecb8cc9cad873 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184508 Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Tommi <tommi@webrtc.org> Cr-Commit-Position: refs/heads/master@{#32150}
2020-09-21 14:31:23 +02:00
RTC_DCHECK_RUN_ON(&sequence_checker_);
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
rotation_applied_ = rotation_applied;
}
void VideoSourceSinkController::SetResolutionAlignment(
int resolution_alignment) {
Remove use of asyncinvoker from WebRtcVideoSendStream. This turned out to be a bit complicated, mostly related to the tests, but here's what's changed: * No AsyncInvoker (and avoid ClearInternal) in WebRtcVideoSendStream (WVSS) * The reason it was there is due to a "design leak" from VideoSourceSinkController/VideoStreamEncoder where the former uses locks in all methods and is unaware of a threading model. That design affected downstream objects, pushed the need for an async hop into WVSS and added a lock. A suggestion was made to address this in a follow-up change, here: https://webrtc-review.googlesource.com/c/src/+/165684 * All methods in VideoSourceSinkController are now called on a known and checked sequence and this CL removes the lock. This also makes checking state consistent (i.e. calling a getter twice in a row on the same sequence, will always return the same value, avoiding race with other threads). * Handling of reporting state changes from the encoder queue to the VSSC, is done by VideoStreamEncoder. * VideoSendStreamImpl is still instantiated on the incorrect thread [1] but has two initialization steps [2]. The second one already runs on the right thread. Addressing that TODO [1] is something we should do but it has side effects to consider. For the purposes of this CL the steps relating to the encoder (setting the sink pointer) have been moved to [2]. [1] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;l=94 [2] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;drc=f4a9991cce74a37d006438ec0e366313ed33162e;l=115 Bug: webrtc:11222, webrtc:11908 Change-Id: Ie46d46e3a52bbe225951b4bd580ecb8cc9cad873 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184508 Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Tommi <tommi@webrtc.org> Cr-Commit-Position: refs/heads/master@{#32150}
2020-09-21 14:31:23 +02:00
RTC_DCHECK_RUN_ON(&sequence_checker_);
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
resolution_alignment_ = resolution_alignment;
}
void VideoSourceSinkController::SetResolutions(
std::vector<rtc::VideoSinkWants::FrameSize> resolutions) {
RTC_DCHECK_RUN_ON(&sequence_checker_);
resolutions_ = std::move(resolutions);
}
Remove use of asyncinvoker from WebRtcVideoSendStream. This turned out to be a bit complicated, mostly related to the tests, but here's what's changed: * No AsyncInvoker (and avoid ClearInternal) in WebRtcVideoSendStream (WVSS) * The reason it was there is due to a "design leak" from VideoSourceSinkController/VideoStreamEncoder where the former uses locks in all methods and is unaware of a threading model. That design affected downstream objects, pushed the need for an async hop into WVSS and added a lock. A suggestion was made to address this in a follow-up change, here: https://webrtc-review.googlesource.com/c/src/+/165684 * All methods in VideoSourceSinkController are now called on a known and checked sequence and this CL removes the lock. This also makes checking state consistent (i.e. calling a getter twice in a row on the same sequence, will always return the same value, avoiding race with other threads). * Handling of reporting state changes from the encoder queue to the VSSC, is done by VideoStreamEncoder. * VideoSendStreamImpl is still instantiated on the incorrect thread [1] but has two initialization steps [2]. The second one already runs on the right thread. Addressing that TODO [1] is something we should do but it has side effects to consider. For the purposes of this CL the steps relating to the encoder (setting the sink pointer) have been moved to [2]. [1] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;l=94 [2] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;drc=f4a9991cce74a37d006438ec0e366313ed33162e;l=115 Bug: webrtc:11222, webrtc:11908 Change-Id: Ie46d46e3a52bbe225951b4bd580ecb8cc9cad873 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184508 Reviewed-by: Henrik Boström <hbos@webrtc.org> Commit-Queue: Tommi <tommi@webrtc.org> Cr-Commit-Position: refs/heads/master@{#32150}
2020-09-21 14:31:23 +02:00
// RTC_EXCLUSIVE_LOCKS_REQUIRED(sequence_checker_)
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
rtc::VideoSinkWants VideoSourceSinkController::CurrentSettingsToSinkWants()
const {
rtc::VideoSinkWants wants;
wants.rotation_applied = rotation_applied_;
// `wants.black_frames` is not used, it always has its default value false.
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
wants.max_pixel_count =
rtc::dchecked_cast<int>(restrictions_.max_pixels_per_frame().value_or(
std::numeric_limits<int>::max()));
wants.target_pixel_count =
restrictions_.target_pixels_per_frame().has_value()
? absl::optional<int>(rtc::dchecked_cast<int>(
restrictions_.target_pixels_per_frame().value()))
: absl::nullopt;
wants.max_framerate_fps =
restrictions_.max_frame_rate().has_value()
? static_cast<int>(restrictions_.max_frame_rate().value())
: std::numeric_limits<int>::max();
wants.resolution_alignment = resolution_alignment_;
wants.max_pixel_count =
std::min(wants.max_pixel_count,
rtc::dchecked_cast<int>(pixels_per_frame_upper_limit_.value_or(
std::numeric_limits<int>::max())));
wants.max_framerate_fps =
std::min(wants.max_framerate_fps,
frame_rate_upper_limit_.has_value()
? static_cast<int>(frame_rate_upper_limit_.value())
: std::numeric_limits<int>::max());
wants.resolutions = resolutions_;
VideoStreamEncoder configuring source/sink with VideoSourceController. This is part of the work for making VideoStreamEncoder responsible for configuring its source/sink and limiting the responsibility of OveruseFrameDetectorResourceAdaptationModule to only output relevant VideoSourceRestrictions. BEFORE THIS CL Prior to this CL, OveruseFrameDetector was responsible for performing AddOrUpdateSink() on the source, which it did using its nested class VideoSourceProxy. AddOrUpdateSink() could happen for both adaptation and non-adaptation related reasons. For example: - Adaptation related: AdaptUp() or AdaptDown() happens, causing updated VideoSourceRestrictions. - Non-adaptation related: VideoStreamEncoder asks the module to reconfigure the source/sink for it, such as with SetMaxFramerateAndAlignment() or SetWantsRotationApplied(). AFTER THIS CL AddOrUpdateSink() is performed by VideoSourceController, which is owned by VideoStreamEncoder. Any reconfiguration has to go through the VideoStreamEncoder. This means that: - Non-adaptation related settings happen between VideoStreamEncoder and VideoSourceController directly (without going through the adaptation module). - Adaptation related changes can be expressed in terms of VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule only has to output the restrictions and not know or care about other source/sink settings. For now, VideoSourceController has to know about DegradationPreference. In a future CL, the DegradationPreference logic should move back to the adaptation module. The VideoSourceRestrictions are fully capable of expressing all possible source/sink values without the "modifier" that is the degradation preference. Bug: webrtc:11222 Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802 Commit-Queue: Henrik Boström <hbos@webrtc.org> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org> Reviewed-by: Evan Shrubsole <eshr@google.com> Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:27:18 +01:00
return wants;
}
} // namespace webrtc