webrtc_m130/video/video_quality_test.h

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

148 lines
5.5 KiB
C
Raw Normal View History

Refactoring full stack and loopback tests Refactoring full stack, video and screenshare tests to use the same code basis for parametrization and initialization. This patch is done on top of recently commited full stack graphs CL https://codereview.webrtc.org/1289933003/, but virtually no changes have been made to full_stack_plot.py nor to the VideoAnalyzer in full stack, except moving it to video_quality_test.cc. Also, full_stack_samples.cc (build target) was removed and replaced with -output_filename and -duration cmdline arguments in video_loopback and screenshare_loopback. The important things to review: - video_quality_test.h Is the structure of Params good? (examples of usage can be found in full_stack.cc, video_loopback.cc and screenshare_loopback.cc) - video_quality_test.cc Is the initialization correct? The case for using Analyzer and using local renderer are different, can they be further merged? - webrtc_tests.gypi Reproducing the different bitrate settings the full stack and loopback tests had was a little bit tricky. To support both simultaneously, I added BitrateConfig to the Params struct, as well as separate start_bitrate and target_bitrate flags for loopback tests. Note: Side-by-side diff for video_quality_test.cc compares that file directly with the old full_stack.cc, so changes to VideoAnalyzer are clearly visible. Note: Recent CL I've committed added -num_temporal_layers and -sl_discard_threshold args to loopback tests. This was removed here. Support for streams and SVC will be added in a CL following this one. Review URL: https://codereview.webrtc.org/1308403003 Cr-Commit-Position: refs/heads/master@{#9969}
2015-09-17 05:30:24 -07:00
/*
* Copyright (c) 2015 The WebRTC project authors. All Rights Reserved.
*
* Use of this source code is governed by a BSD-style license
* that can be found in the LICENSE file in the root of the source
* tree. An additional intellectual property rights grant can be found
* in the file PATENTS. All contributing project authors may
* be found in the AUTHORS file in the root of the source tree.
*/
#ifndef VIDEO_VIDEO_QUALITY_TEST_H_
#define VIDEO_VIDEO_QUALITY_TEST_H_
Refactoring full stack and loopback tests Refactoring full stack, video and screenshare tests to use the same code basis for parametrization and initialization. This patch is done on top of recently commited full stack graphs CL https://codereview.webrtc.org/1289933003/, but virtually no changes have been made to full_stack_plot.py nor to the VideoAnalyzer in full stack, except moving it to video_quality_test.cc. Also, full_stack_samples.cc (build target) was removed and replaced with -output_filename and -duration cmdline arguments in video_loopback and screenshare_loopback. The important things to review: - video_quality_test.h Is the structure of Params good? (examples of usage can be found in full_stack.cc, video_loopback.cc and screenshare_loopback.cc) - video_quality_test.cc Is the initialization correct? The case for using Analyzer and using local renderer are different, can they be further merged? - webrtc_tests.gypi Reproducing the different bitrate settings the full stack and loopback tests had was a little bit tricky. To support both simultaneously, I added BitrateConfig to the Params struct, as well as separate start_bitrate and target_bitrate flags for loopback tests. Note: Side-by-side diff for video_quality_test.cc compares that file directly with the old full_stack.cc, so changes to VideoAnalyzer are clearly visible. Note: Recent CL I've committed added -num_temporal_layers and -sl_discard_threshold args to loopback tests. This was removed here. Support for streams and SVC will be added in a CL following this one. Review URL: https://codereview.webrtc.org/1308403003 Cr-Commit-Position: refs/heads/master@{#9969}
2015-09-17 05:30:24 -07:00
#include <map>
#include <memory>
Refactoring full stack and loopback tests Refactoring full stack, video and screenshare tests to use the same code basis for parametrization and initialization. This patch is done on top of recently commited full stack graphs CL https://codereview.webrtc.org/1289933003/, but virtually no changes have been made to full_stack_plot.py nor to the VideoAnalyzer in full stack, except moving it to video_quality_test.cc. Also, full_stack_samples.cc (build target) was removed and replaced with -output_filename and -duration cmdline arguments in video_loopback and screenshare_loopback. The important things to review: - video_quality_test.h Is the structure of Params good? (examples of usage can be found in full_stack.cc, video_loopback.cc and screenshare_loopback.cc) - video_quality_test.cc Is the initialization correct? The case for using Analyzer and using local renderer are different, can they be further merged? - webrtc_tests.gypi Reproducing the different bitrate settings the full stack and loopback tests had was a little bit tricky. To support both simultaneously, I added BitrateConfig to the Params struct, as well as separate start_bitrate and target_bitrate flags for loopback tests. Note: Side-by-side diff for video_quality_test.cc compares that file directly with the old full_stack.cc, so changes to VideoAnalyzer are clearly visible. Note: Recent CL I've committed added -num_temporal_layers and -sl_discard_threshold args to loopback tests. This was removed here. Support for streams and SVC will be added in a CL following this one. Review URL: https://codereview.webrtc.org/1308403003 Cr-Commit-Position: refs/heads/master@{#9969}
2015-09-17 05:30:24 -07:00
#include <string>
#include <vector>
Refactoring full stack and loopback tests Refactoring full stack, video and screenshare tests to use the same code basis for parametrization and initialization. This patch is done on top of recently commited full stack graphs CL https://codereview.webrtc.org/1289933003/, but virtually no changes have been made to full_stack_plot.py nor to the VideoAnalyzer in full stack, except moving it to video_quality_test.cc. Also, full_stack_samples.cc (build target) was removed and replaced with -output_filename and -duration cmdline arguments in video_loopback and screenshare_loopback. The important things to review: - video_quality_test.h Is the structure of Params good? (examples of usage can be found in full_stack.cc, video_loopback.cc and screenshare_loopback.cc) - video_quality_test.cc Is the initialization correct? The case for using Analyzer and using local renderer are different, can they be further merged? - webrtc_tests.gypi Reproducing the different bitrate settings the full stack and loopback tests had was a little bit tricky. To support both simultaneously, I added BitrateConfig to the Params struct, as well as separate start_bitrate and target_bitrate flags for loopback tests. Note: Side-by-side diff for video_quality_test.cc compares that file directly with the old full_stack.cc, so changes to VideoAnalyzer are clearly visible. Note: Recent CL I've committed added -num_temporal_layers and -sl_discard_threshold args to loopback tests. This was removed here. Support for streams and SVC will be added in a CL following this one. Review URL: https://codereview.webrtc.org/1308403003 Cr-Commit-Position: refs/heads/master@{#9969}
2015-09-17 05:30:24 -07:00
#include "api/fec_controller.h"
#include "api/rtc_event_log/rtc_event_log_factory.h"
#include "api/task_queue/task_queue_base.h"
#include "api/task_queue/task_queue_factory.h"
#include "api/test/frame_generator_interface.h"
#include "api/test/video_quality_test_fixture.h"
#include "api/video/video_bitrate_allocator_factory.h"
#include "call/fake_network_pipe.h"
#include "media/engine/internal_decoder_factory.h"
#include "media/engine/internal_encoder_factory.h"
#include "test/call_test.h"
#include "test/layer_filtering_transport.h"
#include "video/video_analyzer.h"
#ifdef WEBRTC_WIN
#include "modules/audio_device/win/core_audio_utility_win.h"
#include "rtc_base/win/scoped_com_initializer.h"
#endif
Refactoring full stack and loopback tests Refactoring full stack, video and screenshare tests to use the same code basis for parametrization and initialization. This patch is done on top of recently commited full stack graphs CL https://codereview.webrtc.org/1289933003/, but virtually no changes have been made to full_stack_plot.py nor to the VideoAnalyzer in full stack, except moving it to video_quality_test.cc. Also, full_stack_samples.cc (build target) was removed and replaced with -output_filename and -duration cmdline arguments in video_loopback and screenshare_loopback. The important things to review: - video_quality_test.h Is the structure of Params good? (examples of usage can be found in full_stack.cc, video_loopback.cc and screenshare_loopback.cc) - video_quality_test.cc Is the initialization correct? The case for using Analyzer and using local renderer are different, can they be further merged? - webrtc_tests.gypi Reproducing the different bitrate settings the full stack and loopback tests had was a little bit tricky. To support both simultaneously, I added BitrateConfig to the Params struct, as well as separate start_bitrate and target_bitrate flags for loopback tests. Note: Side-by-side diff for video_quality_test.cc compares that file directly with the old full_stack.cc, so changes to VideoAnalyzer are clearly visible. Note: Recent CL I've committed added -num_temporal_layers and -sl_discard_threshold args to loopback tests. This was removed here. Support for streams and SVC will be added in a CL following this one. Review URL: https://codereview.webrtc.org/1308403003 Cr-Commit-Position: refs/heads/master@{#9969}
2015-09-17 05:30:24 -07:00
namespace webrtc {
class VideoQualityTest : public test::CallTest,
public VideoQualityTestFixtureInterface {
Refactoring full stack and loopback tests Refactoring full stack, video and screenshare tests to use the same code basis for parametrization and initialization. This patch is done on top of recently commited full stack graphs CL https://codereview.webrtc.org/1289933003/, but virtually no changes have been made to full_stack_plot.py nor to the VideoAnalyzer in full stack, except moving it to video_quality_test.cc. Also, full_stack_samples.cc (build target) was removed and replaced with -output_filename and -duration cmdline arguments in video_loopback and screenshare_loopback. The important things to review: - video_quality_test.h Is the structure of Params good? (examples of usage can be found in full_stack.cc, video_loopback.cc and screenshare_loopback.cc) - video_quality_test.cc Is the initialization correct? The case for using Analyzer and using local renderer are different, can they be further merged? - webrtc_tests.gypi Reproducing the different bitrate settings the full stack and loopback tests had was a little bit tricky. To support both simultaneously, I added BitrateConfig to the Params struct, as well as separate start_bitrate and target_bitrate flags for loopback tests. Note: Side-by-side diff for video_quality_test.cc compares that file directly with the old full_stack.cc, so changes to VideoAnalyzer are clearly visible. Note: Recent CL I've committed added -num_temporal_layers and -sl_discard_threshold args to loopback tests. This was removed here. Support for streams and SVC will be added in a CL following this one. Review URL: https://codereview.webrtc.org/1308403003 Cr-Commit-Position: refs/heads/master@{#9969}
2015-09-17 05:30:24 -07:00
public:
Revert "Revert "Enables PeerConnectionFactory using external fec controller"" This reverts commit 00733015fafbbc61ddc12dfdc88b21a9fcd9d122. Reason for revert: The reason for a downstream test failure on the original commit and a workaround has been found. Solution is to keep a PeerConnectionFactory constructor implementation as the same as before. Original change's description: > Revert "Enables PeerConnectionFactory using external fec controller" > > This reverts commit 4f07bdb25567d8ef528311e0b50a62c61d543fc3. > > Reason for revert: Speculatively reverting, because downstream test is now hitting "PeerConnectionFactory.initialize was not called before creating a PeerConnectionFactory" error, even though it did call initialize. I don't see how any change in this CL could cause that, but it's the only CL on the blamelist, and it does modify PeerConnectionFactory.java > > Original change's description: > > Enables PeerConnectionFactory using external fec controller > > > > Bug: webrtc:8799 > > Change-Id: Ieb2cf6163b9a83844ab9ed4822b4a7f1db4c24b8 > > Reviewed-on: https://webrtc-review.googlesource.com/43961 > > Commit-Queue: Ying Wang <yinwa@webrtc.org> > > Reviewed-by: Stefan Holmer <stefan@webrtc.org> > > Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> > > Reviewed-by: Niels Moller <nisse@webrtc.org> > > Reviewed-by: Sami Kalliomäki <sakal@webrtc.org> > > Cr-Commit-Position: refs/heads/master@{#22038} > > TBR=sakal@webrtc.org,kwiberg@webrtc.org,nisse@webrtc.org,stefan@webrtc.org,yinwa@webrtc.org > > Change-Id: I95868c35d6f9973e0ebf563814cd71d0fcbd433d > No-Presubmit: true > No-Tree-Checks: true > No-Try: true > Bug: webrtc:8799 > Reviewed-on: https://webrtc-review.googlesource.com/54080 > Reviewed-by: Taylor Brandstetter <deadbeef@webrtc.org> > Commit-Queue: Taylor Brandstetter <deadbeef@webrtc.org> > Cr-Commit-Position: refs/heads/master@{#22040} TBR=deadbeef@webrtc.org,sakal@webrtc.org,kwiberg@webrtc.org,nisse@webrtc.org,stefan@webrtc.org,yinwa@webrtc.org Bug: webrtc:8799 Change-Id: If9f3292bfcc739782967530c49f006d0abbc38a8 Reviewed-on: https://webrtc-review.googlesource.com/55400 Commit-Queue: Ying Wang <yinwa@webrtc.org> Reviewed-by: Ying Wang <yinwa@webrtc.org> Cr-Commit-Position: refs/heads/master@{#22100}
2018-02-20 12:50:27 +01:00
explicit VideoQualityTest(
std::unique_ptr<InjectionComponents> injection_components);
void RunWithAnalyzer(const Params& params) override;
void RunWithRenderers(const Params& params) override;
const std::map<uint8_t, webrtc::MediaType>& payload_type_map() override {
return payload_type_map_;
}
Refactoring full stack and loopback tests Refactoring full stack, video and screenshare tests to use the same code basis for parametrization and initialization. This patch is done on top of recently commited full stack graphs CL https://codereview.webrtc.org/1289933003/, but virtually no changes have been made to full_stack_plot.py nor to the VideoAnalyzer in full stack, except moving it to video_quality_test.cc. Also, full_stack_samples.cc (build target) was removed and replaced with -output_filename and -duration cmdline arguments in video_loopback and screenshare_loopback. The important things to review: - video_quality_test.h Is the structure of Params good? (examples of usage can be found in full_stack.cc, video_loopback.cc and screenshare_loopback.cc) - video_quality_test.cc Is the initialization correct? The case for using Analyzer and using local renderer are different, can they be further merged? - webrtc_tests.gypi Reproducing the different bitrate settings the full stack and loopback tests had was a little bit tricky. To support both simultaneously, I added BitrateConfig to the Params struct, as well as separate start_bitrate and target_bitrate flags for loopback tests. Note: Side-by-side diff for video_quality_test.cc compares that file directly with the old full_stack.cc, so changes to VideoAnalyzer are clearly visible. Note: Recent CL I've committed added -num_temporal_layers and -sl_discard_threshold args to loopback tests. This was removed here. Support for streams and SVC will be added in a CL following this one. Review URL: https://codereview.webrtc.org/1308403003 Cr-Commit-Position: refs/heads/master@{#9969}
2015-09-17 05:30:24 -07:00
static void FillScalabilitySettings(
Params* params,
size_t video_idx,
const std::vector<std::string>& stream_descriptors,
int num_streams,
size_t selected_stream,
int num_spatial_layers,
int selected_sl,
InterLayerPredMode inter_layer_pred,
const std::vector<std::string>& sl_descriptors);
// Helper static methods.
static VideoStream DefaultVideoStream(const Params& params, size_t video_idx);
static VideoStream DefaultThumbnailStream();
static std::vector<int> ParseCSV(const std::string& str);
Refactoring full stack and loopback tests Refactoring full stack, video and screenshare tests to use the same code basis for parametrization and initialization. This patch is done on top of recently commited full stack graphs CL https://codereview.webrtc.org/1289933003/, but virtually no changes have been made to full_stack_plot.py nor to the VideoAnalyzer in full stack, except moving it to video_quality_test.cc. Also, full_stack_samples.cc (build target) was removed and replaced with -output_filename and -duration cmdline arguments in video_loopback and screenshare_loopback. The important things to review: - video_quality_test.h Is the structure of Params good? (examples of usage can be found in full_stack.cc, video_loopback.cc and screenshare_loopback.cc) - video_quality_test.cc Is the initialization correct? The case for using Analyzer and using local renderer are different, can they be further merged? - webrtc_tests.gypi Reproducing the different bitrate settings the full stack and loopback tests had was a little bit tricky. To support both simultaneously, I added BitrateConfig to the Params struct, as well as separate start_bitrate and target_bitrate flags for loopback tests. Note: Side-by-side diff for video_quality_test.cc compares that file directly with the old full_stack.cc, so changes to VideoAnalyzer are clearly visible. Note: Recent CL I've committed added -num_temporal_layers and -sl_discard_threshold args to loopback tests. This was removed here. Support for streams and SVC will be added in a CL following this one. Review URL: https://codereview.webrtc.org/1308403003 Cr-Commit-Position: refs/heads/master@{#9969}
2015-09-17 05:30:24 -07:00
protected:
std::map<uint8_t, webrtc::MediaType> payload_type_map_;
Refactoring full stack and loopback tests Refactoring full stack, video and screenshare tests to use the same code basis for parametrization and initialization. This patch is done on top of recently commited full stack graphs CL https://codereview.webrtc.org/1289933003/, but virtually no changes have been made to full_stack_plot.py nor to the VideoAnalyzer in full stack, except moving it to video_quality_test.cc. Also, full_stack_samples.cc (build target) was removed and replaced with -output_filename and -duration cmdline arguments in video_loopback and screenshare_loopback. The important things to review: - video_quality_test.h Is the structure of Params good? (examples of usage can be found in full_stack.cc, video_loopback.cc and screenshare_loopback.cc) - video_quality_test.cc Is the initialization correct? The case for using Analyzer and using local renderer are different, can they be further merged? - webrtc_tests.gypi Reproducing the different bitrate settings the full stack and loopback tests had was a little bit tricky. To support both simultaneously, I added BitrateConfig to the Params struct, as well as separate start_bitrate and target_bitrate flags for loopback tests. Note: Side-by-side diff for video_quality_test.cc compares that file directly with the old full_stack.cc, so changes to VideoAnalyzer are clearly visible. Note: Recent CL I've committed added -num_temporal_layers and -sl_discard_threshold args to loopback tests. This was removed here. Support for streams and SVC will be added in a CL following this one. Review URL: https://codereview.webrtc.org/1308403003 Cr-Commit-Position: refs/heads/master@{#9969}
2015-09-17 05:30:24 -07:00
// No-op implementation to be able to instantiate this class from non-TEST_F
// locations.
void TestBody() override;
// Helper methods accessing only params_.
std::string GenerateGraphTitle() const;
void CheckParamsAndInjectionComponents();
// Helper methods for setting up the call.
void CreateCapturers();
std::unique_ptr<test::FrameGeneratorInterface> CreateFrameGenerator(
size_t video_idx);
void SetupThumbnailCapturers(size_t num_thumbnail_streams);
std::unique_ptr<VideoDecoder> CreateVideoDecoder(
const SdpVideoFormat& format);
std::unique_ptr<VideoEncoder> CreateVideoEncoder(const SdpVideoFormat& format,
VideoAnalyzer* analyzer);
void SetupVideo(Transport* send_transport, Transport* recv_transport);
void SetupThumbnails(Transport* send_transport, Transport* recv_transport);
void StartAudioStreams();
void StartThumbnails();
void StopThumbnails();
void DestroyThumbnailStreams();
// Helper method for creating a real ADM (using hardware) for all platforms.
rtc::scoped_refptr<AudioDeviceModule> CreateAudioDevice();
void InitializeAudioDevice(Call::Config* send_call_config,
Call::Config* recv_call_config,
bool use_real_adm);
void SetupAudio(Transport* transport);
Refactoring full stack and loopback tests Refactoring full stack, video and screenshare tests to use the same code basis for parametrization and initialization. This patch is done on top of recently commited full stack graphs CL https://codereview.webrtc.org/1289933003/, but virtually no changes have been made to full_stack_plot.py nor to the VideoAnalyzer in full stack, except moving it to video_quality_test.cc. Also, full_stack_samples.cc (build target) was removed and replaced with -output_filename and -duration cmdline arguments in video_loopback and screenshare_loopback. The important things to review: - video_quality_test.h Is the structure of Params good? (examples of usage can be found in full_stack.cc, video_loopback.cc and screenshare_loopback.cc) - video_quality_test.cc Is the initialization correct? The case for using Analyzer and using local renderer are different, can they be further merged? - webrtc_tests.gypi Reproducing the different bitrate settings the full stack and loopback tests had was a little bit tricky. To support both simultaneously, I added BitrateConfig to the Params struct, as well as separate start_bitrate and target_bitrate flags for loopback tests. Note: Side-by-side diff for video_quality_test.cc compares that file directly with the old full_stack.cc, so changes to VideoAnalyzer are clearly visible. Note: Recent CL I've committed added -num_temporal_layers and -sl_discard_threshold args to loopback tests. This was removed here. Support for streams and SVC will be added in a CL following this one. Review URL: https://codereview.webrtc.org/1308403003 Cr-Commit-Position: refs/heads/master@{#9969}
2015-09-17 05:30:24 -07:00
void StartEncodedFrameLogs(VideoReceiveStream* stream);
virtual std::unique_ptr<test::LayerFilteringTransport> CreateSendTransport();
virtual std::unique_ptr<test::DirectTransport> CreateReceiveTransport();
std::vector<std::unique_ptr<rtc::VideoSourceInterface<VideoFrame>>>
thumbnail_capturers_;
Clock* const clock_;
const std::unique_ptr<TaskQueueFactory> task_queue_factory_;
RtcEventLogFactory rtc_event_log_factory_;
test::FunctionVideoDecoderFactory video_decoder_factory_;
std::unique_ptr<VideoDecoderFactory> decoder_factory_;
test::FunctionVideoEncoderFactory video_encoder_factory_;
test::FunctionVideoEncoderFactory video_encoder_factory_with_analyzer_;
std::unique_ptr<VideoBitrateAllocatorFactory>
video_bitrate_allocator_factory_;
std::unique_ptr<VideoEncoderFactory> encoder_factory_;
std::vector<VideoSendStream::Config> thumbnail_send_configs_;
std::vector<VideoEncoderConfig> thumbnail_encoder_configs_;
std::vector<VideoSendStream*> thumbnail_send_streams_;
std::vector<VideoReceiveStream::Config> thumbnail_receive_configs_;
std::vector<VideoReceiveStream*> thumbnail_receive_streams_;
int receive_logs_;
int send_logs_;
Params params_;
std::unique_ptr<InjectionComponents> injection_components_;
// Set non-null when running with analyzer.
std::unique_ptr<VideoAnalyzer> analyzer_;
// Note: not same as similarly named member in CallTest. This is the number of
// separate send streams, the one in CallTest is the number of substreams for
// a single send stream.
size_t num_video_streams_;
#ifdef WEBRTC_WIN
// Windows Core Audio based ADM needs to run on a COM initialized thread.
// Only referenced in combination with --audio --use_real_adm flags.
std::unique_ptr<ScopedCOMInitializer> com_initializer_;
#endif
Refactoring full stack and loopback tests Refactoring full stack, video and screenshare tests to use the same code basis for parametrization and initialization. This patch is done on top of recently commited full stack graphs CL https://codereview.webrtc.org/1289933003/, but virtually no changes have been made to full_stack_plot.py nor to the VideoAnalyzer in full stack, except moving it to video_quality_test.cc. Also, full_stack_samples.cc (build target) was removed and replaced with -output_filename and -duration cmdline arguments in video_loopback and screenshare_loopback. The important things to review: - video_quality_test.h Is the structure of Params good? (examples of usage can be found in full_stack.cc, video_loopback.cc and screenshare_loopback.cc) - video_quality_test.cc Is the initialization correct? The case for using Analyzer and using local renderer are different, can they be further merged? - webrtc_tests.gypi Reproducing the different bitrate settings the full stack and loopback tests had was a little bit tricky. To support both simultaneously, I added BitrateConfig to the Params struct, as well as separate start_bitrate and target_bitrate flags for loopback tests. Note: Side-by-side diff for video_quality_test.cc compares that file directly with the old full_stack.cc, so changes to VideoAnalyzer are clearly visible. Note: Recent CL I've committed added -num_temporal_layers and -sl_discard_threshold args to loopback tests. This was removed here. Support for streams and SVC will be added in a CL following this one. Review URL: https://codereview.webrtc.org/1308403003 Cr-Commit-Position: refs/heads/master@{#9969}
2015-09-17 05:30:24 -07:00
};
} // namespace webrtc
#endif // VIDEO_VIDEO_QUALITY_TEST_H_