Since HW codecs are not as well-behaved as SW codecs, we need a
way to disable the EXPECT_EQ's in the VideoProcessor integration tests
for the former. This CL introduces such an ability.
BUG=webrtc:6634
Review-Url: https://codereview.webrtc.org/2710913004
Cr-Commit-Position: refs/heads/master@{#17166}
Prior to this CL, the encoding/decoding in the VideoProcessor integration
tests were run "online", in the sense that rate allocations could be
changed in between frames. This is useful for evaluating the rate control
of SW codecs, which is one of the reasons for the existence of these
integration tests in the first place.
This CL adds a batch mode, in which the tests are run "offline". The two
main differences to the original mode are: 1) rate control metrics are
calculated after the fact, and 2) no rate allocation changes are allowed
during the test. Difference 1) is the reason for this CL, as HW codecs
that are pipelining will not work well when rate control metrics are
calculated right after a frame has been sent for encode. Difference 2)
is a side effect of the introduction of the batch mode. If we want to
be able to support online rate allocation for pipelining HW codecs in
the future, this can be introduced by adding a delay between encoding
and rate allocation. This was not deemed necessary at this point in time,
and hence this CL does not do that.
The batch mode is only intended to be used for manual experimentation
on devices with HW codecs, and the integration tests running on the
bots should thus NOT use batch mode.
BUG=webrtc:6634
Review-Url: https://codereview.webrtc.org/2707023008
Cr-Commit-Position: refs/heads/master@{#17164}
This CL removes most of the global frame state in VideoProcessor and
replaces that with a vector of frame states. This is useful for pipelining
codecs, where the encoded/decoded frame may not be immediately outputted
after it has been sent to the codec.
The callers (VideoProcessorIntegrationTest and video_quality_measurement)
still call VideoProcessor in a sequential fashion. A follow-up CL will be
submitted that enables batch mode in VideoProcessorIntegrationTest.
Note that VideoProcessor is still not thread safe. Currently, we can run
fairly well on Android due to the synchronicity of our MediaCodec wrapper,
but we still cannot run on iOS due to async issues. This will be fixed in
the future.
BUG=webrtc:6634
Review-Url: https://codereview.webrtc.org/2711133002
Cr-Commit-Position: refs/heads/master@{#17084}
Reason for revert:
The issue is now hopefully fixed.
Original issue's description:
> Revert of Add QP for FFmpeg H264 decoder. (patchset #4 id:200001 of https://codereview.webrtc.org/2649133007/ )
>
> Reason for revert:
> Let's revert this while we investigate a problem in H264 bitstream parser.
>
> Original issue's description:
> > Add QP for FFmpeg H264 decoder.
> >
> > BUG=webrtc:6541
> >
> > Review-Url: https://codereview.webrtc.org/2649133007
> > Cr-Commit-Position: refs/heads/master@{#16942}
> > Committed: 879f4f6c31
>
> TBR=sprang@webrtc.org
> # Not skipping CQ checks because original CL landed more than 1 days ago.
> BUG=webrtc:6541, chromium:697795
>
> Review-Url: https://codereview.webrtc.org/2726973003
> Cr-Commit-Position: refs/heads/master@{#16974}
> Committed: 4c6df8893eTBR=sprang@webrtc.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG=webrtc:6541, chromium:697795
Review-Url: https://codereview.webrtc.org/2735733002
Cr-Commit-Position: refs/heads/master@{#17061}
Reason for revert:
Let's revert this while we investigate a problem in H264 bitstream parser.
Original issue's description:
> Add QP for FFmpeg H264 decoder.
>
> BUG=webrtc:6541
>
> Review-Url: https://codereview.webrtc.org/2649133007
> Cr-Commit-Position: refs/heads/master@{#16942}
> Committed: 879f4f6c31TBR=sprang@webrtc.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG=webrtc:6541, chromium:697795
Review-Url: https://codereview.webrtc.org/2726973003
Cr-Commit-Position: refs/heads/master@{#16974}
This CL is broken out from a future "real" CL, that introduces
support for pipelining HW codecs to VideoProcessor. I order to
simplify the reviewing of that CL a bit, some of the cleanups are
split out here.
No functional changes are intended.
BUG=webrtc::6634
Review-Url: https://codereview.webrtc.org/2709123004
Cr-Commit-Position: refs/heads/master@{#16909}
This CL harmonizes and improves the naming of the config structs
in the VideoProcessor tests. It should have no functional implications.
CodecConfigPars -> CodecParams:
This struct mainly contains codec settings.
QualityMetrics -> QualityThresholds:
This struct contains thresholds against which the calculated
PSNR and SSIM metrics are compared to.
RateControlMetrics -> RateControlThresholds:
This struct contains thresholds against which the calculated
rate control metrics are compared to.
BUG=webrtc:6634
Review-Url: https://codereview.webrtc.org/2703333004
Cr-Commit-Position: refs/heads/master@{#16794}
The average QP of encoded frames is printed in Stats::PrintSummary.
plot_webrtc_test_logs.py: Add QP to plots.
BUG=webrtc:6634
Review-Url: https://codereview.webrtc.org/2709613005
Cr-Commit-Position: refs/heads/master@{#16790}
Reason for revert:
Necessary calls were "protected" by RTC_DCHECKs, that were optimized away in some release builds.
Replacing the RTC_DCHECKs with EXPECTs.
Original issue's description:
> Revert of Add optional visualization file writers to VideoProcessor tests. (patchset #4 id:220001 of https://codereview.webrtc.org/2700493006/ )
>
> Reason for revert:
> Breaks downstream project.
>
> Original issue's description:
> > Add optional visualization file writers to VideoProcessor tests.
> >
> > The purpose of this visualization CL is to add the ability to record
> > video at the source, after encode, and after decode, in the VideoProcessor
> > tests. These output files can then be replayed and used as a subjective
> > complement to the objective metric plots given by the existing Python
> > plotting script.
> >
> > BUG=webrtc:6634
> >
> > Review-Url: https://codereview.webrtc.org/2700493006
> > Cr-Commit-Position: refs/heads/master@{#16738}
> > Committed: 872104ac41
>
> TBR=asapersson@webrtc.org,sprang@webrtc.org,kjellander@webrtc.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=webrtc:6634
>
> Review-Url: https://codereview.webrtc.org/2708103002
> Cr-Commit-Position: refs/heads/master@{#16745}
> Committed: 2a8135a174TBR=asapersson@webrtc.org,sprang@webrtc.org,kjellander@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
BUG=webrtc:6634
Review-Url: https://codereview.webrtc.org/2706123003
Cr-Commit-Position: refs/heads/master@{#16769}
plot_webrtc_test_logs.py: Add number of used cores to figure title.
BUG=webrtc:6634
Review-Url: https://codereview.webrtc.org/2706753005
Cr-Commit-Position: refs/heads/master@{#16756}
Reason for revert:
Breaks downstream project.
Original issue's description:
> Add optional visualization file writers to VideoProcessor tests.
>
> The purpose of this visualization CL is to add the ability to record
> video at the source, after encode, and after decode, in the VideoProcessor
> tests. These output files can then be replayed and used as a subjective
> complement to the objective metric plots given by the existing Python
> plotting script.
>
> BUG=webrtc:6634
>
> Review-Url: https://codereview.webrtc.org/2700493006
> Cr-Commit-Position: refs/heads/master@{#16738}
> Committed: 872104ac41TBR=asapersson@webrtc.org,sprang@webrtc.org,kjellander@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6634
Review-Url: https://codereview.webrtc.org/2708103002
Cr-Commit-Position: refs/heads/master@{#16745}
The purpose of this visualization CL is to add the ability to record
video at the source, after encode, and after decode, in the VideoProcessor
tests. These output files can then be replayed and used as a subjective
complement to the objective metric plots given by the existing Python
plotting script.
BUG=webrtc:6634
Review-Url: https://codereview.webrtc.org/2700493006
Cr-Commit-Position: refs/heads/master@{#16738}
This CL adds the ability to _create_ HW codecs (Android and iOS) in the
VideoProcessor integration tests. Since the VideoProcessor class is not thread
safe yet, this CL does not add the ability to _use_ HW codecs in the tests. A
follow-up CL is planned that will add this ability.
This CL further adds a separate build target which is used to separate the
"plot" versions of the integration tests from the "correctness" versions. The
former will be run manually on devices, whereas the latter are used on the
trybots/buildbots to find regressions in the SW codecs. The underlying test
is the same, however.
BUG=webrtc:6634
Review-Url: https://codereview.webrtc.org/2695653002
Cr-Commit-Position: refs/heads/master@{#16716}
No point in measuring the time needed to write dropped frames to disk.
BUG=webrtc:6634
Review-Url: https://codereview.webrtc.org/2696503003
Cr-Commit-Position: refs/heads/master@{#16629}
Add tests (plot_videoprocessor_integrationtest.cc) to be used to plot stats from (not yet used).
Move VideoProcessorIntegrationTest fixture to separate file. To be used by plot_videoprocessor_integrationtest.cc.
BUG=webrtc:6634
Review-Url: https://codereview.webrtc.org/2643853002
Cr-Commit-Position: refs/heads/master@{#16528}
Reason for revert:
Fix the problem.
Original issue's description:
> Revert of Add QP sum stats for received streams. (patchset #10 id:180001 of https://codereview.webrtc.org/2649133005/ )
>
> Reason for revert:
> Breaks downstream build.
>
> Original issue's description:
> > Add QP sum stats for received streams.
> >
> > This is not implemented yet in any of the decoders.
> >
> > BUG=webrtc:6541
> >
> > Review-Url: https://codereview.webrtc.org/2649133005
> > Cr-Commit-Position: refs/heads/master@{#16475}
> > Committed: ff0e72fd16
>
> TBR=hta@webrtc.org,hbos@webrtc.org,sprang@webrtc.org,magjed@webrtc.org,stefan@webrtc.org,sakal@webrtc.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=webrtc:6541
>
> Review-Url: https://codereview.webrtc.org/2680893002 .
> Cr-Commit-Position: refs/heads/master@{#16480}
> Committed: 69fb2cca4dTBR=hta@webrtc.org,hbos@webrtc.org,sprang@webrtc.org,magjed@webrtc.org,stefan@webrtc.org,skvlad@webrtc.org
BUG=webrtc:6541
Review-Url: https://codereview.webrtc.org/2681663005
Cr-Commit-Position: refs/heads/master@{#16511}
Reason for revert:
Breaks downstream build.
Original issue's description:
> Add QP sum stats for received streams.
>
> This is not implemented yet in any of the decoders.
>
> BUG=webrtc:6541
>
> Review-Url: https://codereview.webrtc.org/2649133005
> Cr-Commit-Position: refs/heads/master@{#16475}
> Committed: ff0e72fd16TBR=hta@webrtc.org,hbos@webrtc.org,sprang@webrtc.org,magjed@webrtc.org,stefan@webrtc.org,sakal@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6541
Review-Url: https://codereview.webrtc.org/2680893002 .
Cr-Commit-Position: refs/heads/master@{#16480}
This is not implemented yet in any of the decoders.
BUG=webrtc:6541
Review-Url: https://codereview.webrtc.org/2649133005
Cr-Commit-Position: refs/heads/master@{#16475}
Moves webrtc/common_video/rotation.h and parts of
webrtc/common_video/include/video_frame_buffer.h and
webrtc/video_frame.h, and adds to a new GN target api:video_frame_api.
BUG=webrtc:5880
Review-Url: https://codereview.webrtc.org/2517173004
Cr-Commit-Position: refs/heads/master@{#15993}
Reason for revert:
Trying to re-enable this test as we're now using a newer Clang version (289944-2).
Original issue's description:
> Disable flaky test VideoProcessorIntegrationTest.ProcessNoLossChangeFrameRateFrameDropVP9
>
> This test is flaky on all platforms, not just Android. Disabling it entirely until webrtc:6057 is fixed.
>
> BUG=webrtc:6057
>
> Committed: https://crrev.com/bb66ec35739830847bfb0146cd029ca41421b2d8
> Cr-Commit-Position: refs/heads/master@{#15594}
TBR=marpan@webrtc.org,sprang@webrtc.org,skvlad@webrtc.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG=webrtc:6057
Review-Url: https://codereview.webrtc.org/2603993002
Cr-Commit-Position: refs/heads/master@{#15821}
Reason for revert:
Crashes perf tests, e.g.,
./out/Debug/webrtc_perf_tests --gtest_filter='FullStackTest.ScreenshareSlidesVP8_2TL_VeryLossyNet'
dies with an assert related to rtc::Optional.
Original issue's description:
> Delete VideoFrame default constructor, and IsZeroSize method.
>
> This ensures that the video_frame_buffer method never can return a
> null pointer.
>
> BUG=webrtc:6591
>
> Committed: https://crrev.com/bfcf561923a42005e4c7d66d8e72e5932155f997
> Cr-Commit-Position: refs/heads/master@{#15574}
TBR=magjed@webrtc.org,stefan@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6591
Review-Url: https://codereview.webrtc.org/2574123002
Cr-Commit-Position: refs/heads/master@{#15597}
This test is flaky on all platforms, not just Android. Disabling it entirely until webrtc:6057 is fixed.
BUG=webrtc:6057
Review-Url: https://codereview.webrtc.org/2568743007
Cr-Commit-Position: refs/heads/master@{#15594}
This ensures that the video_frame_buffer method never can return a
null pointer.
BUG=webrtc:6591
Review-Url: https://codereview.webrtc.org/2541863002
Cr-Commit-Position: refs/heads/master@{#15574}
This brings QualityScaler much more in line with OveruseFrameDetector.
The two classes are conceptually similar, and should be used in the
same way. The biggest changes in this CL are:
- Quality scaling is now only done in ViEEncoder and not in each
encoder implementation separately.
- QualityScaler now checks the average QP asynchronously, instead of
having to be polled on each frame.
- QualityScaler is no longer responsible for actually scaling the frames,
but has a callback to ViEEncoder that it uses to express it's desire
for lower resolution.
BUG=webrtc:6495
Review-Url: https://codereview.webrtc.org/2398963003
Cr-Commit-Position: refs/heads/master@{#15286}
Pass the selected cricket::VideoCodec to H264EncoderImpl::H264EncoderImpl. The cricket::VideoCodec contains relevant information for H264 about selected profile and packetization mode.
BUG=chromium:600254,webrtc:6402, webrtc:6337
Review-Url: https://codereview.webrtc.org/2474993002
Cr-Commit-Position: refs/heads/master@{#15270}
This is yet another reland of https://codereview.webrtc.org/2434073003/
including two fixes:
1. SimulcastRateAllocator did not handle the screenshare settings properly for numSimulcastStreams = 1. Additional test case was added for that.
2. In VideoSender, when rate allocation is updated after setting a new VideoCodec config, only update the state of the EncoderParameters, but don't actually run SetRateAllocation on the encoder itself. This caused some problems upstreams.
Please review only the changes after patch set 1.
Original description:
Extract bitrate allocation of spatial/temporal layers out of codec impl.
This CL makes a number of intervowen changes:
* Add BitrateAllocation struct, that contains a codec independent view
of how the target bitrate is distributed over spatial and temporal
layers.
* Adds the BitrateAllocator interface, which takes a bitrate and frame
rate and produces a BitrateAllocation.
* A default (non layered) implementation is added, and
SimulcastRateAllocator is extended to fully handle VP8 allocation.
This includes capturing TemporalLayer instances created by the
encoder.
* ViEEncoder now owns both the bitrate allocator and the temporal layer
factories for VP8. This allows allocation to happen fully outside of
the encoder implementation.
This refactoring will make it possible for ViEEncoder to signal the
full picture of target bitrates to the RTCP module.
BUG=webrtc:6301
R=stefan@webrtc.org
Review URL: https://codereview.webrtc.org/2510583002 .
Cr-Commit-Position: refs/heads/master@{#15105}
Chrome coding standard now discourages use of references to smart
pointers. This cl updates some recent methods to the new conventions.
BUG=webrtc:6672
Review-Url: https://codereview.webrtc.org/2477233004
Cr-Commit-Position: refs/heads/master@{#15028}
Reason for revert:
Seems to be causing flakiness in perf test:
FullStackTest.ScreenshareSlidesVP8_2TL_LossyNet
Original issue's description:
> Reland of Issue 2434073003: Extract bitrate allocation ...
>
> This is a reland of https://codereview.webrtc.org/2434073003/ including
> some fixes for failing test cases.
>
> Original description:
>
> Extract bitrate allocation of spatial/temporal layers out of codec impl.
>
> This CL makes a number of intervowen changes:
>
> * Add BitrateAllocation struct, that contains a codec independent view
> of how the target bitrate is distributed over spatial and temporal
> layers.
>
> * Adds the BitrateAllocator interface, which takes a bitrate and frame
> rate and produces a BitrateAllocation.
>
> * A default (non layered) implementation is added, and
> SimulcastRateAllocator is extended to fully handle VP8 allocation.
> This includes capturing TemporalLayer instances created by the
> encoder.
>
> * ViEEncoder now owns both the bitrate allocator and the temporal layer
> factories for VP8. This allows allocation to happen fully outside of
> the encoder implementation.
>
> This refactoring will make it possible for ViEEncoder to signal the
> full picture of target bitrates to the RTCP module.
>
> BUG=webrtc:6301
>
> Committed: https://crrev.com/647bf43dcb2fd16fccf276bd94dc4400728bb405
> Cr-Commit-Position: refs/heads/master@{#15023}
TBR=mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6301
Review-Url: https://codereview.webrtc.org/2491393002
Cr-Commit-Position: refs/heads/master@{#15026}
This is a reland of https://codereview.webrtc.org/2434073003/ including
some fixes for failing test cases.
Original description:
Extract bitrate allocation of spatial/temporal layers out of codec impl.
This CL makes a number of intervowen changes:
* Add BitrateAllocation struct, that contains a codec independent view
of how the target bitrate is distributed over spatial and temporal
layers.
* Adds the BitrateAllocator interface, which takes a bitrate and frame
rate and produces a BitrateAllocation.
* A default (non layered) implementation is added, and
SimulcastRateAllocator is extended to fully handle VP8 allocation.
This includes capturing TemporalLayer instances created by the
encoder.
* ViEEncoder now owns both the bitrate allocator and the temporal layer
factories for VP8. This allows allocation to happen fully outside of
the encoder implementation.
This refactoring will make it possible for ViEEncoder to signal the
full picture of target bitrates to the RTCP module.
BUG=webrtc:6301
Review-Url: https://codereview.webrtc.org/2488833004
Cr-Commit-Position: refs/heads/master@{#15023}
Reason for revert:
Breaks perf tests.
Original issue's description:
> Extract bitrate allocation of spatial/temporal layers out of codec impl.
>
> This CL makes a number of intervowen changes:
>
> * Add BitrateAllocation struct, that contains a codec independent view
> of how the target bitrate is distributed over spatial and temporal
> layers.
>
> * Adds the BitrateAllocator interface, which takes a bitrate and frame
> rate and produces a BitrateAllocation.
>
> * A default (non layered) implementation is added, and
> SimulcastRateAllocator is extended to fully handle VP8 allocation.
> This includes capturing TemporalLayer instances created by the
> encoder.
>
> * ViEEncoder now owns both the bitrate allocator and the temporal layer
> factories for VP8. This allows allocation to happen fully outside of
> the encoder implementation.
>
> This refactoring will make it possible for ViEEncoder to signal the
> full picture of target bitrates to the RTCP module.
>
> BUG=webrtc:6301
>
> Committed: https://crrev.com/8f46c679d24a05b3f08e02c6d91ec9637f34e24f
> Cr-Commit-Position: refs/heads/master@{#14998}
TBR=stefan@webrtc.org,perkj@webrtc.org,mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6301
Review-Url: https://codereview.webrtc.org/2489843002
Cr-Commit-Position: refs/heads/master@{#15001}
This CL makes a number of intervowen changes:
* Add BitrateAllocation struct, that contains a codec independent view
of how the target bitrate is distributed over spatial and temporal
layers.
* Adds the BitrateAllocator interface, which takes a bitrate and frame
rate and produces a BitrateAllocation.
* A default (non layered) implementation is added, and
SimulcastRateAllocator is extended to fully handle VP8 allocation.
This includes capturing TemporalLayer instances created by the
encoder.
* ViEEncoder now owns both the bitrate allocator and the temporal layer
factories for VP8. This allows allocation to happen fully outside of
the encoder implementation.
This refactoring will make it possible for ViEEncoder to signal the
full picture of target bitrates to the RTCP module.
BUG=webrtc:6301
Review-Url: https://codereview.webrtc.org/2434073003
Cr-Commit-Position: refs/heads/master@{#14998}
This refactoring allows runtime checks that functions that access
codec specific information are using the correct union member.
The API also allows replacing the union with another implementation
without changes at calling sites.
BUG=webrtc:6603
Review-Url: https://codereview.webrtc.org/2001533003
Cr-Commit-Position: refs/heads/master@{#14775}
