This CL adds major render pipeline changes to the AEC3 code. The reason
for these are that
1) It allows the echo removal unit to receive information about the content
in bands beyond band 0, thereby allowing removal of high-frequency
echoes
2) It allows more controlled handling of the render buffers, allowing proper
buffer behaviour during capture glitches and clock-drift.
Unfortunately, the render pipeline caused a lot of related changes in much
of the rest of the AEC3 files. Most of these are, however, caused by
a change of class name.
Another unfortunate effect of this CL, is that a number of unittest cease to
compile. I chose to temporarily solve that by removing them from the
build using #if/#endif. The reason for that is that those will anyway again
need to be changed in the next review, and doing like this avoids them
having to be reviewed twice.
BUG=webrtc:6018
Review-Url: https://codereview.webrtc.org/2784023002
Cr-Commit-Position: refs/heads/master@{#17547}
Reason for revert:
I will try to reland next week because it is causing some problems.
Original issue's description:
> To accommodate some downstream WebRTC users we need to loosen
> the coupling between our code and the //third_party/protobuf.
>
> This includes using typedefs to define strings instead of
> assuming std::string.
>
> After this refactoring it will be possible to link with other
> protobuf implementations than the current one.
>
> We moved the PRESUBMIT check to another CL [1]. The goal of this
> presubmit is to avoid the direct usage of google::protobuf outside
> of the webrtc/base/protobuf_utils.h header file.
>
> [1] - https://codereview.webrtc.org/2753823003/
>
> BUG=webrtc:7340
> NOTRY=True
>
> Review-Url: https://codereview.webrtc.org/2747863003
> Cr-Commit-Position: refs/heads/master@{#17466}
> Committed: 16ab93b952TBR=kjellander@webrtc.org,henrik.lundin@webrtc.org,kwiberg@webrtc.org,michaelt@webrtc.org,peah@webrtc.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG=webrtc:7340
Review-Url: https://codereview.webrtc.org/2786363002
Cr-Commit-Position: refs/heads/master@{#17483}
the coupling between our code and the //third_party/protobuf.
This includes using typedefs to define strings instead of
assuming std::string.
After this refactoring it will be possible to link with other
protobuf implementations than the current one.
We moved the PRESUBMIT check to another CL [1]. The goal of this
presubmit is to avoid the direct usage of google::protobuf outside
of the webrtc/base/protobuf_utils.h header file.
[1] - https://codereview.webrtc.org/2753823003/
BUG=webrtc:7340
NOTRY=True
Review-Url: https://codereview.webrtc.org/2747863003
Cr-Commit-Position: refs/heads/master@{#17466}
Currently no lock is taken when returning echo likelihood stats, which causes a race condition between the thread getting the stats and the thread running the echo detector. This CL resolves the issue by adding locking.
BUG=webrtc:7346
Review-Url: https://codereview.webrtc.org/2749973003
Cr-Commit-Position: refs/heads/master@{#17270}
AEC3 and AEC2 are separate submodules in APM. This CL ensures that AEC3
deactivates AEC2 if both are active at the same time.
BUG=webrtc:6018
Review-Url: https://codereview.webrtc.org/2675863004
Cr-Commit-Position: refs/heads/master@{#16443}
that the level of the output in the audio processing
module is monitored. This CL adds that.
BUG=webrtc:6181, webrtc:6183, webrtc:6220
Review-Url: https://codereview.webrtc.org/2549143004
Cr-Commit-Position: refs/heads/master@{#15718}
It will be followed by a number of other CLs that extends this framework.
BUG=webrtc:6018
Review-Url: https://codereview.webrtc.org/2567513003
Cr-Commit-Position: refs/heads/master@{#15593}
Reason for revert:
The downstream problem is now fixed, and this should be good to land again.
Original issue's description:
> Revert of APM: Change 3 UMA metrics to fewer but linearly distributed buckets (patchset #2 id:20001 of https://codereview.webrtc.org/2547593002/ )
>
> Reason for revert:
> Breaks down-stream dependencies.
>
> Original issue's description:
> > APM: Change 3 UMA metrics to fewer but linearly distributed buckets
> >
> > In this change WebRTC.Audio.ApmCaptureInputLevel{Average,Peak} are
> > changed to 64 buckets, while WebRTC.Audio.AgcLevel is changed to 50
> > buckets. All three are changed to have linear spacing between buckets.
> >
> > Also, the metrics are renamed to avoid stats conflicts because of different bucket schemes:
> > - WebRTC.Audio.AgcLevel -> WebRTC.Audio.AgcSetLevel
> > - WebRTC.Audio.ApmCaptureInputLevelAverage -> WebRTC.Audio.ApmCaptureInputLevelAverageRms
> > - WebRTC.Audio.ApmCaptureInputLevelPeakRms -> WebRTC.Audio.ApmCaptureInputLevelPeakRms
> >
> > BUG=webrtc:6622
> > CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_chromium_rel_ng
> >
> > Committed: https://crrev.com/49715fe3be17d8579586d5bc954d626126d53415
> > Cr-Commit-Position: refs/heads/master@{#15418}
>
> TBR=peah@webrtc.org,rkaplow@chromium.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=webrtc:6622
>
> Committed: https://crrev.com/63407a9b6ae6f3fc096e01d64e46c6d21d86b517
> Cr-Commit-Position: refs/heads/master@{#15420}
TBR=peah@webrtc.org,rkaplow@chromium.org
BUG=webrtc:6622
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_chromium_rel_ng
Review-Url: https://codereview.webrtc.org/2551863003
Cr-Commit-Position: refs/heads/master@{#15442}
This change enables experimentation with the clipping minimum level
parameter in the gain control.
BUG=webrtc:6622
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_chromium_rel_ng
Review-Url: https://codereview.webrtc.org/2543753006
Cr-Commit-Position: refs/heads/master@{#15426}
Reason for revert:
Breaks down-stream dependencies.
Original issue's description:
> APM: Change 3 UMA metrics to fewer but linearly distributed buckets
>
> In this change WebRTC.Audio.ApmCaptureInputLevel{Average,Peak} are
> changed to 64 buckets, while WebRTC.Audio.AgcLevel is changed to 50
> buckets. All three are changed to have linear spacing between buckets.
>
> Also, the metrics are renamed to avoid stats conflicts because of different bucket schemes:
> - WebRTC.Audio.AgcLevel -> WebRTC.Audio.AgcSetLevel
> - WebRTC.Audio.ApmCaptureInputLevelAverage -> WebRTC.Audio.ApmCaptureInputLevelAverageRms
> - WebRTC.Audio.ApmCaptureInputLevelPeakRms -> WebRTC.Audio.ApmCaptureInputLevelPeakRms
>
> BUG=webrtc:6622
> CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_chromium_rel_ng
>
> Committed: https://crrev.com/49715fe3be17d8579586d5bc954d626126d53415
> Cr-Commit-Position: refs/heads/master@{#15418}
TBR=peah@webrtc.org,rkaplow@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6622
Review-Url: https://codereview.webrtc.org/2548333002
Cr-Commit-Position: refs/heads/master@{#15420}
In this change WebRTC.Audio.ApmCaptureInputLevel{Average,Peak} are
changed to 64 buckets, while WebRTC.Audio.AgcLevel is changed to 50
buckets. All three are changed to have linear spacing between buckets.
Also, the metrics are renamed to avoid stats conflicts because of different bucket schemes:
- WebRTC.Audio.AgcLevel -> WebRTC.Audio.AgcSetLevel
- WebRTC.Audio.ApmCaptureInputLevelAverage -> WebRTC.Audio.ApmCaptureInputLevelAverageRms
- WebRTC.Audio.ApmCaptureInputLevelPeakRms -> WebRTC.Audio.ApmCaptureInputLevelPeakRms
BUG=webrtc:6622
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.linux:linux_chromium_rel_ng
Review-Url: https://codereview.webrtc.org/2547593002
Cr-Commit-Position: refs/heads/master@{#15418}
This CL adds WebRTC.Audio.ApmCaptureInputLevelAverage and
WebRTC.Audio.ApmCaptureInputLevelPeak. The metrics are updated once
every 10 seconds.
BUG=webrtc:6622
Review-Url: https://codereview.webrtc.org/2534473004
Cr-Commit-Position: refs/heads/master@{#15300}
There's no longer any need to make the two arguments have the same
signedness, so we can drop the "u" suffix on literal integer
arguments.
NOPRESUBMIT=true
BUG=webrtc:6645
Review-Url: https://codereview.webrtc.org/2535593002
Cr-Commit-Position: refs/heads/master@{#15280}
the APM parameters to the high-pass filter.
The introduction will be done in three steps:
1) This CL which introduces the new scheme and
changes the code in webrtcvoiceengine.cc to use it.
2) Introduce the scheme into upstream code.
3) Remove the HighPassFilter interface in APM.
BUG=webrtc::6220, webrtc::6296, webrtc::6297, webrtc::6181, webrtc::5298
Review-Url: https://codereview.webrtc.org/2415403002
Cr-Commit-Position: refs/heads/master@{#15197}
This adds two command-line flags to audioproc_f: -red and -red_graph, which can be used to enable/disable the RED, and to set the output path for the graph. The graph is generated as a python script that depends on matplotlib and numpy to display the graph.
BUG=webrtc:6525
Review-Url: https://codereview.webrtc.org/2486763002
Cr-Commit-Position: refs/heads/master@{#15069}
In the new APM statistics interface, the default values did not match those previously used in AudioSendStream::Stats.
BUG=webrtc:6525
Review-Url: https://codereview.webrtc.org/2469783002
Cr-Commit-Position: refs/heads/master@{#14896}
This CL does not contain the actual algorithm, but only creates an empty processing component and connects the right signals to it. The algorithm will be added in a follow-up CL.
BUG=webrtc:6525
Review-Url: https://codereview.webrtc.org/2405403003
Cr-Commit-Position: refs/heads/master@{#14820}
Reason for revert:
This CL breaks internal dependencies.
Original issue's description:
> New statistics interface for APM
>
> This adds functions to enable and retrieve statistics from APM. These functions are not yet called, which will happen in a follow-up CL.
>
> BUG=webrtc:6525
>
> Committed: https://crrev.com/8b8d3e4c30e8ea3846b58dfd36d1fd35a7799df4
> Cr-Commit-Position: refs/heads/master@{#14810}
TBR=peah@webrtc.org,solenberg@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6525
Review-Url: https://codereview.webrtc.org/2456333002
Cr-Commit-Position: refs/heads/master@{#14814}
aecdump recordings in an efficient manner, it is
important to be able to use a standardized analysis
script. For this to be feasible, data log points should
be present.
This CL adds those logpoints as well as the framework
needed to for those to work.
BUG=webrtc:6564
Review-Url: https://codereview.webrtc.org/2457783003
Cr-Commit-Position: refs/heads/master@{#14812}
This adds functions to enable and retrieve statistics from APM. These functions are not yet called, which will happen in a follow-up CL.
BUG=webrtc:6525
Review-Url: https://codereview.webrtc.org/2433153003
Cr-Commit-Position: refs/heads/master@{#14810}
Several subcomponents inside APM copy render audio from
the render side to the capture side using the same
pattern. Currently this is done independently for the
submodules.
This CL moves the the AGC functionality for this into
APM.
BUG=webrtc:5298, webrtc:6540
Review-Url: https://codereview.webrtc.org/2444283002
Cr-Commit-Position: refs/heads/master@{#14770}
the render side to the capture side using the same
pattern. Currently this is done independently for the
submodules.
This CL moves the the AECM functionality for this into
APM.
BUG=webrtc:5298, webrtc:6540
Review-Url: https://codereview.webrtc.org/2444793005
Cr-Commit-Position: refs/heads/master@{#14768}
the render side to the capture side using the same
pattern. Currently this is done independently for the
submodules.
This CL moves the the AEC functionality for this into
APM.
BUG=webrtc:5298, webrtc:6540
Review-Url: https://codereview.webrtc.org/2427553003
Cr-Commit-Position: refs/heads/master@{#14726}
to the functionality in the audio processing module.
Therefore, it should be a pure interface.
This CL ensures that is the case.
BUG=webrtc:6515
Review-Url: https://codereview.webrtc.org/2406193002
Cr-Commit-Position: refs/heads/master@{#14608}
receive a signal level to use initially, instead of the
default initial signal level.
The initial form of the CL
(https://codereview.webrtc.org/2254973003/) was reverted
due to down-stream dependencies. These have been resolved,
but the CL needed to be revised according to the new scheme
for passing parameters to the audio processing module.
Therefore, please review this CL as if it is new.
TBR=aleloi@webrtc.org
BUG=webrtc:6386
Review-Url: https://codereview.webrtc.org/2337083002
Cr-Commit-Position: refs/heads/master@{#14579}
Before this CL, the IntelligibilityEnhancer introduced a processing delay to the lower band, without compensating for it in the higher bands. This CL corrects this.
BUG=b/30780909
R=henrik.lundin@webrtc.org, peah@webrtc.org
Review URL: https://codereview.webrtc.org/2320833002 .
Cr-Commit-Position: refs/heads/master@{#14311}
methods inside the audio processing module for the purpose of
increasing code readability.
BUG=
Review-Url: https://codereview.webrtc.org/2335633002
Cr-Commit-Position: refs/heads/master@{#14269}
This requires marking a bunch of compile-time constants "constexpr"
instead of just "const".
Review-Url: https://codereview.webrtc.org/2335483003
Cr-Commit-Position: refs/heads/master@{#14199}
behavior of the audio processing module is quite complex and hard to
implement in a threadsafe and efficient manner. Therefore a new
scheme for setting the parameters in the audio processing module is
introduced in this CL.
The idea is to roll this scheme out gradually and as a first functionality
in the audio processing module where this is applied the level controller
was chosen. This CL includes the replacement of the Config-based
level controller scheme with the new scheme.
TBR=henrik.lundin@webrtc.org, solenberg@webrtc.org,
BUG=webrtc:5298
Review-Url: https://codereview.webrtc.org/2338493002
Cr-Commit-Position: refs/heads/master@{#14190}
Reason for revert:
Interface change in the mock breaks downstream code.
Original issue's description:
> The current scheme for setting parameters and specifying the behavior
> of the audio processing module is quite complex and hard to implement
> in a threadsafe and efficient manner. Therefore a new scheme for setting
> the parameters in the audio processing module is introduced in this CL.
>
> The idea is to roll this scheme out gradually and as a first functionality
> in the audio processing module where this is applied the level controller
> was chosen. This CL includes the replacement of the Config-based
> level controller scheme with the new scheme.
>
> BUG=webrtc:5298
>
> Committed: https://crrev.com/c8bbe3fe9aad9e9a1189a42dcaa8f5d6c261ecc8
> Cr-Commit-Position: refs/heads/master@{#14171}
TBR=solenberg@webrtc.org,henrik.lundin@webrtc.org,peah@webrtc.org
BUG=webrtc:5298
NOTRY=True
Review-Url: https://codereview.webrtc.org/2334583002
Cr-Commit-Position: refs/heads/master@{#14177}
functionalities doing sample-rate conversions:
-The implicit resampling done in the AudioBuffer CopyTo,
CopyFrom, InterleaveTo and DeinterleaveFrom methods.
-The multi-band splitting scheme.
The selection of rates in these have been difficult and
complicated, partly due to that the APM API which allows
for activating the APM submodules without notifying
the APM.
This CL adds functionality that for each capture frame
polls all submodules for whether they are active or not
and compares this against a cached result.
Furthermore, new functionality is added that based on the
results of the comparison do a reinitialization of the APM.
This has several advantages
-The code deciding on whether to analysis and synthesis is
needed for the bandsplitting can be much simplified and
centralized.
-The selection of the processing rate can be done such as
to avoid the implicit resampling that was in some cases
unnecessarily done.
-The optimization for whether an output copy is needed
that was done to improve performance due to the implicit
resampling is no longer needed, which simplifies the
code and makes it less error-prone in the sense that
is no longer neccessary to keep track of whether any
module has changed the signal.
Finally, it should be noted that the polling of the state
for all the submodules was done previously as well, but in
a less obvious and distributed manner.
BUG=webrtc:6181, webrtc:6220, webrtc:5298, webrtc:6296, webrtc:6298, webrtc:6297
Review-Url: https://codereview.webrtc.org/2304123002
Cr-Commit-Position: refs/heads/master@{#14175}
of the audio processing module is quite complex and hard to implement
in a threadsafe and efficient manner. Therefore a new scheme for setting
the parameters in the audio processing module is introduced in this CL.
The idea is to roll this scheme out gradually and as a first functionality
in the audio processing module where this is applied the level controller
was chosen. This CL includes the replacement of the Config-based
level controller scheme with the new scheme.
BUG=webrtc:5298
Review-Url: https://codereview.webrtc.org/2292863002
Cr-Commit-Position: refs/heads/master@{#14171}
when building the code for ARM.
The intention is to follow up this CL with other CLs that
further addresses the internal resampling in APM
BUG=webrtc:6181
Review-Url: https://codereview.webrtc.org/2265473003
Cr-Commit-Position: refs/heads/master@{#13974}
Reason for revert:
This caused build breakage due to upstream dependencies.
These dependencies need to be resolved before landing the CL.
Original issue's description:
> This CL adds functionality in the level controller to
> receive a signal level to use initially, instead of the
> default initial signal level.
>
> BUG=
>
> Committed: https://crrev.com/57fec1d828113241186e78710ec5e851cc1a0e81
> Cr-Commit-Position: refs/heads/master@{#13931}
TBR=henrik.lundin@webrtc.org,aleloi@webrtc.org,solenberg@webrtc.org,henrika@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=
Review-Url: https://codereview.webrtc.org/2283793002
Cr-Commit-Position: refs/heads/master@{#13936}
receive a signal level to use initially, instead of the
default initial signal level.
BUG=
Review-Url: https://codereview.webrtc.org/2254973003
Cr-Commit-Position: refs/heads/master@{#13931}
It does:
-Handle saturations in a better manner by adding different gain change
step sizes for upwards and downwards changes, as well as when there
is saturation.
-Handle conditions with initial noise-only regions in a better way by
setting a high initial peak level estimate which is gradually reduced until
certainty about the peak level is achieved.
-Limit the maximum gain to limit noise amplification, and to reflect that it
initially is intended to be used in cascade with the fixed digital AGC mode.
-Lower the maximum allowed stationary noise floor to reduce the risk of
excessive noise amplification.
-Lower the target gain to reduce the risk of causing the AEC on the other
end to fail due to high playout levels triggering nonlinearities.
This also reduces the risk for saturation.
-Handle the noise-only regions in a better manner.
NOTRY=true
TBR=aleloi
BUG=webrtc:5920
Review-Url: https://codereview.webrtc.org/2111553002
Cr-Commit-Position: refs/heads/master@{#13350}
