Before this change, the APM limiter used in FrameCombiner (a
sub-component of AudioMixer) only gets to process the data when the
number of non-muted streams is >1. If this number varies between <=1
and >1, the limiter's view of the data will have gaps during the
periods with <= 1 active stream.
This leads to discontinuities in the applied gain. These
discontinuities cause clicks in the output audio. This change
activates APM limiter processing based on the number of audio streams,
independently of their mutedness status.
BUG=chromium:695993
Review-Url: https://codereview.webrtc.org/2776113002
Cr-Commit-Position: refs/heads/master@{#17442}
AudioMixerImpl::CreateWithOutputRateCalculatorAndLimiter(rate_calculator, bool limiter)
was added to create a mixer without the limiter subcomponent. Calling
it "Create with ... *and* limiter" is counterintuitive.
Renamed to simply 'Create'.
TBR=solenberg@webrtc.org
BUG=webrtc:7167
Review-Url: https://codereview.webrtc.org/2709523006
Cr-Commit-Position: refs/heads/master@{#16755}
The APM limiter is a component for keeping the audio from clipping by smoothly reducing the amplitude of the audio samples. It can be rather expensive because of band-splitting & merging. Also, experiments indicate that it is of questionable benefit (adding several sources of human speech almost never cause clipping).
To optionally disable the limiter, this CL does some refactoring on the (quite large) AudioMixerImpl. Functionality related to actual addition of frames and handling AudioFrame meta-data (sample_rate, num_channels, samples_per_channel, time_stamp, elapsed_time_ms) is broken out in a new sub-component called FrameCombiner.
The FrameCombiner is initialized with a 'use_limiter' flag. To create a mixer without using the APM limiter
Inside of FrameCombiner, the meta-data handling and the audio sample addition are kept divided from each other.
This also fixes a few minor GN issues so that warnings do not have to be suppressed.
BUG=webrtc:7167
Review-Url: https://codereview.webrtc.org/2692333002
Cr-Commit-Position: refs/heads/master@{#16742}
This CL breaks out the output sample rate calculation from
webrtc::AudioMixerImpl. A new OutputRateCalculator interface is added
to make the sample rate configurable. There are at least three reasons
for this change:
1. The mixer will be used for an internal project, in which no
resampling is done after the mixing. There the sample rate should
be static. Currently, it can differ across mix iterations and
depends on the number of audio sources. If there are no sources,
the WebRTC mixer behavior is to produce silence at 48 kHz.
2. A planned change to WebRTC will make audio processing steps
happen at constant sample rates. A configurable sample rate
calculator will make the transition simpler for the mixer.
3. The current mixer design is a single large file. Behavior is not
always simple to change (e.g. as in this case to mix at a
constant rate), unrelated behavior can be broken, reusing the
mixer in internal projects is tricky. Using DI for the sample
rate calculation solves parts of these issues.
Changes:
The protected mixer c-tor now takes
unique_ptr<OutputRateCalculator>. The current output rate calculation
is moved to DefaultOutputRateCalculator. A new factory method
AudioMixerImpl::CreateWithOutputRateCalculator is added. The old
factory method passes the default rate calculator.
BUG=webrtc:6346
Review-Url: https://codereview.webrtc.org/2557713006
Cr-Commit-Position: refs/heads/master@{#15472}
This CL is in preparation to move the AudioFrame into webrtc/api. The
AudioFrame is a POD type used for representing 10ms of audio. It
appears as a parameter and return value of interfaces being migrated
to webrtc/api, in particular AudioMixer.
Here, methods operator+=, operator>>=, Mute are
moved into a new target webrtc/audio/utility/audio_frame_operations,
and dependencies are changed to use
the new versions. The old AudioFrame methods are marked deprecated.
The audio frame utilities in webrtc/modules/utility:audio_frame_operations
are also moved to the new location.
TBR=kjellander@webrtc.org
BUG=webrtc:6548
NOPRESUBMIT=True
Review-Url: https://codereview.webrtc.org/2424173003
Cr-Commit-Position: refs/heads/master@{#15413}
In the AudioMixerImpl implementation, removing a source never fails
and the return value is always true (see audio_mixer/audio_mixer_impl.cc).
A return value of |false| signaled that removing a source failed for
some reason. We have come to the conclusion that
* we don't know how to handle a return value of |false|
* we can't think of why an alternative implementation would need to
signal failure when removing a stream.
To avoid having a status code that is never read, never acted upon and
probably never set to anything but |true|, we change ::RemoveSource to
not have a return value.
NOTRY=True
BUG=webrtc:6346
Review-Url: https://codereview.webrtc.org/2506173003
Cr-Commit-Position: refs/heads/master@{#15150}
This change changes mixing to be done at the lowest possible
APM-native rate that does not lead to quality loss. An Audio
Processing-native rate is one of 8, 16, 32, or 48 kHz. Mixing at a
lower sampling rate and avoiding resampling can in many cases lead to
big efficiency improvements, as reported by experiments.
This CL also fixes a design issue with the AudioMixer: audio at
non-native rates is no longer fed to the APM instance which is the
limiter.
NOTRY=True
BUG=webrtc:6346
Review-Url: https://codereview.webrtc.org/2458703002
Cr-Commit-Position: refs/heads/master@{#14980}
The mixer allocates an audio frame for each added data source. This
audio frame was deallocated when a source was removed from the
mixer. Source removal could happen during the mixing, and the existing
locking scheme (and the Clang thread checker) was not sufficient to
prevent a data race.
After this change, the mixer doesn't release its lock until it is
finished with the sources' Audio frames. Since multi-threaded access to
the mixer only happens when a source is added or removed, we believe
that this change wouldn't have any noticeable performance impact.
NOTRY=True
BUG=webrtc:6346
Review-Url: https://codereview.webrtc.org/2439283002
Cr-Commit-Position: refs/heads/master@{#14744}
Simplify the AudioMixer::Source interface and update the mixer
implementation to the new interface.
Instead of asking a mixer source to provide a pointer to an AudioFrame
during each mixing iteration, a mixer should supply a pointer to its
own AudioFrame.
This simplifies lifetime issues as sources do not give away an
internal pointer.
Implementation: when an audio source is added, the mixer allocates a
new AudioFrame. The audio frame is kept together in the internal class
SourceStatus together with the audio source pointer until the source
is removed.
NOTRY=True
BUG=webrtc:6346
Review-Url: https://codereview.webrtc.org/2420913002
Cr-Commit-Position: refs/heads/master@{#14713}
This change is due to an incorrect understanding of the threading
model in Chrome. The new AudioMixer has a thread checker to ensure
that mixing is always done from a single thread. Mixing is done on the
Audio Output Thread. When run in Chrome, it can change. Even if the thread
changes, there is never more than one audio thread, and mixing is done
sequentially.
The threading checks and variable access checks are replaced with
rtc::RaceChecker counterparts.
NOTRY=True
BUG=webrtc:6346
Review-Url: https://codereview.webrtc.org/2437913003
Cr-Commit-Position: refs/heads/master@{#14712}
Changed mixability status into AddSource/RemoveSource. Added 'ssrc()'
method to the MixerSource interface. Removed unnecessary member 'num_audio_sources_' and made the mixer be refcounted.
BUG=webrtc:6346
NOTRY=True
Review-Url: https://codereview.webrtc.org/2408683002
Cr-Commit-Position: refs/heads/master@{#14612}
MixerAudioSource is moved to AudioMixerImpl::Source. Structures and methods of the MixerAudioSource interface have been renamed. The RemixFrame method has added checks and is moved to audio_frame_manipulator.h
BUG=webrtc:6346
Review-Url: https://codereview.webrtc.org/2396803004
Cr-Commit-Position: refs/heads/master@{#14600}
Reason for revert:
breaks chromium FYI
Original issue's description:
> Made MixerAudioSource a pure interface.
>
> This required quite a few small changes in the mixing algorithm
> structure, the mixer interface and the mixer unit tests.
>
> BUG=webrtc:6346
>
> Committed: https://crrev.com/2ae5fdff86b784545cbd724de54bb5ffedde1adf
> Cr-Commit-Position: refs/heads/master@{#14567}
TBR=ivoc@webrtc.org,solenberg@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6346
Review-Url: https://codereview.webrtc.org/2394253003
Cr-Commit-Position: refs/heads/master@{#14568}
This required quite a few small changes in the mixing algorithm
structure, the mixer interface and the mixer unit tests.
BUG=webrtc:6346
Review-Url: https://codereview.webrtc.org/2396483002
Cr-Commit-Position: refs/heads/master@{#14567}
1. Use of const in all variable declarations where it is possible
2. Variable names and function arguments changed from CamelCase to match code style
3. A few stale comments removed.
4. Chromium clang plugin check added (now possible thanks to kwiberg@'s work on common.h)
5. Disallow constructor macros added.
NOTRY=true
Review-Url: https://codereview.webrtc.org/2294263002
Cr-Commit-Position: refs/heads/master@{#14120}
Methods are named more consistently and have a more consistent
signatures. The call structure of mixing is slightly
simplified. Anonymous participants are also ramped up.
NOTRY=True
Review-Url: https://codereview.webrtc.org/2298163002
Cr-Commit-Position: refs/heads/master@{#14110}
Removed the OutputMixer part of the new mixer and renamed the new
mixer from NewAudioConferenceMixer to AudioMixer.
NOTRY=True
Review-Url: https://codereview.webrtc.org/2249213005
Cr-Commit-Position: refs/heads/master@{#13883}
