This change changes mixing to be done at the lowest possible
APM-native rate that does not lead to quality loss. An Audio
Processing-native rate is one of 8, 16, 32, or 48 kHz. Mixing at a
lower sampling rate and avoiding resampling can in many cases lead to
big efficiency improvements, as reported by experiments.
This CL also fixes a design issue with the AudioMixer: audio at
non-native rates is no longer fed to the APM instance which is the
limiter.
NOTRY=True
BUG=webrtc:6346
Review-Url: https://codereview.webrtc.org/2458703002
Cr-Commit-Position: refs/heads/master@{#14980}
This change will allow for a audio source to report its sampling rate
to the audio mixer. It is needed in order to mix at a lower sampling
rate. Mixing at a lower sampling rate can in many cases lead to big
efficiency improvements, as reported by experiments.
The code affected is all implementations of the Source interface:
AudioReceiveStream and a mock class. The AudioReceiveStream now
queries its underlying voe::Channel object for the needed frequency.
Note that the changes to the mixing algorithm are done in a later CL.
BUG=webrtc:6346
NOTRY=True
TBR=solenberg@webrtc.org
Review-Url: https://codereview.webrtc.org/2448113009
Cr-Commit-Position: refs/heads/master@{#14839}
Introduce a convention on categorizing GN targets:
1. Production code
2. Tests
3. Examples
4. Tools
The first two have targets spread out all over the tree,
while the latter are isolated to examples/ and tools/ directories.
Another new convention: Each directory's BUILD.gn file shall contain
a target named similar to the directory name. This target shall
contain the 'most common' production code, i.e. so that most
consumers of the directory can depend on only the directory
(which implicitly means that target in GN).
//webrtc:webrtc_tests is changed to depend on all WebRTC tests.
From now on, it's necessary to add new test targets to this dependency
tree in order to get them compiled.
Two new group targets are created:
//webrtc/modules/audio_coding:audio_coding_tests
//webrtc/modules/audio_processing:audio_processing_tests
to reduce the long list of tests in //webrtc:webrtc_tests.
Visibility on //webrtc:webrtc and //webrtc:webrtc_tests is restricted
to the root target, to avoid circular dependencies due to the monolithic
property of these targets (a problem we've had in the past).
The 'root' target at the top level is renamed to 'default', which means GN will
build this target instead of _all_ generated targets
(see https://chromium.googlesource.com/chromium/src/+/master/tools/gn/docs/faq.md#Can-I-control-what-targets-are-built-by-default).
This target now depends on everything we want to build, meaning all targets now
explicitly needs to be wired up from the root target in order to get build.
Having this, the number of compiled objects on Android is decreased from 8855 to 6276. It also gives us better control over our build.
BUG=webrtc:6440
TESTED=git cl try --clobber
NOTRY=True
Review-Url: https://codereview.webrtc.org/2441383002
Cr-Commit-Position: refs/heads/master@{#14821}
The mixer allocates an audio frame for each added data source. This
audio frame was deallocated when a source was removed from the
mixer. Source removal could happen during the mixing, and the existing
locking scheme (and the Clang thread checker) was not sufficient to
prevent a data race.
After this change, the mixer doesn't release its lock until it is
finished with the sources' Audio frames. Since multi-threaded access to
the mixer only happens when a source is added or removed, we believe
that this change wouldn't have any noticeable performance impact.
NOTRY=True
BUG=webrtc:6346
Review-Url: https://codereview.webrtc.org/2439283002
Cr-Commit-Position: refs/heads/master@{#14744}
Simplify the AudioMixer::Source interface and update the mixer
implementation to the new interface.
Instead of asking a mixer source to provide a pointer to an AudioFrame
during each mixing iteration, a mixer should supply a pointer to its
own AudioFrame.
This simplifies lifetime issues as sources do not give away an
internal pointer.
Implementation: when an audio source is added, the mixer allocates a
new AudioFrame. The audio frame is kept together in the internal class
SourceStatus together with the audio source pointer until the source
is removed.
NOTRY=True
BUG=webrtc:6346
Review-Url: https://codereview.webrtc.org/2420913002
Cr-Commit-Position: refs/heads/master@{#14713}
This change is due to an incorrect understanding of the threading
model in Chrome. The new AudioMixer has a thread checker to ensure
that mixing is always done from a single thread. Mixing is done on the
Audio Output Thread. When run in Chrome, it can change. Even if the thread
changes, there is never more than one audio thread, and mixing is done
sequentially.
The threading checks and variable access checks are replaced with
rtc::RaceChecker counterparts.
NOTRY=True
BUG=webrtc:6346
Review-Url: https://codereview.webrtc.org/2437913003
Cr-Commit-Position: refs/heads/master@{#14712}
The AudioMixer is now split in a mixer and audio source interface part, which has moved to webrtc/api, and a default implementation part, which lies in webrtc/modules.
This change makes it possible to create other mixer implementations and is a first step to facilitate passing down a mixer from outside of WebRTC.
It will also create less build dependencies when the new mixer has replaced the old one.
NOTRY=True
TBR=henrik.lundin@webrtc.org
BUG=webrtc:6346
Review-Url: https://codereview.webrtc.org/2411313003
Cr-Commit-Position: refs/heads/master@{#14705}
Changed mixability status into AddSource/RemoveSource. Added 'ssrc()'
method to the MixerSource interface. Removed unnecessary member 'num_audio_sources_' and made the mixer be refcounted.
BUG=webrtc:6346
NOTRY=True
Review-Url: https://codereview.webrtc.org/2408683002
Cr-Commit-Position: refs/heads/master@{#14612}
MixerAudioSource is moved to AudioMixerImpl::Source. Structures and methods of the MixerAudioSource interface have been renamed. The RemixFrame method has added checks and is moved to audio_frame_manipulator.h
BUG=webrtc:6346
Review-Url: https://codereview.webrtc.org/2396803004
Cr-Commit-Position: refs/heads/master@{#14600}
Reason for revert:
breaks chromium FYI
Original issue's description:
> Made MixerAudioSource a pure interface.
>
> This required quite a few small changes in the mixing algorithm
> structure, the mixer interface and the mixer unit tests.
>
> BUG=webrtc:6346
>
> Committed: https://crrev.com/2ae5fdff86b784545cbd724de54bb5ffedde1adf
> Cr-Commit-Position: refs/heads/master@{#14567}
TBR=ivoc@webrtc.org,solenberg@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6346
Review-Url: https://codereview.webrtc.org/2394253003
Cr-Commit-Position: refs/heads/master@{#14568}
This required quite a few small changes in the mixing algorithm
structure, the mixer interface and the mixer unit tests.
BUG=webrtc:6346
Review-Url: https://codereview.webrtc.org/2396483002
Cr-Commit-Position: refs/heads/master@{#14567}
This CL deletes the old and not used video defines in
engine_configurations.h and pre-pends voice_ to indicate there are only
voice/audio defines left in the file.
BUG=none
R=solenberg@webrtc.org
Review URL: https://codereview.webrtc.org/2401673002 .
Cr-Commit-Position: refs/heads/master@{#14558}
This changes most non-test related rtc_source_set targets to be
rtc_static_library instead. Targets without any .cc files are excluded.
This should bring back the build behavior we used to have with GYP
(i.e. same symbols exported in the libjingle_peerconnection.a file, which
are used by some downstream projects).
After doing an Android build with these changes:
$ nm --defined-only -g -C out/Release/lib.unstripped/libjingle_peerconnection_so.so | grep -i createpeerconnectionf
00077c51 T Java_org_webrtc_PeerConnectionFactory_nativeCreatePeerConnectionFactory
$ nm --defined-only -g -C out/Release/obj/webrtc/api/libjingle_peerconnection.a | grep -i createpeerconnectionf
00000001 T webrtc::CreatePeerConnectionFactory(rtc::Thread*, rtc::Thread*, rtc::Thread*, webrtc::AudioDeviceModule*, cricket::WebRtcVideoEncoderFactory*, cricket::WebRtcVideoDecoderFactory*)
00000001 T webrtc::CreatePeerConnectionFactory()
See https://chromium.googlesource.com/chromium/src/+/master/tools/gn/docs/cookbook.md#Note-on-static-libraries
for more details on this.
NOTICE: This should be further cleaned up in the future, to reduce
binary bloat and unnecessary linking time. Right now it's more
important to restore the desired build output though.
BUG=webrtc:6410, chromium:630755
Review-Url: https://codereview.webrtc.org/2361623004
Cr-Commit-Position: refs/heads/master@{#14364}
1. Use of const in all variable declarations where it is possible
2. Variable names and function arguments changed from CamelCase to match code style
3. A few stale comments removed.
4. Chromium clang plugin check added (now possible thanks to kwiberg@'s work on common.h)
5. Disallow constructor macros added.
NOTRY=true
Review-Url: https://codereview.webrtc.org/2294263002
Cr-Commit-Position: refs/heads/master@{#14120}
Methods are named more consistently and have a more consistent
signatures. The call structure of mixing is slightly
simplified. Anonymous participants are also ramped up.
NOTRY=True
Review-Url: https://codereview.webrtc.org/2298163002
Cr-Commit-Position: refs/heads/master@{#14110}
Remove common_inherited_config from the targets and add it to the
template instead.
BUG=webrtc:6187
NOTRY=True
Review-Url: https://codereview.webrtc.org/2311843002
Cr-Commit-Position: refs/heads/master@{#14069}
Remove common_config from the targets' config and add
it to the template instead.
BUG=webrtc:6187
NOTRY=True
Review-Url: https://codereview.webrtc.org/2300413002
Cr-Commit-Position: refs/heads/master@{#14063}
Defines the rtc_executable, rtc_source_set, rtc_test and
rtc_static_library templates.
These templates provide no functionality yet, but will enable common
configuration to be introduced, avoiding repetition in every target
Changes summary:
- Prepend rtc_ to test, source_set, executable and static_library targets
- Change "configs -= [" to "suppressed_configs += ["
- Include webrtc/build/webrtc.gni where it wasn't included yet
- Delete import("//testing/test.gni"), since rtc_test makes it unnecessary.
BUG=webrtc:6187
TBR=henrik.lundin@webrtc.org,tommi@webrtc.org
NOTRY=True
Review-Url: https://codereview.webrtc.org/2301053002
Cr-Commit-Position: refs/heads/master@{#14043}
Removed the OutputMixer part of the new mixer and renamed the new
mixer from NewAudioConferenceMixer to AudioMixer.
NOTRY=True
Review-Url: https://codereview.webrtc.org/2249213005
Cr-Commit-Position: refs/heads/master@{#13883}
Changes to the mixer unittests:
Removed the tests related to the former 'OutputMixer', as it's going
to be removed. Removed incorrect comparison tests with the old mixer
because doing identical mixing decisions with the old mixer proved
unviable.
When the new mixer went from kMaximumAmountOfMixedAudioSources in the
last iteration to kMaximumAmountOfMixedAudioSources+1, it could hit an
RTC_NOTREACHED(); Added fix to mixer and test
AudioMixer.RampedOutSourcesShouldNotBeMarkedMixed that covers that
case.
NOTRY=True
Review-Url: https://codereview.webrtc.org/2253153004
Cr-Commit-Position: refs/heads/master@{#13880}
Added a level indicator to the new mixer. The level indicator is
webrtc::voe::AudioLevel. It computes the current audio level, which is
used all the way up to peerconnection.
This is part of the project to rewrite the old conference mixer and
output mixer.
NOTRY=True
Review-Url: https://codereview.webrtc.org/2230823004
Cr-Commit-Position: refs/heads/master@{#13878}
Mostly, it's about replacing mutable reference arguments with pointer
arguments, and replacing C style casts with C++ style casts.
Review-Url: https://codereview.webrtc.org/2056653002
Cr-Commit-Position: refs/heads/master@{#13849}
Because passing ownership in raw pointers makes kittens cry.
This also means we can ditch the Destroy functions and the protected
destructors. (Well, almost. We need to keep the old CreateFilePlayer
and DestroyFilePlayer around for a little while longer because of an
external caller.)
Review-Url: https://codereview.webrtc.org/2049683003
Cr-Commit-Position: refs/heads/master@{#13797}
Reason for revert:
Reverting, because it turns out that third-party code was using webrtc::FilePlayer. I'm not at all sure that this is something WebRTC ought to be exporting, but since we did export it, we have to live with it for now.
Original issue's description:
> Move FilePlayer and FileRecorder to Voice Engine
>
> Because Voice Engine was the only user.
>
> (This has been landed twice before, as
> https://codereview.webrtc.org/2037623002 and
> https://codereview.webrtc.org/2240163002. Third time's a charm!)
>
> NOPRESUBMIT=True
> TBR=kjellander@webrtc.org
>
> Committed: https://crrev.com/427ce3d86f6328dc994f84a15c28bb7bfbaa46ef
> Cr-Commit-Position: refs/heads/master@{#13777}
TBR=
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
Review-Url: https://codereview.webrtc.org/2245413002
Cr-Commit-Position: refs/heads/master@{#13779}
Reason for revert:
Breaks downstream code, so revert again. Yay.
Original issue's description:
> Move FilePlayer and FileRecorder to Voice Engine
>
> Because Voice Engine was the only user.
>
> (This is a re-land of https://codereview.webrtc.org/2037623002, which
> had to be reverted.)
>
> NOPRESUBMIT=True
>
> Committed: https://crrev.com/dc65ea29b3270ad418050658ad962ddd33ee70c1
> Cr-Commit-Position: refs/heads/master@{#13757}
TBR=perkj@webrtc.org,kjellander@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
Review-Url: https://codereview.webrtc.org/2245153002
Cr-Commit-Position: refs/heads/master@{#13758}
Also fixed one small chromium-style error in the new mixer.
NOTRY=True
Review-Url: https://codereview.webrtc.org/2234293002
Cr-Commit-Position: refs/heads/master@{#13752}
The folder structure is now as was agreed on in the 'Slim and Modular
WebRTC' effort. Also added some dependencies that were previously in
another part of the tree.
NOTRY=True
Review-Url: https://codereview.webrtc.org/2238803002
Cr-Commit-Position: refs/heads/master@{#13742}
Removed resampler from NewAudioConferenceMixer and AudioMixer (which started as a copy of former OutputMixer). This is part of the mixer rewrite project. In particular,
this is one of the steps required to have a single mixing component
instead of two doing the same thing.
The next planned change (which is not part of this CL) is to plug in the
new mixer (NewAudioConferenceMixer) into AudioState and AudioDeviceModule.
NOTRY=True
Review-Url: https://codereview.webrtc.org/2221443002
Cr-Commit-Position: refs/heads/master@{#13674}
The new version is much shorter than the old one, and hopefully easier
to read. This is part of the effort to rewrite the old mixer.
NOTRY=True
Review-Url: https://codereview.webrtc.org/2132563002
Cr-Commit-Position: refs/heads/master@{#13570}
Reason for revert:
Multiple definitions of webrtc::MockMixerParticipant::MockMixerParticipant() during linking of modules_unittests. Please investigate and resubmit.
Original issue's description:
> Rewrote UpdateToMix in the audio mixer.
>
> The new version is much shorter than the old one, and hopefully easier
> to read. This is part of the effort to rewrite the old mixer.
>
> Committed: https://crrev.com/2942e240f4a985752714dac18c141064c97696d4
> Cr-Commit-Position: refs/heads/master@{#13568}
TBR=ossu@webrtc.org,ivoc@webrtc.org,aleloi@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
Review-Url: https://codereview.webrtc.org/2195633002
Cr-Commit-Position: refs/heads/master@{#13569}
The new version is much shorter than the old one, and hopefully easier
to read. This is part of the effort to rewrite the old mixer.
Review-Url: https://codereview.webrtc.org/2132563002
Cr-Commit-Position: refs/heads/master@{#13568}
Memory frames are now expected to be owned by the mixing participants.
Review-Url: https://codereview.webrtc.org/2127763002
Cr-Commit-Position: refs/heads/master@{#13554}
OutputMixer and AudioConferenceMixer communicated via a callback. OutputMixer implemented an AudioMixerOutputReceiver interface, which defines the callback function NewMixedAudio. This has been removed and replaced by a simple function in the new mixer. The audio frame with mixed audio is now copied one time less. I have also removed one forward declaration.
Review-Url: https://codereview.webrtc.org/2111293003
Cr-Commit-Position: refs/heads/master@{#13550}
