webrtc_m130/audio/audio_transport_impl.h

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

108 lines
3.9 KiB
C
Raw Normal View History

/*
* Copyright (c) 2016 The WebRTC project authors. All Rights Reserved.
*
* Use of this source code is governed by a BSD-style license
* that can be found in the LICENSE file in the root of the source
* tree. An additional intellectual property rights grant can be found
* in the file PATENTS. All contributing project authors may
* be found in the AUTHORS file in the root of the source tree.
*/
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
#ifndef AUDIO_AUDIO_TRANSPORT_IMPL_H_
#define AUDIO_AUDIO_TRANSPORT_IMPL_H_
#include <memory>
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
#include <vector>
#include "api/audio/audio_mixer.h"
#include "api/scoped_refptr.h"
#include "common_audio/resampler/include/push_resampler.h"
#include "modules/async_audio_processing/async_audio_processing.h"
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
#include "modules/audio_device/include/audio_device.h"
#include "modules/audio_processing/include/audio_processing.h"
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
#include "modules/audio_processing/typing_detection.h"
#include "rtc_base/synchronization/mutex.h"
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
#include "rtc_base/thread_annotations.h"
namespace webrtc {
class AudioSender;
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
class AudioTransportImpl : public AudioTransport {
public:
AudioTransportImpl(
AudioMixer* mixer,
AudioProcessing* audio_processing,
AsyncAudioProcessing::Factory* async_audio_processing_factory);
AudioTransportImpl() = delete;
AudioTransportImpl(const AudioTransportImpl&) = delete;
AudioTransportImpl& operator=(const AudioTransportImpl&) = delete;
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
~AudioTransportImpl() override;
int32_t RecordedDataIsAvailable(const void* audioSamples,
const size_t nSamples,
const size_t nBytesPerSample,
const size_t nChannels,
const uint32_t samplesPerSec,
const uint32_t totalDelayMS,
const int32_t clockDrift,
const uint32_t currentMicLevel,
const bool keyPressed,
uint32_t& newMicLevel) override;
int32_t NeedMorePlayData(const size_t nSamples,
const size_t nBytesPerSample,
const size_t nChannels,
const uint32_t samplesPerSec,
void* audioSamples,
size_t& nSamplesOut,
int64_t* elapsed_time_ms,
int64_t* ntp_time_ms) override;
void PullRenderData(int bits_per_sample,
int sample_rate,
size_t number_of_channels,
size_t number_of_frames,
void* audio_data,
int64_t* elapsed_time_ms,
int64_t* ntp_time_ms) override;
void UpdateAudioSenders(std::vector<AudioSender*> senders,
int send_sample_rate_hz,
size_t send_num_channels);
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
void SetStereoChannelSwapping(bool enable);
bool typing_noise_detected() const;
private:
void SendProcessedData(std::unique_ptr<AudioFrame> audio_frame);
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
// Shared.
AudioProcessing* audio_processing_ = nullptr;
// Capture side.
// Thread-safe.
const std::unique_ptr<AsyncAudioProcessing> async_audio_processing_;
mutable Mutex capture_lock_;
std::vector<AudioSender*> audio_senders_ RTC_GUARDED_BY(capture_lock_);
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
int send_sample_rate_hz_ RTC_GUARDED_BY(capture_lock_) = 8000;
size_t send_num_channels_ RTC_GUARDED_BY(capture_lock_) = 1;
bool typing_noise_detected_ RTC_GUARDED_BY(capture_lock_) = false;
bool swap_stereo_channels_ RTC_GUARDED_BY(capture_lock_) = false;
PushResampler<int16_t> capture_resampler_;
TypingDetection typing_detection_;
// Render side.
Replace AudioConferenceMixer with AudioMixer. This CL re-routes audio through AudioMixer instead of AudioConferenceMixer. This is done without any modifications to VoiceEngine. Previously, output audio was polled by an AudioDevice through an AudioTransport pointer, which was an instance of VoEBaseImpl. VoiceEngineImpl sent the request for data on to OutputMixer and further to AudioConferenceMixer. This CL changes the audio flow to an AudioDevice. We reconfigure the AudioDevice to have another AudioTransport pointer, which points to an AudioTransportProxy. The AudioTransportProxy is responsible for feeding mixed data to the AudioProcessing component for echo cancellation, and to resample the audio data after AudioProcessing and before it is sent to the AudioDevice. The set up of the audio path was previously done during VoiceEngine initialization. Now it is changed in the AudioState constructor. This list shows where audio-path-related VoiceEngine functionality has been moved: OutputMixer --> AudioTransportProxy VoiceEngineImpl --> AudioState, AudioTransportProxy SharedData --> AudioState Channel --> AudioReceiveStream, ChannelProxy, Channel AudioState owns the new mixer and connects it to AudioTransport and AudioDevice on initialization. The audio input source is AudioReceiveStream, which registers itself with the mixer (which it gets from AudioState) on Start and Stop. # Since the AudioTransport interface contains non-const references. NOPRESUBMIT=True BUG=webrtc:6346 Review-Url: https://codereview.webrtc.org/2436033002 Cr-Commit-Position: refs/heads/master@{#15193}
2016-11-22 06:42:53 -08:00
rtc::scoped_refptr<AudioMixer> mixer_;
AudioFrame mixed_frame_;
// Converts mixed audio to the audio device output rate.
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
PushResampler<int16_t> render_resampler_;
};
} // namespace webrtc
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
#endif // AUDIO_AUDIO_TRANSPORT_IMPL_H_