webrtc_m130/audio/audio_state.h

99 lines
3.0 KiB
C
Raw Normal View History

/*
* Copyright (c) 2015 The WebRTC project authors. All Rights Reserved.
*
* Use of this source code is governed by a BSD-style license
* that can be found in the LICENSE file in the root of the source
* tree. An additional intellectual property rights grant can be found
* in the file PATENTS. All contributing project authors may
* be found in the AUTHORS file in the root of the source tree.
*/
#ifndef AUDIO_AUDIO_STATE_H_
#define AUDIO_AUDIO_STATE_H_
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
#include <map>
#include <memory>
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
#include "audio/audio_transport_impl.h"
#include "audio/null_audio_poller.h"
#include "audio/scoped_voe_interface.h"
#include "call/audio_state.h"
#include "rtc_base/constructormagic.h"
#include "rtc_base/criticalsection.h"
#include "rtc_base/refcount.h"
#include "rtc_base/thread_checker.h"
#include "voice_engine/include/voe_base.h"
namespace webrtc {
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
class AudioSendStream;
namespace internal {
class AudioState final : public webrtc::AudioState {
public:
explicit AudioState(const AudioState::Config& config);
~AudioState() override;
AudioProcessing* audio_processing() override {
RTC_DCHECK(config_.audio_processing);
return config_.audio_processing.get();
}
AudioTransport* audio_transport() override {
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
return &audio_transport_;
}
void SetPlayout(bool enabled) override;
void SetRecording(bool enabled) override;
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
Stats GetAudioInputStats() const override;
void SetStereoChannelSwapping(bool enable) override;
VoiceEngine* voice_engine();
rtc::scoped_refptr<AudioMixer> mixer();
bool typing_noise_detected() const;
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
void AddSendingStream(webrtc::AudioSendStream* stream,
int sample_rate_hz, size_t num_channels);
void RemoveSendingStream(webrtc::AudioSendStream* stream);
private:
// rtc::RefCountInterface implementation.
void AddRef() const override;
rtc::RefCountReleaseStatus Release() const override;
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
void UpdateAudioTransportWithSendingStreams();
rtc::ThreadChecker thread_checker_;
rtc::ThreadChecker process_thread_checker_;
const webrtc::AudioState::Config config_;
// We hold one interface pointer to the VoE to make sure it is kept alive.
ScopedVoEInterface<VoEBase> voe_base_;
// Reference count; implementation copied from rtc::RefCountedObject.
// TODO(nisse): Use RefCountedObject or RefCountedBase instead.
mutable volatile int ref_count_ = 0;
// Transports mixed audio from the mixer to the audio device and
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
// recorded audio to the sending streams.
AudioTransportImpl audio_transport_;
// Null audio poller is used to continue polling the audio streams if audio
// playout is disabled so that audio processing still happens and the audio
// stats are still updated.
std::unique_ptr<NullAudioPoller> null_audio_poller_;
Remove voe::TransmitMixer TransmitMixer's functionality is moved into the AudioTransportProxy owned by AudioState. This removes the need for an AudioTransport implementation in VoEBaseImpl, which means that the proxy is no longer a proxy, hence AudioTransportProxy is renamed to AudioTransportImpl. In the short term, AudioState needs to know which AudioDeviceModule is used, so it is added in AudioState::Config. AudioTransportImpl needs to know which AudioSendStream:s are currently enabled to send, so AudioState maintains a map of them, which is reduced into a simple vector for AudioTransportImpl. To encode and transmit audio, AudioSendStream::OnAudioData(std::unique_ptr<AudioFrame> audio_frame) is introduced, which is used in both the Chromium and standalone use cases. This removes the need for two different instances of voe::Channel::ProcessAndEncodeAudio(), so there is now only one, taking an AudioFrame as argument. Callers need to allocate their own AudioFrame:s, which is wasteful but not a regression since this was already happening in the voe::Channel functions. Most of the logic changed resides in AudioTransportImpl::RecordedDataIsAvailable(), where two strange things were found: 1. The clock drift parameter was ineffective since apm->echo_cancellation()->enable_drift_compensation(false) is called during initialization. 2. The output parameter 'new_mic_volume' was never set - instead it was returned as a result, causing the ADM to never update the analog mic gain (https://cs.chromium.org/chromium/src/third_party/webrtc/voice_engine/voe_base_impl.cc?q=voe_base_impl.cc&dr&l=100). Besides this, tests are updated, and some dead code is removed which was found in the process. Bug: webrtc:4690, webrtc:8591 Change-Id: I789d5296bf5efb7299a5ee05a4f3ce6abf9124b2 Reviewed-on: https://webrtc-review.googlesource.com/26681 Commit-Queue: Fredrik Solenberg <solenberg@webrtc.org> Reviewed-by: Oskar Sundbom <ossu@webrtc.org> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org> Cr-Commit-Position: refs/heads/master@{#21301}
2017-12-15 16:42:15 +01:00
struct StreamProperties {
int sample_rate_hz = 0;
size_t num_channels = 0;
};
std::map<webrtc::AudioSendStream*, StreamProperties> sending_streams_;
RTC_DISALLOW_IMPLICIT_CONSTRUCTORS(AudioState);
};
} // namespace internal
} // namespace webrtc
#endif // AUDIO_AUDIO_STATE_H_