This change reduces the number of times the Android hardware video
encoder is reconfigured when making an outgoing call. With this change,
the encoder should only be initialized once as opposed to the ~3 times
it happens currently.
Before the fix, the following sequence of events caused the extra
reconfigurations:
1. After the SetLocalDescription call, the WebRtcVideoSendStream is created.
All frames from the camera are dropped until the corresponding
VideoSendStream is created.
2. SetRemoteDescription() triggers the VideoSendStream creation. At
this point, the encoder is configured for the first time, with the
frame dimensions set to a low resolution default (176x144).
3. When the first video frame is received from the camera after the
VideoSendStreamIsCreated, the encoder is reconfigured to the correct
dimensions. If we are using the Android hardware encoder, the default
configuration is set to encode from a memory buffer (use_surface=false).
4. When the frame is passed down to the encoder in
androidmediaencoder_jni.cc EncodeOnCodecThread(), it may be stored in
a texture instead of a memory buffer. In this case, yet another
reconfiguration takes place to enable encoding from a texture.
5. Even if the resolution and texture flag were known at the start of
the call, there would be a reconfiguration involved if the camera is
rotated (such as when making a call from a phone in portrait orientation).
The reason for that is that at construction time, WebRtcVideoEngine2
sets the VideoSinkWants structure parameter to request frames rotated
by the source; the early frames will then arrive in portrait resolution.
When the remote description is finally set, if the rotation RTP extension
is supported by the remote receiver, the source is asked to provide
non-rotated frames. The very next frame will then arrive in landscape
resolution with a non-zero rotation value to be applied by the receiver.
Since the encoder was configured with the last (portrait) frame size,
it's going to need to be reconfigured again.
The fix makes the following changes:
1. WebRtcVideoSendStream::OnFrame() now caches the last seen frame
dimensions, and whether the frame was stored in a texture.
2. When the encoder is configured the first time
(WebRtcVideoSendStream::SetCodec()) - the last seen frame dimensions
are used instead of the default dimensions.
3. A flag that indicates if encoding is to be done from a texture has
been added to the webrtc::VideoStream and webrtc::VideoCodec structs,
and it's been wired up to be passed down all the way to the JNI code in
androidmediaencoder_jni.cc.
4. MediaCodecVideoEncoder::InitEncode is now reading the is_surface
flag from the VideoCodec structure instead of guessing the default as
false. This way we end up with the correct encoder configuration the
first time around.
5. WebRtcVideoSendStream now takes an optimistic guess and requests non-
rotated frames when the supported RtpExtensions list is not available.
This makes the "early" frames arrive non-rotated, and the cached dimensions
will be correct for the common case when the rotation extension is supported.
If the other side is an older endpoint which does not support rotation,
the encoder will have to be reconfigured - but it's better to penalize the
uncommon case rather than the common one.
Review-Url: https://codereview.webrtc.org/2067103002
Cr-Commit-Position: refs/heads/master@{#13173}
This CL adds support for an extension on RTP frames to allow the sender
to specify the minimum and maximum playout delay limits.
The receiver makes a best-effort attempt to keep the capture-to-render delay
within this range. This allows different types of application to specify
different end-to-end delay goals. For example gaming can support rendering
of frames as soon as received on receiver to minimize delay. A movie playback
application can specify a minimum playout delay to allow fixed buffering
in presence of network jitter.
There are no tests at this time and most of testing is done with chromium
webrtc prototype.
On chromoting performance tests, this extension helps bring down end-to-end
delay by about 150 ms on small frames.
BUG=webrtc:5895
Review-Url: https://codereview.webrtc.org/2007743003
Cr-Commit-Position: refs/heads/master@{#13059}
Currently there are two structs that are identical and track extension details:
webrtc::RtpExtension
cricket::RtpHeaderExtension
The use of the structs is mixed in the code to track the extensions being
supported. This results in duplicate definition of
the URI constants and there is code to convert between the two structs.
Clean up to use a single RtpHeader throughout the codebase. The actual location
of RtpHeader may change in future (perhaps to be located in api/). Additionally,
this CL renames some of the constants to clarify Uri and Id use.
BUG= webrtc:5895
Review-Url: https://codereview.webrtc.org/1984983002
Cr-Commit-Position: refs/heads/master@{#12924}
Adds negotiation of rtx codecs for red and vp9. To keep backwards
compatibility with older Chrome versions, this change includes two
hacks:
1. Red packets will be retransmitted over the rtx codec associated with
vp8 if no rtx codec is associated with red. This is how Chrome does
it today and ensures that we still can send red over rtx to older
versions.
2. If rtx packets associated with the media codec (vp8/vp9 etc) are
received and red has been negotiated, we will assume that the sender
incorrectly has packetized red inside the rtx header associated with
media. We will therefore restore it with the red payload type
instead, which ensures that we can still receive rtx associated with
red from old versions.
Offering multiple rtx codecs to older versions should not be a problem
since old versions themselves only try to negotiate rtx for vp8.
R=pbos@webrtc.orgTBR=mflodman@webrtc.org
BUG=webrtc:4024
TEST=Verified by running apprtc and emulating packet loss between Chrome with and without the patch.
Review URL: https://codereview.webrtc.org/1649493004 .
Cr-Commit-Position: refs/heads/master@{#11472}
Also refactor packet router to use a map rather than iterate over all
rtp modules for each packet sent.
BUG=webrtc:4311
Review URL: https://codereview.webrtc.org/1247293002
Cr-Commit-Position: refs/heads/master@{#9670}
Mostly, it's about moving constructors and descructors to the .cc
files, so that they won't be inlined everywhere.
The reason this CL is so big is that a lot of code was using
common_types.h without declaring a dependency on webrtc_common, which
broke the build once common_types.h started to depend on
common_types.cc.
BUG=163
R=kjellander@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/26089004
Cr-Commit-Position: refs/heads/master@{#8516}
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8516 4adac7df-926f-26a2-2b94-8c16560cd09d
Reasons for revert:
1. glaznev discovered potentially related problems using the Android AppRTCDemo.
2. We're trying to do an M41 webrtc roll in Chromium, and this CL is risky.
> Support associated payload type when registering Rtx payload type.
>
> Major changes include,
> - Add associated payload type for SetRtxSendPayloadType & SetRtxReceivePayloadType.
> - Receiver: Restore RTP packets by the new RTX-APT map.
> - Sender: Send RTP packets by checking RTX-APT map.
> - Add RTX payload type for RED in the default codec list.
>
> BUG=4024
> R=pbos@webrtc.org, stefan@webrtc.org
> TBR=mflodman@webrtc.org
>
> Review URL: https://webrtc-codereview.appspot.com/26259004
>
> Patch from Changbin Shao <changbin.shao@intel.com>.
TBR=pbos@webrtc.org, stefan@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/33829004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8033 4adac7df-926f-26a2-2b94-8c16560cd09d
