It do the following:
The SurfaceTexture.updateTexImage() calls are moved from the video renderers into MediaCodecVideoDecoder, and the destructor of the texture frames will signal MediaCodecVideoDecoder that the frame has returned. This CL also removes the SurfaceTexture from the native handle and only exposes the texture matrix instead, because only the video source should access the SurfaceTexture.
It moves the responsibility of calculating the decode time to Java.
Patchset2 Refactor MediaCodecVideoDecoder to drop frames if a texture is not released.
R=magjed@webrtc.org
Review URL: https://codereview.webrtc.org/1440343002 .
Cr-Commit-Position: refs/heads/master@{#10706}
This makes it clearer this code not meant to be used as an API.
I could not find any use of this in downstream code.
BUG=webrtc:5095
TESTED=git cl try -c --bot=android_compile_rel --bot=linux_compile_rel --bot=win_compile_rel --bot=mac_compile_rel --bot=ios_rel --bot=linux_gn_rel --bot=win_x64_gn_rel --bot=mac_x64_gn_rel --bot=android_gn_rel -m tryserver.webrtc
R=stefan@webrtc.orgTBR=magjed@webrtc.org
Review URL: https://codereview.webrtc.org/1440873005 .
Cr-Commit-Position: refs/heads/master@{#10699}
Not fully tested yet. Verified in test loopback application
with fake VP9 codec factory.
Assume that encoder generates bitstream in non flexible mode with
one temporal and one spatial layers.
R=magjed@webrtc.org
Review URL: https://codereview.webrtc.org/1451953002 .
Cr-Commit-Position: refs/heads/master@{#10695}
To avoid breaking downstream, the "interface" directories were copied
into a new "common_video/include" dir. The old headers got pragma
warnings added about deprecation (a very short deprecation since I plan
to remove them as soon downstream is updated).
The header guards are also identical to avoid mixing them up in the transition.
BUG=webrtc:5095
TESTED=Passing compile-trybots with --clobber flag:
git cl try --clobber --bot=win_compile_rel --bot=linux_compile_rel --bot=android_compile_rel --bot=mac_compile_rel --bot=ios_rel --bot=linux_gn_rel --bot=win_x64_gn_rel --bot=mac_x64_gn_rel --bot=android_gn_rel -m tryserver.webrtc
Review URL: https://codereview.webrtc.org/1418913006
Cr-Commit-Position: refs/heads/master@{#10659}
Reason for revert:
Causes fallback to SW decoder if a renderer is put in the background.
Original issue's description:
> Patchset 1 is a pure
> revert of "Revert of "Android MediaCodecVideoDecoder: Manage lifetime of texture frames" https://codereview.webrtc.org/1378033003/
>
> Following patchsets move the responsibility of calculating the decode time to Java.
>
> TESTED= Apprtc loopback using H264 and VP8 on N5, N6, N7, S5
>
> Committed: https://crrev.com/9cb8982e64f08d3d630bf7c3d2bcc78c10db88e2
> Cr-Commit-Position: refs/heads/master@{#10597}
TBR=magjed@webrtc.org,glaznev@webrtc.org
NOPRESUBMIT=true
NOTREECHECKS=true
Review URL: https://codereview.webrtc.org/1441363002 .
Cr-Commit-Position: refs/heads/master@{#10637}
This reverts commit 12f680214e28dc5f0a13ac8afc0d1445f89e67e6.
Original cl in https://codereview.webrtc.org/1396073003/
Prepare MediaCodecVideoEncoder for surface textures.
This refactors MediaVideoEncoder to prepare for adding support to encode from textures. The C++ layer does not have any functional changes.
- Moves ResetEncoder to always work on the codec thread
- Adds use of ThreadChecker.
- Change Java MediaEncoder.Init to return true or false and introduce method getInputBuffers.
- Add simple unit test for Java MediaCodecVideoEncoder.
The pure revert of the revert is in patchset 1.
Patchset 2, moves getting the input buffer to before storing pending timestamps etc to fix b/24984012.
BUG=webrtc:4993 b/24984012
Review URL: https://codereview.webrtc.org/1406203002
Cr-Commit-Position: refs/heads/master@{#10622}
revert of "Revert of "Android MediaCodecVideoDecoder: Manage lifetime of texture frames" https://codereview.webrtc.org/1378033003/
Following patchsets move the responsibility of calculating the decode time to Java.
TESTED= Apprtc loopback using H264 and VP8 on N5, N6, N7, S5
Review URL: https://codereview.webrtc.org/1422963003
Cr-Commit-Position: refs/heads/master@{#10597}
JNI already has jstring<->UTF8 string conversion, so using that should
save ~1mb off android binaries (ICU is *large*), probably around
300-400k after compression.
BUG=
Review URL: https://codereview.webrtc.org/1430023005
Cr-Commit-Position: refs/heads/master@{#10545}
Plus log first few decoder frames in and out events.
BUG=b/25287910
Review URL: https://codereview.webrtc.org/1423843005
Cr-Commit-Position: refs/heads/master@{#10439}
The purpose with this change is to support older API levels by replacing EGL14 (API lvl 17) with EGL10 (API lvl 1). The main purpose is to lower API lvl requirement for SurfaceViewRenderer from API lvl 17 to API lvl 15. Also, camera texture capture will work on API lvl < 17 (and texture encode/decode in MediaCodec, but we don't use MediaCodec below API lvl 18?).
GLSurfaceView/VideoRendererGui is already using EGL10.
EGL 1.1 - 1.4 added new functionality, but won't affect performance. We don't need the functionality, so there should be no reason to not use EGL 1.0.
I have profiled AppRTCDemo with Qualcomm Trepn Profiler on a Nexus 5 and Nexus 6 and couldn't see any difference.
Specifically, this CL:
* Update EglBase to use EGL10 instead of EGL14.
* Update imports from EGL14 to EGL10 in a lot of files (plus changing import order in some cases).
* Update VideoCapturerAndroid to always support texture capture.
Review URL: https://codereview.webrtc.org/1396013004
Cr-Commit-Position: refs/heads/master@{#10378}
This reverts commit 90754174d98d6b71fd4aaed897bd54980f7e59c4.
Revert "Fix use of scaler in MediaCodecVideoEncoder"
This reverts commit ec93628e75fdb81f23635b39b5f3da846bcefd21.
R=magjed@webrtc.orgTBR=glaznev@webrtc.org
BUG=webrtc:4993 b/24984012
Review URL: https://codereview.webrtc.org/1407263002 .
Cr-Commit-Position: refs/heads/master@{#10300}
The code that depends on the reverted CL is disabled but not removed. NativeHandleImpl is reverted to the previous implementation, and the new implementation is renamed to NativeTextureHandleImpl. Texture capture can not be used anymore, because it will crash in peerconnection_jni.cc.
Reason for revert:
Increased HW decoder latency and crashes related to that. Also suspected cause of video tearing.
Original issue's description:
> This CL should be the last one in a series to finally
> unblock camera texture capture.
>
> The SurfaceTexture.updateTexImage() calls are moved from
> the video renderers into MediaCodecVideoDecoder, and the
> destructor of the texture frames will signal
> MediaCodecVideoDecoder that the frame has returned. This
> CL also removes the SurfaceTexture from the native handle
> and only exposes the texture matrix instead, because only
> the video source should access the SurfaceTexture.
>
> BUG=webrtc:4993
> R=glaznev@webrtc.org, perkj@webrtc.org
>
> Committed: https://crrev.com/91b348c7029d843e06868ed12b728a809c53176c
> Cr-Commit-Position: refs/heads/master@{#10203}
TBR=glaznev
BUG=webrtc:4993
Review URL: https://codereview.webrtc.org/1394103005
Cr-Commit-Position: refs/heads/master@{#10288}
This CL should reduce the number of timeouts for dequeueInputBuffer() which results in the log "MediaCodecVideo: dequeueInputBuffer error" followed by software fallback for VP8/VP9 and codec restart for H264.
A timeout always happen for dequeueInputBuffer() when frames_received_ > frames_decoded_ + num_input_buffers. The following code tries to drain the decoder before enqueuing more input buffers:
// Try to drain the decoder and wait until output is not too
// much behind the input.
if (frames_received_ > frames_decoded_ + max_pending_frames_) {
ALOGV("Received: %d. Decoded: %d. Wait for output...",
frames_received_, frames_decoded_);
if (!DeliverPendingOutputs(jni, kMediaCodecTimeoutMs,
true /* dropFrames */)) {
ALOGE << "DeliverPendingOutputs error";
return ProcessHWErrorOnCodecThread();
}
if (frames_received_ > frames_decoded_ + max_pending_frames_) {
ALOGE << "Output buffer dequeue timeout";
return ProcessHWErrorOnCodecThread();
}
...
}
However, for H264, |max_pending_frames_| can currently be larger than the number of input buffers so that the code above is never executed. This CL limits |max_pending_frames_| to the number of input buffers.
TBR=glaznev
BUG=b/24867188,b/24864151
Review URL: https://codereview.webrtc.org/1394303005
Cr-Commit-Position: refs/heads/master@{#10273}
This make small refactorings to MediaVideoEncoder to prepare for adding support to encode from textures. The C++ layer does not have any functional changes.
- Moves ResetEncoder to always work on the codec thread
- Adds use of ThreadChecker.
- Change Java MediaEncoder.Init to return true or false and introduce method getInputBuffers.
- Add simple unit test for Java MediaCodecVideoEncoder.
BUG=webrtc:4993
Review URL: https://codereview.webrtc.org/1396073003
Cr-Commit-Position: refs/heads/master@{#10250}
These are the necessary changes in C++ related to the video capturer necessary to capture to a surface texture.
It does not handle scaling / cropping yet though.
BUG=
R=magjed@webrtc.org
Review URL: https://codereview.webrtc.org/1395673003 .
Cr-Commit-Position: refs/heads/master@{#10218}
This adds support for capturing to a texture in the Java part of VideoCapturerAndroid.
After this cl, the C++ also needs modification.
https://codereview.webrtc.org/1375953002/ contains the idea and have a working version where textures can be rendered in local preview.
BUG=webrtc:4993
R=magjed@webrtc.org
Review URL: https://codereview.webrtc.org/1383413002 .
Cr-Commit-Position: refs/heads/master@{#10213}
This CL should be the last one in a series to finally unblock camera texture capture.
The SurfaceTexture.updateTexImage() calls are moved from the video renderers into MediaCodecVideoDecoder, and the destructor of the texture frames will signal MediaCodecVideoDecoder that the frame has returned. This CL also removes the SurfaceTexture from the native handle and only exposes the texture matrix instead, because only the video source should access the SurfaceTexture.
BUG=webrtc:4993
R=glaznev@webrtc.org, perkj@webrtc.org
Review URL: https://codereview.webrtc.org/1378033003 .
Cr-Commit-Position: refs/heads/master@{#10203}
The Java PeerConnection maintains a cached list of Java RtpSenders
and RtpReceivers so that the same objects are returned every time
getSenders() or getReceivers() is called. They are disposed of when
PeerConnection.dispose() is called, which will also dispose their
referenced MediaStreamTracks.
Review URL: https://codereview.webrtc.org/1368143002
Cr-Commit-Position: refs/heads/master@{#10189}
This CL shouldn't make any functional changes. It adds a new VideoCapturerAndroid.nativeCreateVideoCapturer() instead of always using VideoCapturer.nativeCreateVideoCapturer(). The purpose is to simplify androidvideocapturer_jni and VideoCapturerAndroid.create(). This way, it is possible to use the ctor instead of VideoCapturerAndroid.init() to initialize variables, and they can be made final etc.
R=perkj@webrtc.org
Review URL: https://codereview.webrtc.org/1360173002 .
Cr-Commit-Position: refs/heads/master@{#10171}
This CL separates the types and code paths for textures vs byte buffers in MediaCodecVideoDecoder.dequeueOutputBuffer() and MediaCodecVideoDecoder::DeliverPendingOutputs(). The purpose is to prepare for lifetime management of textures received from the SurfaceTexture.
This CL is a part of the plan in https://codereview.webrtc.org/1357923002/.
BUG=webrtc:4993
Review URL: https://codereview.webrtc.org/1379383002
Cr-Commit-Position: refs/heads/master@{#10156}
This CL should not change the behaviour of the decoder. The purpose is to prepare for lifetime management of textures received from the SurfaceTexture. The main change is to only use exceptions for error signaling in MediaCodecVideoDecoder.dequeueOutputBuffer() and MediaCodecVideoDecoder.releaseOutputBuffer(), not both exceptions and error return values.
BUG=webrtc:4993
R=perkj@webrtc.org
Review URL: https://codereview.webrtc.org/1383983003 .
Cr-Commit-Position: refs/heads/master@{#10148}
Reason for revert:
Relanding with SetConfiguration not pure virtual.
Original issue's description:
> Revert of Adding PeerConnectionInterface::SetConfiguration method. (patchset #4 id:60001 of https://codereview.webrtc.org/1317353005/ )
>
> Reason for revert:
> Broke FYI bots because SetConfiguration is pure virtual and MockPeerConnectionImpl doesn't implement it. Need to reland with SetConfiguration not pure virtual.
>
> Original issue's description:
> > Adding PeerConnectionInterface::SetConfiguration method.
> >
> > Also updated the JNI and Objective-C bindings. Later, will have a CL to
> > remove UpdateIce, which this method effectively replaces.
> >
> > BUG=webrtc:4945
> >
> > Committed: https://crrev.com/70702afbcb8418fe93747e7ed63bcbf5e56b90e9
> > Cr-Commit-Position: refs/heads/master@{#10040}
>
> TBR=guoweis@webrtc.org,pthatcher@webrtc.org
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=webrtc:4945
>
> Committed: https://crrev.com/7603c76ab077b1e2033bb179595129bd96797345
> Cr-Commit-Position: refs/heads/master@{#10041}
TBR=guoweis@webrtc.org,pthatcher@webrtc.org,tkchin@webrtc.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:4945
Review URL: https://codereview.webrtc.org/1361273002
Cr-Commit-Position: refs/heads/master@{#10112}
The purpose of this CL is to use jlongFromPointer() for converting frame pointers to jlong instead of implicit casts which is not safe.
In order to respect constness, I had to make a small helper function for this.
BUG=webrtc:4993
R=perkj@webrtc.org
Review URL: https://codereview.webrtc.org/1373233002 .
Cr-Commit-Position: refs/heads/master@{#10108}
Reason for revert:
The top row in the video stream from the camera is messed up. The byte[] pointer is not the same as GetDirectBufferAddress() apparently.
Original issue's description:
> Android VideoCapturer: Send ByteBuffer instead of byte[]
>
> The purpose with this CL is to replace GetByteArrayElements() and ReleaseByteArrayElements() with GetDirectBufferAddress().
>
> R=hbos@webrtc.org
>
> Committed: https://crrev.com/cb3649b40b3fd6d5bbb0a92003b717e46ce90924
> Cr-Commit-Position: refs/heads/master@{#10091}
TBR=hbos@webrtc.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
Review URL: https://codereview.webrtc.org/1377783002
Cr-Commit-Position: refs/heads/master@{#10103}
p2ptransportchannel. This CL does not use the new policy yet.
BUG=
Review URL: https://codereview.webrtc.org/1369773003
Cr-Commit-Position: refs/heads/master@{#10092}
The purpose with this CL is to replace GetByteArrayElements() and ReleaseByteArrayElements() with GetDirectBufferAddress().
R=hbos@webrtc.org
Review URL: https://codereview.webrtc.org/1372813002 .
Cr-Commit-Position: refs/heads/master@{#10091}
This cl moves back loading java SurfaceTextureHolder to the ClassReferenceHolder and use FindClass through ClassReferenceHolder. Without this, jni->FindClass returns nullptr in surfacetexturehelper_jni.cc.
BUG=webrtc:4993
R=magjed@webrtc.org
Review URL: https://codereview.webrtc.org/1370013002 .
Cr-Commit-Position: refs/heads/master@{#10086}
This cl adds a C++ counterpart of the Java SurfaceTextureHandler. It can be used for creating a webrtc::VideoFrames from a native handle and also guarantee that the Java SurfaceTexture is notified when the VideoFrame is no longer in use.
BUG=webrtc:4993
R=magjed@webrtc.org
Review URL: https://codereview.webrtc.org/1366413003 .
Cr-Commit-Position: refs/heads/master@{#10084}
Android hardware H.264 seems to keep a steady high-QP flow instead of
dropping frames, so framedrops aren't sufficient to detect a bad state
where downscaling would be beneficial.
BUG=webrtc:4968
R=magjed@webrtc.org, stefan@webrtc.org
Review URL: https://codereview.webrtc.org/1364253002 .
Cr-Commit-Position: refs/heads/master@{#10078}
Reason for revert:
Broke FYI bots because SetConfiguration is pure virtual and MockPeerConnectionImpl doesn't implement it. Need to reland with SetConfiguration not pure virtual.
Original issue's description:
> Adding PeerConnectionInterface::SetConfiguration method.
>
> Also updated the JNI and Objective-C bindings. Later, will have a CL to
> remove UpdateIce, which this method effectively replaces.
>
> BUG=webrtc:4945
>
> Committed: https://crrev.com/70702afbcb8418fe93747e7ed63bcbf5e56b90e9
> Cr-Commit-Position: refs/heads/master@{#10040}
TBR=guoweis@webrtc.org,pthatcher@webrtc.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:4945
Review URL: https://codereview.webrtc.org/1361263002
Cr-Commit-Position: refs/heads/master@{#10041}
Also updated the JNI and Objective-C bindings. Later, will have a CL to
remove UpdateIce, which this method effectively replaces.
BUG=webrtc:4945
Review URL: https://codereview.webrtc.org/1317353005
Cr-Commit-Position: refs/heads/master@{#10040}
