网站建设类公司新闻,泉州做网站联系方式,深圳住房和建设,巩义网站建设托管前言#xff1a;
MediaCodec 支持 texture mode#xff0c;即MediaCodec解码video完毕后把 yuv 数据填入 GPU 共享出来的 graphic buffer 里面#xff0c;app 会把 video 的 yuv数据 和 ui 的数据通过通过软件渲染组件(opengl等)发送给GPU 进行一并渲染。这样做的效率较低
MediaCodec 支持 texture mode即MediaCodec解码video完毕后把 yuv 数据填入 GPU 共享出来的 graphic buffer 里面app 会把 video 的 yuv数据 和 ui 的数据通过通过软件渲染组件(opengl等)发送给GPU 进行一并渲染。这样做的效率较低但是稳定性较好。且性能取决于GPU的性能。
另外一种是 surface mode就是常用的模式这种模式下 app 不再获取 yuv 数据而是只负责输送 es 数据并决定 render 还是 discard 解码后的数据且是根据 bufferid 来判断的根本没有机会接触解码后的 yuv 数据所有的一切都在平台完成。这种办法效率搞且功耗也很低因为不需要GPU的参与。大部分情况下都是这种模式。
本文简单理清一下 Graphic Buffer 的框架以对 texture mode 中的 Graphic Buffer 申请流程有一个大致的概念。 Graphic Buffer 的申请
始于 ANativeWindow 类
frameworks/native/libs/nativewindow/include/system/window.h struct ANativeWindow
{
#ifdef __cplusplusANativeWindow(): flags(0), minSwapInterval(0), maxSwapInterval(0), xdpi(0), ydpi(0){common.magic ANDROID_NATIVE_WINDOW_MAGIC;common.version sizeof(ANativeWindow);memset(common.reserved, 0, sizeof(common.reserved));}/* Implement the methods that spANativeWindow expects so that itcan be used to automatically refcount ANativeWindows. */void incStrong(const void* /*id*/) const {common.incRef(const_castandroid_native_base_t*(common));}void decStrong(const void* /*id*/) const {common.decRef(const_castandroid_native_base_t*(common));}
#endifstruct android_native_base_t common;/* flags describing some attributes of this surface or its updater */const uint32_t flags;/* min swap interval supported by this updated */const int minSwapInterval;/* max swap interval supported by this updated */const int maxSwapInterval;/* horizontal and vertical resolution in DPI */const float xdpi;const float ydpi;/* Some storage reserved for the OEMs driver. */intptr_t oem[4];/** Set the swap interval for this surface.** Returns 0 on success or -errno on error.*/int (*setSwapInterval)(struct ANativeWindow* window,int interval);/** Hook called by EGL to acquire a buffer. After this call, the buffer* is not locked, so its content cannot be modified. This call may block if* no buffers are available.** The window holds a reference to the buffer between dequeueBuffer and* either queueBuffer or cancelBuffer, so clients only need their own* reference if they might use the buffer after queueing or canceling it.* Holding a reference to a buffer after queueing or canceling it is only* allowed if a specific buffer count has been set.** Returns 0 on success or -errno on error.** XXX: This function is deprecated. It will continue to work for some* time for binary compatibility, but the new dequeueBuffer function that* outputs a fence file descriptor should be used in its place.*/int (*dequeueBuffer_DEPRECATED)(struct ANativeWindow* window,struct ANativeWindowBuffer** buffer);/** hook called by EGL to lock a buffer. This MUST be called before modifying* the content of a buffer. The buffer must have been acquired with* dequeueBuffer first.** Returns 0 on success or -errno on error.** XXX: This function is deprecated. It will continue to work for some* time for binary compatibility, but it is essentially a no-op, and calls* to it should be removed.*/int (*lockBuffer_DEPRECATED)(struct ANativeWindow* window,struct ANativeWindowBuffer* buffer);/** Hook called by EGL when modifications to the render buffer are done.* This unlocks and post the buffer.** The window holds a reference to the buffer between dequeueBuffer and* either queueBuffer or cancelBuffer, so clients only need their own* reference if they might use the buffer after queueing or canceling it.* Holding a reference to a buffer after queueing or canceling it is only* allowed if a specific buffer count has been set.** Buffers MUST be queued in the same order than they were dequeued.** Returns 0 on success or -errno on error.** XXX: This function is deprecated. It will continue to work for some* time for binary compatibility, but the new queueBuffer function that* takes a fence file descriptor should be used in its place (pass a value* of -1 for the fence file descriptor if there is no valid one to pass).*/int (*queueBuffer_DEPRECATED)(struct ANativeWindow* window,struct ANativeWindowBuffer* buffer);/** hook used to retrieve information about the native window.** Returns 0 on success or -errno on error.*/int (*query)(const struct ANativeWindow* window,int what, int* value);/** hook used to perform various operations on the surface.* (*perform)() is a generic mechanism to add functionality to* ANativeWindow while keeping backward binary compatibility.** DO NOT CALL THIS HOOK DIRECTLY. Instead, use the helper functions* defined below.** (*perform)() returns -ENOENT if the what parameter is not supported* by the surfaces implementation.** See above for a list of valid operations, such as* NATIVE_WINDOW_SET_USAGE or NATIVE_WINDOW_CONNECT*/int (*perform)(struct ANativeWindow* window,int operation, ... );/** Hook used to cancel a buffer that has been dequeued.* No synchronization is performed between dequeue() and cancel(), so* either external synchronization is needed, or these functions must be* called from the same thread.** The window holds a reference to the buffer between dequeueBuffer and* either queueBuffer or cancelBuffer, so clients only need their own* reference if they might use the buffer after queueing or canceling it.* Holding a reference to a buffer after queueing or canceling it is only* allowed if a specific buffer count has been set.** XXX: This function is deprecated. It will continue to work for some* time for binary compatibility, but the new cancelBuffer function that* takes a fence file descriptor should be used in its place (pass a value* of -1 for the fence file descriptor if there is no valid one to pass).*/int (*cancelBuffer_DEPRECATED)(struct ANativeWindow* window,struct ANativeWindowBuffer* buffer);/** Hook called by EGL to acquire a buffer. This call may block if no* buffers are available.** The window holds a reference to the buffer between dequeueBuffer and* either queueBuffer or cancelBuffer, so clients only need their own* reference if they might use the buffer after queueing or canceling it.* Holding a reference to a buffer after queueing or canceling it is only* allowed if a specific buffer count has been set.** The libsync fence file descriptor returned in the int pointed to by the* fenceFd argument will refer to the fence that must signal before the* dequeued buffer may be written to. A value of -1 indicates that the* caller may access the buffer immediately without waiting on a fence. If* a valid file descriptor is returned (i.e. any value except -1) then the* caller is responsible for closing the file descriptor.** Returns 0 on success or -errno on error.*/int (*dequeueBuffer)(struct ANativeWindow* window,struct ANativeWindowBuffer** buffer, int* fenceFd);/** Hook called by EGL when modifications to the render buffer are done.* This unlocks and post the buffer.** The window holds a reference to the buffer between dequeueBuffer and* either queueBuffer or cancelBuffer, so clients only need their own* reference if they might use the buffer after queueing or canceling it.* Holding a reference to a buffer after queueing or canceling it is only* allowed if a specific buffer count has been set.** The fenceFd argument specifies a libsync fence file descriptor for a* fence that must signal before the buffer can be accessed. If the buffer* can be accessed immediately then a value of -1 should be used. The* caller must not use the file descriptor after it is passed to* queueBuffer, and the ANativeWindow implementation is responsible for* closing it.** Returns 0 on success or -errno on error.*/int (*queueBuffer)(struct ANativeWindow* window,struct ANativeWindowBuffer* buffer, int fenceFd);/** Hook used to cancel a buffer that has been dequeued.* No synchronization is performed between dequeue() and cancel(), so* either external synchronization is needed, or these functions must be* called from the same thread.** The window holds a reference to the buffer between dequeueBuffer and* either queueBuffer or cancelBuffer, so clients only need their own* reference if they might use the buffer after queueing or canceling it.* Holding a reference to a buffer after queueing or canceling it is only* allowed if a specific buffer count has been set.** The fenceFd argument specifies a libsync fence file decsriptor for a* fence that must signal before the buffer can be accessed. If the buffer* can be accessed immediately then a value of -1 should be used.** Note that if the client has not waited on the fence that was returned* from dequeueBuffer, that same fence should be passed to cancelBuffer to* ensure that future uses of the buffer are preceded by a wait on that* fence. The caller must not use the file descriptor after it is passed* to cancelBuffer, and the ANativeWindow implementation is responsible for* closing it.** Returns 0 on success or -errno on error.*/int (*cancelBuffer)(struct ANativeWindow* window,struct ANativeWindowBuffer* buffer, int fenceFd);
};上述代码中有很多函数指针比如 int (*dequeueBuffer)(struct ANativeWindow* window,struct ANativeWindowBuffer** buffer, int* fenceFd);
这些指针的具体实现在其子类中有实现和设置这里是一种 C 的 hook 写法而不是 c 的虚函数写法可能出于兼容性考虑。
\frameworks\native\libs\nativewindow\ANativeWindow.cpp 里面并不是 ANativeWindow的实现而是对其封装比如
int ANativeWindow_dequeueBuffer(ANativeWindow* window, ANativeWindowBuffer** buffer, int* fenceFd) {return window-dequeueBuffer(window, buffer, fenceFd);
}
可以看到这里调用了 ANativeWindow的dequeueBuffer函数而这个函数在 ANativeWindow 结构里是一个函数指针而这个函数指针又指向哪里呢 Hook的实现者Surface
\frameworks\native\libs\gui\include\gui\Surface.h
\frameworks\native\libs\gui\Surface.cpp
class Surface: public ANativeObjectBaseANativeWindow, Surface, RefBase
{}
Surface 是 ANativeWindow的子类但是考虑到 ANativeWindow通过 函数指针的方式实现多态所以Surface里面一定有设置Hook的地方那就是在构造函数里。
Surface::Surface(const spIGraphicBufferProducer bufferProducer, bool controlledByApp): mGraphicBufferProducer(bufferProducer),mCrop(Rect::EMPTY_RECT),mBufferAge(0),mGenerationNumber(0),mSharedBufferMode(false),mAutoRefresh(false),mSharedBufferSlot(BufferItem::INVALID_BUFFER_SLOT),mSharedBufferHasBeenQueued(false),mQueriedSupportedTimestamps(false),mFrameTimestampsSupportsPresent(false),mEnableFrameTimestamps(false),mFrameEventHistory(std::make_uniqueProducerFrameEventHistory()) {// Initialize the ANativeWindow function pointers.ANativeWindow::setSwapInterval hook_setSwapInterval;ANativeWindow::dequeueBuffer hook_dequeueBuffer;ANativeWindow::cancelBuffer hook_cancelBuffer;ANativeWindow::queueBuffer hook_queueBuffer;ANativeWindow::query hook_query;ANativeWindow::perform hook_perform;ANativeWindow::dequeueBuffer_DEPRECATED hook_dequeueBuffer_DEPRECATED;ANativeWindow::cancelBuffer_DEPRECATED hook_cancelBuffer_DEPRECATED;ANativeWindow::lockBuffer_DEPRECATED hook_lockBuffer_DEPRECATED;ANativeWindow::queueBuffer_DEPRECATED hook_queueBuffer_DEPRECATED;
至此所有调用 ANativeWindow_xxx 的入口都会被导入 Surface 里的相应接口中进入 hook_xxx。
int Surface::hook_dequeueBuffer(ANativeWindow* window,ANativeWindowBuffer** buffer, int* fenceFd) {Surface* c getSelf(window);return c-dequeueBuffer(buffer, fenceFd);
}
进而在进入 Surface 的 xxx 函数。从 xxx 函数开始和 graphic service 打交道就正式开始了。
int Surface::dequeueBuffer(android_native_buffer_t** buffer, int* fenceFd) {ATRACE_CALL();status_t result mGraphicBufferProducer-dequeueBuffer(buf, fence, reqWidth, reqHeight,reqFormat, reqUsage, mBufferAge,enableFrameTimestamps ? frameTimestamps: nullptr);}Graphic Buffer 服务
Surface 的 所有和 Graphic Buffer 相关的接口都会使用到成员 mGraphicBufferProducer
spIGraphicBufferProducer mGraphicBufferProducer;
这个成员的类型一看就是一个 binder Interface继续看 IGraphicBufferProducer。
\frameworks\native\libs\gui\include\gui\IGraphicBufferProducer.h
class IGraphicBufferProducer : public IInterface
{
public:using HGraphicBufferProducerV1_0 ::android::hardware::graphics::bufferqueue::V1_0::IGraphicBufferProducer;using HGraphicBufferProducerV2_0 ::android::hardware::graphics::bufferqueue::V2_0::IGraphicBufferProducer;。。。virtual status_t dequeueBuffer(int* slot, spFence* fence, uint32_t w, uint32_t h,PixelFormat format, uint64_t usage, uint64_t* outBufferAge,FrameEventHistoryDelta* outTimestamps) 0;。。。}
至此一个binder 模型已经形成。 Surface::mGraphicBufferProducer as Bp --IGraphicBufferProducer BINDER-- ? as Bn 那么这里的Bn是什么呢只需要在源码里搜一下哪些类继承IGraphicBufferProducer 即可。其中一个常用的类就是 BufferQueueProducer。
\frameworks\native\libs\gui\include\gui\BufferQueueProducer.h
class BufferQueueProducer : public BnGraphicBufferProducer,private IBinder::DeathRecipient {。。。virtual status_t queueBuffer(int slot,const QueueBufferInput input, QueueBufferOutput* output);
。。。}
继承自BnGraphicBufferProducerBn开头一看就知道是server端。
status_t BufferQueueProducer::dequeueBuffer(int* outSlot, spandroid::Fence* outFence,uint32_t width, uint32_t height, PixelFormat format,uint64_t usage, uint64_t* outBufferAge,FrameEventHistoryDelta* outTimestamps) {spGraphicBuffer graphicBuffer new GraphicBuffer(width, height, format, BQ_LAYER_COUNT, usage,{mConsumerName.string(), mConsumerName.size()});status_t error graphicBuffer-initCheck();}
在上面的代码里有 new GraphicBuffer 和 initCheck() 这两个动作这里就是在创建管理对象并进行初始化检查。 内存管理
\frameworks\native\libs\ui\include\ui\GraphicBuffer.h
class GraphicBuffer: public ANativeObjectBaseANativeWindowBuffer, GraphicBuffer, RefBase,public FlattenableGraphicBuffer
{GraphicBuffer(uint32_t inWidth, uint32_t inHeight, PixelFormat inFormat,uint32_t inUsage, std::string requestorName Unknown);}
\frameworks\native\libs\ui\GraphicBuffer.cpp
GraphicBuffer::GraphicBuffer(uint32_t inWidth, uint32_t inHeight, PixelFormat inFormat,uint32_t inLayerCount, uint64_t inUsage, std::string requestorName): GraphicBuffer() {mInitCheck initWithSize(inWidth, inHeight, inFormat, inLayerCount, inUsage,std::move(requestorName));
}
status_t GraphicBuffer::initWithSize(uint32_t inWidth, uint32_t inHeight,PixelFormat inFormat, uint32_t inLayerCount, uint64_t inUsage,std::string requestorName)
{GraphicBufferAllocator allocator GraphicBufferAllocator::get();uint32_t outStride 0;status_t err allocator.allocate(inWidth, inHeight, inFormat, inLayerCount,inUsage, handle, outStride, mId,std::move(requestorName));if (err NO_ERROR) {mBufferMapper.getTransportSize(handle, mTransportNumFds, mTransportNumInts);width static_castint(inWidth);height static_castint(inHeight);format inFormat;layerCount inLayerCount;usage inUsage;usage_deprecated int(usage);stride static_castint(outStride);}return err;
}
上面创建 Graphic Buffer 的过程中会有一个 initWithSize 这是初始化的过程在初始化过程里又又 allocator.allocate可见是一个内存分配的过程。 内存分配 GraphicBufferAllocator
\frameworks\native\libs\ui\include\ui\GraphicBufferAllocator.h
\frameworks\native\libs\ui\GraphicBufferAllocator.cpp
class GraphicBufferAllocator : public SingletonGraphicBufferAllocator
{}
allocate 方法如下
status_t GraphicBufferAllocator::allocate(uint32_t width, uint32_t height,PixelFormat format, uint32_t layerCount, uint64_t usage,buffer_handle_t* handle, uint32_t* stride,uint64_t /*graphicBufferId*/, std::string requestorName)
{。。。status_t error mAllocator-allocate(width, height, format, layerCount, usage, 1, stride, handle);
。。。}
mAllocator 是什么
sphardware::graphics::allocator::V2_0::IAllocator mAllocator;
又是一个 Interface 这次是 hardware 的因此其实现是由硬件完成的。具体细节参考 /hardware/interfaces/graphics/allocator/2.0/
