Graphic Architecture
本篇文章是基于谷歌有關Graphic的一篇概覽文章的翻譯：http://source.android.com/devices/graphics/architecture.html
大量文字以及術語的了解基于自身的了解，可能并不準确。文中有部分英文原文我也不能準确了解，對于這種語句，我在翻譯的語句後加了（？）符号。
This document describes the essential elements of Android’s “system-level” graphics architecture, and how it is used by the application framework and multimedia system. The focus is on how buffers of graphical data move through the system. If you’ve ever wondered why SurfaceView and TextureView behave the way they do, or how Surface and EGLSurface interact, you’ve come to the right place.
這篇文檔描述了android系統的子子產品Graphic的總體架構，以及APP Framework層和多媒體系統如何使用Graphic子產品的過程。這篇文章的重點在于講述Graphic的buffer資料如何在系統内部傳輸的。如果你曾經對SurfaceView和TextureView工作方式表示好奇，如果你希望了解Surface和EGLSurface的互動方式，那麼朋友，你來對地方了。
Some familiarity with Android devices and application development is assumed. You don’t need detailed knowledge of the app framework, and very few API calls will be mentioned, but the material herein doesn’t overlap much with other public documentation. The goal here is to provide a sense for the significant events involved in rendering a frame for output, so that you can make informed choices when designing an application. To achieve this, we work from the bottom up, describing how the UI classes work rather than how they can be used.
閱讀這篇文章前，我們假設你已經對android裝置和應用開發有了一定的了解。你不需要了解app framework層的大量知識，文中會涉及少量api，但是所涉及的材料并不會跟其它文檔有很大重疊。這篇文章重點講解在一幀的渲染過程中的重要步驟，目的在于使你在開發應用程式時做出更明智的選擇。為實作這個目标，我們将自下向上的講解相關的UI類是如何工作的，至于如何使用這些類，則不在我們的講解範圍内。
We start with an explanation of Android’s graphics buffers, describe the composition and display mechanism, and then proceed to the higher-level mechanisms that supply the compositor with data.
我們從解釋android Graphic buffer開始講起，描述了buffer合成和顯示的原理，然後，我們将在更高的層面講解這些資料合成的原理。
This document is chiefly concerned with the system as it exists in Android 4.4 (“KitKat”). Earlier versions of the system worked differently, and future versions will likely be different as well. Version-specific features are called out in a few places.
這篇文章主要是基于android KK的，早期系統和後面的系統在一些細節方面會有一些不同。
At various points I will refer to source code from the AOSP sources or from Grafika. Grafika is a Google open-source project for testing; it can be found at https://github.com/google/grafika. It’s more “quick hack” than solid example code, but it will suffice.
BufferQueue and gralloc
To understand how Android’s graphics system works, we have to start behind the scenes. At the heart of everything graphical in Android is a class called BufferQueue. Its role is simple enough: connect something that generates buffers of graphical data (the “producer”) to something that accepts the data for display or further processing (the “consumer”). The producer and consumer can live in different processes. Nearly everything that moves buffers of graphical data through the system relies on BufferQueue.
我們将從具體的場景來了解android Graphic 系統的運作。整個繪制系統的核心是一個叫做BufferQueue的類。它的作用其實很簡單：将一些可以生産繪制資料（buffers of graphical data）的子產品（producer）和一些将繪制資料顯示出來或做進一步處理的子產品（consumer）相連。生産者和消費者可以存在于不同的程序内。幾乎所有和buffers of graphical data移動的過程都依賴與BufferQueue類。
The basic usage is straightforward. The producer requests a free buffer (dequeueBuffer()), specifying a set of characteristics including width, height, pixel format, and usage flags. The producer populates the buffer and returns it to the queue (queueBuffer()). Sometime later, the consumer acquires the buffer (acquireBuffer()) and makes use of the buffer contents. When the consumer is done, it returns the buffer to the queue (releaseBuffer()).
這個類基本的用法很簡單。生産者申請一塊空閑的buffer（dequeueBuffer()），在申請時指定寬度，高度，像素的格式，以及使用的用途的一系列參數。生産者填充緩沖區後，将它送還給隊列（queueBuffer()）。之後，消費者申請buffer (acquireBuffer())，然後使用對應的buffer資料。當消費者使用完成後，它将buffer傳回給隊列（releaseBuffer()）。
Most recent Android devices support the “sync framework”. This allows the system to do some nifty thing when combined with hardware components that can manipulate graphics data asynchronously. For example, a producer can submit a series of OpenGL ES drawing commands and then enqueue the output buffer before rendering completes. The buffer is accompanied by a fence that signals when the contents are ready. A second fence accompanies the buffer when it is returned to the free list, so that the consumer can release the buffer while the contents are still in use. This approach improves latency and throughput as the buffers move through the system.
最新的android裝置支援一種叫做sync framework的技術。這允許系統結合硬體來實作對Graphic資料的異步操作。比如說，一個生産者可以一次性送出一系列的OpenGL ES繪制指令，然後在渲染結束前将其入隊到輸出緩沖區中。Buffer會被一個fence保護，當内容準備好後，發出信号。當buffer回到free清單時（有第二個fence保護），是以消費者在内容仍在使用的時候釋放緩沖區。這種方法提高了buffer在系統中移動的速度和吞吐量。
Some characteristics of the queue, such as the maximum number of buffers it can hold, are determined jointly by the producer and the consumer.
隊列其他的一些特性，比如說它能擁有的最大緩沖區數量，則有生産者和消費者共同決定。
The BufferQueue is responsible for allocating buffers as it needs them. Buffers are retained unless the characteristics change; for example, if the producer starts requesting buffers with a different size, the old buffers will be freed and new buffers will be allocated on demand.
BufferQueue負責緩沖區的配置設定。除非buffer的一些屬性發生變化，否則buffers将被保留。舉例說，如果生成者申請一個大小不同的buffers，舊的buffers将被釋放，而新的buffers将重新被申請。
The data structure is currently always created and “owned” by the consumer. In Android 4.3 only the producer side was “binderized”, i.e. the producer could be in a remote process but the consumer had to live in the process where the queue was created. This evolved a bit in 4.4, moving toward a more general implementation.
這個資料結構一直被消費者建立并“持有”。在4.3版本時，隻有生成者一端是“binder化的”，也就是說生産者這端可以在遠端程序（binder的另一側程序）裡，而消費者必須在隊列被建立的程序裡。這在KK版本裡面有了一定程式的改進。
Buffer contents are never copied by BufferQueue. Moving that much data around would be very inefficient. Instead, buffers are always passed by handle.
BufferQueue 永遠不會拷貝Buffer的資料，因為移動如此多的資料效率将十分低下，buffers隻會以句柄的方式被傳遞。
Gralloc HAL
The actual buffer allocations are performed through a memory allocator called “gralloc”, which is implemented through a vendor-specific HAL interface (see hardware/libhardware/include/hardware/gralloc.h). The alloc()function takes the arguments you’d expect — width, height, pixel format — as well as a set of usage flags. Those flags merit closer attention.
事實上，緩沖區的配置設定是由一個叫做gralloc的記憶體配置設定子產品控制的，這個子產品是是由具體廠商來實作的一個HAL接口（參見 hardware/libhardware/include/hardware/gralloc.h）。使用alloc函數被傳入你所需要的參數—寬度，高度，像素類型，以及用途的标志（usage flags）。
The gralloc allocator is not just another way to allocate memory on the native heap. In some situations, the allocated memory may not be cache-coherent, or could be totally inaccessible from user space. The nature of the allocation is determined by the usage flags, which include attributes like: • how often the memory will be accessed from software (CPU) • how often the memory will be accessed from hardware (GPU) • whether the memory will be used as an OpenGL ES (“GLES”) texture • whether the memory will be used by a video encoder
這個gralloc allocator并不是僅僅在native heap上配置設定記憶體。在一些場景中，配置設定的記憶體很可能并非緩存一緻性（所謂緩存一緻性，是指保留在高速緩存中的共享資源，保持資料一緻性的機制）的，或者是從使用者空間不可達的。配置設定到的記憶體具有哪些特性，取決于在建立時傳入的usage flags：
• 從軟體層次來通路這段記憶體的頻率（CPU）
• 從硬體層次來通路這段記憶體的頻率（GPU）
• 這段記憶體是否被用來做OpenGL ES的材質（GLES）
• 這段記憶體是否會被拿來做視訊的編碼
For example, if your format specifies RGBA 8888 pixels, and you indicate the buffer will be accessed from software — meaning your application will touch pixels directly — then the allocator needs to create a buffer with 4 bytes per pixel in R-G-B-A order. If instead you say the buffer will only be accessed from hardware and as a GLES texture, the allocator can do anything the GLES driver wants — BGRA ordering, non-linear “swizzled” layouts, alternative color formats, etc. Allowing the hardware to use its preferred format can improve performance. Some values cannot be combined on certain platforms. For example, the “video encoder” flag may require YUV pixels, so adding “software access” and specifying RGBA 8888 would fail. The handle returned by the gralloc allocator can be passed between processes through Binder.
比如說，如果你設定了RGBA 8888的像素格式，并且你設定了緩沖區被軟體通路（這意味着你的程式可以直接修改像素的資料），那麼allocator會建立一個四位元組的緩沖區，其中順序按照R-G-B-A的存儲順序來排列。。。。。設定成硬體推薦的資料格式可以提高性能。
一些數值的組合在某些特定的平台是不被允許的。比如說，video encoder對應的可能是YUV的資料格式，是以如果我們如果加入software access，并且指定資料格式為RGBA 8888就有可能失敗。
gralloc allocator建立的緩沖區的句柄将通過binder在不同程序之間傳輸。
SurfaceFlinger and Hardware Composer
Having buffers of graphical data is wonderful, but life is even better when you get to see them on your device’s screen. That’s where SurfaceFlinger and the Hardware Composer HAL come in.
擁有Graphic資料的緩沖區很美妙，但是如果你能看到它們顯示在螢幕上才更是讓你覺得人生完美。是時候讓SurfaceFlinger 和 Hardware Composer HAL登場了。
SurfaceFlinger’s role is to accept buffers of data from multiple sources, composite them, and send them to the display. Once upon a time this was done with software blitting to a hardware framebuffer (e.g./dev/graphics/fb0), but those days are long gone.
SurfaceFlinger的工作是接受來自不同來源的緩沖區資料，将這些資料混合，然後發送資料到顯示裝置上。曾幾何時，這些功能是由軟體直接複制資料到硬體的framebuffer上（e.g./dev/graphics/fb0），但這樣的日子早已一去不複返。
*When an app comes to the foreground, the WindowManager service asks SurfaceFlinger for a drawing surface. SurfaceFlinger creates a “layer”
- the primary component of which is a BufferQueue - for which SurfaceFlinger acts as the consumer. A Binder object for the producer side is passed through the WindowManager to the app, which can then start sending frames directly to SurfaceFlinger. (Note: The WindowManager uses the term “window” instead of “layer” for this and uses “layer” to mean something else. We’re going to use the SurfaceFlinger terminology. It can be argued that SurfaceFlinger should really be called LayerFlinger.)*
當一個app轉到前端，WindowManager服務會要求SurfaceFlinger繪制一個surface。SurfaceFlinger将建立一個layer（它的主要組成部分是一個BufferQueue），而實際上SurfaceFlinger扮演了一個消費者的角色。一個生産者一側的binder對象通過WindowManager傳輸給了app，是以它可以直接向SurfaceFlinger發送幀資料（注意：WindowManager實際上使用的是window這個術語，而不是layer這個術語。但是我們這裡将使用SurfaceFlinger體系下的術語，可以說SurfaceFlinger更應該被稱作是LayerFlinger）。
For most apps, there will be three layers on screen at any time: the “status bar” at the top of the screen, the “navigation bar” at the bottom or side, and the application’s UI. Some apps will have more or less, e.g. the default home app has a separate layer for the wallpaper, while a full-screen game might hide the status bar. Each layer can be updated independently. The status and navigation bars are rendered by a system process, while the app layers are rendered by the app, with no coordination between the two.
對于大多數app來說，螢幕上一般總是有三個layer：螢幕上方的status bar，螢幕下方的navigation bar（實際上很多品牌的手機并沒有navigation bar，比如三星），以及應用本身的UI。一些應用的layer可能會有不同，比如home app的桌面會有一個獨立的layer，而一直全螢幕的遊戲可能不會有status bar。每個layer都是獨立的被更新。status和navigation bars是被系統程序渲染的，而app的layer則被app渲染，這二者之間并不會有什麼協同作業。
Device displays refresh at a certain rate, typically 60 frames per second on phones and tablets. If the display contents are updated mid-refresh, “tearing” will be visible; so it’s important to update the contents only between cycles. The system receives a signal from the display when it’s safe to update the contents. For historical reasons we’ll call this the VSYNC signal.
裝置的顯示重新整理頻率是一個特定的值，一般是60幀每秒。如果顯示的内容重新整理不夠迅速，就可能出現顯示撕裂的情況。因為按照周期來更新顯示的内容至關重要。當顯示系統可以安全的更新内容時，它會發送一個信号給系統。基于某種曆史上的原因，我們将這個信号稱之為VSYNC信号。
The refresh rate may vary over time, e.g. some mobile devices will range from 58 to 62fps depending on current conditions. For an HDMI-attached television, this could theoretically dip to 24 or 48Hz to match a video. Because we can update the screen only once per refresh cycle, submitting buffers for display at 200fps would be a waste of effort as most of the frames would never be seen. Instead of taking action whenever an app submits a buffer, SurfaceFlinger wakes up when the display is ready for something new.
裝置的重新整理率可能随時間變化，基于不同的場景，一些型号的重新整理率可能在58到62之間變化。對于一個連接配接了HDMI的電視，這個值理論上可以下降到24或者48。因為我們隻能在每個重新整理周期上更新螢幕内容，如果我們以200fps的頻率來送出buffer的資料，那麼由于大多數的資料并不會被顯示，這無疑是一種浪費。因為我們不會在每次app送出buffer資料時就做相應操作，隻會在顯示系統可以接受資料時才喚醒SurfaceFlinger。
When the VSYNC signal arrives, SurfaceFlinger walks through its list of layers looking for new buffers. If it finds a new one, it acquires it; if not, it continues to use the previously-acquired buffer. SurfaceFlinger always wants to have something to display, so it will hang on to one buffer. If no buffers have ever been submitted on a layer, the layer is ignored.
當VSYNC信号到達時，SurfaceFlinger會周遊它的layer清單來查找新的buffer。如果查找到一個，SurfaceFlinger将請求它（acquires），否則的話，SurfaceFlinger将繼續使用之前的資料。SurfaceFlinger總是需要一些資料來顯示，是以它依賴于一個buffer（？）。如果一個layer沒有buffer被送出，那麼這個layer将被忽略。
Once SurfaceFlinger has collected all of the buffers for visible layers, it asks the Hardware Composer how composition should be performed.
一旦SurfaceFlinger已經收集到了所有可見layer的buffer，它将請求Hardware Composer來執行混合的操作。
Hardware Composer
The Hardware Composer HAL (“HWC”) was first introduced in Android 3.0 (“Honeycomb”) and has evolved steadily over the years. Its primary purpose is to determine the most efficient way to composite buffers with the available hardware. As a HAL, its implementation is device-specific and usually implemented by the display hardware OEM.
HWC是從android 3.0版本引入的，在過去的數年中逐漸變得穩定。它的作用是使用現有的硬體選擇最有效的方式來合成緩沖區。做為一個HAL層接口，它的内容是由顯示硬體裝置廠商來具體實作的。
The value of this approach is easy to recognize when you consider “overlay planes.” The purpose of overlay planes is to composite multiple buffers together, but in the display hardware rather than the GPU. For example, suppose you have a typical Android phone in portrait orientation, with the status bar on top and navigation bar at the bottom, and app content everywhere else. The contents for each layer are in separate buffers. You could handle composition by rendering the app content into a scratch buffer, then rendering the status bar over it, then rendering the navigation bar on top of that, and finally passing the scratch buffer to the display hardware. Or, you could pass all three buffers to the display hardware, and tell it to read data from different buffers for different parts of the screen. The latter approach can be significantly more efficient.
如果設想一下”overlay planes.”的場景，那麼這個方法的價值是顯而易見的。”overlay planes”的作用是在display hardware而不是GPU中同時混合不同的buffer。打比方說，典型場景下，螢幕上方的status bar，螢幕下方的navigation bar，以及應用本身的UI。每個layer都有自己獨立的buffer。你可以通過逐漸繪制每個layer到緩沖區裡的方式來合成，最後将緩沖區的資料傳遞給顯示硬體裝置；或者，你也可以将每個layer資料分别傳給顯示硬體裝置，然後告知顯示硬體裝置從不同的緩沖區中讀取資料。顯然後一種方法更有效率。
*As you might expect, the capabilities of different display processors vary significantly. The number of overlays, whether layers can be rotated or blended, and restrictions on positioning and overlap can be difficult to express through an API. So, the HWC works like this:
- SurfaceFlinger provides the HWC with a full list of layers, and asks, “how do you want to handle this?”
- The HWC responds by marking each layer as “overlay” or “GLES composition.”
- SurfaceFlinger takes care of any GLES composition, passing the output buffer to HWC, and lets HWC handle the rest.*
如你所料，不同顯示處理器之間的性能有巨大的差距。很多Overlay， layer被旋轉或者混合，是以一個api很難準确表達在位置和遮蓋上的限制。是以，HWC子產品是這樣運作的：
1.SurfaceFlinger給HWC提供一份完整的layer清單，然後問，“你打算如何處理？”
2.HWC将每個layer标記為overlay或者GLES composition然後回複給SurfaceFlinger
3.SurfaceFlinger來處理被标記為GLES composition的layer，将處理之後的資料傳輸給HWC，并且讓HWC子產品來處理剩下的工作。
Since the decision-making code can be custom tailored by the hardware vendor, it’s possible to get the best performance out of every device.
因為硬體廠商可以自己定制decision-making的代碼，是以每台機器達到性能最優成為了可能。
Overlay planes may be less efficient than GL composition when nothing on the screen is changing. This is particularly true when the overlay contents have transparent pixels, and overlapping layers are being blended together. In such cases, the HWC can choose to request GLES composition for some or all layers and retain the composited buffer. If SurfaceFlinger comes back again asking to composite the same set of buffers, the HWC can just continue to show the previously-composited scratch buffer. This can improve the battery life of an idle device.
當螢幕上沒有任何東西變化時，Overlay planes的效率并不如GL composition的效率高。當overlay的内容中有很多透明的像素，或者重疊的layer在一起被混合時，這種差距尤其明顯。在這種情況下，HWC會請求讓GLES composition來處理部分或者全部的layer，并且保留混合後的buffer。如果Surfaceflinger又來請求混合相同的buffer時，HWC會直接顯示之前儲存的混合好的buffer。這麼做将可以提升裝置待機時間。
Devices shipping with Android 4.4 (“KitKat”) typically support four overlay planes. Attempting to composite more layers than there are overlays will cause the system to use GLES composition for some of them; so the number of layers used by an application can have a measurable impact on power consumption and performance.
搭載了KK的android裝置一般支援四條overlay planes。如果我們嘗試混合更多的layer時，系統會使用GLES composition來處理其中的部分；是以一個應用使用了多少layer會影響到系統的功耗和性能。
You can see exactly what SurfaceFlinger is up to with the command adb shell dumpsys SurfaceFlinger. The output is verbose. The part most relevant to our current discussion is the HWC summary that appears near the bottom of the output:
你可以通過adb shell dumpsys SurfaceFlinger這個指令來檢視Surfaceflinger具體使用了什麼。這個指令的輸出十分的長，其中和我們上面探讨的問題關連最深的是HWC的一段總結，這段一般在輸出内容的底部：
type | source crop | frame name
------------+-----------------------------------+--------------------------------
HWC | [ 0.0, 0.0, 320.0, 240.0] | [ 48, 411, 1032, 1149] SurfaceView
HWC | [ 0.0, 75.0, 1080.0, 1776.0] | [ 0, 75, 1080, 1776] com.android.grafika/com.android.grafika.PlayMovieSurfaceActivity
HWC | [ 0.0, 0.0, 1080.0, 75.0] | [ 0, 0, 1080, 75] StatusBar
HWC | [ 0.0, 0.0, 1080.0, 144.0] | [ 0, 1776, 1080, 1920] NavigationBar
FB TARGET | [ 0.0, 0.0, 1080.0, 1920.0] | [ 0, 0, 1080, 1920] HWC_FRAMEBUFFER_TARGET
This tells you what layers are on screen, whether they’re being handled with overlays (“HWC”) or OpenGL ES composition (“GLES”), and gives you a bunch of other facts you probably won’t care about (“handle” and “hints” and “flags” and other stuff that we’ve trimmed out of the snippet above). The “source crop” and “frame” values will be examined more closely later on.
從這裡我們可以看到那些顯示在螢幕上的layer，是被overlays (“HWC”)處理，還是被OpenGL ES composition (“GLES”)處理，另外還有一些我們目前不太關注的别的屬性（”handle” and “hints” and “flags”還有别的一些屬性，我們沒有粘帖在上面的輸出中）。我們會在後面詳細解釋”source crop” and “frame”這兩個值的含義。
The FB_TARGET layer is where GLES composition output goes. Since all layers shown above are using overlays, FB_TARGET isn’t being used for this frame. The layer’s name is indicative of its original role: On a device with/dev/graphics/fb0 and no overlays, all composition would be done with GLES, and the output would be written to the framebuffer. On recent devices there generally is no simple framebuffer, so the FB_TARGET layer is a scratch buffer. (Note: This is why screen grabbers written for old versions of Android no longer work: They’re trying to read from The Framebuffer, but there is no such thing.)
FB_TARGET這個layer就是由GLES composition輸出組成。這個layer上面的其餘layer都是由overlay渲染而成，是以在這一幀裡面，FB_TARGET并沒有被使用。這個layer的名字表明了初始的角色：一個在/dev/graphics/fb0 的裝置，所有的合成工作由GLES來完成，然後輸入将會被寫入framebuffer中。在目前的裝置上并沒有一個單純的framebuffer，所有這個FB_TARGET layer 實際上是一個scratch buffer(這就是為什麼在android早期版本上寫的一些螢幕截圖工具現在不能正常工作的原因：程式試圖從framebuffer中讀取資料，但是現在已經沒有了framebuffer).
The overlay planes have another important role: they’re the only way to display DRM content. DRM-protected buffers cannot be accessed by SurfaceFlinger or the GLES driver, which means that your video will disappear if HWC switches to GLES composition.
overlay planes有另外一個重要的作用：這是顯示DRM内容的唯一方法。受保護的DRM視訊的buffer是無法被Surfaceflinger或者GLES來讀取的，這意味着如果你使用GLES而不是HWC的話，你的視訊将無法播放。
The Need for Triple-Buffering
To avoid tearing on the display, the system needs to be double-buffered: the front buffer is displayed while the back buffer is being prepared. At VSYNC, if the back buffer is ready, you quickly switch them. This works reasonably well in a system where you’re drawing directly into the framebuffer, but there’s a hitch in the flow when a composition step is added. Because of the way SurfaceFlinger is triggered, our double-buffered pipeline will have a bubble.
為了避免出現畫面撕裂，系統需要雙重緩沖：前台緩沖在顯示時，背景緩沖則在準備中。在接收到VSYNC信号後，如果背景緩沖已經準備好，你就可以迅速切換到上面。如果你總是直接向framebuffer繪入資料，那麼這種工作方式就足夠了。但是，當我們加入一個合成步驟後，這樣就會有問題。Because of the way
SurfaceFlinger is triggered, our double-buffered pipeline will have a bubble. Suppose frame N is being displayed, and frame N+1 has been acquired by SurfaceFlinger for display on the next VSYNC. (Assume frame N is composited with an overlay, so we can’t alter the buffer contents until the display is done with it.) When VSYNC arrives, HWC flips the buffers. While the app is starting to render frame N+2 into the buffer that used to hold frame N, SurfaceFlinger is scanning the layer list, looking for updates. SurfaceFlinger won’t find any new buffers, so it prepares to show frame N+1 again after the next VSYNC. A little while later, the app finishes rendering frame N+2 and queues it for SurfaceFlinger, but it’s too late. This has effectively cut our maximum frame rate in half.
假設frame N正在被顯示，而frame N+1已經被Surfaceflinger擷取用于下一次VSYNC發生時的顯示（假設frame N使用了overylay來做渲染，是以顯示處理完成之前，我們沒辦法修改buffer的内容）。當VSYNC信号到來時，HWC投遞了緩沖區。當app開始渲染frame N+2 到Frame N用過的緩沖區内時，Surfaceflinger開始檢查layer清單，檢視是否有更新。此時Surfaceflinger并不會發現任何新的buffer，是以它會準備在下一個VSYNC到來時繼續顯示N+1幀的内容。一段時間後，app結束了N+2幀的渲染，然後将資料傳給Surfaceflinger，但是此時已經為時太晚。這将導緻我們最大幀率縮減為一半。
We can fix this with triple-buffering. Just before VSYNC, frame N is being displayed, frame N+1 has been composited (or scheduled for an overlay) and is ready to be displayed, and frame N+2 is queued up and ready to be acquired by SurfaceFlinger. When the screen flips, the buffers rotate through the stages with no bubble. The app has just less than a full VSYNC period (16.7ms at 60fps) to do its rendering and queue the buffer. And SurfaceFlinger / HWC has a full VSYNC period to figure out the composition before the next flip. The downside is that it takes at least two VSYNC periods for anything that the app does to appear on the screen. As the latency increases, the device feels less responsive to touch input.
三重緩沖可以解決我們的這個問題。VSYNC信号之前，幀N已經被顯示，幀N+1已經合成完畢（或者計劃進行overlay），等待被顯示，而幀N+2已經在排隊等候被Surfaceflinger擷取。When the screen flips, the buffers rotate through the stages with no bubble.App有略少于一個完整VSYNC周期的時間（當幀率為60時，這個時間為16.7毫秒）去做它的渲染工作并且将buffer入隊。在下一個VSYNC到來之前，Surfaceflinger/HWC有一個完整的VSYNC周期去完成合成的工作。壞消息是，app将内容顯示在螢幕上，将需要花費兩個VSYNC的周期。因為延遲增加了，是以裝置會顯得會觸摸事件的響應不夠靈敏。
SurfaceFlinger with BufferQueue
Figure 1. SurfaceFlinger + BufferQueue
The diagram above depicts the flow of SurfaceFlinger and BufferQueue. During frame:
上面的圖表描述了SurfaceFlinger and BufferQueue的處理流程，在每一幀中：
1.red buffer fills up, then slides into BufferQueue
2.after red buffer leaves app, blue buffer slides in, replacing it
3.green buffer and systemUI shadow-slide into HWC (showing that SurfaceFlinger still has the buffers, but now HWC has prepared them for display via overlay on the next VSYNC).
The blue buffer is referenced by both the display and the BufferQueue. The app is not allowed to render to it until the associated sync fence signals.*
1.紅色的緩沖區填滿後，進入BufferQueue中
2.當紅色緩沖區離開app後，藍色的緩沖區進入并代替了它
3.綠色緩沖區和SystemUI的資料進入HWC（這裡顯示Surfaceflinger依然持有這些緩沖區，但是現在HWC已經準備好在一個VSYNC到來時，将資料通過overlay顯示在螢幕上了）
藍色的緩沖區同時被顯示和BufferQueue引用，是以在相關的同步信号到來前，app是不能在這塊緩沖區上渲染的。
On VSYNC, all of these happen at once:
當VSYNC到來時，以下操作同時發生：
1.Red buffer leaps into SurfaceFlinger, replacing green buffer
2.Green buffer leaps into Display, replacing blue buffer, and a dotted-line green twin appears in the BufferQueue
3.The blue buffer’s fence is signaled, and the blue buffer in App empties
4.Display rect changes from to
1.紅色的緩沖區進入Surfaceflinger，取代了綠色緩沖區
2.綠色緩沖區取代了藍色緩沖區，開始顯示，同時圖中虛線連接配接的，綠色緩沖區的複制儲存在了BufferQueue中
3.藍色緩沖區的fence被解除，進入到了App empties**中
4.顯示内容從藍色緩沖區+SystemUI變成了綠色緩沖區+systemUI
The System UI process is providing the status and nav bars, which for our purposes here aren’t changing, so SurfaceFlinger keeps using the previously-acquired buffer. In practice there would be two separate buffers, one for the status bar at the top, one for the navigation bar at the bottom, and they would be sized to fit their contents. Each would arrive on its own BufferQueue.
SystemUI提供了狀态欄和導航欄，我們這裡認為它是不變的，是以Surfaceflinger使用了前面儲存的buffer。而實際上，這裡會有兩個獨立的buffer，一個屬于上面的狀态欄，一個屬于下面的導航欄，并且他們的大小和内容是比對的。每一個都會獨立達到自己的BufferQueue中。
The buffer doesn’t actually “empty”; if you submit it without drawing on it you’ll get that same blue again. The emptying is the result of clearing the buffer contents, which the app should do before it starts drawing.
這裡的buffer并非真的是空的，如果你不在上面繪制而是直接送出的話，你将會得到一個同樣的藍色緩沖區。App在繪制執行應該先執行清空緩沖區的指令，這将會buffer變空。
We can reduce the latency by noting layer composition should not require a full VSYNC period. If composition is performed by overlays, it takes essentially zero CPU and GPU time. But we can’t count on that, so we need to allow a little time. If the app starts rendering halfway between VSYNC signals, and SurfaceFlinger defers the HWC setup until a few milliseconds before the signal is due to arrive, we can cut the latency from 2 frames to perhaps 1.5. In theory you could render and composite in a single period, allowing a return to double-buffering; but getting it down that far is difficult on current devices. Minor fluctuations in rendering and composition time, and switching from overlays to GLES composition, can cause us to miss a swap deadline and repeat the previous frame.
通過讓合成不占用一整個VSYNC時間的辦法，我們可以降低延遲。如果合成是由overlay來實作的，那麼它幾乎不需要消耗CPU和GPU時間。但我們不能依賴于此，是以我們需要一點額外的時間。如果app在兩個VSYNC信号中間開始渲染，而surfaceFlinger直到VSYNC到達前的幾毫秒才進行了HWC的設定（譯者注：setUpHWComposer調用，也就是把需要顯示的layer資料準備好，報給HWC子產品來決定使用誰來合成），那麼我們可以将延遲從2幀降到1.5幀。理論上來說我們可以讓渲染和合成在一個周期内，這樣雙重緩沖區足矣（譯者注：的确，理論上來說如果這個過程不消耗時間的話，app在VSYNC之後dequeue到buffer，開始渲染，然後在這個VSYNC時間内完成渲染，要求合成，合成如果瞬間完成，的确不需要多一個VSYNC周期，兩個周期足矣，但這要求太高了）；但這對目前的裝置來說要求太高了，渲染和合成時一點微小的耗時變化（使用GLES而不是HWC來合成），都會導緻錯過更新時間，導緻重複顯示上一幀。
SurfaceFlinger’s buffer handling demonstrates the fence-based buffer management mentioned earlier. If we’re animating at full speed, we need to have an acquired buffer for the display (“front”) and an acquired buffer for the next flip (“back”). If we’re showing the buffer on an overlay, the contents are being accessed directly by the display and must not be touched. But if you look at an active layer’s BufferQueue state in the dumpsys SurfaceFlinger output, you’ll see one acquired buffer, one queued buffer, and one free buffer. That’s because, when SurfaceFlinger acquires the new “back” buffer, it releases the current “front” buffer to the queue. The “front” buffer is still in use by the display, so anything that dequeues it must wait for the fence to signal before drawing on it. So long as everybody follows the fencing rules, all of the queue-management IPC requests can happen in parallel with the display.
Surfaceflinger buffer的處理過程展示了我們前面提過的fence-based buffer的管理過程。如果畫面高速的變化，我們需要申請一個緩沖區用于顯示（front），同時需要申請一個緩沖區用于下一幀（back）。如果顯示的buffer是被overlay使用的，那麼這裡面的内容是直接被顯示系統讀取的，是以不能被修改。但是如果你通過dumpsys SurfaceFlinger指令來check一個活動的layer的BufferQueue狀态時，你會看到一個acquired buffer, 一個queued buffer, 還有一個free buffer.這是因為，當Surfaceflinger申請一個新的back buffer時，它釋放了front buffer給隊列。但是這個front buffer依然被display使用，是以任何想要在繪制之前dequeue這段buffer的程序，都必須等待fence signal的通知。隻要每個人都遵守這套規則，所有的同步隊列管理IPC請求都可以在顯示系統中被并行的處理。
Virtual Displays
SurfaceFlinger supports a “primary” display, i.e. what’s built into your phone or tablet, and an “external” display, such as a television connected through HDMI. It also supports a number of “virtual” displays, which make composited output available within the system. Virtual displays can be used to record the screen or send it over a network.
Surfaceflinger支援一個主顯示，也支援一個額外的顯示，比如一個通過HDMI線連接配接的電視機。同時它也支援一些虛拟的顯示，虛拟顯示可被用于錄制螢幕或者通過網絡發送。
Virtual displays may share the same set of layers as the main display (the “layer stack”) or have its own set. There is no VSYNC for a virtual display, so the VSYNC for the primary display is used to trigger composition for all displays.
虛拟電視可以有跟主顯示相同的layer，也可以有它自己的layer stack。但是虛拟顯示并沒有VSYNC，是以主顯示的VSYNC将用于觸發所有顯示的合成工作。
In the past, virtual displays were always composited with GLES. The Hardware Composer managed composition for only the primary display. In Android 4.4, the Hardware Composer gained the ability to participate in virtual display composition.
在過去，虛拟顯示的合成一直是由GLES來完成的。HWC僅僅用于主顯示，但是在KK，HWC也可以參與虛拟顯示的合成工作了。
As you might expect, the frames generated for a virtual display are written to a BufferQueue.
正如你所料，虛拟顯示的幀是被寫入了一個BufferQueue的。
Surface and SurfaceHolder
The Surface class has been part of the public API since 1.0. Its description simply says, “Handle onto a raw buffer that is being managed by the screen compositor.” The statement was accurate when initially written but falls well short of the mark on a modern system.
Surface類從1.0開始就是公開api的一部分。它的描述是這樣的：處理被螢幕合成器管理的raw buffer。這句話在當時被寫下時（1.0時代）是準确的，但是在當代的作業系統的标準下這句話已經遠遠落後。
The Surface represents the producer side of a buffer queue that is often (but not always!) consumed by SurfaceFlinger. When you render onto a Surface, the result ends up in a buffer that gets shipped to the consumer. A Surface is not simply a raw chunk of memory you can scribble on.
Surface代表了一個buffer queue的生産者一側，這個buffer queue一般被（但不是總是）被Surfaceflinger來消費。當你向一個Surface渲染時，結果最終在一個緩沖區内被運送到消費者那裡。一個Surface并不是一個可以任意修改的簡單raw記憶體資料塊。
The BufferQueue for a display Surface is typically configured for triple-buffering; but buffers are allocated on demand. So if the producer generates buffers slowly enough — maybe it’s animating at 30fps on a 60fps display — there might only be two allocated buffers in the queue. This helps minimize memory consumption. You can see a summary of the buffers associated with every layer in the dumpsys SurfaceFlinger output.
一個顯示surface的bufferQueue一般被配置為三重緩沖區，但是緩沖區是按需配置設定的。是以如果生産者生産緩沖區足夠緩慢（比如在一個重新整理率60的裝置上隻有30的重新整理率），這種情況下可能隊列中隻有兩個被配置設定的緩沖區，這樣可以有效的降低記憶體使用。通過指令dumpsys SurfaceFlinger，你可以看到每個layer關連的buffer的彙總。
Canvas Rendering
Once upon a time, all rendering was done in software, and you can still do this today. The low-level implementation is provided by the Skia graphics library. If you want to draw a rectangle, you make a library call, and it sets bytes in a buffer appropriately. To ensure that a buffer isn’t updated by two clients at once, or written to while being displayed, you have to lock the buffer to access it. lockCanvas() locks the buffer and returns a Canvas to use for drawing, and unlockCanvasAndPost() unlocks the buffer and sends it to the compositor.
曾經，所有的渲染工作都可以由軟體來完成，在今天你依然可以這麼做。底層的實作是由Skia庫來實作的。如果你想繪制一個矩形，你調用一個庫函數，函數就會設定好緩沖區中的資料。為了確定buffer同時被兩個用戶端同時更新，或者在顯示時被寫入，你需要在使用它之前鎖定這塊buffer。函數lockCanvas會鎖定一塊緩沖區并且傳回一個canvas用來繪制，函數unlockCanvasAndPost函數解鎖緩沖區，并且把它發送給合成器。
As time went on, and devices with general-purpose 3D engines appeared, Android reoriented itself around OpenGL ES. However, it was important to keep the old API working, for apps as well as app framework code, so an effort was made to hardware-accelerate the Canvas API. As you can see from the charts on the Hardware Acceleration page, this was a bit of a bumpy ride. Note in particular that while the Canvas provided to a View’s onDraw() method may be hardware-accelerated, the Canvas obtained when an app locks a Surface directly with lockCanvas() never is.
随着時間的推移，帶有通用3D加速引擎的裝置出現了。Android圍繞OpenGL ES做了調整。然而，保證舊的API可以運作同樣重要。是以我們努力使得Canvas的API支援硬體加速。如你在Hardware Acceleration頁面所能看到的圖表一樣，這是一段艱苦的旅程。特别要注意的是，當一個Canvas提供到一個View的onDraw方法時，它可能是硬體加速的；而你通過lockCanvas方法擷取到的Canvas則絕不可能是硬體加速的。
When you lock a Surface for Canvas access, the “CPU renderer” connects to the producer side of the BufferQueue and does not disconnect until the Surface is destroyed. Most other producers (like GLES) can be disconnected and reconnected to a Surface, but the Canvas-based “CPU renderer” cannot. This means you can’t draw on a surface with GLES or send it frames from a video decoder if you’ve ever locked it for a Canvas.
當你為了使用Canvas而鎖定一個Surface的時候，”CPU renderer”連接配接到了BufferQueue的生産者一端，直到Surface被銷毀才會斷開。大多數其他的生産者（比如GLES）可以斷開連接配接，并且重新連接配接到一個Surface之上；但是基于CPU渲染的Canvas不行。這意味着，一旦你為了使用一個Canvas而lock了一個Surface，你就不能使用GLES繪制這個Surface，你也不能将視訊解碼器生成的幀發送給它。
The first time the producer requests a buffer from a BufferQueue, it is allocated and initialized to zeroes. Initialization is necessary to avoid inadvertently sharing data between processes. When you re-use a buffer, however, the previous contents will still be present. If you repeatedly call lockCanvas() and unlockCanvasAndPost() without drawing anything, you’ll cycle between previously-rendered frames.
第一次生産者從BufferQueue中請求一個buffer時，它被配置設定并且被初始化為空。為了避免出現程序間不經意的分享資料，初始化是必要的。因為當你重新使用一個buffer時，之前的内容可能還在那裡。如果你重複調用 lockCanvas() 和unlockCanvasAndPost()函數而不繪制任何東西的話，你将會循環顯示前面渲染過的幀。
The Surface lock/unlock code keeps a reference to the previously-rendered buffer. If you specify a dirty region when locking the Surface, it will copy the non-dirty pixels from the previous buffer. There’s a fair chance the buffer will be handled by SurfaceFlinger or HWC; but since we need to only read from it, there’s no need to wait for exclusive access.
Surface的lock/unlock代碼保持了上次渲染過的buffer的引用。如果你在lock時指定了髒區域，那麼它會将前一個緩沖區内非髒區域的像素拷貝過來。有相當大的可能這塊buffer正在被Surfaceflinger或者HWC處理，但是因為我們隻是要從中讀取内容，是以我們沒必要一直等待互斥鎖。
The main non-Canvas way for an application to draw directly on a Surface is through OpenGL ES. That’s described in the EGLSurface and OpenGL ES section.
一個app不通過Canvas這個方法，而直接在Surface上繪制的辦法是通過OpenGL ES。我們将在EGLSurface and OpenGL ES 這一節中講到這個問題。
SurfaceHolder
Some things that work with Surfaces want a SurfaceHolder, notably SurfaceView. The original idea was that Surface represented the raw compositor-managed buffer, while SurfaceHolder was managed by the app and kept track of higher-level information like the dimensions and format. The Java-language definition mirrors the underlying native implementation. It’s arguably no longer useful to split it this way, but it has long been part of the public API.
一些在surface上工作的東西需要一個SurfaceHolder，尤其是SurfaceView。初始的想法是，Surface代表了raw格式的，被混合器管理的緩沖區，而SurfaceHolder被app管理。這樣，app可以在比如大小以及格式等更高層面上來處理問題。Java層定義了一個底層實作的上層鏡像。這種分層方法目前已經不再有意義，但是它已經長時間成為了公共API中的一部分。
Generally speaking, anything having to do with a View will involve a SurfaceHolder. Some other APIs, such as MediaCodec, will operate on the Surface itself. You can easily get the Surface from the SurfaceHolder, so hang on to the latter when you have it.
一般而言，對View所做的一切事情都需要通過一個SurfaceHolder。其他的一些api，比如MediaCodec，将直接在Surface上操作。你可以很容易的從一個SurfaceHolder中擷取一個Surface。
APIs to get and set Surface parameters, such as the size and format, are implemented through SurfaceHolder.
擷取和設定Surface參數的一些API，比如大小和格式，都是通過SurfaceHolder實作。
EGLSurface and OpenGL ES
OpenGL ES defines an API for rendering graphics. It does not define a windowing system. To allow GLES to work on a variety of platforms, it is designed to be combined with a library that knows how to create and access windows through the operating system. The library used for Android is called EGL. If you want to draw textured polygons, you use GLES calls; if you want to put your rendering on the screen, you use EGL calls.
OpenGL ES定義了一組Graphic的渲染API。它并沒有定義一個視窗系統。為了讓GLES可以工作在不同的平台之上，它設計了一個庫，這個庫知道如何在指定的作業系統上建立和使用視窗。Android上的這個庫叫做EGL。如果你想繪制一個多邊形，那麼使用GLES的函數；如果你想要将它渲染到螢幕上，你需要使用EGL的調用。
Before you can do anything with GLES, you need to create a GL context. In EGL, this means creating an EGLContext and an EGLSurface. GLES operations apply to the current context, which is accessed through thread-local storage rather than passed around as an argument. This means you have to be careful about which thread your rendering code executes on, and which context is current on that thread.
在你使用GLES做事之前，你需要建立一個GL的上下文。具體針對EGL，這意味着建立一個EGLContext 和一個 EGLSurface。GLES的操作作用在目前的上下文之中，而上下文的通路更多的依賴本地線程的存儲而不是參數的傳遞。這意味着你需要關注你的渲染代碼執行在哪個線程之上，并且這個線程的目前上下文是什麼。
The EGLSurface can be an off-screen buffer allocated by EGL (called a “pbuffer”) or a window allocated by the operating system. EGL window surfaces are created with the eglCreateWindowSurface() call. It takes a “window object” as an argument, which on Android can be a SurfaceView, a SurfaceTexture, a SurfaceHolder, or a Surface — all of which have a BufferQueue underneath. When you make this call, EGL creates a new EGLSurface object, and connects it to the producer interface of the window object’s BufferQueue. From that point onward, rendering to that EGLSurface results in a buffer being dequeued, rendered into, and queued for use by the consumer. (The term “window” is indicative of the expected use, but bear in mind the output might not be destined to appear on the display.)
EGLSurface可以是一個由EGL配置設定的離屏緩沖區（”pbuffer”）或者一個由作業系統配置設定的視窗緩沖區。EGL window Surface由eglCreateWindowSurface()函數建立。它持有一個視窗對象做為參數，在Android系統上，這個對象可能是一個SurfaceView，一個SurfaceTexture，一個SurfaceHolder，或者一個Surface—-所有的這些下面都有一個BuffferQueue。當你調用這個函數時，ELG建立了一個新的EGLSurface對象，并且将它連接配接到一個視窗對象的BufferQueue的生産者接口上。從這一刻開始，渲染到一個EGLSurface上将導緻一個buffer經曆出隊，渲染，入隊供消費者使用 這個過程。（這裡屬于window被使用，但這隻是一個預期，實際上，輸出可能并不顯示在螢幕上）
EGL does not provide lock/unlock calls. Instead, you issue drawing commands and then call eglSwapBuffers()to submit the current frame. The method name comes from the traditional swap of front and back buffers, but the actual implementation may be very different.
EGL并沒有提供lock/unlock的調用。你需要調用繪制指令，然後調用eglSwapBuffers()函數去送出目前的幀。這個方法名字的來源是傳統的交換前後緩沖區，但是目前實際的實作可能會有很大的不同。
Only one EGLSurface can be associated with a Surface at a time — you can have only one producer connected to a BufferQueue — but if you destroy the EGLSurface it will disconnect from the BufferQueue and allow something else to connect.
一個EGLSurface一次隻能關連一個Surface—-一次隻能有一個生産者連接配接到一個BufferQueue上—但是你可以銷毀這個EGLSurface，使得它和BufferQueue的連接配接斷開，這樣就可以用其他的東西連接配接這個BufferQueue了。
A given thread can switch between multiple EGLSurfaces by changing what’s “current.” An EGLSurface must be current on only one thread at a time.
一個給定的線程可以通過設定哪個是Current的方法來在不同的EGLSurfaces間切換，一個線程同時隻能有一個EGLSurface作為current。
The most common mistake when thinking about EGLSurface is assuming that it is just another aspect of Surface (like SurfaceHolder). It’s a related but independent concept. You can draw on an EGLSurface that isn’t backed by a Surface, and you can use a Surface without EGL. EGLSurface just gives GLES a place to draw.
一個普通的誤解是，很多人把EGLSurface當做是Surface的另一種表現（就像是SurfaceHolder）。他們二者之間有關系，但是這是兩個獨立的概念。你可以在一個EGLSurface上繪制而不需要一個Surface的支援，你也可以不通過EGL而使用一個Surface。EGLSurface隻不過給GLES提供了一個繪制的地方而已。
ANativeWindow
The public Surface class is implemented in the Java programming language. The equivalent in C/C++ is the ANativeWindow class, semi-exposed by the Android NDK. You can get the ANativeWindow from a Surface with the ANativeWindow_fromSurface() call. Just like its Java-language cousin, you can lock it, render in software, and unlock-and-post.
公共的Surface類是由Java實作的。在C/C++層對應的是ANativeWindow類，半暴漏在Android NDK中。你可以通過使用 ANativeWindow_fromSurface()從Surface中獲得一個ANativeWindow。就像他的java表親一樣，你可以lock，使用軟體渲染，然後unlock-and-post.
To create an EGL window surface from native code, you pass an instance of EGLNativeWindowType to eglCreateWindowSurface(). EGLNativeWindowType is just a synonym for ANativeWindow, so you can freely cast one to the other.
為了從本地代碼中建立一個EGL window surface，你需要給eglCreateWindowSurface()方法傳遞一個EGLNativeWindowType執行個體。EGLNativeWindowType等同于ANativeWindow，是以你可以在二者之間自由的轉換。
The fact that the basic “native window” type just wraps the producer side of a BufferQueue should not come as a surprise.
事實上，native window的本質不過是BufferQueue在生産者一側的包裝罷了。
SurfaceView and GLSurfaceView
Now that we’ve explored the lower-level components, it’s time to see how they fit into the higher-level components that apps are built from.
現在我們已經研究了底層的一些元件，是時候來看下更高層次上元件是如何工作的了。
The Android app framework UI is based on a hierarchy of objects that start with View. Most of the details don’t matter for this discussion, but it’s helpful to understand that UI elements go through a complicated measurement and layout process that fits them into a rectangular area. All visible View objects are rendered to a SurfaceFlinger-created Surface that was set up by the WindowManager when the app was brought to the foreground. The layout and rendering is performed on the app’s UI thread.
一個Android的app Framework層的UI是從視圖上而來的基于對象的層次結構。大多數的細節對我們的讨論來說無關緊要，但是了解一下過程依然是對我們有幫助的：即UI元素是如何通過負責的測量和布局過程來将他們部署在一個矩形區域裡面的。所有可見的view對象都呈現給了Surfaceflinger—當app由背景轉到前台後，通過WindowManager建立了Surface。Layout和渲染都是在app的UI線程裡面執行的。
Regardless of how many Layouts and Views you have, everything gets rendered into a single buffer. This is true whether or not the Views are hardware-accelerated.
根據你有多少layout和view，每個對象都在一個獨立的buffer中渲染。無論是否使用硬體加速，都是如此。
A SurfaceView takes the same sorts of parameters as other views, so you can give it a position and size, and fit other elements around it. When it comes time to render, however, the contents are completely transparent. The View part of a SurfaceView is just a see-through placeholder.
一個SurfaceView有跟其他view一樣的一些參數，是以你可以設定它的位置和大小等等。當它被渲染時，我們可以認為他的所有内容都是透明的。SurfaceView的視圖部分隻不過是一個透明的占位區域。
When the SurfaceView’s View component is about to become visible, the framework asks the WindowManager to ask SurfaceFlinger to create a new Surface. (This doesn’t happen synchronously, which is why you should provide a callback that notifies you when the Surface creation finishes.) By default, the new Surface is placed behind the app UI Surface, but the default “Z-ordering” can be overridden to put the Surface on top.
當SurfaceView的元件即将變為可見時，Framework層要求WindowManager請求Surfaceflinger建立一個新的Surface（這個過程是異步發生的，這就是為什麼你應該提供一個回調函數，這樣當Surface建立完成時你才能得到通知）。預設情況下，新建立的Surface在app UI Surface的下面，但是Z軸順序可能将這個Surface放在上面。
Whatever you render onto this Surface will be composited by SurfaceFlinger, not by the app. This is the real power of SurfaceView: the Surface you get can be rendered by a separate thread or a separate process, isolated from any rendering performed by the app UI, and the buffers go directly to SurfaceFlinger. You can’t totally ignore the UI thread — you still have to coordinate with the Activity lifecycle, and you may need to adjust something if the size or position of the View changes — but you have a whole Surface all to yourself, and blending with the app UI and other layers is handled by the Hardware Composer.
渲染在這個Surface（SurfaceView的surface）上的内容将由Surfaceflinger來混合，而不是由app。這才是SurfaceView的真正作用：這個surface可以被一個獨立的線程或者程序來渲染，和app UI上其他的渲染工作分開，這些緩沖區資料将直接傳遞給Surfaceflinger。當然你不能完全忽略UI線程—-你依然要和activity的生命周期保持一緻，并且一旦view的大小或者位置發生了改變，你可能也需要做些調整—但是你擁有了一個完整的Surface，并且這個Surface和app UI以及其他layer的混合工作将由HWC來完成。
It’s worth taking a moment to note that this new Surface is the producer side of a BufferQueue whose consumer is a SurfaceFlinger layer. You can update the Surface with any mechanism that can feed a BufferQueue. You can: use the Surface-supplied Canvas functions, attach an EGLSurface and draw on it with GLES, and configure a MediaCodec video decoder to write to it.
值得一提的是，新建立的surface實際上是生産者端，而消費者端則是一個Surfaceflinger的layer。你可以通過任何可以填充BufferQueue的途徑來更新這個surface。你可以：使用Surface提供的Canvas相關的函數，附加一個EGLSurface然後使用GLES在上面繪制，配置一個MediaCodec 視訊解碼器直接在上面寫資料。
Composition and the Hardware Scaler Now that we have a bit more context, it’s useful to go back and look at a couple of fields from dumpsys SurfaceFlinger that we skipped over earlier on. Back in the Hardware Composer discussion, we looked at some output like this:
我們現在有了更多的上下文知識，是以讓我們回去看看前面講到dumpsys SurfaceFlinger時我們忽略的幾個字段。我們來看看如下的幾個輸出：
type | source crop | frame name
------------+-----------------------------------+--------------------------------
HWC | [ 0.0, 0.0, 320.0, 240.0] | [ 48, 411, 1032, 1149] SurfaceView
HWC | [ 0.0, 75.0, 1080.0, 1776.0] | [ 0, 75, 1080, 1776] com.android.grafika/com.android.grafika.PlayMovieSurfaceActivity
HWC | [ 0.0, 0.0, 1080.0, 75.0] | [ 0, 0, 1080, 75] StatusBar
HWC | [ 0.0, 0.0, 1080.0, 144.0] | [ 0, 1776, 1080, 1920] NavigationBar
FB TARGET | [ 0.0, 0.0, 1080.0, 1920.0] | [ 0, 0, 1080, 1920] HWC_FRAMEBUFFER_TARGET
This was taken while playing a movie in Grafika’s “Play video (SurfaceView)” activity, on a Nexus 5 in portrait orientation. Note that the list is ordered from back to front: the SurfaceView’s Surface is in the back, the app UI layer sits on top of that, followed by the status and navigation bars that are above everything else. The video is QVGA (320x240).
這個抓取自Nexus 5豎屏模式，當在Grafika裡面播放視訊的時候。注意這清單是按照從後到前的順序排列的：SurfaceView的Surface在最後面，app ui layer在上面，然後是狀态欄和導航欄。視訊是QVGA的。
The “source crop” indicates the portion of the Surface’s buffer that SurfaceFlinger is going to display. The app UI was given a Surface equal to the full size of the display (1080x1920), but there’s no point rendering and compositing pixels that will be obscured by the status and navigation bars, so the source is cropped to a rectangle that starts 75 pixels from the top, and ends 144 pixels from the bottom. The status and navigation bars have smaller Surfaces, and the source crop describes a rectangle that begins at the the top left (0,0) and spans their content.
“source crop”訓示了Surface的buffer中要被SurfaceFlinger顯示的部分。App UI的surface大小是整個顯示的大小（1080*1920），但是由于需要顯示狀态欄和導航欄，是以從上面裁剪了75個像素，從下面裁剪了144個像素。
The “frame” is the rectangle where the pixels end up on the display. For the app UI layer, the frame matches the source crop, because we’re copying (or overlaying) a portion of a display-sized layer to the same location in another display-sized layer. For the status and navigation bars, the size of the frame rectangle is the same, but the position is adjusted so that the navigation bar appears at the bottom of the screen.
Frame一欄是指最終顯示在螢幕上的位置。APP UI因為是完全相同位置的拷貝，是以這個值和前一列完全相同。而對于狀态欄和導航欄，大小和前面一列是相似的，但是位置已經發生了改變。
Now consider the layer labeled “SurfaceView”, which holds our video content. The source crop matches the video size, which SurfaceFlinger knows because the MediaCodec decoder (the buffer producer) is dequeuing buffers that size. The frame rectangle has a completely different size — 984x738.
現在我們來看下SurfaceView這個layer，這個layer裡面是視訊的内容。source crop一列和視訊的大小一緻，SurfaceFlinger之是以知道這個資訊是因為MediaCodec解碼器申請的出隊的buffer的大小就是這麼大。而Frame則有一個完全不同的大小：984*738.
SurfaceFlinger handles size differences by scaling the buffer contents to fill the frame rectangle, upscaling or downscaling as needed. This particular size was chosen because it has the same aspect ratio as the video (4:3), and is as wide as possible given the constraints of the View layout (which includes some padding at the edges of the screen for aesthetic reasons).
SurfaceFlinger處理了這種大小的不同，通過縮放緩沖區資料的方式來填充到frame中，根據需要放大或者縮小。之是以選擇這個特殊的大小，是因為這個大小和視訊有相同的高寬比，并且在View Layout允許的寬度下盡可能的大（基于美觀的考慮，在螢幕邊緣也放置了一些填充物。）。
If you started playing a different video on the same Surface, the underlying BufferQueue would reallocate buffers to the new size automatically, and SurfaceFlinger would adjust the source crop. If the aspect ratio of the new video is different, the app would need to force a re-layout of the View to match it, which causes the WindowManager to tell SurfaceFlinger to update the frame rectangle.
如果我們在同一個surface上面播放了一個不同大小的視訊，那麼底層的BufferQueue會重新使用新的大小重新配置設定緩沖區，SurfaceFlinger也會調整它的source crop。如果新的視訊的高寬比也發生了變化，app需要強制要求View來re-layout來比對目前大小，這将導緻WindowManager通知SurfaceFlinger來更新每個frame矩形的大小。
If you’re rendering on the Surface through some other means, perhaps GLES, you can set the Surface size using the SurfaceHolder#setFixedSize() call. You could, for example, configure a game to always render at 1280x720, which would significantly reduce the number of pixels that must be touched to fill the screen on a 2560x1440 tablet or 4K television. The display processor handles the scaling. If you don’t want to letter- or pillar-box your game, you could adjust the game’s aspect ratio by setting the size so that the narrow dimension is 720 pixels, but the long dimension is set to maintain the aspect ratio of the physical display (e.g. 1152x720 to match a 2560x1600 display). You can see an example of this approach in Grafika’s “Hardware scaler exerciser” activity.
如果你使用這個Surface來做其他用途，比如GLES，那麼你可以通過調用SurfaceHolder#setFixedSize() 函數來設定Surface的大小。比如你可以設定一個遊戲的大小為1280*720，這樣當你去運作在一個大小為2K或者4K的螢幕上時，你可以顯著的減少需要填充的像素的數目。顯示處理器會來處理縮放。。。。
GLSurfaceView
The GLSurfaceView class provides some helper classes that help manage EGL contexts, inter-thread communication, and interaction with the Activity lifecycle. That’s it. You do not need to use a GLSurfaceView to use GLES.
GLSurfaceView類提供一個輔助類，這些類可以幫助我們管理EGL的上下文，線程通信以及與activity生命周期的互動。當然，你不需要GLSurfaceView就可以使用GLES。
For example, GLSurfaceView creates a thread for rendering and configures an EGL context there. The state is cleaned up automatically when the activity pauses. Most apps won’t need to know anything about EGL to use GLES with GLSurfaceView.
舉例來說，GLSurfaceView建立了一個用來渲染和管理EGL上下文的線程。當activity pause時，它會自動清空所有的狀态。大多數應用通過GLSurfaceView來使用GLES時，不需要了解任何和EGL有關的事情。
In most cases, GLSurfaceView is very helpful and can make working with GLES easier. In some situations, it can get in the way. Use it if it helps, don’t if it doesn’t.
在大多數情況下，GLSurfaceView對處理GLES來說很有幫助。但是在一些情況下，它可能是一種阻礙。僅僅在你需要的時候使用它。
SurfaceTexture
The SurfaceTexture class is a relative newcomer, added in Android 3.0 (“Honeycomb”). Just as SurfaceView is the combination of a Surface and a View, SurfaceTexture is the combination of a Surface and a GLES texture. Sort of.
SurfaceTexture類是從Android 3.0開始引入的。就像是SurfaceView是一個Surface和view的組合一樣，SurfaceTexture某種程度上是一個Surface和GLES材質的組合。
When you create a SurfaceTexture, you are creating a BufferQueue for which your app is the consumer. When a new buffer is queued by the producer, your app is notified via callback (onFrameAvailable()). Your app calls updateTexImage(), which releases the previously-held buffer, acquires the new buffer from the queue, and makes some EGL calls to make the buffer available to GLES as an “external” texture.
當你建立了一個SurfaceTexture時，你建立了一個BufferQueue，而你的app則是消費者。當一個新的buffer被生産者入隊後，你的app将會被回調函數通知（onFrameAvailable()）。你的app會調用updateTexImage()函數，釋放了先前持有的buffer，從隊列中acquire新的buffer，做了一些EGL調用，使得一些buffer可以作為額外的材質被GLES擷取。
External textures (GL_TEXTURE_EXTERNAL_OES) are not quite the same as textures created by GLES (GL_TEXTURE_2D). You have to configure your renderer a bit differently, and there are things you can’t do with them. But the key point is this: You can render textured polygons directly from the data received by your BufferQueue.
額外的材質（GL_TEXTURE_EXTERNAL_OES）跟由GLES自身建立的材質（GL_TEXTURE_2D）有一些不同。需要配置renderer略有不同，也有一些事情是不能做的。但是我們關注的重點是：我們可以使用從BufferQueue中收到的資料直接渲染多邊形。
You may be wondering how we can guarantee the format of the data in the buffer is something GLES can recognize — gralloc supports a wide variety of formats. When SurfaceTexture created the BufferQueue, it set the consumer’s usage flags to GRALLOC_USAGE_HW_TEXTURE, ensuring that any buffer created by gralloc would be usable by GLES.
你可以好奇，我們如何保證buffer中的資料格式可以被GLES正确讀取—-要知道gralloc可是支援各種各樣的格式。當SurfaceTexture建立BufferQueue時，它設定消費者的usage flags是GRALLOC_USAGE_HW_TEXTURE，這保證了任何由gralloc建立的buffer都是可以被GLES使用的。
Because SurfaceTexture interacts with an EGL context, you have to be careful to call its methods from the correct thread. This is spelled out in the class documentation.
因為SurfaceTexture要和EGL上下文互動，是以務必保證調用的方法來自正确的線程，這個在類的說明文檔中已經指出（SurfaceTexture objects may be created on any thread. updateTexImage() may only be called on the thread with the OpenGL ES context that contains the texture object. The frame-available callback is called on an arbitrary thread, so unless special care is taken updateTexImage() should not be called directly from the callback.）。
If you look deeper into the class documentation, you will see a couple of odd calls. One retrieves a timestamp, the other a transformation matrix, the value of each having been set by the previous call to updateTexImage(). It turns out that BufferQueue passes more than just a buffer handle to the consumer. Each buffer is accompanied by a timestamp and transformation parameters.
如果你對這個類的文檔做更深的研究，你會發現一些古怪的函數。一個檢索一個時間戳，而另外一個則變換一個矩陣，這兩個值都是在前面那個updateTexImage()調用時被改變的（？）。這說明BufferQueue傳給消費者的不僅僅是一個buffer的句柄。每一個buffer都伴随着一個時間戳參數和一個transformation參數。
The transformation is provided for efficiency. In some cases, the source data might be in the “wrong” orientation for the consumer; but instead of rotating the data before sending it, we can send the data in its current orientation with a transform that corrects it. The transformation matrix can be merged with other transformations at the point the data is used, minimizing overhead.
Transformation參數用于提高效率。在一些情況下，提供給消費者的源資料可能方向是錯誤的，但是相比發送之前旋轉資料，我們可以發送一個目前方向的資料，同時一起發送一個transform參數用以變換。transformation matrix（變換矩陣）可以跟其他一些變換參數一起傳遞，用以減小系統開銷。
The timestamp is useful for certain buffer sources. For example, suppose you connect the producer interface to the output of the camera (with setPreviewTexture()). If you want to create a video, you need to set the presentation time stamp for each frame; but you want to base that on the time when the frame was captured, not the time when the buffer was received by your app. The timestamp provided with the buffer is set by the camera code, resulting in a more consistent series of timestamps.
時間戳則是為了明确具體的buffer源資料。舉例來說，假設你将camera的輸入做為生産者（通過setPreviewTexture）。如果你先要建立一段視訊，你需要給每一幀打上時間戳；你想要基于幀被錄制的時間，而不是app收到buffer的時間。是以這個時間戳是被camera的代碼提供的，這些時間戳将更加準确一緻。
SurfaceTexture and Surface
If you look closely at the API you’ll see the only way for an application to create a plain Surface is through a constructor that takes a SurfaceTexture as the sole argument. (Prior to API 11, there was no public constructor for Surface at all.) This might seem a bit backward if you view SurfaceTexture as a combination of a Surface and a texture.
如果你仔細看過api說明的話，你會發現建立一個簡易Surface的唯一辦法是使用一個帶有SurfaceTexture參數的構造函數（在API11之前的版本，Surface根本就沒有一個公有的構造函數）。如果你認為SurfaceTexture是Surface和texture的組合的話，似乎就有點落後。
Under the hood, SurfaceTexture is called GLConsumer, which more accurately reflects its role as the owner and consumer of a BufferQueue. When you create a Surface from a SurfaceTexture, what you’re doing is creating an object that represents the producer side of the SurfaceTexture’s BufferQueue.
在底層，SurfaceTexture實際上是一個GLConsumer，這個名字更能反應出它作為一個BufferQueue的持有者和消費者的角色。當你使用一個SurfaceTexture來建立一個Surface時，你實際上做的，其實是建立了BufferQueue（SurfaceTexture持有的）的生産者一側。
Figure 2.Grafika’s continuous capture activity
In the diagram above, the arrows show the propagation of the data from the camera. BufferQueues are in color (purple producer, cyan consumer). Note “Camera” actually lives in the mediaserver process.
Encoded H.264 video goes to a circular buffer in RAM in the app process, and is written to an MP4 file on disk using the MediaMuxer class when the “capture” button is hit.
如上圖所示，箭頭顯示了從相機開始資料的傳輸。圖中紫色的是生産者，而藍色的則是消費者。注意Camera實際上是在mediaserver程序中的。編碼過的H.264 video存入app程序的一個環形緩沖區内，并且通過MediaMuxer類被寫入一個硬碟上的MP4檔案中（？）。
All three of the BufferQueues are handled with a single EGL context in the app, and the GLES operations are performed on the UI thread. Doing the SurfaceView rendering on the UI thread is generally discouraged, but since we’re doing simple operations that are handled asynchronously by the GLES driver we should be fine. (If the video encoder locks up and we block trying to dequeue a buffer, the app will become unresponsive. But at that point, we’re probably failing anyway.) The handling of the encoded data — managing the circular buffer and writing it to disk — is performed on a separate thread.
（上圖中）所有的三組BufferQueue都是由app中同一個EGL上下文來處理的，GLES的操作實在UI線程内執行的。在UI線程中來做SurfaceView的渲染工作一般是不推薦的。但是鑒于我們隻需要做一些簡單的操作，（其餘的）由GLES來做異步的處理，是以應該問題不大（如果視訊編碼器死鎖，那麼我們試圖擷取Buffer的操作将被阻塞，app将會發生ANR。但是在這種情況下，無論如何錯誤都會發生）。編碼好的資料的處理—-管理這個環形緩沖區和将它寫到磁盤上——是在一個單獨的線程上的。
The bulk of the configuration happens in the SurfaceView’s surfaceCreated() callback. The EGLContext is created, and EGLSurfaces are created for the display and for the video encoder. When a new frame arrives, we tell SurfaceTexture to acquire it and make it available as a GLES texture, then render it with GLES commands on each EGLSurface (forwarding the transform and timestamp from SurfaceTexture). The encoder thread pulls the encoded output from MediaCodec and stashes it in memory.
大量的配置工作發生在SurfaceView的 surfaceCreated()函數被調用時。EGLContext被建立，用于顯示和視訊編碼器的EGLSurfaces也同樣被建立。當新的一幀來臨時，我們通知SurfaceTexture來擷取并且使之變為GLES texture，然後使用GLES指令來在各個EGLSurface上渲染（從SurfaceTexture來發送transform 和 timestamp）。解碼器線程從MediaCodec取出資料并且把它存放在記憶體裡。
TextureView
The TextureView class was introduced in Android 4.0 (“Ice Cream Sandwich”). It’s the most complex of the View objects discussed here, combining a View with a SurfaceTexture.
TextureView類是由Android 4.0引入的。它是我們目前介紹過的最複雜的類對象，由一個View混合了一個SurfaceTexture而成。
Recall that the SurfaceTexture is a “GL consumer”, consuming buffers of graphics data and making them available as textures. TextureView wraps a SurfaceTexture, taking over the responsibility of responding to the callbacks and acquiring new buffers. The arrival of new buffers causes TextureView to issue a View invalidate request. When asked to draw, the TextureView uses the contents of the most recently received buffer as its data source, rendering wherever and however the View state indicates it should.
讓我們回憶一下，SurfaceTexture實際上是一個‘GL consumer’，消費Graphic緩沖區的資料，并且使他們可以作為紋理。TextureView包裝了一個SurfaceTexture，接管了它響應回調函數和申請新的緩沖區的工作。新的緩沖區的到來使得TextureView發出一個view重繪的請求。當被要求繪制時，TextureView使用了最新收到的緩沖區資料作為資料來源，渲染任何view狀态訓示它做的工作。
You can render on a TextureView with GLES just as you would SurfaceView. Just pass the SurfaceTexture to the EGL window creation call. However, doing so exposes a potential problem.
你可以在TextureView上使用GLES渲染，就像你在SurfaceView上做的一樣。隻需要在EGL window建立時，把SurfaceTexture傳遞過去。但是，這樣做會暴露一個潛在的問題。
In most of what we’ve looked at, the BufferQueues have passed buffers between different processes. When rendering to a TextureView with GLES, both producer and consumer are in the same process, and they might even be handled on a single thread. Suppose we submit several buffers in quick succession from the UI thread. The EGL buffer swap call will need to dequeue a buffer from the BufferQueue, and it will stall until one is available. There won’t be any available until the consumer acquires one for rendering, but that also happens on the UI thread… so we’re stuck.
大多數情況下，BufferQueue在不同的程序間傳遞資料。當我們在TextureView上使用GLES渲染時，生産者和消費者在一個程序内，而且他們很可能被同一個線程來處理。假設我們連續的UI線程上送出資料，EGL需要從BufferQueue中dequeue一個Buffer，但是在知道一個buffer可用時，這個操作才會完成。但是除非消費者acquire了一個buffer用于渲染，否則不會有任何可用的buffer，但這個過程同樣發生了UI線程中。現在我們悲劇了。
The solution is to have BufferQueue ensure there is always a buffer available to be dequeued, so the buffer swap never stalls. One way to guarantee this is to have BufferQueue discard the contents of the previously-queued buffer when a new buffer is queued, and to place restrictions on minimum buffer counts and maximum acquired buffer counts. (If your queue has three buffers, and all three buffers are acquired by the consumer, then there’s nothing to dequeue and the buffer swap call must hang or fail. So we need to prevent the consumer from acquiring more than two buffers at once.) Dropping buffers is usually undesirable, so it’s only enabled in specific situations, such as when the producer and consumer are in the same process.
解決這個問題的辦法是：讓BufferQueue保證始終都有一個buffer是可以被dequeued的，這樣這個過程才不會阻塞。如何才能保證這一點？一個方法是當BufferQueue上有新的buffer被queued時，就丢棄掉之前queue的緩沖區，我們還要設定最小緩沖區數量的限制和最大acquired緩沖區數量的限制（如果你的隊列隻有三個緩沖區，但是三個緩沖區都被消費者acquired，這樣我們就不能dequeue到任何buffer，阻塞就發生了。是以我們需要阻止消費者acquire兩個以上的緩沖區）丢幀一般是不可接受的，是以這個方法隻能用在一些特殊的場景裡，比如當生産者和消費者在一個程序裡面的時候。