Table of Contents
1 Introduction 1
1.1 Problem Statement 1
1.2 Delimitations 1
1.3 Thesis Outline 2
2 3D Game Engine Overview 3
2.1 Resource Handling 3
2.1.1 Models 3
2.1.2 Textures 4
2.1.3 Shaders 5
2.1.4 Materials 6
2.1.5 Animations 6
2.2 Scenegraphs 6
2.3 Rendering 7
2.3.1 Methods 8
2.3.2 View Frustum Culling 9
2.3.3 Occlusion Culling 10
2.3.4 Spatial Acceleration Structures 11
2.3.5 Hardware Specific Optimisations 12
3 Graphics Libraries 13
3.1 OpenGL 13
3.1.1 Versions 1.0 – 1.5 13
3.1.2 Versions 2.0 – 2.1 15
3.2 OpenGL ES 15
3.2.1 Versions 1.0 – 1.1 16
3.2.2 Version 2.0 16
4 Graphics Hardware 18
4.1 PC Hardware 18
4.2 Mobile Hardware 18
5 Approach 21
5.1 3D Game Engine 21
5.1.1 The Effect Files 21
5.1.2 The Model Files 23
5.1.3 Scene Representation 24
5.1.4 Scene Management 25
5.1.5 Animations 25
5.1.6 Built-in and Custom Uniforms 26
5.1.7 Scene Rendering 28
5.2 The Demon Demo 29
5.2.1 OpenGL ES 1.1 Adaptation 30
5.2.2 Symbian OS Adaptation 31
5.3 3D Kodo Game Prototype 31
6 Evaluation 34
6.1 3D Game Engine 34
6.1.1 Resource Handling 34
6.1.2 Data Files 35
6.1.3 Scene Management 36
6.1.4 Rendering 37
6.2 Demon Demo 37
6.3 3D Kodo Game Prototype 38
7 Conclusions and Further Work 40
Chapter 1 - Introduction
1 Introduction
Mobile phones constitute an interesting hardware platform for game developers since so
many people always carry a phone around with them. However, mobile phones are
generally not specifically designed to support gaming, which poses problems for game
developers. One of these problems has been that mobile phones traditionally have
provided comparatively simple graphics. This thesis aims to evaluate the graphics
capabilities of current and upcoming mobile phones, specifically focused on 3D
graphics using the OpenGL ES graphics programming interface. OpenGL ES is an
adaptation of the OpenGL industry standard aimed at embedded systems, and is
available in two main versions: 1.1 and 2.0. This thesis is mainly focused on the later
version which supports more advanced graphical effects through the use of shader
programs.
The goals for the project were to examine how to develop mobile games with OpenGL
ES 2.0 and how three-dimensional graphics and shader effects can successfully be used
in mobile games. This was accomplished by developing a 3D game engine on top of a
OpenGL ES 2.0 emulator library currently available and using this engine to create a
game prototype. The differences between PCs and mobile phones, between 2D and 3D
games, and between 3D graphics with and without shaders were evaluated.
1.1 Problem Statement
My main research question for the thesis was:
What are the specific technical considerations relating to graphics that apply when
developing 3D games for mobile phones?
This thesis is specifically focused on graphics, and does not cover other areas such as
game design, sound or input. Furthermore, since most games are built on top of a game
engine and game engines handle the technical details of the platform, this thesis puts
much focus on the design of game engines for mobile phones.
1.2 Delimitations
Since the project was a collaboration between two students, Erik Olsson and Mikael
Gustavsson, the background information in chapters 2 – 4 are common to both reports.
The remainder of the theses are individual. We made the following delimitations
regarding the individual parts of the theses:
Mikael will focus on the construction of a 3D graphics engine based on the OpenGL ES
2.0 graphics library.
Erik will focus on evaluating the limitations of the platform from the experiences gained
while creating the game prototype.
1
Chapter 1 - Introduction
1.3 Thesis Outline
The thesis can be divided into two main parts; background and implementation/
evaluation.
Background provides some background information about graphics libraries, 3D game
engines and 3D graphics hardware in general and is referred to from other parts of the
thesis. The background part contains the following chapters:
3D Game Engine Overview
This chapter describes 3D game engines in general, as well as the different parts they
are commonly constructed of.
Graphics Libraries
This chapter describes the OpenGL and OpenGL ES graphic libraries, including an
overview of the version histories.
Graphics Hardware
A description of the development of graphics hardware, both for mobile phones and PC
for comparison.
Implementation/evaluation describes the details of our implementation of the engine and
game prototype, as well as an evaluation of the results. The chapters of this part is as
follows:
Approach
Implementation details, focusing on the 3D graphics engine.
Evaluation
An evaluation of our results as well as some more general discussion about 3D graphics
and games for mobile phones and how the limitations of the hardware affect the design
of game engines for mobile platforms.
Conclusions and Further Work
Conclusions of our work and suggestions for further study.
2
Chapter 2 - 3D Game Engine Overview
2 3D Game Engine Overview
A 3D Game Engine is a software component which is intended to perform certain tasks,
such as handling resources, scenegraphs and rendering.
2.1 Resource Handling
Resources are the content of the scene, i.e. what eventually is drawn on the screen.
Resources may include models, textures, shaders, materials and animations.
2.1.1 Models
Models are the geometries of the scene, which are bound to objects in the scenegraph.
Models can be generated through code, but most often they are read from a file, which
usually has been exported from a modelling application, such as Autodesk's Maya. The
geometry consists of vertex data, as well as a surface description which describes how
vertices are connected to create primitives such as polygons, triangle strips, lines or
points. Figure below shows an example model.
Vertices have different types of attributes, one of which is position. Other common
attributes are normals, texture coordinates, colours, tangents, binormals, bone weights
and bone indices. A vertex normal defines the surface normal at a specific position and
is most often used for lighting calculations. Texture coordinates are used to map a
texture image onto the surface. Tangents and binormals, together with normals, form the
basis of tangent space, which is sometimes referred to as surface space. Tangent space
is used for bump map lighting calculations. Bone weights and bone indices can be used
to deform a geometry in a non-rigid way, such as bending an arm of a character. This is
called skinning, and is most often used in animation.
3
Figure 1: A model of the famous Utah Teapot, shown here in wireframe mode
Chapter 2 - 3D Game Engine Overview
2.1.2 Textures
Originally, textures were two-dimensional images which were mapped onto surfaces.
Nowadays they are better thought of as general containers of data to be used during
rendering. Textures can be one-, two- or three-dimensional. When data is read from a
texture it is usually filtered to make the output continuous and reduce aliasing.
Textures can be used for a number of different purposes, for example:
● Diffuse map – defines the colour of the surface, i.e. the original use of textures.
By far the most common. See figure 2 above.
● Detail map – adds more fine-grained details than a diffuse texture and is usually
repeated across the surface with a high frequency.
● Specular map – defines the reflectiveness of the surface. Usually monochrome.
● Emissive map – defines the light emittance of the surface, which enables the
surface to glow regardless of external light sources.
● Ambient occlusion map – defines the accessibility of the surface and has to be
calculated with regards to the surrounding geometry. Points surrounded by a
large amount of geometry has low accessibility and becomes dark.
● Normal map – stores surface normals, which can be used in bump mapping to
give the illusion of a much more detailed geometry than what is actually used.
● Light map – stores a pre-calculated lighting of the scene. This technique is on
the decline due to the increasing dynamic nature of game worlds.
● Depth map – stores the depth of a scene as seen from some view. Often rendered
from a light position to be used in shadow calculations.
4
Figure 2: A textured Utah Teapot
Chapter 2 - 3D Game Engine Overview
● Environment map – stores a view of the environment as seen from an object or
the origin of the scene, either in the form of a sphere map (for instance a
photograph taken with a fish-eye lens) or as a cube map (a set of 6 images, one
for each direction along the coordinate axes). Environment maps are usually
mapped on objects to make them appear reflective.
Generally, hardware limits how many textures that can be used simultaneously when
rendering in real time. Textures can have a number of different channels, grey-scale
textures, also known as luminance textures, only have one channel, while colour
textures most often has four; red, green, blue and alpha (RGBA).
2.1.3 Shaders
Shaders are short programs that are usually executed on a graphics processing unit
(GPU). They combine the power of dedicated hardware with the versatility of a
software renderer. Different shader units are arranged in a pipeline, a typical example
can be seen in figure 3. Shaders receive data in the form of attributes and uniforms.
Attributes vary with every element that is processed and are provided either from the
previous shader in the pipeline or by the engine. Common attributes are described in
chapter 2.1.1. Uniforms on the other hand vary at most once per draw call. Typical
examples of uniforms are object properties such as position, texture bindings and
material properties. On modern hardware there are up to three types of shader units
available: vertex shaders, geometry shaders and fragment shaders.
Vertex shaders operate on individual vertices, and receive vertex attributes from the
engine. The vertex shader can generate or modify any vertex attributes, such as
position, colour or texture coordinates. Common usages include making a tree's
branches sway in the wind, moving raindrops and skinning a character model.
Geometry shaders operate on individual primitives, such as polygons, points or lines
and receive input from a vertex shader. The geometry shader can emit zero or more
primitives. Common usages include geometry tessellating, generating particle polygons
from points or extruding shadow volumes.
Fragment shaders, sometimes referred to as pixel shaders, operate on individual
fragments. The ouput of the fragment shader is the colour of a pixel written to the frame
buffer. The fragment shader also has the ability to discard a fragment so that it is not
written to the frame buffer. Common usages include per pixel lighting, bump mapping
and reflections.
When shader hardware is not available, a fixed function pipeline must be used. This
pipeline can usually be set up to perform basic calculations such as per vertex lighting,
rigid transformations and blending of several textures. Many effects can be
accomplished both with and without shaders, but shaders provide a much wider range of
effects and better image quality.
5
Figure 3: A shader pipeline
Chapter 2 - 3D Game Engine Overview
2.1.4 Materials
A material is basically a collection of textures, shaders and uniform values. Materials
are often exported from modelling applications or from shader studio applications such
as NVIDIA's FX Composer or AMD's RenderMonkey. These applications however
create and modify what they refer to as effects. Effects are best thought of as material
templates; for example, a fur effect might be used in several fur materials with different
colours or textures.
2.1.5 Animations
Animations can be divided into three groups, node animations, blend shapes and bone
animations, and span a certain number of frames. Animation data is specified either for
each frame, or at a number of keyframes. When a keyframed animation is played back,
data of two adjacent keyframes are interpolated to produce data for the frames in-
between in order to keep the animation fluent.
Node animations modifies the position, rotation or scale of nodes in the scenegraph.
Less commonly used in games, where it is primarily adopted for cut scenes and camera
movements.
Blend shapes are a sequence of models, which define the positions of the object's
vertices. Require much memory but little processing power, and were often used to
animate characters in older games.
Bone animations modifies the position, rotation or scale of certain nodes known as
bones. Vertices in a model are linked to one or many of these bones in order to follow
their movement. This technique is known as skinning. Bones are usually linked in a
hierarchical manner to affect each other, mimicking the behaviour of, for example, a
skeleton.
2.2 Scenegraphs
A scenegraph is a data structure that arranges the logical and often spatial representation
of a graphical scene [1]. A scenegraph contains a number of linked nodes, usually in
such a way that a node can have multiple child nodes, but only one parent, thus making
it a directed graph, see figure 4 for a simple example scenegraph. In order to be useful,
the graph should also be acyclic. Nodes can be divided into two categories, group
nodes, which may have children, and leaf nodes, which may not. There can be
numerous types of nodes, for instance transform nodes, object/model nodes, light nodes,
camera nodes and emitter nodes for particle systems.
6
Chapter 2 - 3D Game Engine Overview
A transform node is a group node which represents a transform relative to its parent
node. This arranges the scene in a hierarchical structure, which is useful for numerous
reasons, such as moving a complex object by only moving the parent node.
An object node, or a model node, is a leaf node that represents a graphical object that
can be rendered. It has references to a mesh and a material resource.
All leaf nodes, such as those for objects, lights, cameras and emitters receive transforms
from a parent transform node.
Scenegraphs are built and modified as the game world changes, but parts are often
loaded from files that have been exported from a modelling application or a custom
world editor.
2.3 Rendering
In 1986, Jim Kajiya introduced the rendering equation [3], which is a general integral
equation for the lighting of any surface.
L
o
x ,
= L
e
x ,
∫
f
r
x ,
' ,
L
i
x ,
'
'⋅
n d
'
The equation describes the outgoing light (L
o
) in any direction from any position on a
surface, which is the sum of the emitted light (L
e
) and the reflected light. The reflected
light itself is the sum of the incoming light (L
i
) from all directions, multiplied by the
surface reflection and cosine of the incident angle. All methods of calculating lighting in
modern computer graphics can be seen as approximations of this equation.
There are several ways of rendering a scene, such as ray-tracing and radiosity. Such
methods allow for advanced lighting effects and global illumination [4]. Global
illumination takes the environment into consideration so that effects such as reflections,
refractions and light bleeding are possible. However, global illumination is generally
7
Figure 4: A simple scenegraph of a car [2]
Chapter 2 - 3D Game Engine Overview
considered too slow to be applied to games. This section will hence focus on using
hardware accelerated rasterisation [4], which is the most common method employed by
games. Although rasterisation only directly supports local lighting effects, which only
considers the actual surface and light sources, modern games include many global
effects such as shadows and reflections. This, however, makes the process of rendering
a modern game a very complex task.
2.3.1 Methods
Drawing relies on the use of a depth buffer, also called z-buffer, which is a buffer that
associates a depth value with each pixel in the frame buffer [1],[4]. This allows the
drawing of objects to be performed in any order, since a pixel is written to the frame
buffer only if it is closer to the viewpoint than what is currently stored in the depth
buffer. This is called hidden surface removal.
The basic way of rendering a scene is as follows:
1. Traverse the scene graph in a depth-first-order, concatenating node transforms
with the resulting parent transform.
2. When an object node is reached, draw the object using its associated material
and model.
This approach unfortunately has several problems, some of which being that it cannot
render dynamic lights, dynamic shadows or transparent objects correctly. To handle
dynamic lights, the light node transforms have to be known before any objects are
drawn. Dynamic shadows are even more problematic since they require the use of
several rendering passes. Due to the nature of the depth buffer on current graphics
hardware, transparent objects has to be sorted back-to-front and drawn after the opaque
objects. The following method is an example of how to address these problems:
1. Traverse the scene graph and concatenate the node transforms, put the lights and
their transforms in a list, and put all objects and their transforms in another list.
2. For each light that casts shadows, render the scene as seen from the light source
to one or several depth maps.
3. Sort the list of objects, so that transparent objects are sorted back-to-front and
placed after opaque objects.
4. Draw the objects, using information from the list of lights and the light's depth
maps for lighting and shadow calculations.
There are several alternatives to this method, mainly related to lighting and shadows.
There are two common methods of drawing dynamic shadows in games, shadow
mapping and shadow volumes [4]. Shadow mapping uses the depth maps generated in
step 2, shadow volumes do not.
While the method listed above calculates lighting per-object-per-light (POPL); the
alternative, per-light-per-object (PLPO), is also common. If this method is used, the
fourth step in the previous method is replaced with the following two steps:
4. Draw the objects lit by the ambient lighting in the scene.
5. For each light, draw the scene again lit by only this light, additively blending the
result into the frame buffer.
8
Không có nhận xét nào:
Đăng nhận xét