Posts RSS Comments RSS Digg Technorati Blinklist Furl reddit 63 Posts and Comments till now
This wordpress theme is downloaded from wordpress themes website.

Archive for the 'Graphics & GPU' Category

Vulkan 1.0 released

I started writing DirectX/OpenGL code when I took multiple college classes in computer graphics and game programming (in 2001 or 2002).  In the years since then, I’ve continuously been exposed to DirectX/OpenGL through my GPU modeling job and through hobby projects.

So I have experience writing code for different versions of DirectX/OpenGL APIs and seeing their evolution.  In the time of my experience, there’s been some major API changes such as the move from fixed function pipeline to programmable pipeline (shader programs) (GLSL, HLSL).

Yesterday was another big change – Vulkan 1.0 was officially released.  Vulkan is lower level (like Microsoft DirectX 12 and Apple Metal), but it’s also cross-platform and cross-vendor.  I’m really into my Unreal Engine 4 project for the time being (and probably all of 2016), but I at least did some reading about Vulkan.  Here are a few links I enjoyed reading to get a taste of Vulkan:


API reference:

Detailed Overview:

Samples – Hello World Triangle


Fix minor camera bug (with GLM camera)

My previous post’s Release 001 had a camera bug such that turning right from the initial state was broken.  You can see this if you run the WebGL (Emscripten) previous post’s Release 001 demo – just press “E” to turn right a little.

The bug was related to mixing up degrees and radians.  Basically what happened was that I used a simple camera from this tutorial ( ) (by Tom Dalling) to generate my viewProj matrix.  The tutorial is dated 2013/01/21 and uses GLM ( ).  But GLM released 2014/11/30 changed GLM from degrees to radians, which broke the camera.  See GLM release notes “Transition from degrees to radians compatibility break and GLM help” here ( ).

I fixed this tiny bug (and I started rewriting/refactoring the camera).  I also added keyboard controls “R” to look up and “F” to look down.  Here’s a WebGL (Emscripten) Release 002 with the fix – Release 002

Emscripten build target (JavaScript asm.js WebGL)

I’ve posted a WebGL version of my recent “Tile Map Geometry” post.  It was built using Emscripten, which is just one of the build targets for my (C++, SDL2, GLES2, CMake) project.  For editing the web code (HTML, CSS, JavaScript) not generated by Emscripten, I tried out Brackets (

Rather than embed the JavaScript directly in this WordPress post, I uploaded it to a separate area labeled release “001”.  Click the following screen shot – it’s a link.  For future versions I’ll do the same thing – post a release at “..\pemproj\grfxdemossdl2015\" and link to it from a WordPress post.

Controls: WASD moves, QE turns, ZX flies (up, down).


Tile Map Geometry

Today I implemented tile map geometry.

Basically the way it works is I have a class TileFloorData that stores a 2D array of Tile’s.  For now, each struct Tile contains a height.  In the future, Tile will also have (a) texture ID(s).  For rendering, TiledFloor is an renderable Entity that generates geometry based on its TileFloorData.

To start with, I just did the geometry (no textures yet).  The geometry is designed such that the TileFloorData can be drawn with a single draw call – so it’s a single list of vertex positions, a single list of vertex colors, and a single list of indices.  The geometry is also designed to work with a texture atlas.  So each tile has 5 faces (we ignore the bottom face) and 4 verts per face (so 20 verts per tile).  Vertices are not shared between faces – because we want independent texture coordinates for texture atlas lookup.  Each tile face also has 6 indices per face (so 30 indices per tile).

For GLES2 core profile, glDrawElements() can only use GL_UNSIGNED_BYTE or GL_UNSIGNED_SHORT (ie GL_UNSIGNED_INT is not allowed), so that’s a max of 65536 vertices.  With 20 verts per tile, that’s a max of 3276 tiles, which is only 57×57 tiles.  If we want more tiles than that, then we can break it up into multiple draw calls (such as four draw calls for 114×114 tiles).  Or we could use glDrawArrays() in which case we’d need 30 verts per tile (instead of 20).

Just for fun…  A single vertex has 3 coordinates per position and 3 values (red, green, blue) for color.  Each of these values is a GLfloat.  So a single vertex has (sizeof(GLfloat) is 4 bytes) * (3 + 3 is 6) = 24 bytes.  Each tile has 20 verts, so that’s 480 bytes per tile.  57×57 tiles * 480 bytes per tile would be 1,559,520 bytes or a little under 1.5 MB.  Using GLshort, indices are only 2 bytes each, so 2 bytes * 30 indices per tile = 60 bytes per tile.  57×57 tiles * 60 bytes per tile would be 194,940 bytes.  For both vertices + indices, 57*57*(480+60) = 1,754,460 bytes or a little over 1.67 MB.

Here’s some screen shots – 10×10 stair-step (incrementing) heights, 10×10 random heights, 50×50 random heights.  For the 50×50 example, I had to increase my zFar plane from 100 to 200.  The fourth screen shot shows that with 58×58 we lost some tiles.  This is because on GLushort overflow it wraps around, so any index value of 65536 or greater will wrap-around and cause us to redraw tiles we already drew (ie 65536 becomes 0, 65537 becomes 1, etc).

The final screen shot shows Z-fighting.  This happens because adjacent tiles share bottom verts and their top verts are on the same plane, so we have overlapping triangles (tile faces) on the same plane.  However, we only see this from the bottom side of the tile map.  In the final game, these won’t be visible to the player, so they will all be back face culled.  The top side doesn’t have this problem, because you only see the tallest part of tile sides (everything else gets drawn over).

 image image image image image

C++, SDL2, OpenGL ES 2 (GLES2), CMake, Git

Background, CMake

Two C++ OpenGL side projects I’ve posted about in the last two years are…  One uses (Qt + OpenGL + Qt Creator with qmake .pro build files) for an STL model viewer in Qt Creator for (Windows, Mac OS X, Linux).  The other uses (SDL2 + OpenGL) for cross-platform demos such as a bookshelf style grid view and a shadow mapping demo (using assimp for models).  The build system for this latter project was a bit messy because I started from SDL2 sample projects for (Windows Visual Studio, Linux make files, Android nmake, XCode OSX, XCode iOS).  This made sense at the time (because my real job keeps me busy so I wanted a faster short-term path to see my cross-platform code running).  However, I decided I wanted a cleaner long-term build solution, so I gave this SDL2 project a complete “reboot”.

I started a new (but similar) project from scratch using CMake.  This was a great way to get some experience using CMake.  At my job, we use Visual Studio for Windows and Makefiles for Linux (with GCC).  For this side project there’s a lot more native build systems, so it makes more sense to use a tool like CMake (or SCons etc).


To make the cross-platform aspect simpler at this stage, I’m doing all the OpenGL stuff with OpenGL ES 2 core profile (obviously in the long-term I could have multiple rendering paths).  The current list of platforms is – Windows (ANGLE), Linux, OS X, Android, iOS, Emscripten.  I kind of wanted to use GLES3 because of new features like (vertex array objects) and (texture arrays).  However, there is currently better cross-platform support for GLES2.  In particular, for Emscripten GLES3, better WebGL 2 browser support is required.

ANGLE is also currently lacking in GLES3 support.  Wikipedia says OpenGL 4.3 provides full compatibility with OpenGL ES 3.0, but I’m not sure whether it’s 100% true.  Plus I’m eventually planning to use ANGLE for Windows Mobile 10 support (or Universal Windows Platform) too.

OSX is lacking in terms of GLES2 support.  GL_ARB_ES2_compatibility does not provide full compatibility with GLES2.  I found that even simple GLES2 code requires changes to work on OSX – for example, a VAO (vertex array object) must be bound for OSX, while GLES2 core does not support VAOs.  GL_ARB_ES2_compatibility Overview states “will ease the process of porting applications from OpenGL ES 2.0 to OpenGL”, so it’s not full ES2 support.

So for OSX I will need some conditionals (or an alternate rendering path).  Or I could wait for ANGLE to add GLES2 support.  Or I could try MetalGL.  With the release of OSX 10.11 (just a few days ago), OSX now supports Metal, and there is a project MetalGL that can map GLES2 to Metal.  The day OSX 10.11 came out (9/30) I was able to run the demo (DrawLoadDemo) for MetalGL on OSX.  I was able to connect my own project to it (using CMake), but there is some compatibility issue with SDL2.  I’m not yet sure how easy it will be to get MetalGL to work with SDL2 on OSX.  TBD on this.

Qt Creator

I’m using Qt Creator as my primary editor and IDE (when possible).  Qt Creator has support for CMake.


I’m using Git (instead of Subversion).  For basic use, the main difference between Git and Subversion is that Git is not designed around a centralized server.  So when I do a git commit, I also do a git push to push my local commit to the online central server.  Another obvious difference I noticed is that when I check svn log using TortoiseSVN it takes a long time to get the log from the central server (unless you use something to cache it).

Modern C++

An interesting aside is that Linus Torvald (who initially designed and developed git) likes C better than C++.  I actually have some sympathy with that view – C is simpler than C++ (so it’s less likely to result in pointlessly complex code) and it’s lower level (so it’s easier to know what the compiler will do).  Plus my job (GPU modeling) relates to systems software, and systems software programmers (eg computer engineers) tend to like lower layer code (ie closer to the hardware).

For my job we haven’t upgraded our project yet from Visual Studio 2010.

For this side project, I’m making a conscious effort to use more modern C++ features (eg C++11) when useful.

Screen Shots

I’ll figure out later how best to post the Emscripten version (uses JavaScript and WebGL).  For now I’ll just share a screenshot.  I’m also sharing a photo of the same demo running on a Kindle Fire HD 6.  This simple demo also runs on Linux, Windows, iOS, OSX.  A lot of the changes (ie the rewrite) I describe here relate to the cross-platform support (eg CMake usage), so I’m also including a screen shot of CMake in Qt Creator.  I’m also  including a screen shot of SourceTree for Git

image image image  image

Next Steps

I had a great time and learned a lot building this cross-platform infrastructure from the ground up.  I’m tempted to just focus more on that aspect.  I could make it an open source project, focus on making it more robust and better designed as a generic engine base.  Make it simpler and more streamlined for others to use for their projects.  Take feedback and contributions.  Add features and general-use functionality that isn’t specific to a particular application.  Integrate additional libraries (make some parts optional).  It might become more like a game engine or similar to Marmalade ( ).

Another path I’d love to pursue is writing additional graphics demos (eg beyond the basic shadow map demo seen in an earlier post).

However, the path I’ve decided to focus on next is to make a playable game.  I’ve been playing “Clash of Clans”, so I’ve decided to write a cross-platform tower defense game similar to CoC.  Here’s a random screen shot of CoC to give an idea – it’s an isometric tile-based tower defense strategy game.  I’m making it 3D so the rendering may look more like “Galaxy Control”.

image image

To keep things simple, I’m planning to focus on the rendering, basic UI, and a basic single player demo.  This means things like (art, sound, networking, content, game design details) are lower priority (at least initially).  I suspect if my goal was to release a commercial game as a solo developer, then using a pre-existing game engine (like Unity) (or at least Marmalade) would make more sense.  And going all crazy with the ground-up cross-platform support wouldn’t be necessary.  I also have a busy real job (GPU model development) and this project is just for fun and learning…  So we’ll see how far I get :-).  However far I get, I’ll at least expect to get some more great experience and learning 🙂

Qt OpenGL model viewer (STL)

My previous post contains notes for some reading I did to learn about Qt’s OpenGL support.  The post before that mentions integrating Open Asset Import Library (assimp) into my OpenGL SDL project.  Continuing on both themes, I wrote (from scratch) a simple STL model viewer with Qt (when I wrote this post, I used Qt Creator 3.2.0 with Qt 5.3.1).  Uses modern OpenGL (no legacy fixed function).  Uses Qt’s OpenGL classes layer when available.  Otherwise, it uses standard OpenGL.

Features include:
* load STL models and lays them out in a row with bounding box based spacing
* mouse left rotate model, mouse right translate camera, mouse wheel zoom camera, WASD rotates orbital camera (spherical coordinates)
* basic Phong lighting
* use per-face normals in STL file, or generate per-face normals, or generate per-vertex normals
* uniform scales each model within bounding box range 15x15x15 to 25x25x25
* calculate surface area, calculate volume

image image image

For this project, I originally thought of Qt as mostly an alternative to SDL – it gives an OpenGL context and user input (mouse, keyboard).  However, Qt also focuses on cross-platform GUI support, and on classes that are an alternative to C++ standard library (eg QString, QVector, QHash, QFile).  Qt has helper functions related to 3D math (QVector3D, QMatrix4x4) that integrate well with Qt’s OpenGL classes (QOpenGLWidget, QOpenGLShaderProgram, QOpenGLShader) (for my SDL project, I used GLM).  Finally, I used the cross-platform IDE Qt Creator (for my SDL project, I used Visual Studio for Windows, GCC for Linux, XCode for Mac OS X, XCode for iOS, Android SDK with ADT for Android).  So Qt made a lot of things simpler for this project.  The project’s C++ code is only about 1000 lines and one .ui file (note most of those lines are comments and formatting).  It was faster and easier to develop than the cross-platform SDL project.  The .pro build file (Qt qmake project file) is amazingly small (just a few lines).  I faced less platform-specific nuances, build setup, #ifdef’s, etc (with Qt versus my SDL project).

For volume, I summed the signed volume of a tetrahedron (for each triangle).  See (Efficient Feature Extraction for 2D/3D Objects in Mesh Representation, Cha Zhang and Tsuhan Chen ) ( –> ) ( ) .

For generating smooth normals…  STL specifies a triangle list with per-face normals (or we generate per-face normals).  We create a QHash that maps vertex position to (sum of un-normalized face normals).  The magnitude of cross product equals the area of a parallelogram with the vectors for sides, and that area is just double the area of the triangle.  Then we iterate the QHash (using the list of positions as the keys), and normalize each value (sum of un-normalized face normals), and append that value to the normals list (QVector<QVector3D>).

More on generating smooth normals…  If you look at my smooth normals screenshot, you’ll see that the cube doesn’t look very good – the lighting is strange and the edges look rounded.  Another idea I’ll plan to try later is an angle-weighted average.  Some links I found useful…

This post likes angle-weighted average:

This paper compares methods and seems to like MWA (mean weighted by angle):

This post mentions more ideas (including weight-by-angle):

This excellent tutorial argues that angle weights are better than area weights:

This tutorial video describes using smooth shading for shallow angles:

Here’s another algorithm that uses angle as a threshold:

Generating normals should be a fun topic to learn more about.  It sounds like there might not be one standard simple solution.  So I’ll plan to look for more reading etc with searches like (research smooth normals) and (research generating normals).

Qt OpenGL notes

Notes on using OpenGL with Qt’s OpenGL, focus on Qt 5+ and QOpenGLWidget (not the older QGLWidget).  Qt’s QOpenGL* support offers integration with Qt, and aims to provide helper classes that make cross-platform development with Qt & OpenGL simpler & easier (faster development time).  QOpenGL* classes are based on OpenGL ES 2.0 and designed to be able to use ANGLE.  You can use a mix of QOpenGL* helpers with direct standard raw OpenGL calls.

Qt’s support to give an OpenGL context to display in a GUI with mouse/keyboard input (in this way it is similar to SDL or GLUT).  Qt also has helper classes for OpenGL, and you can still mix this with native gl*() function calls.  Qt OpenGL helpers reduces #ifdef’s for platforms and for OpenGL desktop vs. OpenGL ES, and also gives some convenience functionality (below I list some examples relating to core profiles, extensions, debug/log, timer, shader programs).

The actual notes I put here are not really designed to be useful to anyone, except maybe as a reference to myself to remind me of some stuff I read when first getting exposed to Qt’s integration with OpenGL.  This also just shows an example of some stuff I read, and how I sometimes use notes to help me organize my thoughts.  My brain can only hold so much at once, so I’ll often have lots of notes and to-do lists (paper and electronic).  The PDF notes I took on a plane without WiFi.

Qt with ANGLE (OpenGL ES 2.0) versus desktop OpenGL

Qt 5 uses ANGLE as the default renderer for its OpenGL ES 2.0 API wrapper.  This gives high performance OpenGL compatibility to Windows desktops by translating calls to Direct3D, which has better driver support.  The Direct3D 9.0c backend means it supports (AMD GPU from 2005) and (NVIDIA from 2004) on Windows.  If the host computer runs Windows 7+ (or Vista with update), then even if the host computer has no GPU (or limited GPU), ANGLE will call DirectX 11 which will use WARP.  Qt does not recommend using ANGLE for Windows XP.  Microsoft Windows XP ended mainstream support Apr 2009 and extended support Apr 2014.
* QOpenGLContext makes cross-platform OpenGL context easy (then you can use raw opengl if you want)
* Qt gives GUI and keyboard/mouse input
* helpers: QMatrix4x4, QOpenGLxxx (Context, Buffer, VAO, Texture, FBO, Shader, ShaderProgram, DebugLogger)
* eg QOpenGLVertexArrayObject points desktop to GL_ARB_vertex_array_object, ES to GL_OES_vertex_array_object
* some Qt OpenGL wrappers are lacking, but you can still use raw OpenGL code
* Qt comes with lots of cross-platform libraries besides GUI
* Qt 5 compile with "Desktop GL" or "ES 2"; warning ES 2 is different
* some Windows users don’t upgrade their GPU drivers, so OpenGL ES 2 with ANGLE helps by using Direct3D
* warning modern GL requires desktop OpenGL (ANGLE is currently only for OpenGL ES 2.0) (ES 3.0 in progress)
* Qt Quick has QML scripting language; Qt Quick renders GUI with OpenGL; relevant to mobile UI
* Qt 5’s OpenGL wrappers versus desktop OpenGL
* Qt 5’s OpenGL wrappers are built on top of OpenGL ES 2.0 to facilitate portability
* Using desktop OpenGL with Qt may require more development effort
* QGL* classes are deprecated since QGLWidget can now be replaced by QOpenGLWidget
* Dynamic OpenGL implementation selection on application startup is now available on Windows when configured with -opengl dynamic. This allows using either opengl32.dll or ANGLE’s OpenGL ES 2.0 implementation in Qt and applications without the need for two separate builds of the binaries.

OpenGL in Qt 5.1 introduction (from March 2013)
* Qt 5.0 use QOpenGLWidget (QOpenGL*) instead of older QGLWidget (QGL*)
* see QOpenGL* at
* QOpenGLFunctions – common subset of OpenGL 2 & OpenGL ES 2
* QOpenGLContext::versionFunctions() – example getting OpenGL 4.3 core profile functions, QOpenGLFunctions_4_3_Core
* QOpenGLContext::hasExtension(), extensions() – check for supported extensions
* QOpenGLContext::getProcAddress() – perform manual resolution of entry points (OpenGL extension function) (simpler than GLEW, GLee)
* QtOpenGLExtensions – static lib with OpenGL extensions
* QOpenGLBuffer – per-vertex attribute data & element index buffers
* QOpenGLVertexArrayObject, VAO (Vertex Array Object) – manages a set of VBOs (vertex buffer objects), vertex layouts, IBO (Index Buffer Object)
* comments section (Grumpy OpenGL) questions whether one should use Qt GL wrapper classes, but suggest barriers to using pure OpenGL with Qt
* timer queries help analyze OpenGL (GPU) performance for profiling or adaptive rendering
* OpenGLTimerQuery simple wrapper on GL, QOpenGLTimeMonitor makes timer query easier
* traditional OpenGL debug uses glGetError(), recently GL_KHR_debug extension
* QOpenGLDebugLogger, messageLogged(), errors, info messages, use SIGNAL SLOT, simpler logging
* QOpenGLDebugLogger::SynchronousLogging mode allows you to break to stack with OpenGL function call error
* QOpenGLShader, QOpenGLShaderProgram
* example heightmap terrain
* QGuiApplication + QWindow + QOpenGLContext basically the same as GLUT or SDL
* code example mixes QOpenGL* helper classes with raw opengl gl*() calls such as glClear(), glDrawArrays(), glViewPort(), glEnable(), glClearColor()

Hello GL Example – included in Qt Creator Examples
* example uses QGLWidget (not the newer QOpenGLWidget)
* display OpenGL scene as a QWidget, use SIGNALs/SLOTs, mouse/keyboard input
* slots used for mouse GUI input to control rotation
* initializeGL() init, paintGL() draws, updateGL() triggers draw, resizeGL() updates viewport and projection matrix
* uses some legacy functions like glMatrixMode(), glLoadIdentity(), glOrthof(), glRotatef(), glMaterial() etc
* ex QtLogo class holds vertex data
* ex class Window (QWidget) is a container to hold the QGLWidget plus GUI sliders and keyboard input
* GUI sliders also connect() SIGNALs and SLOTs to control rotation
* QGLWidget is a QWidget, so you can also draw 2D on it with QPainter
* lists some legacy functions to avoid

Introduction to using OpenGL, focus on using OpenGL with Qt
* tutorial introduces (or reviews) basic OpenGL (computer graphics) concepts, and explains simple examples using Qt

* view volume, perspective projection, orthographic projectionm ch 2
* linear algebra: vec4 (x y z w), m4x4, affine transformations (translate scale rotate) (stretch shear reflect) (project)
* mat mul not commutative (order matters)
* coordinate system (frame concept, basis); eg model space, world space, camera space, screen space coordinates
* model-view-proj => normalized device coordinates x y z (-1 to 1)
* vert spec -> VS -> PA -> clip cull -> RS  -> FS -> Per Sample (OM) -> framebuffer/screen
* C++ GL* types, GL_* #define’s, gl*() functions
* GLSL types (vec4, mat4, sampler2D etc) (array, struct, xyzw rgba stpq)
* GLSL storage qualifiers (attribute, unform, in out (formerly varying))
* GLSL built-in variables gl_Position, gl_FragColor, FS can "discard", #version 330 etc
* GLSL main(), functions don’t return values – they use parameter qualifiers (in out inout)
* GLSL no recursion, no pointers, constant array indexes, for loop compile-time iteration count, type cast with constructor
* QGLWidget, implement initializeGL(), resizeGL() (viewport, projection), paintGL() renders scene; QT += opengl, ch 3
* ex class GlWidget : public QGLWidget, Q_OBJECT, QGLShaderProgram, QMatrix4x4, QVector<QVector3D> verts, QGLFormat, ch 3
* depth test, face cull, clear color; ex mixes glEnable(), qglClearColor(), QGLShaderProgram, QGLShader, QVector3D
* uniform mat4 mvpMatrix; in vec4 vertex; void main(void) { gl_Position = mvpMatrix * vertex); }
* uniform vec4 color; out vec4 fragColor; void main(void) { fragColor = color; }
* QMatrxi4x4::perspective() & glViewport() same aspect ratio
* QGLShaderProgram::bind(), setUniformValue(), setAttributeArray(), enableAttributeArray(), disableAttributeArray(), release()
* mouse input: implement QWidget::mousePressEvent(), mouseMoveEvent(), wheelEvent()
* QMatrix4x4::lookAt(), rotate(); ex rotate camera about origin while looking at origin (distance away from origin), ch 3.2
* VS: uniform mat4 mvpMatrix; in vec4 vertex; in vec4 color; out vec4 varyingColor; void main(void) { varyingColor = color; gl_Position = mvpMatrix * vertex; }
* FS: in vec4 varyingColor; out vec4 fragColor; void main(void) { fragColor = varyingColor; }
* C++: shaderProgram.setAttributeArray("color", colors.constData()); shaderProgram.enableAttributeArray("color"); … shaderProgram.disableAttributeArray("color");
* ex uses 12 triangles (3 vertes per triangle) for cube drawn with GL_TRIANGLES, ch 3.3
* textures (0,0) BL to (1,1) TR; glGenTextures(), glDeleteTextures(), glBindTexture() or QGLWidget::bindTexture(), GL_TEXTUREi, glActiveTexture(), ch 3.4
* VS: uniform mat4 mvpMatrix; in vec4 vertex; in vec2 textureCoordinate; out vec2 varyingTextureCoordinate;
  void main(void) { varyingTextureCoordinate = textureCoordinate; gl_Position = mvpMatrix * vertex; }
* FS: uniform sampler2D texture; in vec2 varyingTextureCoordinate; out vec4 fragColor;
  void main(void) { fragColor = texture2D(texture, varyingTextureCoordinate); }
* C++: QVector<QVector2D> textureCoordinates; GLuint texture; … texture = bindTexture(QPixmap(":/texture.png"));
* C++: setUniformValue() glBindTexture() setAttributeArray() enableAttributeArray() glDrawArrays() disableAttributeArray() release()
* ex defines glActiveTexture() manually via (#include <GL/glext.h>), but normally you would use GLEW (#include <GL/glew.h>), ch 3.4
* ex renders textured cube with Phong shading (based on light’s pos & color, object’s texture & material) & the light source, ch 3.5
* seperate shader programs for cube versus spotlight
* phong lighting equation uses ambient diffuse specular for each light & each object’s material (for ambient, diffuse, specular)
* diffuse uses normalized direction vector fragment to light & surface normal
* specular uses direction of light reflected at this point & vector from fragment to camera
* the vectors are calculated for each vertex in VS, then interpolated for FS … page 34
* ch 3.6 buffer object

Adding some 3D models with Open Asset Import Library (assimp)

The build setup for assimp turned out to be a little more effort than I’d expected.  My goal was cross-platform support most likely by building assimp from source, but I was hoping to focus on getting a proof of concept working first using a Windows binary release.  Unfortunately, assimp-3.1.1 released 2014-06-14 was not as easy to setup as the assimp-3.0 releases from 2012-07-10, so I ended up using assimp-3.0 (for first pass).  The assimp-3.1.1 Windows release binaries only worked with older visual studio (not with 2012), and assimp-3.1.1 code release did not include a vcxproj, and running cmake gave me an error.  assimp-3.0 came with a vcxproj file, so I started from that.

I imported this vcxproj into my solution.  To build assimp as a static library, I also needed to download boost.  Using the static library caused link errors relating to C++ standard library, see ( ).  So I built assimp as a dll (instead of a lib).  (“There’s no STL used in the binary DLL/SO interface, so it doesn’t care whether your application uses the same STL settings or not”).  I also added to assimp -> properties -> Configuration -> C/C++ -> Preprocessor -> ASSIMP_BUILD_DLL_EXPORT.  This enabled __decclspec(dllexport)’s needed when building the dll.  Without ASSIMP_BUILD_DLL_EXPORT, I was able to build assimp.dll but there was no corresponding assimp.lib (which I need to statically load, aka implicitly link to assimp.dll).

image image image

Once I got the build working, the real fun began.  I skimmed a few assimp tutorials and some of the official assimp documentation, but I found this one ( ) to be a great starting point.  I used that as a guide along with my existing shadow map objects (class Entity extends HelloWorldBox).  I started with a Cube.obj model file for debugging.  I started by printing (SDL_Log()) the values that I read (using assimp) from Cube.obj file.  From there, it was easy to use those values (position, normal) in the same manner.  Once I got Cube.obj working, I verified an obj model with more verts would work (utah teapot) (wt_teapot.obj).  Then I created a scene with a few different imported models.  The model files I used had normals, but my current shadow mapping example has shadows without lighting (eg no Phong Illumination), so I used the normals as colors.

image image image image image

In modern graphics (API’s and GPU’s), each vertex has a position and 0 to N parameters, and is drawn using (shaders, textures, primitive assembly & other graphics modes).  So a 3d model file format would at least need some way to specify vertex data.  Additional data needed for a model may include textures, mesh hierarchies, animations, deformations.

One simple plan text file format is Wavefront OBJ (.obj), see ( ).  OBJ specifies vertex positions, specific types of attributes (texture UV, normals), and primitives (eg faces).  OBJ only specifies geometry, but you could have separate files for other things like textures and materials (Material Template Library).  More details on OBJ here ( ).

For some applications (certain games), it might make more sense to design a custom file format (or choose one that matches your needs), then write an export plugin (such as for Blender), and an importer (for your application).  OBJ might be a good choice for an educational project, since it’s standard and simple and plain text.  But for a commercial project, you probably want to use a more optimized format.  For some applications (such as tools for 3d modeling), there is a genuine need to load lots of different 3D model file formats – which is why assimp supports a list ( ).

Shadow Mapping

As described in this blog category’s previous posts (cross-platform SDL demos), I did initial development on this project in Nov Dec of 2013.  After the holiday breaks, I was back to being busy with my day job, and doing other learning on the side (such as more general Computer Science algorithms) (one example is algorithms for trees and tries).  Then this weekend, I resurrected my “cross-platform SDL demos” project and implemented shadow mapping.  (Aside – I wrote this on my laptop because in the background I was replacing my desktop computer’s motherboard and reinstalling the OS etc).s

In other blog posts (CG intro Harvard), I’ve mentioned the 2008 Harvard extension school introduction to computer graphics video lectures ( ) ( ).  Lecture 7 gives a great overview of shadow mapping.

Screen shot from 55m 36s shows the idea of doing two draw calls – one from the perspective of the light (draws to the shadow map), and one from the perspective of the camera (draws to the screen).  Screen shot from 54m shows that an object (a position in the vertex shader, or pixel shader) shares object space and world space in both spaces (light space and eye space).  If you want to know, for a particular pixel (fragment) in camera space whether that pixel is in shadow, you need to check the shadow map using that pixel’s position in light space.

vlcsnap-2014-08-24-16h04m12s215 vlcsnap-2014-08-24-16h02m49s33

Another of my favorite learning resources, Essential Mathematics for Games & Interactive Applications – A Programmer’s Guide (by James M. Van Verth, Lars M. Bishop), doesn’t go into shadow mapping. Section 8.12 (second edition page 367) (in the Lighting chapter) has a one page overview that explains how shadows are different than lighting. The book explains – “Shadowing is generally a multipass technique over the entire scene”.  Also – “Since the real core of shadowing methods lie in the structure of the two-pass algorithms rather than in the mathematics of lighting, the details of this shadowing algorithms are beyond the scope of this book”.

I mention this because one interesting thing about my shadow mapping implementation is that I do not implement lighting (ie no Phong illumination, no bump mapping).  Shadow mapping does an extra off-screen draw call (from the perspective of the light) to create the shadow map. Then each of your scene’s on-screen draw calls (from the perspective of the camera) need to check the shadow map in their shader code, to determine whether a given pixel fragment is in shadow.  It’s optional whether you also do other things in your shader (eg texturing, Phong lighting, bump mapping).

To implement shadow mapping in my SDL project, I looked at this tutorial ( ).  One quirk I noticed with the sample code is that it’s “depthModelMatrix” and "ModelMatrix” happen to both be the identity matrix, so it is okay to do (depthMVP = depthProjectionMatrix * depthViewMatrix * depthModelMatrix), then (depthBiasMVP = biasMatrix*depthMVP). However, for my project, I use each object’s own model matrix in the second draw call.

Shadow mapping screen shots with light rays perpendicular to the boxes.  Also shows rendering the shadow map (depth values drawn from the light’s perspective) to the screen:

image image image image

When I moved the light to a different angle, I got horrible shadow acne:

image image image

One way to avoid the shadow acne is to use a depth bias (polygon offset) when rendering the shadow map.
glPolygonOffset(2.0f, 1.0f);

image image

There’s a good article here ( ) to give more ideas on how to improve a shadow map implementation to reduce artifacts related to precision, aliasing, shadow acne, shimmering edges, peter panning.

Simple texture example

I loaded three simple bmp’s and blended them with a GLSL fragment shader:
out_Color = texture(sam0, ex_TexCoord)/3.0 + texture(sam1, ex_TexCoord)/3.0 + texture(sam2, ex_TexCoord)/3.0;

This was just to make sure I had a cross-platform texture example for SDL2 with OpenGL 4 (OpenGL ES 2 or 3) on (Windows, Linux, OS X, iOS, Android).  I loaded pixels from bmp file with this simple code ( ).  SLD2 doesn’t include any image loading functionality; it’s FAQ ( ) references SDL_image.  The uncompressed source code for SDL2_image-2.0.0 is 27.7 MB (31.3 MB on disk).  I may later integrate SDL2_image (or some other image loading library), but for now I am just using bmp’s.  Another option would be to use KTX file format (an official Khronos OpenGL & OpenGL ES texture file format) or DDS; this link ( ) says NVIDIA OpenGL driver supports GPU acceleration for compressed textures with DDS but not for KTX (as of Aug 14 2013).  Or I could even write code that converts from BMP (etc) to a custom format file, and code that loads the texture from that custom format file.

image image

Another relevant texture issue is “texture atlas” and “array texture”.  This is relevant to the example of a thumbnail viewer, because the folder could contain an arbitrary number of image files (png bmp etc).

My AMD Radeon HD 6520G (HP laptop) has a GL_MAX_COMBINED_TEXTURE_IMAGE_UNITS value of 32 (checked on Windows 7).  According to, the Asus Nexus 7 has a value of 16 for GL_MAX_COMBINED_TEXTURE_IMAGE_UNITS.  A naïve approach of using one texture unit per thumbnail would limit you to only one GL_MAX_COMBINED_TEXTURE_IMAGE_UNITS thumbnails (loaded at a time).  To render more thumbnails (than GL_MAX_COMBINED_TEXTURE_IMAGE_UNITS), you would require state changes.

A more sane approach would be to fit all the thumbnails on one texture with a texture atlas, or to use hardware support for array textures.  My AMD Radeon HD 6520G (HP laptop) has a value of 8192 for GL_MAX_ARRAY_TEXTURE_LAYERS.  According to ( ), array textures (EXT_texture_array) was core since OpenGL 3.0.  OpenGL ES 3.0 lists array textures as a new feature.  Unfortunately, OpenGL ES 2.0 does not appear to support array textures, except as an extension – see ( ).

Next »