3DMark 2005

Written by - on 09 November 2009. Posted in DirectX 9.0b-9.0с Shaders ver 2.0b

Released: 2006
Developer: Futuremark Corporation
Platform: PC
Minimum system requirements:
Operating system: Microsoft Windows 2000 or XP operating system
Processor: x86 compatible processor with MMX support, 2000MHz
RAM: (512MB recommended)
DIRECT X: DirectX9.0c or later (required)

DirectX9 and Shader Models

With the evolution of GPU functionality, developers need to take advantage of their advanced capabilities both to improve scene quality and to gain additional performance.
Microsoft, the creator of DirectX, has been remarkably flexible in allowing two leading GPU developers to spawn a whole set of shader models that take advantage of advanced GPU functionality beyond the basic requirements of Shader Model 2.0. The industry needed a standardized approach to extending the core capabilities of DirectX 9, and the result was Shader Model 2.0a, Shader Model 2.0b, and Shader Model 3.0.

For example, in Far Cry, when calculating illumination using model 3.0 pixel shaders, a maximum of 4 light sources are calculated in one pass, when using model 2.0b - for 3, when using model 2.0 - for one light source.

So, with the transition of games to a new development method, a new approach to performance evaluation has appeared: instead of testing video cards in absolutely identical conditions, it is necessary to use for each specific accelerator the shader model that most fully uses its capabilities. This is how the developers at Futuremark saw 3Mark05.

3DMark05: Stages of the Path

Futuremark, calling its benchmarks "The Gamer's Benchmark", is constantly adding support for new technologies and developing the functionality of its 3DMarks. Test packages from Futuremark, which first appeared in late 1998, over time have become the main tool for measuring the performance of video cards and evaluating the balance of power for a wide variety of people, from ordinary enthusiasts to managers of the largest companies.

3DMark99 - focused on the speed of texturing and polygon processing.
3DMark2000 - received support for hardware transformation of polygons, along with this, the complexity of scenes increased.
3DMark2001- received support for vertex and pixel shaders 1.1 and additionally increased scene complexity - now tens of thousands of polygons were present in scenes.
3DMark03 - used shaders 1.x and 2.0. Only one of the gaming tests did not use pixel shaders at all. All other gaming tests used DirectX8 pixel and vertex shaders extensively, and the last, most difficult, gaming test heavily used Model 2.0 shaders. The complexity of the scenes has increased to hundreds of thousands of polygons.
3DMark05- raised the level of technology even higher. The package uses only model 2.0 and higher shaders, and all shaders can be executed in any of the profiles corresponding to models 2.0, 2.0a, 2.0b and 3.0. The complexity of the scenes has increased: now, on average, there can be more than one million polygons in the frame.

The main difference between 3DMark05 and previous versions of the test package is the use of shaders of at least model 2.0 and the choice of the optimal rendering path for each of the video cards.
Patric Ojala of Futuremark gives this example: “We use several custom shaders, for example, in the first play test, the shader corresponding to model 3.0 uses dynamic execution control and stops when it detects the fact that the surface is not lit. Another example is a shader using Depth Stencil Textures.”

Graphics engine: using shaders

Futuremark's previous 3DMark incarnations used modified versions of MAX-FX, but for 3DMark05 the company has developed a new graphics engine. All shaders used in scenes are written in high-level language - HLSL. These shaders are not executed directly, before being sent to the accelerator they need to be compiled, i.e., translated into a language more understandable to the GPU and its driver. DirectX offers several profiles - several optimal configurations for the shader compiler, designed for GPUs with different functionality. So, say, for ATI RADEON 9700 PRO profile PS 2_0/VS 2_0 will be used, and for NVIDIA GeForce 6800 Ultra - PS 3_0/VS 3_0. The latest generation of GPUs exceed the basic requirements of DirectX 9.0, and despite the fact that they support lower profiles, say, PS 2_0/ VS 2_0, they will default to the profile that makes the most use of their functionality. The same applies to graphics processors that will appear after the release of 3DMark05 - for them, based on the list of features provided by the drivers, the profiles that use their functionality to the fullest will be selected.

So, in the new reincarnation of the test package, Futuremark moves even further away from the idea of ​​comparing video cards in absolutely identical conditions. This is not surprising: all modern video cards support the basic requirements of DirectX 9.0, but above the basic requirements, all have completely different functionality. Putting them in the same conditions is incorrect: these identical conditions in different cases will be optimal for some video cards and suboptimal for others. Instead of all this, 3DMark05 makes the most of the capabilities of each video card by choosing the most functional profiles for each GPU.
Nevertheless, for those who are still interested in comparing the work of video cards in the same conditions, the ability to select a profile for compiling HLSL shaders has been introduced. In this way, you can force the video card to work with a less functional profile, say, NVIDIA GeForce 6800 Ultra will not use shaders 3.0, but the scene rendering speed will, of course, change.

Graphics Engine: CPU Usage

In gaming tests, Futuremark's new package doesn't use CPU resources for anything other than preparing data for scene building. That is, there are no calculations related to game physics, logic or AI - “artificial intelligence” - in game tests.
Most of the built-in tests in normal games are organized in the same way: for the duration of playing a demo recording and measuring speed, AI, physics, etc. are calculated. turns off. For example, in Doom3, the built-in test is organized in this way.
So, in terms of the use of CPU resources in gaming tests, Futuremark developers tried to get closer to real gaming tests, and this approach seems to be quite justified, because 3DMark is, first of all, a test of video cards, not CPUs.

Graphics Engine: Shadow Calculation System

Dynamic shadows appeared in 3DMark 2001 scenes - the engine used projected shadow maps. In 3DMark03, in the second and third tests, the graphics engine switched to a different way of rendering shadows, the same as that used by the "great and terrible" Doom3 - calculating the volumes that bound the shadowed areas, and using the template buffer to determine the illumination of objects.
In 3DMark05, the developers moved away from this method of calculating shadows - while providing excellent quality, it nevertheless has a number of drawbacks. For each object that should cast a shadow, you need to create its “shadow volume” - a polygonal model, the faces of which from the side of the light source are the edges of the object itself, and from the sides - the silhouette of the object extended from the light source to infinity. Finding the faces that form the silhouette of an object and are subject to extrusion is a difficult task performed by the CPU, and the more complex the object, that is, the more polygons are included in the calculation, the more time it takes to create the “shadow volume”.
Further use of these invisible "shadow volumes" is associated with the need to draw them into the template buffer, which significantly increases the load on the GPU in terms of rendering speed. And the more objects cast shadows, the greater the load on the GPU.

The shadow calculation method used in 3DMark05 is free from these shortcomings. 3DMark05 uses a type of shadow maps called "perspective shadow maps", PSM, to calculate dynamic shadows, with its own modifications to minimize the manifestation of their characteristic flaws.
When calculating dynamic shadows using a shadow map, the scene building steps are something like this:

-First, the scene is built from the position of the light source. When building, textures, pixel shaders, etc. are not used, since all that is needed at this stage is the Z value, that is, the distance of the scene pixels from the light source. This value for each pixel is written to the output buffer in floating point format. The higher the size of this buffer and the more accurate the data representation format, the better the result will be.
-After calculating the shadow map, the scene is built in the usual way, from the position of the camera. To determine the illumination of pixels, pixel shaders are used: in the shader, for each pixel of the scene, the corresponding pixel of the shadow map is determined and the distance from the scene pixel to the light source is calculated. If this distance is equal to or less than the value stored in the shadow map, then the pixel is illuminated. If this distance is larger, then it is obvious that some of the elements of the scene turned out to be closer to the light source when calculating the shadow map, and the pixel turned out to be shaded.

The main advantage of calculating shadows using PSM is that to calculate dynamic shadows using shadow maps, no additional calculations are required by the CPU, and the number of calculations does not depend on the complexity of the scene - invisible, but resource-consuming "shadow volumes" are not added .
Modern pixel processors support long and complex shaders, which allows one pass to determine the shading of each pixel in relation to several light sources at once, i.e., further reduce the amount of work.
Plus, this method uses pixel processors, and it is their performance that has been growing at the fastest pace lately - faster than the power of vertex processors, CPU performance, memory bus speed, or texture sampling speed.

This method, of course, has its own weaknesses, but the developers at Futuremark assure that their PSM modification is well suited for a wide variety of scenes and light sources.
Shadow maps for directional light sources are calculated in a resolution of 2048x2048, shadow maps are saved in floating point format, R32F or D24X8. For these lights, the graphics engine calculates two shadow maps, one for objects closer to the camera and one for the rest of the scene. Thus, an increased accuracy of shadow calculation is achieved in those places where it is most noticeable, i.e., close to the camera, and the ability to calculate shadows for the rest of the scene is preserved. However, even this is sometimes not enough to completely avoid the appearance of artifacts - in the third 3DMark05 game test, shading artifacts are noticeable on rock areas located almost parallel to the sun's rays.

Pay attention to the shadow cast by the cables near the "fin" of the flying ship, and to the fragment of the canyon wall.

The developers note that these are not driver errors or hardware problems, this is a manifestation of one of the weaknesses of the shadow mapping method.
For non-directional light sources, the 3DMark05 graphics engine builds six shadow maps in R32F format 512x512 in size, placing the light source in the center of an imaginary cube, building shadow maps for 6 faces of this cube, and thus reducing this case to the case of a directional light source.

The selection of values ​​from the shadow map in the pixel shader in the presence of hardware support for Percentage Closest Filtering (PCF) and Depth Stencil Textures (DST) is performed using PCF, that is, in fact, with the usual bilinear filtering, and if there is no hardware support for PCF, filtering is performed directly in the shader, for this, 4 values ​​closest to the reference point are selected and averaged from the shadow map.
These approaches provide slightly different results, both in terms of performance and image quality, but the developers at Futuremark are confident that DST and PCF, that is, hardware support and hardware filtering of shadow maps, should be used whenever possible, since the most Major game developers are already using these GPU features, and the demand for these features will only increase in the future.

So, enough details. Finally, let's move on to the description of the tests.

Game Test 1: Return to Proxycon :

The first gaming test definitely belongs to the action games section: space pirates attack the Proxycon cargo ship again.

Reflecting the scenes of classic shooters, Game Test 1: Return to Proxycon combines fairly large rooms with narrow corridors, and a large number of foot soldiers fighting at the same time brings the game situation closer to multiplayer games.

Most of the surfaces in Game Test 1: Return to Proxycon use shader-defined "metallic" materials with Blinn-Fong lighting calculations. The exponents required for the calculation are not mathematically calculated in the shaders; instead, samples from a pre-calculated table are used.
In total, the scene has 8 light sources that cast shadows: 2 directional light sources, for which shadow maps of 2048x2048 are calculated, and six non-directional sources, for which shadow maps of 512x512x6 are calculated.

Game Test 2: Firefly Forest :

This test is a good example of an open space scene using a lot of vegetation. The scene is relatively small, but full of details to the limit.
Moonlight night. The ground is covered with dense grass, the branches of trees are slightly noticeably swaying in a light breeze ...

The display of dense vegetation on the ground is implemented in a dynamic way: the concentration and level of detail of the ground vegetation changes with the movement of the camera. Grass leaves are displayed only where they are needed, and this allows you to reduce the load on the GPU, while maintaining the visual experience at the highest level. 

The ground surface in this test is rendered using the "metal" shader from the first test, but with the addition of basic and detailed color/normal maps. The tree branches material does not use bump and specular maps, but has a color cube map. The sky is rendered using a shader that simulates light scattering.
Moonlight is a directional light source that casts dynamic shadows. Shadows are calculated using a shadow map with a resolution of 2048x2048. The magical firefly illuminates the grass and trees as an omnidirectional light source with a 512x512x6 cube shadow map.

Game Test 3: Canyon Flight :

The last game test stands out for its large open spaces - in this scene, a Julvernian flying ship sails over the waves through a canyon guarded by a real sea devil.

The water surface is the most remarkable part of this scene. Water not only imitates reflections and refractions, but also has its own transparency value, so that a sea monster moving in the water column looks like it is actually floating in the water column, and not behind cloudy refractive glass.
The shader used to render the water surface is an advanced modification of the "water" shader from 3DMark03, but the water surface is not only a shader. Correct calculation of refractions and reflections, including the correct display of shadows, requires six passes of the graphics accelerator. The shader itself uses readings from normal maps, refraction and reflection. In addition, volumetric fog is used for underwater objects, making them darker and less saturated as they move deeper from the surface of the water.

To enhance the effect of presence in a large open space, fog is used in the scene - thanks to its use, distant rocks look more natural.

The shader used to draw the rocks is called by the developers the most complex shader in 3DMark05 - when combined with the calculation of shadows, it barely fits into the 2.0 model pixel shader specifications. The rock material uses two base textures, two normal maps and a lighting calculation based on the Lambert model.
The scene has one light source - the Sun. Sun shadows are calculated using two shadow maps with a resolution of 2048x2048, one map is used for objects close to the camera, and the second one is used for the rest of the scene.

 CPU Test :

3DMark05, like 3DMark03, uses gaming tests at 640x480 resolution with post-effects disabled and software emulation of vertex shaders to test the CPU speed. This shifts the balance of tests towards increasing the load on the central processor and makes the test results dependent on the speed of the central processor, and not the video card. To ensure that tests run under exactly the same conditions on either system, both CPU Tests use a fixed frame per second scene output mode.

In the first CPU Test, the developers introduced additional calculations assigned to the central processor. Despite the fact that the passage of the ship through the canyon in any conditions is carried out along the same trajectory, this test includes a continuous calculation of the optimal trajectory that envelops the contours of the canyon. The calculations associated with this calculation are performed in a side thread, and this allows you to use the capabilities of multiprocessor systems or processors with HyperThreading.

Fill Rate :

This test has passed into 3DMark05 almost unchanged. Everything that has changed is visible to the naked eye: in order to reduce the requirements for memory bandwidth and highlight the speed of texturing, the developers have reduced the resolution of the used textures to the maximum - now they are boring “cells”.

The test, as usual, has two modes: single texture overlay and multitexturing. In the single texture blending mode, the scene has 64 surfaces with one texture layer on each, and in the multitexturing mode - 8 surfaces with eight textures on each.

Pixel Shader :

This test uses the most sophisticated of 3DMark05's shaders, the rock surface shader in the third gaming test. The shader has been moved from the game test with only one change - shadows are not calculated here.

It is worth recalling that the shader is written in HLSL, the basic requirements for the GPU, like all other 3DMark05 tests, are support for model 2.0 shaders.

The developers note that the results of this test will be determined not only by the speed of pixel processors, but also by the speed of the memory bus - this shader makes heavy use of large volume textures.
As a less memory-dependent alternative to such a shader, developers see a shader that uses mathematical calculations, that is, “creates textures on the fly”, but, firstly, a similar shader has already been used in 3DMark03, and secondly, as Futuremark notes, game developers instead of mathematical calculations in shaders, they are much more willing to use ordinary textures.

Vertex Shader:

The test consists of two parts: in the first part, the speed of a simple transformation of sea monster models is measured - the shader responsible for the transformation may well meet the 1.0 model shader specifications, but the test, following the Futuremark ideology, uses DirectX 9.0.

The second, more complex version of the test uses a complex vertex shader to transform blades of grass. Each blade of grass bends independently of the others under the influence of the "wind" set by the fractal noise calculated on the central processor. In order to reduce the influence of factors such as CPU performance and fill rate on the test results, the calculation of fractal noise is optimized as much as possible, and the blades of grass are moved away from the camera.

 Batch Size Tests :

 Batch Size Tests is a set of tests designed to speed up the rendering of batches of different sizes - groups of polygons sent by the application to the accelerator driver in a single Direct3D function call. In each of these tests, the same number of polygons is drawn, but the polygons are combined into groups of different sizes each time: 8,32,128, 512, 2048 and 32768 polygons.
To ensure that video card drivers do not merge small groups into larger ones for optimization purposes, each initial group of polygons is drawn with its own color - this causes the graphics pipeline to restart when each new group of polygons arrives:

The test reveals how much video card performance is reduced by reloading the graphics pipeline and shows the efficiency of the driver and video card on groups of different sizes - it is known that sending to the accelerator and rendering the same number of polygons with one function call and one group is faster than many small groups.

Each of 3DMark's reincarnations has brought modern graphics cards to their knees with new technologies and more complex scenes, but never before has the "parrot denomination" been accompanied by such a huge jump in image quality. In order to get the maximum impression and appreciate the new tests, of course, you need to watch the demo mode - each game scene in the demo mode is a finished work of art.

It is noteworthy that this time Futuremark clearly divides gaming scenes by genre, and this is noticeable at first glance at the tests - a repetition of the situation with 3DMark03, where the second and third gaming tests behind a different shell were the same inside, did not happen. All game scenes are seriously different, and, at first glance, they should reflect scenes from games in the near and distant future well.
It is also worth noting Futuremark's new approach to testing video cards with different functionality - the use of HLSL shaders and its own optimal profile for each of the GPUs is perhaps the most adequate reflection of the current trends in the gaming industry.