Copying animations between units

Last week started with a thought – “Copying skinning and animations from unit to unit should be trivial, right?”

Let’s see, but first – some intro:

Disposition

All games need to store their assets (models, textures, sfx, etc.). For most games assets are prepared and stored in a way that their engine requires. With a custom engine, this goes a bit other way round – engine is created around common formats of asset formats that engine’s creators chose. Of course each game (and Knights Province is not an exception) might eventually distill required pieces and rearrange them in a way to speed up data loading and enrich with engine-specific features (i.e. KP has add-on info for house building steps or vertex weights for terrain tiles).

I’m mostly using Lightwave 3D format (LWO) for in-game models. I’m fairly experienced with it and when I need to create or tweak a model – Lightwave 3D is my tool of choice. Currently ~90% of models in the game are in LWO format (to be distilled later on, to improve loading times and save space). Since all models were mostly static in early games life and I knew how to model static-objects, I used LW to model placeholder houses and units.

However, game also needs animations, and that’s something I’m not really experienced nor skilled at. So after creating sample unit animation I went to hire freelancers to do better models and animations. Now I had to deal with their preferences, since asking someone to create both good models AND requiring them to convert it to format unknown to them is a tough/expensive thing. Fortunately, there’s a common ground - FBX is kind of a standard file format when it comes to models and animations for games. Most freelancers use it (also OBJ, but it’s less common).

So I was tasked with extracting models and animations from FBX. Since LW can open FBX files, I decided simply to use LW to convert FBX and call it a day. LW animation format is quite complex (and complicated). To extract animation data needed for the game, I wrote a special Python plugin to export animation frames into a text file in compact and straightforward form. Because of all that, existing process of animation import is rather lengthy:

  1. Obtain FBX animation from freelancer
  2. Convert FBX to LWO
  3. Tweak skeleton setup in LW
  4. Export static skeleton from LW to game format with a Python plugin
  5. Export animation same way

In actuality that’s a tedious 19-step process prone to mistakes. It works okay for 2-3 models, but it’s unbearable for anything larger.

So far game has just a bunch of animations. It would be nice to reuse some, e.g. walking unit animation is the same for all citizen (for warriors it’s a bit different, since they walk with regard to carried weapon).

Idea

It would be great to copy common animations from models who have them to the rest (e.g. idle and walk animations).

To make things right, I decided to write FBX animation importer. It’s not strictly necessary for the task, but once I’m at animations, it’s better to fix several adjoined areas. I already had code to read and parse FBX file format (few in-game houses are actually FBX, not LWO models). Expanding the code with reading of skeletal and animation info should not be that hard .. so I thought.

Last week was dedicated to that task. Things learned and reinforced:

  1. Anything that seems trivial could easily take 20 hours because of unforeseen details, auxiliary things and need to wrap it all into a neat “API”.
  2. Even with unforeseen details, if something seems doable (i.e. without involving words “automatically” or “somehow” in its description), it’s really doable, just needs more time.
  3. The way FBX and LWO stack transformation operations is different (and learning that alone cost me a day).
  4. Automation is great. Writing a small dedicated tool to convert, copy and launch preview instead of manually doing the same – saved my sanity.
  5. When something seems about right, but not quite – it’s almost as good as wrong.
  6. Animation bugs are fun:

Plans

Now with swordsman animations working, I intend to:

  1. Re-import existing unit animations (and throw out LWO animation code)
  2. Write methods for copying skinning between models
  3. Tweak couple stray errors in skinning
  4. Have all units have common walk/idle animation in Alpha 10
Posted in Uncategorized | 2 Comments

Alpha 10 enters wip stage

Alpha 10 is entering “work in progress” stage, where it has its features mostly done. This stage usually runs for several months while I’m polishing features (and adding smaller new ones). Come by and take a stab at it in “Knights Province” Discord channel

Posted in Downloads, News | 8 Comments

Alpha 10 features

New features development on Alpha 10 is nearing its completion (more on that in articles end). Here I would like to list and explain features that went into it:

Particle effects
Smoke, dust, fire and mist effects were added to the game. That involved creation of new rendering engine section. Some details could be found in previous article. This includes:

  • Allowed and placed smoke for house work
  • Allowed and placed smoke for damaged houses (6 of 30)
  • Allowed and placed dust effects for plans placement
  • Allowed and placed dust effect for house destruction
  • Allowed emitters that could be placed on terrain in MapEd (e.g. mist and fog)

SSAO
Covered in detail in previous article. This is a big rendering change and it includes:

  • New deferred rendering pipeline
  • SSAO itself
  • New custom antialiasing (FXAA2 and partially FXAA3)

Revised house-placement
Houses can not be placed around trees from all 4 sides now. This makes much more sense than before, where only lower-right tile below the tree was blocking house-placement. On the same topic, there are two ways of placing houses now, from game, where trees are blocking, and from script (static and dynamic) where trees get auto-magically removed (to keep scripting simpler). To keep town-building manageable, see the next change below

Allowed builders to uproot trees and small stones
Long awaited feature. There’s a new “Uproot” command in build menu. Now you can markup trees and small objects to be removed by builders.

Revised fish-catching
This is first iteration on improving fish-catching. Now every shore tile has certain fish count associated with it, that fisherman can catch (since it’s a shore property, it can not be cheated by building many Fishermans houses, although that would increase initial rate of catch). Fish will replenish slowly over time. Intention is to keep fish a good starting food source, but keep it limited, so you will have to invest into other food types when it’s ran out, but to keep Fishermans useful and whole process realistic – now they will keep catching small amounts of fish forever after.

Revised wind effects to act uniformly on flags, particles and map objects
This is purely visual feature. However it’s nice to have

Expanded undo/redo in MapEd (units, houses, etc.)
Better undo/redo is always a good thing! I’ve restructured undo/redo to handle various MapEd changes and will be able to include yet untrackable things more easily later on.

Unified main menu and gameplay options pages
Now you can change most of the same game setting during gameplay and MapEd as well. Except those, that require render reinit – resolution setting and localization (once it’s enabled)

Separate volume controls for sounds (alerts, effects, menu)
Separation and categories are not final yet, but the idea is there.

Sound for messages opened and closed
Found free sound and added it

Changed campaign UIDs to GUIDs (generated automatically)
This should simplify campaign creation a bit. You might need to update old campaigns if you want to see them in Alpha 10. Stock campaigns are updated by me.

Rigged coalmakers house to place/collect smoking heaps
This is half-baked feature until Coalmakers house model is made. For now – just a groundwork for upcoming feature.

A lot of other smaller features and bugfixes, among which:

  • “Replay has ended” should happen on mission end, not on last command given
  • House plan outline disappears when builder pick up tablet until he starts digging
  • Don’t let AI connect new houses to Towers/Campfires
  • MapEd showing error when trying to save SP map as campaign map (or vice versa) due to “slash” in path
  • Replay info would get lost on loading a savegame
  • Occasional “Integer overflow” error caused by (TKMMain.RequestRender > TKMFPSCounter.FrameEnd > TimeGetUsecSince > TimeGetUsec)
  • Game would rarely crash on trying to assign Train task to Blank who is walking out of a house

Non-game changes include:

Setting up Patreon
Finally set up a way of collecting donations for Knights Province development. More details on Patreon page and in one of previous articles.

Setting up itch.io page
One more place for Knights Province presence.

So .. Alpha 10 is mostly feature-complete. It still needs a lot of content placed (e.g. placing fire-damage smoke emitters for 2 dozen houses). Now it will undergo some weeks of Wip builds (uploaded on Discord channel) and then, with good luck, will roll out officially.

Posted in Live progress, News | 7 Comments

Working on render quality – Part 1 (SSAO)

Goals

Ever since the beginning of Knights Province project I wanted it to be visually pleasing. Slightly cartoonish style with bright colors, semi-realistic soft lighting and shading. Ambient Occlusion (AO) is one of those shading effects that add depth and sense of scale to whole scene, tying objects together and highlighting details.

SSAO composition

What is AO

So what is AO exactly and how does it relate to real world? AO is an effect of ambient light spreading between objects (hence the effects name) and also of some dirt accumulation naturally occurring in crevices. Counter to popular belief, AO is not 100% real-world based. Take a look at walls-ceiling corner – it has no dark gradient on it (even worse, some corners seem to gain brightness near the edge due to light reflecting from other side). Still, AO is a nice artistic effect that adds depth to the picture and gives objects more “weight”, making details to stand out more.

How AO is done

AO can be done in several different ways depending on the use-case. It could be raytraced and recalculated every frame (typical for offline rendering, what 3DSMax and Blender and other rendering apps usually do) when quality is the king. It offers superior quality at a cost of speed (typical scene can take 30+ seconds or minutes to calculate). Games can not afford this. For games, AO could be raytraced once and baked into lighting textures (so-called lightmaps). This offers greater speed. Downsides are – only static objects could be baked. Dynamic objects won’t have proper AO on them and on objects they interact with. Oftentimes older games combined lightmaps with fake shadow patches below actors feet/wheels/etc. That looked okay in 2000’s when computational resources were limited. Now, as the computational powers of GPUs have sufficiently increased, dynamic solutions for games could be made – SSAO, HBAO, HBAO+ and other variants. They don’t come that close to offline rendering solutions, but still offer good enough compromise between performance and looks, with some shortcuts taken.

Options

Here’s a short list of options I’ve considered viable for the Knights Province:

  • Pre-baked lightmaps – look best on static objects, but don’t cope well with dynamic ones, needing lots of special-case “hacks” convoluting the render pipeline. They also require either separate UV map or unique UV mapping (which does not mix well with houses reusing textures and materials, cloned trees and terrain)
  • Dynamically baked lightmaps or vector maps – offer more general approach to the problem, but still require a lot of hassle and have noticeable flaws.
  • SSAO and its variants – is even more general solution. It works on top of any geometry and does not care about how it was combined. What you see is what gets AOed – It’s that simple.

Why did I pick SSAO

I’ve picked SSAO for its generality and simple SSAO implementation specifically for its widespread popularity, which must mean that it’s a good enough compromise between ease of implementation, performance and picture quality. There are many tutorials, common problems and solutions to them are well known too. Optimization techniques are plentiful as well.

How SSAO works in general

SSAO works by basically checking each pixels surroundings within a small sphere, to see if they block ambient light from reaching the surface. The more of those surrounding are covered by other objects, the more occluded the pixel is and the darker it will be. Ideally, each surfaces pixel checks whole multitude of directions and ranges to see how occluded it is from ambient space. In practice, this is very costly operation, so only handful of tests are made and their coarse results get averaged out.

Illustration below shows 2D version of SSAO where 3 surface points are checked with hemispheres oriented towards surface normal (reasonable optimization):

SSAO needs DR

In older times, when render was simpler, GPUs were rendering frames in one go and would calculate each pixels color by factoring everything at once. Now SSAO samples a lot of pixels over and over (even for coarse results, every pixel needs to test 16+ surrounding pixels to figure out its shading). Because of that, it makes sense to prerender required information and access it from some buffer. Providing for that is a complex rework of the rendering pipeline. Approach with rendering intermediate buffers is called Deferred Render (DR). Where as before the game drew objects one by one in one go, now all objects need to be drawn into temporary buffers (typically that’s position, normal to surface, color) to later combine them into final picture. DR is already sort of in the game – shadowmaps, water reflections and fog of war are prepared in separate buffers and combined into final rendering.

DR plan

Render pipeline needed some planning in order to find correct place and sequence for each operation. Here’s how render pipeline is planned to be with DR and SSAO:

Current render structure of the game

SSAO optimizations

SSAO is very resource hungry algorithm – it needs to test dozens of pixels for every drawn pixel. Optimizations are vital. Most straightforward ones are – sampling only those points that are above tested surfaces (testing within hemi-sphere instead of whole sphere of directions). Then comes in randomizing of samples – it is done by rotating sampling hemisphere for each next pixel – thus output is more evenly distributed, but noisy. Blurring helps with reducing noise. SSAO has a number of controls, allowing to trade-off quality vs. performance – e.g. reducing number of test-samples used, reducing blur radius. At the moment I plan to add 3 SSAO setting – off, low and high quality.

DR and AA

Unfortunately DR does not mix well with traditional AA techniques, since DR relies on pixel perfect information in buffers, only straightforward AA could work (that is rendering frame X times larger and downscaling it), which is unfortunate for SSAO where each new pixel needs whole lot of sampling from neighbour area. Luckily there are known AA algorithms well suited for DR. FXAA is quite popular and easy to implement.

DR and transparency

DR has certain difficulty coping with semi-transparent surfaces (STS), since depth buffer can only contain a single value per pixel, it is either of surface behind the STS or the STS. Which means that either the surface behind or the STS one gets shaded wrongly. There are smart techniques to deal with that (e.g. using a checkerboard pattern and storing every 2nd pixel from other surface), but for the RTS that has quite few STSs (namely smoke particles and water surface) it is easier to deal with the issue by simply rendering STS on top of DR/AO geometry. STS generally don’t need AA too, since their transparency means they don’t create a lot of sharp pixels in the first place. Even better – STSs rendered on top of normal geometry cover some of it, thus reducing amount of sharp edges that AA will need to smooth out.

Recombining the render

Once AO is computed and applied, the render needs to return back to “normal” functioning to render STSs. This is done by writing RGB and depth values back into a buffer and rendering STSs using default depth test.

(Particle textures are replaced with checkerboards for debug)

In the following articles (which are already in works) I’ll cover FXAA and general render perspectives.

P.S. Thanks to Thibmo for reviewing this article 🙂

Posted in How things work, Live progress | 7 Comments

Knights Province on Patreon

I’ve finally set up Patreon page for Knights Province!

Patreon is a site that allows to collect monthly donations to support and sponsor projects and people. I’ve been asked a lot about setting up donations for the Knights Province. Finally I did it. Last month I did a small test and money are really getting processed and send to my PayPal account. This means I am ready to announce it.

Since there’s no way of estimating how good this Patreon thing will work, my current Patreon plan is to just start collecting donations and seeing how it goes. I’m planning to add donations to funds I pay from my pocket for freelancers work on assets creation for the Knights Province (2D/3D artists, musicians, animators, etc). It would also be nice to buy a glass of beer every now and then, knowing it was gifted to me by Knights Province fans and players 😉

Patreon page link

Regardless of donations or not, I’m still very grateful for your support and interest in my humble hobby project. Thank you!

Posted in News | Leave a comment

Where does smoke go?

One of the features I’m working on for Alpha 10 is particle effects. Those include smoke, dust, fire, mist and others.

Here are some early particle tests from dev tools:

First particles

Smoke affected by wind

Rigging smoke to tavern

Faced difficulties included writing a shader for billboard particles (so they always face the camera) and keeping spawn rate stable. Rigging particles to the game was also a bit complicated.

So now game will include small dust puffs on roadplans and house plans placement as well as lite smoke from tavern when anyone is eating in it.

Next – some damaged houses fires!

Posted in Live progress, Tools | 6 Comments

Working on Caster unit navigation

Been working on unit navigation for Alpha 10. It is still very hard and complicated, but progress makes me happy. Made some terrain collision outlines generation code tonight:

Posted in How things work, Screenshots | Leave a comment

Alpha 9.1

Alpha 9.1 (r5956)
Thanks to everyone sending in crashreports – includes following bugfixes:

+ Fixed non-warrior under attack from a house would cause a crash
+ Fixed removal of decals in MapEd
+ Fixed crash on towers hitting non-warrior units
+ Attempt to fix crash on Wine trying to init OpenGL 3.2 function
+ Allowed to change depth buffer bitdepth and shadowmaps bitdepth
+ Added attempt to recover if unsupported GFX settings were chosen
+ Selecting an animal in Replay would cause a crash
+ Fixed crash on setting group attack orders outside of map bounds in MapEd
+ Fixed group attack position offset in MapEd
+ Fixed render of rubble on house demolish
+ Fixed crash on loading savegame where unit training was cancelled

Download links:

Knights Province Alpha 9.1 (7z package)
Knights Province Alpha 9.1 (installer)

Posted in Downloads | 4 Comments

Alpha 9 is ready!

I’m happy to announce new Alpha of the game!

Alpha 9 changes and improvements:

Three major gameplay mechanics (described in detail in earlier article):

  • Wildlife (aggressive animals and dens)
  • Early army (militia can be trained in Camp)
  • Morale to affect warrior’s attack

Content

  • More great music by Juan I. Goncebat
  • New fight and death sounds
  • New scripted map Walkers (survive Walkers waves)
  • New scripted map Into the Swamps (by Klassix)
  • New skirmish map Horseshoe (with aggressive wolves)
  • New skirmish map Borders (by Klassix)
  • New campaign Orthalm (3 missions, by DarkianMaker)
  • New terrains (yellow grass on dirt, coarse gravel)
  • Diagonal transition for mountains/flat terrain
  • New temp house models (by Icomatix)
  • House building animation for 2 houses (with help from Dah)
  • New palm trees

Improvements

  • Simplified gold production (no coal required)
  • Increased unit walking speed by 25%
  • Reworked AI setup and presets for skirmish
  • Changed defence position setup in MapEd
  • Tweaked AI to handle food production better
  • Tweaked AI to handle offense better
  • Tweaked AI to handle defense better
  • Increased max stone volume up to 6 in one tile
  • Gold and Iron ore can be placed on any mountain type now
  • Fixed AI iron mining
  • Improved iron mining rate
  • Lots of smaller improvements in MapEd
  • Lots of gameplay bugfixes and smaller improvements

As usual – this is still Alpha, there are likely a lot of bugs, flaws, incomplete features, broken mechanics and such. Please report them and also your thoughts on what should change and why.

Download links (updated to latest version):

Knights Province Alpha 9.1 (7z package)
Knights Province Alpha 9.1 (installer)

Posted in Downloads, News | 24 Comments

Palms

Did some low-poly palms for the upcoming Alpha 9:

Posted in Screenshots, Sidenotes | 1 Comment