Edit: I am soon going to a writeup on the rendering. I wrote this post in haste so have not gotten a chance to write much yet but in as few words as possible the new rendering is much faster because it only calculates visible surfaces instead of volumes.  It is based on cell marching and distance field marching.  More info to come soon!

Since the last update:

• Implemented an entirely new ray-based rendering method which includes such benefits as...
• No more chunk loading.  All scene data appears instantly
• No more refreshing of chunks. Changes can occur in realtime, at 60+ FPS.
• Vastly Increased performance and draw distance.
• "Infinite detail" (limited only by floating point precision and algorithmic complexity).
• Ability to dynamically animate any surface or object.
• Successfully integrated with much of the old pipeline which optionally includes various screen-space effects, material rendering, and so forth.
• I also implemented complex fluid dynamics, which include such features as...
• Constant fluid volume (impossible to destroy matter (unless intended)).  Fluids maintain correct proximity and do not lose or gain water.
• Water pressure.  Fluids will balance out (i.e. fill a U-bend evenly) and simulate correct pressure and distribution.
• Free of most artifacts but still needs work.  Particularly notable is that the water does not jitter around as it does in many constant fluid simulations.

I have been fairly busy with "real life" issues, which seems to be the norm these days, but I have managed to get quite a bit done regardless.

Below is one of the earlier tests I put together - rendering terrain based on VQ's generated map.

Added in bilinear filtering to smooth out the water (sorry for all the twitter embeds, I am lazy):

And one more demo of the smooth water:

Since the last update:

• Explosions (temporary effect, needs improvement); Sound effects are also all placeholders by the way
• More intelligent chunk refreshing that reduces redundant chunk refreshes (useful for destruction among other things).
• Rendering spheres of arbitrary size in screen space with real depth, normals, clipping, etc
• Projectiles
• Camera Shake (on jump, on objects hitting ground, fades with distance from camera); it is exaggerated right now
• Zooming in and out to first person mode
• Walking around with collision, realtime smooth movement that can revert to unit space for turn-based combat and so forth.
• Physics (gravity, dampening, rigid body physics with adjustable bounciness/hardness, friction, etc).  Currently operates on axis-aligned boxes (even when boxes are rotated on the Z axis)
• All movement-related effects are now timer-based (rather than frame-based), although you can also force a certain time interval to prevent stutter/frame skipping.
• Jumping
• Lantern placement fixed (raised up inside of houses, no more lantern clusters for more efficient lighting)
• Volumetric textures optimized for lower voxel resolutions (larger shingles, bricks, and boards)
• Better character selection/deselection with dotted line selection effect; now distinguishes between the active entity and the target entity
• Better sprite rendering with no clipping or depth sorting errors
• Labeled all 1600+ items and entities (even though they are just placeholder objects)
• Added context sensitive options to the drop down menu based on which targets (if any) are selected
• Quick experiment with trees: upped the number of limbs to produce dramatically larger trees (as seen below)
• Improved, less naive chunk picking (for rendering) - now uses a flood fill to pick chunks, instead of a bounding cube.
• Many other small improvements I am probably forgetting to list :)

I am still working on many aspects of the AI but only a day or two per week right now - I am trying to not get stuck in any one area as there are many tasks left to cover.  I think this most recent set of changes opens up a lot more modding potential (in particular, for realtime games).  I think it would be pretty easy to make a platformer (like Mario N64), an artillery game (like Scorched Earth or Worms), or many other options (at least, easier than it would have been with strictly unit/turn-based movement).

I came in with the intention of doing videos weekly but as it turns out monthly makes more sense - there are some weeks where not a lot happens visually.  It also saves me time.  Nonetheless I advise you to follow the @voxelquest (or @gavanw) account on Twitter or my Youtube channel for smaller updates in between.

Also, I posted this a bit late for April Fool's, but it is worth reposting I think.  Since I added in first person view, I obviously need the obligatory hand swinging some weapon.  (In this case it is one of the weapons from Hexen):

In spite of this being a joke, I really do dig this style (as did Twitter, apparently) and I would like to perhaps have something similar ultimately.

Hello! It has been over a month since the last update.  I apologize for the delay, but as I have mentioned elsewhere there really was not much to look at as the majority of my work was doing IO in a text-based console.  I have been hard at work on the AI, and now have the logical portion of it mostly complete.  It is capable of making deductions as I have described prior in the forums and elsewhere.  This is a full "scripting" language with compiler, although for the time being it is parsed using rules from JSON which makes it somewhat unwieldy but I just wanted to get a parser working as fast as possible.  It contains many features, and is both a subset and extension of the language Prolog, albeit with different syntax and grammar.

I use quotes on "scripting" language because it is not writing AI scripts like you might see in other games.  Rather it is just specifying a knowledge base with various rules and relations.  You can then pose queries to this knowledge base in order to make deductions and determine whether or not things are true or false.  Additionally you can do more advanced proofs and solve statements for a given set of variables, finding the variables which result in a true set of facts.  If you are interested in learning more about this, I recommend learning Prolog - there is a good free book I used here for reference.

So why not just use an existing solution like Prolog? Well, first of all Prolog is relatively bloated.  My language fits within under 1000 lines of code if you ignore the JSON parsing (less minus comments/whitespace).  It is compiled to integer symbols so it avoids string manipulation and comparison.  I only need a subset of Prolog's functionality, so it would be wasteful to include the entire thing.  Moreover, Prolog is not sufficient on its own.  I must carefully modify the deduction algorithms to support things like score maximization, and thus I must have a full understanding of the code involved.  This would be much harder if I went in and tried to modify an existing Prolog implementation, and more error-prone due to the larger breadth Prolog has over my subset language.

The ultimate goal of this is to build an AI that can make logical deductions and perform surprising, emergent behavior based on things that are not explicitly scripted.  The term I have been throwing around, as have some other games, is a "Game of Thrones" simulator - something capable of weaving together a complex plot in which every NPC has their own set of motivations.

I also added in some billboarding to show sprites in the placeholder boxes, which makes it much easier to figure out what a given object is.  Also reorganized all the items and added in placeholder NPCs and monsters.  Art is by 7soul and available for purchase in most game dev asset stores.

Also, as a quick (dumb) experiment, I messed with voxel warping.  Just 2 lines of code, could have some interesting uses for spell effects and such.

See full update here.

Happy New Year!

It has been a while since the last update, I know. As I may have mentioned prior, aside from the holidays, I have been just trying to get something interesting to show, which is hard when the majority of my work the past few weeks has been on relatively boring back-end stuff.

Here is a video - sorry I was tired when I made it!

Since the last video update, we have:

- Walking around, with collision and auto-climbing/descent of stairs/hills (the power of technology!!!)
- Camera that follows the character, orients itself in your travel direction, and more
- Saving and loading of character poses
- Rendered (fully voxel-based / procedural) character has replaced the placeholder box object.
- Ability to assign materials to each character limb, or other objects
- Material editor has been migrated from the web client. For those who have not seen it, the web client looked like this. The web client had many problems, including performance (slowing down the main client), inability to effectively manage data over a websocket, requiring a web server to run (i.e. a localhost), being generally unwieldy, and more.
- Much more advanced GUI features, data binding, and data serialization. The GUI now supports multiple features to keep everything DRY (Don't Repeat Yourself), keep data and presentation separate, and more. This is important because many times it is not a 1:1 mapping of data to UI - for example the color picker has 6+ components which map to only 3 data values.
- Several more optimizations, in particular some UI optimizations that make it load faster and make it more responsive.
- Many more small fixes and improvements.

In the coming week I am going to work on inventory management, picking up items and dropping them, and rendering additional NPCs and making them walk around.

As usual feel free to ask me anything.

This week I did a pretty huge cleanup on the code (I deleted thousands of lines while retaining the same functionality and more). I also added in collision detection, a context menu (right click menu), and some functions for spawning/removing/editing entities - mostly for the purposes of quickly testing stuff but also useful for the eventual sandbox.

Making the collision work was not exactly trivial but it is a lot easier now that voxels are passed to the CPU and meshed. It looks at all the voxels within a given cell and determines (based on a given threshold) whether or not that cell is mostly solid, air, water, etc. More granular collisions can be done in screenspace, but this is necessary for offscreen or hidden collision testing. This coming week I will be working on camera movement (following an NPC or trackballing around it), in addition to registering collision and behaving accordingly (i.e. climbing stairs automatically if you collide with a step, etc). I will also try to work on managing movement and collision when other NPCs are present.

Also, if you enjoyed the classic Master of Magic, you might want to check out my friend Aaron's game Worlds of Magic. He is running for Indie Game of the Year (in the top 100 now), so throw him a vote if you are feeling generous:

http://www.indiedb.com/games/worlds-of-magic

Also, David Sahlin, who I have known for almost a decade (since I worked on Genesis) sent me his ideas for a mod for VQ, drawing inspiration from the likes of Pokemon, Spore, etc - I think it is a pretty interesting idea and I would love to see someone make a mod like this:

https://prezi.com/8dzstjlitnoi/vq-mod-mythical-creatures/

It has taken some time to get to updating this as I mostly update in the forums and on the KS page, but I am happy to let everyone know that Voxel Quest has been successfully funded!  Also, in the last week of the KS campaign, I added in a perspective camera - this is an early video of it but I am demoing a much improved version this coming Friday.

Click here for the Kickstarter campaign. (wink wink)

Click here for the full gallery.

Hi everyone.
Phew!
It has been about 6 months since my last major update.  In that time I got married and had my first child (it's a girl!).  But aside from real life I've been neck-deep in voxels.


So what has changed? A lot - a huge amount.  There are plenty of visual changes but also tons of things under the hood - not so much (premature) optimization, but really a restructuring of a good portion of the engine to make it easier to work with.


One small, but major change is the support for voxel super-sampling.  Everything you are viewing has been rendered to bitmaps that are 1/4 to 1/8th as large as the bitmaps used for chunks in the previous video.  This reduction in buffers for the chunks has led to a massive decrease in the memory needed to store chunks - since chunks are stored volumetrically, every reduction of 2 decreases memory by a factor of about 8 (2*2*2).  So a reduction of 4 leads to 4*4*4 or 64 times less memory for chunks.

In the image above, I am using 1/8 chunk buffer resolution (8*8*8 or 512 voxels per pixel!).  You would think such super sampling to be excessive, but it makes all the difference visually and ensures that no voxels get dropped (as you may remember in the last video, lowering the voxel resolution led to holes in the plaster and defects in other thin areas). It is also using 1/2 screen buffer resolution (1/2 1080p, stretched to 1080p).  Lowering by a factor of 8 also reduces the number of ray casts and normal calculations needed by a factor of 64.  Total memory usage is only 472 megabytes and it has rendered about 46 billion voxels without unloading anything (you may have notice how quickly chunks unloaded in the old video).  I have a 3 GB video card but I've capped mem usage at 2560 MB to reserve space for the operating system (Windows seems to use usually only a few hundred MB unless a browser is open).

The TL;DR here is that it can now store massive areas in memory without needing to unload anything, without any major drop in visuals.  There is still tons of room for "low-hanging" or trivial optimizations like better front-to back loading.

Another thing that has changed is the way the world is represented.  The old terrain system was based on a heightmap.  The new one is based on macroscopic chunks (you can see each cliff or bluff here is sort of cubic, but rounded off to look more natural.  All this data is pulled from a volumetric buffer and makes it incredibly easy to calculate things like dynamic water.  Right now each unit in this volumetric set is 8x8x8 meters large, but it can be adjusted.  This is big enough to carve out sections of cave or dungeon and ensure that any type of monster will fit in there without hitting the ceiling or walls, making pathfinding more trivial.  Speaking of which, I have added in pathfinding support as well (you can see this at 5:13 to 5:22 in the Kickstarter video).    Chunks do not need to be rendered to perform pathfinding, as shown (first image below).Ensuring that the world is constructed properly and all roads are reachable when you have multiple levels, and ensuring that stairs are spaced enough to not be too steep, and dealing with a million other factors, was a big challenge (see second image below). :) I even tried to implement winding staircases to address the issue of small space within towers, but I pulled that feature for now (third image below).

I've also added in camera tilt - this was mostly to be able to easily see around walls, seen at 7:44 in the KS video.

The UI is another major change.  The UI is tightly bound to the data storage, although the presentation layer and the data storage layer are completely different structures - its not MVC though, a paradigm I hate.  I have built so many UI's I'm sick of it, but I must say I've gotten the hang of it.  The VQ UI system (let's just call it VQUI for now) is built to address many of the headaches I've had in the past with UIs, namely:

• Manual layout: having to manually specify every (x,y) coordinate, margin, etc and make sure things are aligned properly and do not overlap is a major pain.  Many games actually still use manual layout.  VQUI automatically lays out things, just as if you were writing HTML (it uses JSON though).
  
• Binding every single event properly to each button or UI element (i.e. when you click this button, execute this action).  VQ Automatically binds events and lets you grab UI elements with string selectors to get their values and other properties.
  
• Recompiling the whole program for each little GUI change, only to realize some button is one pixel off. VQUI can be hotloaded.
  
• Properly binding data to the UI (say, for example you have a list of inventory items; you only want to store the item id and quantity for each NPC, but when you look at an item in game it brings up an advanced sheet of properties and perhaps some dialogue options).  VQUI supports templates applied to classes of children nodes, even recursively (i.e. a factory).
  
• Saving data that is changed by the player, only to have the data structure or layout change and break a save file.  The data structure in VQUI is "unbreakable" - it doesn't matter if you change the schema of data, it is all stored in a linear fashion with unique ids for every property.  This can be inefficient, but the sanity gains outweigh the performance loss (which is trivial).  Even if you destroy some data key, it will retain it for future loads so nothing breaks in between changes (and you can optionally clean unused data keys on a save). 
  
• Balancing between performance and having a GUI that is dynamic and easy to modify in realtime.
  
• Keeping everything DRY (Don't Repeat Yourself - i.e. minimize copy-paste code, usually via use of templates or CSS-like rules).

Writing your own UI would seem like a bad idea in most circumstances, but I really evaluated every solution and most of them honestly suck.  I would have loved to use browser-based tech like in the old VQ editor, but that turned out to be a really, really bad idea for multiple reasons I won't delve into (hey, I make mistakes like anyone else :) ).  On the bright side, it was a good practice run for what not to do.


One really powerful thing in the UI is that you can autobind shader variables to sliders, as seen in the video (the part where the hue is adjusted).  This works by using a shader preprocessor that I wrote, which already was in there for automating other tasks.  You only have to surround a variable name with "@" symbols, and this will autobind it to a slider after you refresh the shaders in the program.  You don't even need to declare the variable, it will do that for you, so if you quickly want to fiddle with a value, you just throw in one of these special variables.


This also has an important performance side effect - you can bake these values into the shader after you have changed them, so that you don't need to pass in a uniform (the preprocessor just declares a constant value and sticks it in the generated program).  None of this effects the code while you are writing (it modifies the shader in memory, not the file you are working on).  The impact has been huge - I can now tweak values without having to constantly save and refresh the shader.


So, you can see that I've faced many challenges - the devil is in the details as they say.  There were a million other improvements and fixes to the engine, but I can't cover everything - too much other work to do now! I do have most of the pieces in place to actually start building a real, genuine game though - enough of this engine nonsense. :)