[Original article that I wrote can be found here on medium.com if you prefer]
‘ Where are the revolutionaries?
It used to be the younger generation, around college-age, would rethink their parents’ systems: political, business, etc. These kids would look at the buildings around them and imagine what could be accomplished in razing those buildings and creating a new foundation. I don’t see that anymore. I see college-aged kids simply wanting to add a new floor on top of a very rickety shack that should have been discarded ages ago.
Ironically, the people writing these types of manifestos for Computer Science are almost exclusively older. They often wrote that first foundational layer and know that it needs to be overhauled.
It’s true we’ve tried many variations on these themes in the past, but if we do the logical syllogism in our heads, we come invariably to the conclusion this is how the future should be. We know these things. So why give up? What does failure mean other than that we need to continue trying?
It infuriates me that we don’t try — when people say “we’ve tried everything and there’s no way forward.” No, we’re just lazy. Charles Duell never actually said in 1899 that “everything has already been invented,” yet, oh my God, we can’t stop saying it now. So I’m going to ask again, “Where are the revolutionaries?” ’
When we optimize for the dollar rather than the product, we are no longer engineers or artists - we are salesmen.
When we invest based on traction or growth, we are not betting on an improved future, we are banking on an existing present.
I having nothing against making money, I just believe it should be a byproduct of running a good business, not a metric optimized at the expense of everything else. Similarly, I believe in competition, but in a race to the bottom, no one is actually a winner.
To the average person, things probably seem fine right now. In fact, the average person is probably eternally content in the present, whenever that present might be. As someone who clings to the edge of the bell curve, and not necessarily the favorable edge, I want to help you understand that vector between now and the future is discontent.
There was something magic about the 80s and 90s. We were small enough that the individual mattered. We had real heroes. Software was an arcane art that took expertise to ship and worthiness to occupy a store shelf. There was no room for shovelware. There was no ceaseless drip of notifications. We were not glorified drug dealers vying for your attention span. Nobody knew what a dark pattern was. Applications did not harbor their own moral ecosystems dictated by the whims of advertisers and boardrooms.
By necessity everything was smaller, faster, less wasteful, free of bloat. Engineering was the business, not a business expense. It took dark wizardry to cram worlds into kilobytes, and that expertise was valued. Software was not crippled by a myriad of bad platforms, store restrictions, privacy concerns, and permissions.
Somewhere along the way we lost a few things. We lost our freedom, we lost our courage to enact change, we lost our reason to care. We need to care again. We to dream again. We need to feel something, anything, other than passively numb. We need heroes. We need people with brains, guts, hearts, and maybe a loose grip on sanity. We need to challenge things. I refuse to believe this is the best we can do.
The solution is not gradual, it is radical. We need to rethink the operating system, the compiler, the web browser, the internet, and maybe even the computer itself. We need to lay our own fiber if that’s what it takes. We need to ditch legacy, because legacy is not the future. A Formula One car does not have a passenger seat for the horse and a trunk to store the cart in.
We need to rethink the rules. The rules that create perverse incentives and favor monopolies. We need to level the playing field. We need to put the consumer first. Software should not be something that we fight with, that exploits us, that siphons our attention span and our wallet, that pits us against each other. It should augment our abilities and our free time to do other things. We need to hit the reset button.
Right now I probably sound insane, which ok, maybe even par for the course. Reminder: Apple is a quasi-trillion dollar company, started by a crazy homeless person, that competed against juggernauts of its time like IBM. I am definitely not Steve Jobs, but I know what future I want to live in - and I will get there or (literally!) die trying.
When you give up everything that exists right now, the possibilities are virtually endless. There is nothing stopping you from molding the rules in your vision. Want to create your own internet, your own computer, your own blank slate? Go ahead. It will be small, but it will be yours. We are engineers, we are artists, we are gods of our own domain, we are in control. We are not here to stand on the shoulders of giants, we are here to throw stones.
On our net, no domains are yet registered, there are no search engines to compete with, there are no ranking algorithms throwing you into a predestined bucket. There are no advertisements - does that sound crazy? When was the last time you tuned into commercial TV instead of Netflix?
On our grid we own all the fiber and all of the airspace. We paid for it, we put it there, and we own it. What was once an obscene tax paid for a connection will now empower content creators - they are the only reason the connection exists in the first place.
In our system there are rules. Anarchy sounds appealing in theory but that has led us to the present, where corporations abuse the lack of rules or bully people into using their own. I believe it is possible to create a system that maintains freedom while preserving order. It is also possible to create a system that avoids the pitfalls that have led us to where we are now.
In our content, politics are not present unless you are looking for them. The same goes for any other mind-numbing content. Using newfangled technology like tagging, you see what you want to see, rather than corporations showing you what they know will drive engagements or hiding what they believe qualifies for censorship.
In our browser there is no distinction between a native application and a web application. Does that sound dangerous? Good, get out of your comfort zone and quickly learn who to trust; rethink security in a way that does not cripple consumers and creators.
In our interface creative freedom is allowed and encouraged. Material Design and Metro are not gods. We are artists, not cult members. Bring back skeuomorphism, bring back shading, bring back anything that qualifies as variety. Rethink the mechanics of interaction, because no one has bothered to do that for several decades. We can probably come up with a painless way to center elements horizontally and vertically if we are so daring.
In our compilers “write once and run anywhere” is a reality. Performance is peak, package and build systems make sense, frustration is minimal. We are not simply piling on a new standard, we are crushing inferior standards. We will spend our time building rather than fighting three layers of APIs and searching for obscure bugs. Software will only survive based on its quality and merit. Knowledge and experience will matter because strong software does not run on a weak foundation.
We thought we conquered everything but there is still an infinite land left to explore. The point is, the future is nebulous and distant, but the present is whatever we want to make it, starting right now. We are only trapped in a cage of our mental design. There is a future out there waiting for anyone bold enough to claim it.
Although this is not related to Voxel Quest, I thought this piece that I wrote might be worth reposting here, forgive me if it is not!
For those wondering, because the question often comes up, I am still working on Voxel Quest although at a rate probably 100x slower than normal due to life constraints.
At the bottom of this post I have included images and videos from the many versions of VQ, for nostalgia's sake. :)
I released the first version of VQ a while ago, but have not since done much with the other versions. I had been debating what to do about the current situation of the VQ source. As I have probably said a million times at this point, my spare time is borderline non-existent these days between family and work obligations. There are three things I could do:
Edit 2: simcop2387 has kindly put it all in a repository here and bsagdiyev made this bittorrent magnet:?xt=urn:btih:e52b16bdf2eda8b678d5104517dba6ba05fb6089&dn=vqfiles
Edit: use this Dropbox link instead of the one below to get all files. (As of this edit at 11/5/2016 7:50 PST files are still uploading, so give it some time)
Every single snapshot of Voxel Quest (Google Drive link, limited to versions under 20 mb)
All of this code is being released under the zlib license (alternate licenses available on request). Any third party code or libraries used fall under their respective licenses, although from what I recall every third party resource also falls under a liberal license. The one exception is the bitmap resources used (sprites, icons, etc). These free for your personal use but you must buy the corresponding files from 7soul (AKA Henrique Lazarini) if you plan to publish anything (the RPG sprite and monster packs).
I have also included some very old versions of the source, dating back to stuff seen on gavanw.com, long before VQ even had a name.
Some of the bigger snapshots contain all files (usually weighing in at around 70 mb or so), and the smaller ones (around 15-20mb) are just code and other resources. You can copy over resources from the nearest version and it will usually work, or just work with archives that contain all files. Note that you cannot use the same resource files between all versions, necessarily - sometimes the layout or structure of this data would change over time. If you really want to get it up and running fast, you can check out the binary or release folder in a big snapshot, which usually will contain an exe. Not all snapshots are stable. To find a stable snapshot for a particular date, I recommend looking at the videos and picking the date nearest to that of the Youtube publish date on the video. To find controls, which also change over time, you will have to look at the source (most of the time, controls can be found within the singleton file).
If you are opening up one of the newest versions (which use "real" voxels and are computed on the CPU, as seen in recent @voxelquest screenshots), you will have to press "t" to toggle the new render mode. Otherwise, you will be running the old render mode, while at the same time computing voxel chunks on the CPU and it will run slow as hell! To use the ray traced version, I personally recommend the release on or around early August 2015.
You can probably find some helpful notes in the isometric repository - not all will apply, but much of the information there is still relevant to other versions. If you have any missing dll's, its not really "best practice" but you can probably snag them from the iso release on itch.io. (Dump them in the binary folder)
Some older versions are designed to be build on Mac, I think around early 2013 and prior. The rest are designed to be built on Windows, but all versions are portable with a tiny bit of work.
Other notes: people keep telling me they don't want their money back, which is fine - I'm not going to force you to take it back. But I am still honoring my promise to return money to anyone who wishes (and the few who have requested have been paid back so far). Do not feel bad about asking for money back, I am more than happy to return it! You can reach me via Kickstarter, Twitter, or the contact form on this site if you would like a refund of any money that you put into VQ (through KS or otherwise).
One last note: I do not get notifications for the comments on this website, but I do try to read all comments from time to time. I will more likely respond to comments on other channels like Twitter or Youtube as those are easier to manage in a timely manner.
I may have left something out, please let me know if I am forgetting anything!
It is the moment you have all been waiting for!...ish. I am sticking one toe in the water and putting out a very, very unpolished release of the very first version of the engine (the isometric engine, which differs from what I previously said I would release first, due to demand). Other versions are to follow. It was a fair amount of work to prep just this release, as unpolished as it is. And there are many things wrong, which I am mostly aware of (see the README.md). As noted, the point was not to make a perfect release, the point was to get the code in your hands as fast as possible (and I am sorry it has taken this long for me to get it out!). I am hoping that between my help and contributors, it can be turned into something at least a little more polished. So without further ado:
or you can get the executable at:
(Note, I am not charging money, but it obviously costs itch.io to host it, so throw some money at them if you care to).
On a side note, I am still figuring out how I want to earn money (from work, not VQ!). In the short term, I will probably be contracting a bit more, and I have created a new contracting website here (if you have any leads, let me know! - also, it is a rough draft and the splash page is messed up on mobile phones, so view it in a browser for now). Also, if you would like to vouch for me, you can do so here.
My contract at OpenAI is up and it was a great experience - I highly recommend working for them if you get the chance.
I have decided to take a short break and get the source out for VQ. As usual I apologize for it having been quiet, but I had been fairly busy at work.
I may have discussed it elsewhere, but I am aiming on getting a few different releases out for VQ. There were roughly 5 major iterations of the engine (2 of the ray-casting ones were fairly similar). Each release will probably occur separately. The first release I am targeting actually an older version of the engine, as seen below, just because I think it is interesting content-wise (each new iteration often lost some of the content in favor of new incompatible methods for the tech).
Hi all - bet you are wondering what is going on at this point. The short answer is I did not know exactly how to proceed until now, finally having an idea of my employment situation.
I have talked to some people (in the forums, email, and elsewhere) about the current status but otherwise apologize for the silence. Basically for the past month I have been busy hunting for jobs and doing associated prep work or tasks. I also tried to get VQ funded via other means (licensing) but that did not work out. I am moving on to a fulltime job, so I am deciding to cancel the Patreon (and I will refund everyone's money here and elsewhere, of course). I don't feel right taking money when I probably won't have much time or energy to devote to the project.
The good news is that I am making all of Voxel Quest fully open source (MIT or similar, every iteration of the engine will be available), and I am doing a little bit of work to make things ready for this (I am even attempting to buy rights to the sprite sheet that Voxel Quest uses, which is a 3rd party asset). Stay tuned for more on this. I can't express how thankful I am that you chose to support me, and I hope that the years of work I put into VQ and all its source will be a small token of my gratitude. I will probably cancel my Patreon before the next pledge cycle so you don't need to necessarily manage your pledge, it should be automatically canceled soon. The Patreon peaked at $450/month, which I am really surprised at, although I expressed concern with starting it in the first place and turns out those concerns were valid. I will also soon organize a method for refunds (both for Patreon and everything else). Any questions or concerns, as usual, do not hesitate to contact me. Again, my goal is to keep everyone happy so let me know what I can do in that respect.
I am not fully abandoning VQ - I will attempt to provide as much support as I reasonably can to help people understand the source, and I will also probably chip in on the github repository every so often. But overall I will not be making great strides in the short term. I am hoping that someone can eventually make a commercial product with it, and that might kick up interest a bit.
For those curious, I am soon going to start contracting at OpenAI, with potential to move full time.
*Again, I stress that refunds are not required, but if anyone wants one I am offering without judgement.
Additionally, I have read through all of the comments below and I continue to be blown away by people's enthusiasm. I am not replying individually because my website's comment handler is a pain to use for many entries.
First, I just wanted to say "thanks" to everyone for the outpouring of enthusiasm and support. I tried to reply to many people but apologies if I missed you - there were literally hundreds of messages across Twitter, Facebook, email, Kickstarter, this website, etc. I was legitimately, deeply moved by many of the replies.
By popular request, I am not shoving your money back in your hands, it will be optional. I will put together a survey for this (likely when I get a means to start paying people back, be that employment or whatever). I also still need to figure out the logistics of what order people should be paid back in.
Also by popular request, I am putting up a Patreon page, which is located here. I put very little effort into it, but as noted I think my effort is better spent actually working on the game. Previously I did have a patronage page on this website, and even though I never once advertised or mentioned it, people found it on their own and I had over $100/month coming in from this, which really helped. It is also accessible via the "Patrons" link at the top of this site. Old patrons (who donated via Paypal) - please cancel your patronage (or I will do it for you when I get the chance). If you still wish to be a patron (and I really won't judge you either way), you can use the Patreon page. The old list of patrons will remain in place, for the historical record. :)
If you are not already aware of the state of things, I may have to get a job soon so I am hesitant to recommend providing further funds as my pace will be slow. If I get enough money (I mean, via whatever job I take), I might hire someone who costs less than I do to work on it (or at least to help me with certain tasks) - I am also supposed to have someone interning for me this summer but I don't know what that will yield (they volunteered to though). Anyhow, I am still evaluating all options at this point and trying to figure out the best plan of attack. There have been some good suggestions so far, feel free to add any more.
No point in dancing around the issue, I am out of money and trying to figure out what the next step is. Whatever happens, I am continuing to work on VQ, as that does not cost me anything but time. But I might need to allocate much of that time to a paying job, unless I can figure something else out.
The one thing that I have decided, for certain, is that I am returning everyone's money: Kickstarter backers, investors, patrons, and preorders (and anything else). Everyone will keep their rewards regardless. I made this decision before I even launched my Kickstarter campaign, although I've only talked about it with a few people. I am taking this step regardless of success or failure.
If I have to work another job, I will probably begin this return process as soon as possible (and it will take time to accumulate everything). Otherwise I will begin returning money when profitable. I am still planning to release something in the short term, as well as the source code.
I can understand if anybody is disappointed or angry, but I assure you no one has a heavier heart than I do. I invested over $100k of my own money, debt, and equity into this, in addition to about $500k of work (accounting for overtime and opportunity cost). I spent about a decade working in this area without any sort of return (other than it being "fun"), and over the past 3 years I put in well over 10,000 hours. I dragged my family through all of this as well. Nonetheless, if there is anything more I can do, please let me know.
This is not the end, but it is still a depressing position to be in. Still, I can't help but be grateful to have been given an opportunity to work full time on something like this.
As I have mentioned in the past, writing a game engine is a feasible (if unwise) thing to do, if you pick your battles carefully. I simply had too many battles, and too many directions I tried to tackle at once. My todo list never once shrank faster than it grew. Only at the tail end of this did I decide (for better or worse) to burn my entire set of goals and focus on one "simple" goal (although I must stress that nothing is simple).
If I sound like I have a bleak outlook, I don't. I still believe in this and I have something that I am proud of. This is the first time in my life I have worked on something of this magnitude, spanning 50k to 100k lines of code, and not wanted to scrap the entire thing. I've tackled a lot of new ground so I suppose the road is destined to be bumpy, but I continue to learn from my mistakes.
If anyone has any ideas of what I should do next, I am all ears. At the very least, I feel obligated to be transparent about my current situation. There is no outcome that I fear, I mostly just want to do what is best for everyone, even if that simply means getting a job and paying everyone back. I will probably investigate past offers I have gotten for funding, but I don't know if that will necessarily bear fruit.
One last thing I would like to make clear is that nobody owes me sympathy, and I am not even asking for it. My situation is a result of my own choices, and the only thing that I can make better is the future. All things considered, I think this journey has been great so far, and I am curious to see what the future holds.
On a happier note, let's discuss what I've been working on recently.
I've talked about (on Twitch) why I ended up doing yet another technical revision. If it seems crazy that I am doing another revision on the rendering, it might be, but also consider that teams with more members, more experience, and more money are going through the same thing that I am (see this video, if you have not already).
My former method, with the ray marching, was becoming a dead end (on my GPU, a GTX 980, it was dipping to 30 FPS in some cases, at 480x270 no less). It was not scaling, and it was getting to be too large (I had to break it down into two shaders because I exceeded the instruction limit). I had the choice to kill it or try and resurrect its performance somehow. The first thing I tried (and I tried many things) was to get performance acceptable. I couldn't (at least not without creating other serious issues), so I started a new method, which is pretty close to the second iteration of the engine (when it first transitioned from isometric to perspective mode). However, I still use the SDF / ray marching stuff where it makes sense, with animated things and quickly previewing objects, neither of which has a huge performance impact on its own. Overall not a lot of ground lost, especially since I am still using a good deal of code from the SDF stuff (not to mention that the rendering is a relatively small aspect of the code base).
I have transitioned voxel generation to the CPU, using a flood-fill approach that ignores nonvisible voxels and "air" voxels. It is much more flexible and less restrictive than using the GPU for generation. There is no instruction limit, there is not as big a penalty to branching, and there are many more sparse operations that can be done (for one, flood-filling is a sparse algorithm much better suited to the CPU). I also moved to a unified object and destruction model, where all objects and modifications are handled by one algorithm and modifications are voxel-independent, making it easy to do an "infinite" fast undo/redo queue. There is also caching of generated objects, so you only need to generate things once and they get dumped to a scratch disk (ideally these results will be compressed in the future).
I have a new rendering trick that I worked on independently, but I also heard that the team at Media Molecule is doing something similar on Dreams (actually it is pretty funny because our problems and solutions have crossed paths many times, even though we are both working more or less "in the dark"). For lack of a better term, I call it "deferred rasterization."
Polygons are fast, but they consume a lot of vertex memory (one cubic voxel might consume up to 8 vertices and 36 indices depending on the layout). That is up to 272 bytes of data for a single voxel. The naive alternative is to use points (like GL_POINTS) - a single point for a voxel would only consume 16 bytes if a 4-component vector were used, or 32 bytes in my case (two 4-component vectors).
But the problem with naive points is that they have massive amounts of overdraw (assuming they are large enough to fill gaps between them). Especially when looking at objects whose sides run parallel to the view vector. Also, naive points do not look very good - they are essentially tiny billboards.
My (reinvented) solution is to draw points as small as possible (1 pixel large), and grow them in screenspace, using ray casting to determine the cubic boundaries of each point. This is done in a horizontal pass and a vertical pass, as you would do with most kernel type shaders like a blur shader, to reduce the number of texture lookups. In my tests, it is approximately as fast as polygon rasterization of voxels (or faster), but uses a fraction of the memory.
The results speak for themselves. I can now run at 1080p at 120 FPS or greater (again, note that my last method only performed "ok" at 1/4 1080p, 470x270 - at 1080p it dropped below 20 FPS). And I still have plenty of room left for optimization. It also looks cleaner, visually (the SDF stuff often looked too noisy and had many artifacts, not to mention that jutting edges (such as grass) were too costly). I should be able to do VR if I can ever spare the time to implement it. Running at low resolution, I can get 600 to 1000 FPS, so this should scale decently (I know milliseconds are more meaningful, but more people understand FPS as a number).
There are many additional tasks and a lot of cleanup that I am doing as I prepare for the first release. I have had to rework a lot of the physics code because it was simply too unpredictable and not really well suited for voxel collision (Bullet is a great library, but be very careful what you choose to use it for). I couldn't bring myself to write a custom voxel collision handler for Bullet, so prior I was "faking it" and the voxels were not producing "real" collisions or contact pairs - they were simply causing objects to hover above them as if they were exerting a magnetic force. This led to many issues and collision and friction produced a user control feedback that felt wonky. There is something be said for simple, predictable, slightly unrealistic physics though.
I will put together a proper update video as well soon, but for now here are some shots and videos:
I'm going to use this space to reply to the comments, as (unfortunately) my web host is not that great and the comments section is rather annoying to type multiple entries in (and I would be typing the same response over and over). First of all, thank you all for the outpouring of compassion, I was not expecting a response like this. If you don't want your money back, I won't force it on you but the option remains there, and I won't hold it against you.
I don't have any firm plans for the future yet. A lot of people are asking me to put a Patreon up so they can give me more money. I'm kind of hesitant to do that, I feel a bit bad taking money while possibly having to spend much of my time working another job. I will think about it, but I don't have a yes or no reply yet.
Anyhow, just wanted to say thank you everyone, and I swear I'm not getting teary-eyed. ;)
Happy New Year! Ok, maybe a bit late for that, but I still want to do a review of the last year, what went right, what went wrong, and so forth. I will also try to preemptively address some of the common questions I get - please read the bottom of this post before you ask!
So, since the last update, there was so much stuff that I could not even cover it all in the video, and I won't cover most of it here. But here is a high-level list:
What Went Wrong:
What Went Right:
I don't have the greatest feedback mechanisms but it seems people are generally happy and impressed with my work so far. Somehow things are all coming together in the code, in ways I would not expect. I had never written a fluid simulator, yet it turned out well. I had never written a character animation system, but I was pleased with the results. I had never written a networking client or server (beyond trivial examples), yet somehow I did it (even though it is a simple TCP deterministic architecture, it was not trivial to write). I guess if nothing else, I have gained faith that I can tackle pretty much any task, given enough time.
Other than that, not really sure what to put here. :) There are plenty of ups and downs, but I remain confident in the outcome of the future.
The character animation system presented several new challenges. First of all, how do you efficiently ray cast with hundreds of dynamic limbs moving around on the screen? Well, limbs share a common trait: they are attached, and hence in the same proximity. So you only need to compute an intersection for one AABB (or sphere if you prefer) and that will let you know if you are anywhere near to hitting a limb in that body. Then you need not march into every limb of the body, but do an analytical prepass to determine roughly which shapes get hit, and collect some of them and march into those with a more complex algorithm. In this case, everything is just a line-line distance (one line being the view ray, the other line being the bone segment in question).
I had initial fears about passing in a bunch of bone data to the GPU, that the bottleneck would choke performance. But really its not so bad (really no different in practice than passing bone data to a vertex mesh), and you can animate a pretty large crowd - its not going to do Lord of the Rings scale battles, but that is ok (quality vs quantity).
Some of the biggest challenges came from trying to wrangle a physics system. There is a dichotomy between how the physics world wants to behave and how the user wants it to responsively react. This was compounded by trying to do complex animation systems with dynamically-oriented limbs over a custom collider for the voxel data of the terrain.
I'm going to add a few other technical notes soon as well, when I get a moment.
Been working on many things - increased view distance and improved rendering, much better AI / pathfinding, and more - I will post a "real" update on these things soon but for now here is a video and some images: