Helion: The Rising Star
By Eric Claus
Introduction
Doom has a deep history of source ports with different visions, goals, and features since the source code was released back in the 90s. While each project sought to go in various directions, such as Chocolate Doom being a more modernized but otherwise Vanilla client, to GZDoom that adds so many rich features it really is its own engine and has been used in unique game development. Rarely, though, has any source port deviated from a specific idea of Doom’s structure. In the case of Helion, even the development language itself has been changed to create a re-implementation of Doom using the C# language, and a complete shift in the paradigm of rendering away from BSP which it used initially. Helion is a port developed by Nick St. Laurent (doomworld username hobomaster22) and WchrisK that is on version 0.9.3.0 as of this writing and will probably have some updates after this article releases. Helion currently supports vanilla, boom, mbf, mbf 21, and some zdoom line specials in terms of compatibility.
Helion Development: An Interview with Nick St. Laurent
Seeking to understand this up and comer in source port development I reached out to Nick, one of the source port authors, to get a better idea of Helion’s mission and design goals. After messaging we spoke on Discord and Nick was happy to answer the questions I sent their way. His answers provided some excellent insight into the port’s goals and design. Without further ado, here is the Q&A.
The Interview
Eric Claus: What is your goal for Helion? Is it a proof of concept/experimental type port?
Nick: To make an engine that plays Doom well; which includes but is not limited to: playing smoothly, feeling modern, capturing the old aesthetic, and adding quality of life features that do not interfere with any of the previous points. It could be considered a proof of concept a year or two ago, but has evolved beyond that since then in our opinion.
We also try to differentiate our work from a “port”. We did not fork away from the original source code since we are writing C# and the original source code is in C. When you see us calling ourselves an “engine”, we do that for this reason. It does not bother us if someone calls us a port since we know what people are trying to say.
The Doom mapping community has evolved significantly over the past decade. The amount of talent and effort that goes into levels these days is mind blowing. There are so many high quality wads coming out in the limit removing/Boom/MBF21 categories. Content creators have been creating more large and open maps that suffer in source ports today. Our goal for Helion is to allow more people to be able to play these types of levels even if their hardware is slower and/or older and to enable content creators to create even larger levels and push the limits significantly further.
Eric Claus: Is it a more “serious” type of port where you hope for wider adoption as time goes on with longer term development goals?
Nick: “Serious” is a tough word to define since it means different degrees of intensity to different people. I think that it is more serious than most other ports, since we focus heavily on responding to user input and try to hit various deadlines. An example would be that we want to include feature F for release R, and then we will schedule F such that it is done before R. If we don’t get F done before R’s due date, we will postpone the release by a week or however long is needed. This involves manually testing before releases.
We love how smooth, crisp, and solid everything is, and sharing that with others makes us happy. If more people derive great enjoyment, then that is satisfying for us. As such, a wider adoption is preferred.
We also hope that it spurs practical innovation in other ports. We work on Helion because no one else shares our goals. If someone did, we would instead work on that port/engine.
Unfortunately we haven’t seen our work spur innovation in other ports, unless we missed it. While some projects have tried to optimize things like the software renderer, and we think it is great that attempts are made, such projects crash frequently enough that it does not feel playable. We also think that the future of rendering is on a GPU since it’s an entire piece of hardware dedicated specifically to taking care of that task.
I forget which port does it (dsdadoom?) but palette/colormap rendering can be emulated in shaders. Helion currently will map all images from their palette format onto RGBA and draw that, and approximate vanilla light levels as best as we can. This means the colormap isn’t currently used. We however happen to like how this looks.
We think we are also serious since there are a lot of frequent updates. The active development and voluminous output we have makes it feel serious to us. We do have periods of calm and bursts of activity due to real life. Jobs, marriages, and children, unexpected health problems, and life in general are in the equation and that means sometimes things calm down for a short period.
Eric Claus: What was your reasoning to use C# and .NET Core as the codebase? Does that stack play in to your goals of balanced performance and readability?
Nick: As some brief history, originally Helion was made in Java 7-8+ years ago, but had issues with Java’s garbage collector and the associated stutters when the garbage collector ran, which ruined users’ enjoyment. Helion was then remade in C++ to mitigate this problem. However C++ is slow to write correctly and painful to debug compared to higher level languages like Java and C#. It is our hope that using C# allows more people to contribute since it is an easier language to learn and becoming more popular. While C# is managed by Microsoft it has this perception of being propriety and only being for Windows. While this was true at one point, since 2017 C# works across all the major operating systems, and since 2014 is completely free and open source.
There was a decision at some point to get the best of both worlds, which was C#. What made C# win was that it had the high development velocity of Java, but the GC was much more evolved for real-time performance and could handle high amounts of garbage generation without any noticeable stutters. We still had to do object pooling and move to structs years later to increase performance, but we were quite bad compared to now in our garbage generation for the initial version of Helion. It still blows my mind how well C# performed with so much garbage. Our intention was “get it working first, optimize later”, and you care less about how many objects you generate when you want to get it working first.
For anyone non-technical, when you create new objects in C#, you can create them and forget about them since there is a background “garbage collector” that will reclaim any memory you allocated and stopped. This reduces the need for manually managing memory, and increases development velocity since it’s one less thing you have to both worry about and code for. The downside is the garbage collector can run at any point, and depending on how much work it does, this can result in perceptible pauses at unexpected times. Modern C++ has a good solution for this, but there are other significant downsides to C++ that make it less of an optimal fit than C# is currently.
We also bet on .NET continuing to evolve. Microsoft showed what direction they were heading in, which was performance, and it seemed like an intelligent investment in the future. We also tried looking into the future since we realized it would take a while to get to the release date, and by that time, C# would have become even faster. In due time, it did.
C# results in very high performance, development velocity, and readability. There have been tremendous improvements in the runtime optimizer (JIT), especially as of .NET 8 which I have played around with. Due to the optimization improvements that have been made, comparing optimized output from C# with the appropriate C++ counterpart shows usually the same instructions now. The easy inclusion of SSE/AVX intrinsics is something we would like to explore as well.
Eric Claus: Were you originally setting out to make a more efficient rendering engine or did that evolve naturally as you worked on the port?
Nick: It was intended at first to be a general proof of concept engine. Its origins were in trying to be an engine that handled multiplayer optimally since there was a lot of tension in the multiplayer space over stability and quality of life. This meant Helion was intended to be something smaller than it was today, and cater to a niche sub-scene in Doom.
One of the problems we ran into is that some people with low end computers had trouble running anything modern. Hardware from 2009 would somehow be in the single digit frames per second in ports like GZDoom on d2m1 with no mods and default settings. It was confusing that people with hardware from 2009 were running slower than computers in 1993 did, and we set out to fix that. We could take people who had such “slow 2009 manufactured hardware” and ran it with a primitive version of Helion, where they rendered and simulated the world at multiple hundreds of frames per second.
Even nowadays from our last performance profiling, we notice some maps that run 28000% faster than they do in GZDoom.
The original intention was not to make some hyper-efficient renderer, but when people were running the prototypical Helion (around 2018-2019) and suddenly were able to play crazy wads that they were incapable of doing before, they were ecstatic. Due to how happy it made people, the shift towards an optimized renderer became a priority over time, and eventually took over as one of the selling points.
This is not to say that we did not do other optimizations. World simulation had a lot of work done to it as well. An efficient renderer that is bottle-necked by world simulation is pointless unless you want to stare at eye candy maps.
Eric Claus: In removing BSP rendering, what challenges or incompatibilities have you had to deal with?
Nick: Removing the BSP tree for rendering was a challenge itself because of how dynamic Doom is. The traditional BSP tree approach starts clean each render frame and walks the BSP tree and only renders what is visible. This means that any changes from the previous frame to the next are handled automatically. Modern games have static areas that don’t change and then dynamic areas where static areas are the vast majority of the map. Because that static data never changes, it can easily be uploaded to the GPU and the GPU does all the work from there resulting in very fast rendering. Doom allows for everything to be dynamic which requires very different methods for rendering in some cases.
The first part of the solution is to generate a static buffer of the level and upload it to the GPU. This means that Helion simply needs to tell the GPU to draw, and since the GPU handles all the discards incredibly fast, the rendering is very fast. This solves the CPU side bottleneck of constantly generating the world geometry that BSP traversal has. The problem to solve is how to deal with sector movement. Helion accomplishes this by using a separate blockmap for rendering. For example, if the player opens a door, Helion will clear the vertex data in the static arrays for the door sector and upload those chunks so that nothing is drawn from the static buffer. Then the sector is linked to the render blockmap. Helion will traverse the render blockmap in front of the player to dynamically render moving sectors up to the maximum render distance. This is similar to the slower method of generating everything with the BSP but is only done for moving geometry. Once the sector has stopped moving, it is removed from the render blockmap and put back into the static buffer.
Changing light levels was a huge challenge in previous versions of Helion. Light levels were previously part of the vertex data. This makes sense if you are using the BSP and starting clean each frame. The problem in Helion was that a change in a light level would require uploading all the parts of the map that were affected. Some modern maps change very large amounts of sectors and would require massive amounts of the map to be recalculated and uploaded to the GPU, which can be very slow since all the CPU processing is blocking the GPU rendering. An extreme example was Microslaughter Community Project MAP02. The solution to this problem was to use an array of light levels for each sector. This array of sector light levels is modified when a sector’s light level changes and is uploaded each frame. Each vertex has the index of which sector it belongs to in this array and the light level is looked up on the GPU when rendering. Simply, instead of modifying thousands of vertices and uploading to the GPU when a light level changes, Helion will update that one light level on the GPU so that the GPU can quickly look it up.
Another huge challenge was flood filling. The flood filling quirk of Doom’s software rendering is a feature of Doom mapping. In OpenGL emulating the flood fill effect was done using a stencil buffer. The problem we ran into was that without the BSP we did not know what flood filling the player could see. This created a huge problem in maps that used a lot of flood filling, some of which was unintentional and the player would never see it. For example, not setting the upper/lower texture on the backside of a window that the player would never enter, or a floor in a monster closet. Because the OpenGL stencil operation was incredibly expensive, Helion would suffer on maps with a lot of missing upper and lower textures, and even worse time was wasted on these OpenGL stencil calls for flood filling the player would never see. Worse yet, to keep it reasonably performant it became practically impossible to update flood filling at runtime if a sector moved. The final solution for Helion was to manually generate the flood filling on the GPU per pixel. This removed OpenGL stencil call bottleneck and if the flood filling would not be seen then the GPU wouldn’t waste time generating it. This method also easily allowed Helion to update flood filling if the sector moved. This one change alone is what gave some maps a 100% performance increase, for example PUSS X: The Summer of Slaughter MAP32 – No Living Land (SOS_BOOM.wad).
Eric Claus: Do you plan to include any further new features in Helion?
Nick: I think this would be an active area of discussion on where we want to go.
What may be on the horizon is:
More performance improvements
Split-screen multiplayer
UDMF support
Voxel support
Possibly netplay, but lockstep netplay (no near term plans for client/server)
Possibly “dynamic lights”, which seems to be what people call point lights as famously associated with GZDoom, but only if we can do it in a performant manner
Possibly VR support
Eric Claus: What challenges are you facing currently in Helion’s development? What challenges do you think might arise in the future?
Nick: Most of the challenges come from real life intervening in unexpected and unfortunate ways. The rest is standard work of wanting to add some feature and implementing it in a sane fashion.
I think that the main challenge arises from a lot of legacy code where we were originally going to support a lot of mod features that you see in G/ZDoom, but then we focused on going for Boom compatibility instead and put our time into making the core strong. This means there are a lot of things that should be pruned at some point, or maybe completed.
For example, Helion has Decorate support, but it is limited in the sense that only vanilla/boom/mbf21 action functions work.
We also don’t see OpenGL going away in the near future, so there is no immediate push to migrate to something like Vulkan. We don’t think the time spent writing Vulkan would be the highest bang-per-buck for our users.
Eric Claus: Will Helion stay Doom only, or will it expand to Heretic/Hexen and Strife? Will it become or serve as the blueprint of a more generalized engine?
Nick: The intention right now is to stay with Doom. Heretic probably could be added with the decorate support that exists but at the current moment in time this has not been a consideration. Hexen would be a lot to add. Strife is loved by one of the developers, but that would be likely more work that Heretic and Hexen.
The primary reason is that due to how busy real life is, selecting where to spend time becomes important. It likely is the case that adding Heretic, Hexen, and/or Strife support would be massive levels of effort for little perceptible gain.
Eric Claus: Are there any WADs you can think of that Helion is particularly needed for performance?
Nick: I think your question has multiple answers.
One answer is: All of them.
There are people who cannot play Doom at any reasonable frames per second due to hardware constraints. As seen earlier, someone gets single digit FPS on ports like GZDoom on d2m1. To them, our performance means the world since they can play maps and enjoy them without enduring a slideshow. I am biased, but the snappiness of the engine combined with the classic aesthetic and high frame rates make everything feel crisp. I can’t achieve this experience in other ports, there is always something that feels wrong in them. Helion does everything right for me. What is most enjoyable are maps that bring every port to a crawl, but Helion chugs along as if it’s nothing.
A second answer is: everyone who wants to run high frames at decent sized monitors on busy maps. The density of a map will hurt more and more people, and even on this machine which has good 2022+ hardware, Helion is the only port where I can hit consistently high frame rates that never dip under my 144 hz refresh rate.
Eric Claus: On a more personal note, you mentioned in a post in the Helion thread on Doomworld that you were humbled by Doom development. How much do you feel you have grown in your understanding today?
Nick: I grew up playing Doom practically my entire life. Doing software development over the past 15 years and working on Doom Launcher and the Odamex port put me in a position where I thought I knew a lot more about the game than I actually did. Starting Helion from scratch really showed how little I knew. I have lost count of how many debugging sessions I did because a single map exposed some very specific physics quirk that broke map progression in Helion. Every new bug is a new learning experience and gives me more knowledge about how the internals of the game work. It has helped me grow immensely in debugging where I’m trying to isolate all the variables and sections of code that are relevant to determine what differences exist between Helion versus the original Doom code.
Eric Claus: Are there any features or accomplishments you’re particularly proud of so far?
Nick: We are most proud of the performance for new and old hardware alike. The BSP tree has been a given in Doom ports. There have been discussions about ditching the BSP for rendering and the take away was always that it wasn’t possible. Being able to have a functioning port using a more modern static rendering method while retaining all the quirks of Doom and supporting through MBF21 is a huge accomplishment that we are proud of. It was a monumental effort starting from scratch and diverging from the usual C/C++ code bases and we are very proud of how everything has come together.
We are also very proud of the ability to run the game simulation without the renderer. This allows Helion to run automated tests that test everything from item pickups to the most specific physics cases. Each time a bug is encountered in Helion it’s impossible to know what test cases there are off the top of your head. When a bug is encountered, especially in the physics areas, we can feel confident that the change is not breaking the vast amount of special cases that need to be supported. And each time a new case is found we can add it to the test suite that will be tested every release making Helion more robust and significantly less prone to bug fixes and features breaking something else.
Post Interview Thoughts and Personal Recommendations
Sometime while I was working on this article/interview I did actually test Helion myself, and funny enough even helped troubleshoot a bug and helped with a fix for the Linux releases going forward. Helion’s focus on performance is evident. To test the port a bit I ran nuts.wad to see what framerates I could get between Helion and GZDoom to see what differences I could get (disclaimer: settings and capabilities can be different, this is not a dig on GZDoom and I tried to keep things as similar as I could; this was not “scientific” in any way). GZDoom ran fine on hardware rendering and I didn’t have any hitches at all, but the amount of spare frames were noticeable on my setup as Helion was able to often stay at a good 3-4 times the frames pretty often so the performance of Helion is pretty noticeable. Helion does not currently have all the same features of GZDoom however, but for large maps that are compatible Helion would be a great choice to use and could be a boon for those with less capable hardware that still want a good looking experience. Helion doesn’t necessarily present much to the player visibly, but under the hood you have a powerful engine running Doom on all cylinders. I recommend playing around with Helion and seeing how well it works for you, especially players who want to play some chunky maps but have trouble with a heavier port. If anything you will be supporting a project that brings some fresh ideas to the table that may one day inspire future development.
Thanks to:
Nick for answering my questions as well as Nick, and a thanks to all Helion contributors including those not seen in this article. I also wanted to thank Nick for involving me in the process of fixing a Linux version issue I reported, it was a great experience to help troubleshoot and test a fix. I wish you all the best on continued development!
My fellow Wadazine contributors and editors for help in reviewing this article and putting me on the right path for an interview, especially Endless and Lucius Wooding.
The Wadazine as a whole, you guys are great people.
The Afternoon Crew for taking me in to their group and showing interest in my work outside of it also good people.
Useful Links
Helion Doomworld Link: https://www.doomworld.com/forum/topic/132153-helion–renderingc-0940-824-goodbye-bsp-tree-rendering/
Helion Github: https://github.com/Helion-Engine/Helion