0
Started

Conservative/inefficient memory use

Mr. Fusion 2 years ago updated by Tyler Owen (Lead Developer) 2 years ago 7

Since the last few versions I've been observing severely degraded memory performance, excessive load times (especially when leaving the habitat, sometimes so bad that the entire OS starts choking, and other background processes also get interrupted/paused) and a lot of storage activity of paging/asset loading during playing.


For a while I kind of assumed that it's because the game is starting to outgrow my system with only 4 GB memory, but now I actually watched the memory use and it turned out it's not that. The game appears to be capped at using only 1.2 GB memory at all times while on the outside map (approx 850 MB while in the hab, and about 700 MB in the main menu), even when at least another 1 GB of memory is reported being still available, while at the same time it obviously has/needs more data to work with as inditaced by the very frequent loading/paging that didn't happen before.


I suspect that this memory cap forces the game process to page a lot more than it should be if it was allowed to use more memory (ideally as much as is reported by the OS being available), and that this excessive paging also interferes with the high volume asset loading during switching from the hab interior to the exterior map (basically the game likely ends up in a paging race condition where it immediately starts paging out assets it just loaded while trying to load more assets at the same time, completely filling the storage bandwidth and causing the aforementioned OS level choking).


If the working set / VM pool requested by the game process, or memory management in general is something that can be configured in the Unity dev tools, it would worth looking into why the game appears to be requesting less memory than it obviously needs.


As sad as it is, poor / inefficient memory management is the Achilles-heel of Unity, but I normally see Unity games trying to hog literally all the memory available in the system (and in many cases then running practically without any storage activity / paging), while Lacuna Passage does the opposite of that for some reason, denying itself of the amount of memory it would need to run smoothly.

Under review

Hmm... that is odd. I just recorded 2.7 GB memory usage while outside of the habitat. It seems to scale just fine according to memory needs while playing. I have 32 GB of system ram and I'm using the 64bit version of the game (downloads automatically if you're playing on a 64bit windows OS). Are you playing on Mac or PC? The Mac version is a Universal build which should support both 32bit and 64bit Mac OS.


Sadly, you are right that Unity is somewhat notorious for poor memory optimization, though part of that responsibility does come down on the developer. Do you know if there was a specific patch that introduced this problem for you? There honestly should not be much of a difference to memory usage since launch. Very few new game assets have been added, it has mostly been systems and feature additions. In fact, I've found ways to improve memory usage through some batching operations.

PC, Win7 64 bit, and Steam launches the 64 bit version as expected.


I have a suspicion which I didn't have time to test yet. Is it possible that the game (incorrectly) limits its requested working set based on the quality setting in options (it appears to control texture quality, amongst other things, and that's one of the biggest impact factor on memory use in general)? I used to have it at 5 for a very long time, but since I switched to a larger resolution monitor a few months ago, I had to drop it back to 4, being somewhat GPU capped, not to have a lower than pleasant framerate.


And if I think about it, that's kind of when I started to notice a lot more storage access than before, to the point where the game often feels near unplayable since it very frequently and repeatedly freezes for several seconds, or stutters for minutes with constant storage activity, while there's still plenty of available memory left it could use, but it does not.

Planned

That is an interesting theory that I will need to test. I wouldn't have thought that a lower quality setting would cap the memory usage in any way, but you're right that it would likely not require as much memory because of things like texture quality. I'll try out a few things on my system and see if I can recreate the issue. Thanks.

I've now checked the memory use both at quality preset 5 and 6, and it was still capped at 1.2 GB in both cases with the same level of storage activity and OS level choking when exiting the habitat, still with at least 1 GB reported by the OS as being available.


I suspect the cap is being set based on either total system memory (4 GB in my case) or video card memory (1 GB in my case), but I have no idea why. I'm certain the game used to be able to use pretty much all available memory in the past, which resulted in much smoother performance.

This could be a result of updating the Unity engine to a newer version. I'm not really sure yet how to address this, but please keep me updated on your performance issues after the next few patches.

Since this still continues to bug me, as the game hovers just on the border of playability for a considerable part of a play session (near constant drive use for extended periods, 2-3 seconds freezes every few seconds, complete lock ups for anwyhere between 10-30 seconds), I monitored it's memory / page file related performance and compared it to other games of moderate / high resource utilization.


The primary problem I saw (and which is a strong indication that the game is severely memory constrained) is that the game is generating an extreme amount of page faults: about 30-35 million per hour, as opposed to other games with similar memory requirements and use I've monitored in a similar way, which tend to be not higher than between 500,000 - 2 million page faults per hour (and hardly cause any noticeable drive activity and related stuttering / freezing).


Anhother proof is that if I stand still while outside and keep turning around constantly at a steady turn rate, after a turn or two the game starts using the drive constantly and doesn't stop until I stop turning. This indicates that it almost immediately drops graphics related resources once they are not needed (out of view) and thus is forced to reload them again on the next turn, then drop again, then reload again and so on. Effectively, at this point the game will start to run from disk, so to speak.


And while all this goes on, the game never uses more than 1.2 GB physical memory, with another 2 GB left available to it if it wanted to use it, but it does not, instead it relies on high page file utilization and reloading assets from the game files every time they are needed (likely this is what primarily goes on, and the reason why all this is a lot less apparent on a system with a lot of memory, as in, 16-32 GB; there the OS likely ends up having cached most/all game files in memory after a while so the need of actual drive access gets all but eliminated, thus even tho the game technically still rereads those files constantly, they are served from memory instead of actual disk access so the delay all this causes is reduced by at least one magnitude).


The system I run it on is capable of providing a single process with up to 3.2-3.4 GB physical memory (out of a total of 4 GB present) if requested, as another Unity based game demonstrates this, which has a very high memory use, but even in spite of it using literally every bit of physical memory availalbe except what needed for the parts of the OS that absolutelly have to stay in memory and cannot be paged out, it hardly ever just so much as noticeably stutters, let alone completely freezes.


There must be some way to configure how Unity should handle system memory and page file, the level of asset caching and persistence etc, because as of now Lacuna Passage is the worst performing game I've struggled with playing in a long time on this system, even amongst a fair number of Unity based games I played. Even most games that had considerably higher minimum system requirements than what my system has tended to run better, especially in terms of memory management and drive access related stuttering as a result of less system memory present than the minimum indicated.

Started

I have been doing a massive performance improvement pass over the last couple weeks, to the extent that it may result in very few other changes or additions to the game, but I'm really hoping that it helps with the playability of the game in your case. I have drastically reduced overall garbage collection and I've made big improvements to CPU usage as well. I did not really target rendering optimizations since I figured that the existing quality settings do a pretty good job of letting users select what their own system is capable of, but the other improvements are ones that every player should see a benefit from. I was closely monitoring RAM usage as well, and in my play tests there were times where the RAM usage was going over 1.2gb, sometimes up to 1.5gb. I suppose I could devise a test where I add a hidden asset to the game that requires a large amount of RAM just to see what happens, but I haven't tried that yet. Theoretically, it should push that RAM usage up noticeably, but if it would not then somewhere there must be some artificial cap on RAM being applied. I just have no idea where or how that would be possible. Please let me know how the game performs on your system after the next update (next couple days).