Do NOT Optimise (General) Performance

Support Previous Build Reports
1

Currently this game has a massive design flaw - when the game is under load, instead of merely slowing down (but everything still working correctly), various things fail. Symptoms include haulers not hauling, crafters not crafting, miners not mining, scaffolding not being taken down, buildings stalling, hearthlings starving, etc.

Note that I’d estimate that about 15% of the bug reports are all caused by this same massive design flaw; and that all of these bugs are currently being mis-attributed as bugs with mining, or bugs with hauling, or bugs with building, or… when they are not.

By optimising the performance you’ll just hide symptoms temporarily - instead of users having problems at 15+ hearthlings users will have the exact same problems at 25+ hearthlings.

Optimising performance will only make it harder to see symptoms and harder to fix the massive design flaw. Far smarter (as a temporary bug finding technique) would be to deliberately make performance significantly worse (e.g. throw some unnecessary delays into the code) to exacerbate the problem and make it easier to find/solve the real problem.

My plea is: Fix the massive design flaw (then optimise performance). Don’t optimise performance until the massive design flaw is fixed.

1 Like
2

the ai is based off of frames so optimization will improe both.

3

Well, to be correct, they are actually starving or not doing actions ‘because’ they aren’t optimized, a bug will be them doing something wrong, of failing to do it, not when they’re not trying at all

The computer is thinking who should do what, and the timer is still ticking, so if the computer takes 10minutes to determine what a hearthling should do, that hearthling will be doing nothing for 10minutes, in which time they may starve to death, since the hunger timer is still ticking in the back

Bugs and optimisation are two different but related things, but i’m sure the devs know how to seperate one from another, and deal with them in effective ways

3 Likes
4

There is always 2 different concepts: “game time” and “real time”. If the player is using double speed and it takes a tiny fraction of second for the game to figure out what hearthlings should be doing.then “1 second real time = 2 seconds game time”. if it takes 10 minutes for the game to figure out what hearthlings should be doing then “10 minutes real time = 1 second game time”.because the game can’t achieve “1 second real time = 2 second game time” in that case.

Under no circumstances should CPU speed, CPU load or optimisations make any difference to hearthling behaviour or AI or physics simulation. It should only effect game speed and nothing else (e.g. if the game can actually achieve the speed the player selected or runs slower because it can’t achieve that speed)…The game could/should be able to have an “unlimited speed” option where everything happens as fast as possible; and even in that case it still shouldn’t make anything break.

The fact that everything does break when the game is struggling to keep up is what makes this the single most significant bug in the game at the moment.

Like I said, optimising performance won’t fix the bug at all. Instead of everything breaking at 20+ hearthlings on modern/fast CPUs; everything will break at 30+ hearthlings (or 40+ hearthlings, or …). Everything will still break regardless of how much you optimise.

2 Likes
5

Well… That isn’t really the case with SH, if you try playing for a long period of time, you’ll often notice them not doing anything (thinking) and the in-game time going in the same speed as always, it’s weird yes, but it is the case(in my case, no error messages at-all, just them bing slow)

And also, you can’t just somehow ‘improve’ general performance/optimisation without getting rid of bugs, it’s not like there’s a magic ‘faster processing’ button, if you want it to work better and longer/faster, you need to get rid of aspects that bogg the game down and cause those issues in the first place,

The computer never makes ‘mistakes’ it’ll only slow down, a ‘bug’ will be, as i said, a case where there’s an error in the code, something directly fixable, and if there is said bug, the hearthling count will not matter, it will happen even with one hearthling (assuming it’s a task capable with only one, and that type of hearthling)
If the hearthling count mattered, that will be considered a performance issue

In fact, the very thing you are talking about (hearthling not doing anything when the count goes up) is literally a issue with performance, if it was a thinking problem, it’ll happen regardless of the count

2 Likes
6

If you can’t even understand the difference between a performance problem (where everything does work correctly, but slower than it should) and a bug (where things don’t work correctly under certain conditions) then you are simply not qualified to talk to me.

Imagine if the game crashed under certain conditions (whenever the game is struggling to keep up). That would be a bug and not a performance problem because the game should never crash. It would make no difference if the bug occurred at 20+ hearthlings or if the bug occurred at 20000+ hearthlings, the game should never crash and therefore if it does crash it would be a bug.

The bug is that under certain conditions (whenever the game is struggling to keep up), hearthlings fail to haul, fail to do crafting jobs, fail to mine, fail to build and/or fail to eat. It’s a bug because it should never happen under any conditions. It makes no difference if the bug occurs at 20+ hearthlings or if the bug occurs at 20000+ hearthlings, hearthlings should never fail to do things and therefore if they do fail to do things it is a bug.

If it was a performance problem and not a bug; then at 20000+ hearthlings they would all still haul, still do crafting jobs, still do mining, still build and still eat; they’d just do these things very slowly; and even if the game is so slow at 20000+ hearthlings that it’s unplayable it still wouldn’t be a bug because hearthlings are doing everything correctly (but slowly).

1 Like
7

Well, i’m no professional coder or game developer, so yes, i’m not ‘qualified’ to talk like an expert, but i do know what i know, and i’m a person that plays this game, so i do know what I experienced, if i’m wrong(i say something that doesn’t actually happen) point it out for me, i do want to get my thought across but i sure don’t want to be stubborn

What i mean is that the things you talked about(hem not eating or hauling ect) are usually problems that accur to different people with different PC power at differemt points, SH is a game where every single entity has their own AI so the game slowing down to a point where peopel don’t do anything for 10+ minutes because they are thinking, is currently a problem for people with low compuing power, or people with a Ton of hearthlings, and you can see that they don’t ‘fail’ to do try it, they’r simply just too slow and the day is still passing(again, weird i know, but the game time seems to go as usual) you can see it if you pause the game, you’ll see the hearthlings pop into moving animations one by one, and the CPU usage bar at the bottom will clear to green after a while(a few minutes if there’s a lot of them) a bug will be where the CPU bar cleares, but some hearthlings are still not doing anything(thay’ll only do one since that’s all they think of, but if there is a task, they’ll try at-least one, if not, yes, a bug)

that’s why i call it a performance issue, the way that SH is set up(every single entity has AI) doesn’t seem like a place where such issues can be completely removed, even with a lot of fixes and improvements, i wouldn’t imagine a game still working well with 1000+ Hearthlings, they are actively searching for work and danger ect, and how long your game will be playable(not too slow) will depend heavily on your computers power

The crashing in the other hand, are definitely a bug, usually a Memory leak in somehwere piling up until it’t soo much, the devs had mentioned it and i’m not sure if it still exists(probably does) but that’s a case of the game actually crasing after a given time, even computers with beefy CPU and RAM usually get crashes after a long while for no good reason, just a memeory leak, this is what i’d call a bug(at-least one of them) because by the concept(a memory leak) it will still accure in a small amount of hearthlings, just extremely slowly

Other crashes such as crash on load are susually a bug; a badly designed building or a bad use of the console to a building ect they usually are caused by actual bugs in the game that the game simply cannot load, and hearthling count usually doen’t cause it, the logs usually show a bad building load attempt or even a memory issue, or maybe just a corrupted save, thees are definitely a bug, and are things that should be fixable

Again, if I’m definitely wrong on something lets me know, i’d rather be acknowledged as wrong and learn from it

4 Likes
8

This is a classic symptom of a performance issue, not a bug.

I think the reason that you don’t see it as such is because the game doesn’t become unresponsive (that is, you can click around the UI, the UI responds to user events, you can assign new crafting/building orders and so on), but that doesn’t mean that the game isn’t starved for resources. Most likely the AI worker (that is, what is running the AI code) doesn’t have enough time (meaning computing power) to complete its tasks.

Why is it running out of time with X Hearthlings but not with (X-K)? Because the poor AI worker has too much to do and not enough resources to do it, and as consequence the whole chain stalls, and therefore the symptom of that is Heartlings that don’t do anything. However, the game is continuing to be simulated (in-game time passes, sounds play, the UI receives events and so on) because their workers aren’t bogged down.

There are two solutions to this, and one of those involves you paying for a beefier computer, but that becomes unsustainable quickly (because the root causes of the performance issues aren’t solved) for both parties; and the other involves Radiant to write (and/or rewrite) more performant code. However, this is one of the hardest, time-consuming and costly (and not immediately obvious, in some cases) parts of writing code. Of course there’s some give and take in these two solutions, but I think this should be the gist of it.

.

3 Likes
9

A bug doesn’t have to make the game crash to be a bug. A bug is any kind of unintended behavior.

The behavior you describe, where the AI can’t keep up and the game time is still ticking, causing hearthlings to starve and so on is exactly the thing that @Brendan points out as a bad/unintended behavior. Your hearthlings should not die from starvation when they are standing next to food. Not ever. No matter how many hearthlings you have or what kind of PC you’re running the game on. It’s a bug.
Game time should not proceed unless all threads that run the simulation (including the thread that runs the AI) says that it’s ok to proceed. This is not solved by increasing performance. This is solved by handling the problems that occur when there are performance issues so that the end result is still the same.

@HardDiskD In that quote he is describing how the game should work. If it would run slow but still correct then it is a performance issue. But it is not. The AI runs slow but not the game time, causing unintended behavior. That’s a bug. The game time should run as slow as the slowest part of the simulation. Moving the camera, using the UI and assigning tasks can still be fast.

(yes I’m repeating myself but need to go to bed and don’t have time the rewrite it)

1 Like
10

Huh, you have a good point, though the slowing down is still a performance issue, what you mentioned is a real issue, and also kind of a bug, i guess i didn’t consider the game time not slowing as an issue, i’ve played the game for so long that i think i became use to it, and considered it obviouse, and now that you mention it, yeah, i guess the game timer should slow down and sync to the AI to avoid anyone starving to death while there’s foor right in front of their adorable faces

What i was talking about was he fact that they slowed down, but you make a good point, thanks for pointing it out

11

For a modern game, the draw loop [where everything’s rendered to the screen] and the update loop [where all the simulations are calculated] shouldn’t be tied together too much. That is, ideally, the Hearthlings should update for however much “game time” has passed periodically, and the game should be rendering as fast as possible, at a steady framerate. This never quite happens, of course, because it’s still one computer running the whole thing, so there has to be waiting at some point. Otherwise everything gets completely out of sync, which only gets worse in multiplayer.

But because you’re not relying on every frame or update taking a specific amount of time, things can be fudged quite a bit. If that means simplifying Hearthling behavior to keep the graphics going smoothly, that’s okay with me.

Currently Stonehearth doesn’t seem to have any good ways to cheat. An action either happens along with the normal time, or it doesn’t happen at all and the Hearthling idles, or the game lags and the update speed slows. The only way to simplify Hearthling behavior, then, is to make every single expensive behavior run quicker.

It’s possible to run the game at 2x speed, though, because instead of doing the calculations twice every update, it should be calculating once, pretending twice the time as passed. This way you can run the game at 2x speed without taking exactly twice as much CPU. So maybe what really needs to happen when Hearthlings get too expensive is to stagger some of the behavior. You could be clever, even, and run Hearthlings off screen through four times as much of an update every fourth update. You can’t see them, so why do they need to be as accurate as long as they’re working?

Unfortunately, that probably won’t work directly in multiplayer. Everything has to be consistent in multiplayer, or else the client will be showing a world completely different from the server and players get extremely confused. My solution’s far too random and centered on the player.

And Hearthlings can only do so much per single update calculation, no matter how much longer in “game time” you’re updating for.

3 Likes
12

Sure, draw loop is/should be separate from the update loop.
But the progression of the day time (that makes Hearthlings starve) should be a part of the update loop, not the draw loop.

If the update loop can’t keep up with the draw loop it needs to tell the draw-loop that.

What @Brendan is getting at is that the update-loop isn’t bug-free and is causing the issues, not that the update-loop is slow it’s bugged.

I always run my Stonehearth games at 2x or 4x the speed since it feels like everyone is walking around in slow motion otherwise. But that shouldn’t cause the AI to crash, it should just run at 4x the speed also (I have lots of CPU and memory to spare).

13

Adding the two cents of someone that is actually a “programmer”:
A “bug” is anything that is not originally intended by the engineer. You may often notice a bugs’ impact during runtime in many ways, but don’t judge too quickly. Villagers picking up items and dropping them and picking them up again is likely a bug. All villagers doing their jobs more and more rarely is most likely a performance problem with the main thread.

The main thread is executed a certain amount of times per second. It usually handles things like displays, calculations to entities, resources and pathfinding.
All things in a thread have to be done one after another. When there are many things to do, one thread takes longer for a complete run.
When a thread needs more calculation time to a point where it cannot fulfill the amount of runs per second it should, you get “fps drops and lags”.
Pathfinding takes up a lot of calculation and is triggered based on how often the main thread is run. It however also seems to calculate independently of the main thread whenever its triggered.

In general, it seems that the calculations for path finding and jobs take much too long under certain circumstances, slowing down other processes and ultimately leaving hearthlings without a job for longer times. The system simply cuts a calculation when a job takes up too much time or cannot handle the big calculations while it tries to keep up with other things.

5 Likes
14

Thanks for the more ‘experienced’ comment

So, for what i understand in your post is that, while some behaviors(the ones you
Mentioned) are likely bugs, but what we’re talking about(hearthlingns idling and not doing jobs) are most likely performance issues, the inactivity caused by a calculation taking too long and being abandoned, not that the calculation itself was incorrect or inappropriate

since the daytime timer(the one that causes starving and such) seem to run just fine, does that mean that the actions(eating) is too expensive for the computer when there’s a lot of hearthlings and the timer is inexpensive enough to run smoothly

15

That’s the gist of it, yes.

16

There’s 2 different problems - there’s definitely some major performance problems; but there’s also the “everything fails when the game struggles” bug.

The thing is that performance is something that can never be solved - It can only be improved. What I mean is that if they made the game 10 times faster (which is technically possible, but not something I’d consider easy or “economically viable”), then there’ll just be performance problems with 10 times as many hearthlings (or with hardware that’s 10 times slower, or…).

The “everything fails when the game struggles” bug can be solved (and to be perfectly honest, should never have existed).

To be even more honest; I suspect the game’s performance is so bad that (on the fastest computer you can buy) it can’t even handle 15 hearthlings 100% properly (hearthlings doing what they should as soon as they should); and the developers have deliberately caused the “everything starts to fail when the game struggles” bug in a misguided attempt to hide the symptoms of the performance problem. For between 15 to 20 hearthlings, hiding the symptoms works (sort of) because people don’t realise that it’s taking longer for hearthlings to get things done; and beyond that the symptoms get too large to hide and hearthlings start failing to do things completely.

1 Like
17

If you transfer $100 from one bank account to another, and the cash is removed from one bank account but the bank’s computers are struggling that day so the cash never arrives in the other bank account; would you say that’s just a performance problem (“LOL, the bank should buy faster computers!!”)?

Things happening slowly is a performance problem. Things failing to ever happen is a bug.

2 Likes
18

Pause the game and watch your hearthlings getting new business orders.

That worked well for me on performance intensive games.

1 Like
19

okay, honestly, i don’t think you’re really listening to us, but i’ll still say what i will

i hope you red this, but i’ll sum it up for you

what he said means is that if the process(a calculation for a hearthling) is taking too long either because it’s impossible or that the game is lagged too much to handle it
the game will intentionally dismiss the calculation as she considers it as a ‘failed’ calculation

what you are saying is that they should just keep on trying, but if she really keeps trying to calculate every single process that takes too long, that could mean her trying to calculate an impossible calculation in an infinite loop, and that would be a even worse situation for a game

bank acounts and systems are a different thing from this type of process, they don’t ‘have’ to be fast, they just need to be accurate, they have fail-safes that protects them from such problems, they’ll probably send a request, and wait for an answer, and send that they got an answer and if the calls match, they’ll consider a amount of cash as transfered,
this is digital, it’s not like there’s a dollar bill going through a pipe to come out the other side and being dumped in the middle, it only tranferes if bolth request and respond work properly, it won’t lose any money by any chance just by a lagging computer, because it can be as slow as they want, and there can’t be a unsolvable loop in a request and respond environment, it’s just numbers

in stonehearth, it’s a more complex environment, there ARE problems that the game AI can’t handle and will keep fail every time she tries, if you really want her to wait forever for a process just so your hearthling do a job, or you don’t want to pause the game for a little so the AI can figure things out, that meas that you want her to try and solve every impossible calculation that in thrown at her. meaning every single problem that the AI can’t handle will KEEP being calculated, you’ll just accumulate problems until the game is quite literally unplayable

she abandons such things so the game will actually run, and to not slow the entire game for that one or twenty calculations that she simply cannot solve, and as i said, they don’t fail to do it, you can pause the game and see them actually do stuff

2 Likes
20

Then I’m ready to give up on this game already.

If it can’t handle 20-30 hearthlings, how will it handle 50 + invaders + animals + generic wildlife?

How will the game handle pathing through buildings in a town of this size?

That’s from their kickstarter, this is what I backed. If you average 2 per house I count 90 Hearthlings living in that town, excluding the castle and guards.