Theres a huge gap between 19 and 40 you could cap at like 30 or so. Also I have had about 30 citizens and no lag I only have a i7 7700k 3.6 ghz so I would say your good to go.
If you have plenty of ram you should be good to go, if it starts lagging after a while restart the game to clear the memory cache. I play on a less powerful PC and have 42 hearthlings and quite a complex town, it lags a lot sometimes but most of the time it is playable. Turning off light effects helps
It is all to do with your processor, the better it is the more hearthlings you can have, new i7s I’ve heard can handle up to 42 - 45 hearthlings provided you kill all the enemies immediately.
I have a 4 year old i7, I can get 28 but it isn’t very playable, 25 is about the upper limit for me.
RAM is the next limiting factor but if you have 16GB you’ll never max it out.
Make sure you turn lights and shadows off unless you have nice shiny graphics card
Just increase it in increments of 2 or 3 as you get that many hearthlings and see how your computer does, when it starts lagging a bit stop increasing the limit
Another thing that helps minimize lag is to make sure your hearthlings are picking up all the items and putting them in storage chests. Items on the floor or in storage zones use up more computing power.