Is it possible to override a Service?

As I have an issue with my Hearthlings getting stuck in the pits they dig out, I’m trying to make it so that ladders either cost nothing, or use stone resources rather than wood. So far I’ve been unsuccessful at making ladders free, but I did find that if I change the return value of function LadderBuilder:getMaterial() to “stone resource”, I can, indeed, make stone ladders.

The issue now is packaging this into a mod. I’ve been able to override things in the ai/actions folder (like build_ladder_action.luac) without a problem by putting this in my manifest.json:

   "overrides": {
      "stonehearth/ai/actions/build_ladder_adjacent_action.luac" : "file(ai/actions/build_ladder_adjacent_action.luac)"
   }

And that was fine, the override worked. However, the file that contains the LadderBuilder:getMaterial() function is a service. I tried to override it, but…

   "overrides": {
      "stonehearth/services/server/build/ladder_builder.luac" : "file(services/server/build/ladder_builder.luac)"
   }

This does not work. Even copy-pasting the minified LUAC from the original Stonehearth.smod into my (new) file didn’t work. Stonehearth gives me an error: “attempt to call local (obj): a table value”. Modifying the original file inside Stonehearth.smod file works, but overriding it through manifest.json results in that error.

Overriding lua files is officially frowned upon (while I can’t cite any public source, let me tell you that I’ve had a pretty clear response from Radiant in a PM stating that there will be no proper support for overriding lua files… the fact that it works right now is more or less an “accident”).

Because I usually just tell people that it’s bad but never go in an in-depth explanation, let’s do that.

Why overrides-ing lua files is bad

  1. Lua files have no fixed URI. They have no aliases attached to them and are usually not referred to by them (for example, all require functions explicitly do not allow you to specify the path because lua usually references other lua scripts with periods, not with (back-)slashes.)
  2. Stonehearth supports two file extensions for lua: .luac and .lua. As far as I know, there is not a guarantee which file gets called if both are existent.
  3. As with all overrides, overrides can only be done by one mod and should be used very sparingly. If two mods override the same file, it is not defined what happens (as far as I know there won’t be an obvious error, but only one mod will succeed in actually getting its code in place).

What are the alternatives?

Personal recommendation: Don’t override. If there really is no API for what you need, then use Jelly. Jelly is an independent, third-party mod (written by me and by now a few contributors) that aims to solve these issues by expanding the existing API. As with most things, this comes with advantages and disadvantages:

Advantages:

  • Mods don’t need to override lua files anymore
  • Multiple mods can access the same functionality because it’s in an isolated API
  • Because Jelly offers a fixed API, you rarely will notice version updates. Even if the API/services that Jelly uses change, this change will rarely make it to the “client API” (i.e. you). Your mods will need less maintenance.
  • For some special things, Jelly offers even JSON data that can be mixinto’d or overwritten instead. This is an easier/safer approach to handle things.
  • It’s open source - people are invited to make their own contributions.

Disadvantages:

  • Your mod will require that users install a recent version of Jelly, therefore creating a dependency.
  • Because Jelly is independent, it may take some while until it is updated for a new unstable version. Usually this happens within a few days. In-between, Jelly and some of its mods might not work, although this is rather rare.
  • Although most of the code should be documented and there’s a wiki for it, some features could use some documentation love.

In my opinion, the advantages outweigh the disadvantages by a lot, but I’m probably biased, too.


Back to this case: Could Jelly help? Currently, no. The way I see it, there’s two options:

  • You/I create an API for Jelly and push it, giving you access to override the costs for ladders (or adding new ladders, or or or…). I wouldn’t recommend this way, as this is the AI we’re talking about, so…
  • You simply create a new AI action (with higher priority?) that has the same name, mixinto it into the human-mixin and have that one used instead.

For free ladders, I suppose you would need to see what uses this action (I guess it’s a chain like get_item.take_item.move_to.build_ladder).

As for the AI part, I guess Jelly could use some mixinto magic love (maybe something like "!-key at the start of an entry in special areas means that it and key have to be removed) in the AI component that would allow removing things… Not sure what Radiant’s current approach is on improved mixintos (I would guess that because we haven’t heard anything for a long time and there have never been fixed plans, there isn’t much to tell us though).

7 Likes

Thank you, @RepeatPan! I’m looking into Jelly. I realize now the beauty of script extenders, like the ones Silverlock makes/maintains for The Elder Scrolls games. If mods rely on a script extender, then after changing a certain part of the game’s API, only the script extender needs to get updated, rather than every SH mod that used that API.

Regarding free ladders: I did try to remove “get item” from the action chain that’s responsible for creating ladders, and I also commented out the requirement that the Hearthling must be holding the required material for the ladder to build. Unfortunately, I receive many errors – something about some function not returning the right value. As soon as I figure out how to log things, as per a forum post somewhere I’ve seen before, I’ll get back into it.

For now, I will just mod my version of Stonehearth.smod, with a backup handy. I have no idea how to create a new action for my version of ladders yet, and how to non-destructively (i.e. without using overrides) inject it into Stonehearth. I’ve seen the blog post about mixins and mixintos, but the concept still eludes me. I’m fresh off the Javascript full-stack development boat, so Lua is very new to me. Once I sort it out, then I will be able to create publishable mods.

Thanks again for the very detailed answer.

Let’s take a look at mixins and mixintos from the point of view of JSON structures. mixins are basically a request “Please shove that other JSON into me”, while mixintos are a request to be shoved into some other JSON. The rules are basically that mixintos will always add or replace (in that order); Array elements are always added, object keys are always overwritten (if they exist) or added (if they don’t): For a vague C# implementation of what mixintos likely do, see Jofferson’s mergeTokens function.

Mixins and mixintos are really the same; the only difference is the definition. JSONs (or only entities, I am not certain) can request mixins (kind of like includes in other languages) which are then mixed into the file; while manifests can define mixintos to mix some JSON into another JSON (you can think of tricking the target to mixin the file).

The whole mixin/mixinto business has absolutely nothing to do with lua, it’s just a way to merge JSONs (and therefore, data).

2 Likes

That makes sense. So mixins are, like you said, includes, while mixintos are… “outcludes”, both of which you can aim at a specific resource / entity that is defined by a JSON file, which in turn links to scripts or other assets. Tying this back to overriding, when you use overrides, you’re basically changing “raw code” instead of programmatically re-pointing a resource to something else. That’s why only one mod can really do it, multiple mods overriding the same file would cause confusion among the mods. I think I get it.

Yes and not really.

SH operates on a virtual filesystem (of some sorts); after all they need to in order to be able to support archives (.smod) and directories (plain folders) in the same way. By overriding, you basically state “If somebody tries to load resource X, load Y instead”. If two mods do that, you’ll end up with “If somebody tries to load resource X, load Y instead” and “If somebody tries to load resource X, load Z instead” - in which case Z will be loaded rather than X (or Y). Overrides can be used to replace virtually any file, but its effectiveness depends heavily on the file (as said before, lua/luac could be overridden, but mustn’t, and I think that overriding any horde-element will simply be ignored).

Mixins are really like includes, and mixintos are injections of some sort. They can only operate on JSON (“data”), not on any other file type. To patch lua or JS, you need to use monkey patching (or use the API where available), while HTML can be modified with JS, CSS can be simply extended (or modified with JS too). For images, animations and other things, overrides are the only way to go.

Wow, thank you very much. I was beginning to think that I shouldn’t override anything, ever.

I’ve been scouring the forums for this sort of information. Is there any central place where I can find dev information like this? For instance, how can I edit Stonehearth’s HTML/JS? Simply running it in Chrome didn’t do the trick, do I need to run it through a server? I don’t want to clog up the thread with these questions, though, so I’m mostly curious if there’s any place I should be looking to find these sort of answers.

Nope. There’s a few people that know a lot about the internal workings (by reverse engineering, trial and error and watching a few streams) and a few official documents (like the mixinto/override thread), but that’s about it. Because the whole process is very much in flux, only bits are official, some more bits are semi-official (like the documentation they ship with Stonehearth, whose completion varies).

The process is a bit like this: JS is included with a <script> reference in the root UI document, the LESS is loaded/unloaded dynamically with the views (I think ember handles that, so I have no real clue. It just works).

So, if you want to edit existing assets, you either just write your own JS (that gets included at semi-random points, so you might want to wait for the mods-loaded-event… not sure if/when that one fires in JS though) and do the stuff you usually do (jQuery and Ember are at your feet) that does whatever you want or you override the existing ones.

Preferably do JavaScript magic though, that way you can keep mod compatibility. If there’s some specific, somewhat common change you want to do, feel free to propose it to Jelly.

(I’m not really satisfied with Jelly’s JavaScript part currently - overriding files is a hassle, especially if it’s done for just one function. It creates much more maintenance than necessary).

1 Like

I hope it is not true anymore :glum:

I need to overwrite farming_service.lua to add new functionality and I found a way to implement it. Then I was thrown such a catastrophe straight into my face. Not nice.

Well, lots of things have changed in the past two years, and I’m not in the loop.

There’s ways of overriding lua functionality (not files) from mods, there has been since day one (with more or less hacks involved). I think I’ve talked about it somewhere else, and I think I’ve done it in Frostfeast ('15 at the very least?), and maybe some other mods that I’ve worked with.

The basic idea is the following: Since most of that stuff are classes, and those classes are usually returned/somewhere registered in their .lua, you can load them. A class in SH is nothing different than a special table (since in lua, everything’s that’s somewhat structured is a table when you get down to it). You just load the file, get a reference to the class, mess around with the class, and you’re done.

There’s two drawbacks: First, you cannot easily access local variables/functions, which some classes might do. Second, this needs to be done before the class is every instantiated, otherwise stuff goes bork. This is usually not a problem.

So, if you want to override stuff, off you go with monkey patching.

Ugh. It seems overly complicated for my not-much-beyond-beginner LUA skills because what I need to code depends heavily on local variables.

Thanks for clarification, though.

You could always create a new service, MyFarmingService, which would be a copy of the vanilla farming service except with your modifications, and have the game use that instead of the regular service.

1 Like

Seems like a solution, but I have no idea how to perform such a swap. I would appreciate a simple explanation.

Taken from Frostfeast:

function frostfeast:_three_headed_monkeys()
   local victim = radiant.mods.require('stonehearth.services.server.game_master.controllers.encounters.script_encounter')
   victim.start = function(self, ctx, info)
      if info.script then
         local script = radiant.create_controller(info.script, ctx, info.data)
         self._sv.script = script
         radiant.verify(script, "script encounter could not find script %s", info.script)
         script:start(ctx, info.data)
      end

      -- Override: Allow the script to define the next campaign
      if not info.script_triggers_next_encounter then
         ctx.arc:trigger_next_encounter(ctx)
      end
   end
end

function frostfeast:_on_required_loaded()
   self:_three_headed_monkeys()
end

radiant.events.listen(radiant, 'radiant:required_loaded', frostfeast, frostfeast._on_required_loaded)

This is monkey-patching one function in the game master controller (start). If you just need to do a few functions and not a complete rewrite of the service (which is possible too, by just looping over your table and assigning it to victim - FF’15 did it that way), then that should work.

Not sure if there’s a better way yet, but I couldn’t come up with anything nicer. I think technically overrides work now, but… I’m not sure how to feel about it.

I have two more questions.

1] If I understand correctly this will just replace a function so if I want to inject new local variables I am safe to do that while accessing local variables created by other functions is a problem. The functions I need to alter use only global variables as input so this would not be a problem, right?

2] Stupid one: how do I find function and service names? I tried stonehearth.services.server.farming.farming_service and referenced functions by both period and colon and it didn’t work. Farming call handler uses simply stonehearth.farming but it doesn’t work neither.

It’s replacing a class method with some other function that you define. The scope of the function isn’t touched, e.g. if you define local variables, then those are accessible by your function (but your function does not have access to local variables of the original one). In addition, self refers to the instance itself, so saving stuff there is possibly.

Should you wish to keep the original around, local old = victim.start and then just call old(self, ctx, info) (in this example).

The “name” is just the file path containing the service, without extension, separated with dots instead of slashes. If you refer to stonehearth.farming, the service is already initialized and it might be a bit too late to mess around with things, depending on what you want to do.

So my .lua file looks like this and doesn’t work. Now I wonder whether it is because of service load order or me doing something wrong.

plant_lore = {}

function plant_lore:_biome_crops()
   local victim = radiant.mods.require('stonehearth.services.server.farming.farming_service')

victim:_load_initial_crops = function()
   -- modified function goes here
end

victim:_get_crop_list = function(session)
   --- modified function goes here
end

end

function plant_lore:_on_required_loaded()
   self:_biome_crops()
end

radiant.events.listen(radiant, 'radiant:required_loaded', plant_lore, plant_lore._on_required_loaded)

return plant_lore

The service you modify in Frostfeast loads earlier than farming service so loading order should not be a problem.

Your problem is the colon. It’s victim._load_intial_crops, not victim:_load_initial_crops. That’s something that should show up in the error log, however, as it’s a syntax error.

After replacing colons with dots it seems to work because I get an error even if I copy-paste the original function.

Here’s the original function which caused the error while defining local crop_list in line 3.

victim._get_crop_list = function(session)
   local player_id = session.player_id
   local crop_list = self._data.player_crops[player_id]
   if not crop_list then
      -- xxx: look this up from the player info when that is avaiable
      local kingdom = stonehearth.population:get_population(player_id)
                                                :get_kingdom()

      -- start out with the default crops for this player's kingdom.
      crop_list = {}
      local all_crops = self._all_crops
      local kingdom_crops = self._initial_crops[kingdom]
      if kingdom_crops and all_crops then
         for key, crop in pairs(all_crops) do
            crop_list[key] = {
               crop_key = key,
               crop_type = crop.crop_type,
               crop_info = self:get_crop_details(crop.crop_type),
               crop_level_requirement = crop.level_requirement,
               ordinal = crop.ordinal,
               initial_crop = kingdom_crops[key]
            }
         end
      end
      self._data.player_crops[player_id] = crop_list
   end
   return crop_list
end

The problem is self._data is created by initialize() function which I don’t want to alter but in order to access it I have to add it to .lua as well.

victim.initialize = function()
   self._data = {
      -- we probably want different crop inventories per town, but right now there's
      -- at most one town per player (and only 1 player.  ha!).  until we get that
      -- straightend out, let's just assume a player can use all the crops in all
      -- his towns
      player_crops = {}
   }

   self.__saved_variables = radiant.create_datastore(self._data)
   self.__saved_variables:mark_changed()

end

And now I have no idea what to try next because self.__saved_variables is not called anywhere else in farming_service.lua. Error log is the same as above.

Crash course in lua: function foo:bar() end (colon) is equivalent to function foo.bar(self) end (period). Your session isn’t session, it’s self. Therefore, your function definition ought to be victim._get_crop_list = function(self, session) because the original function was/is called like someObject:_get_crop_list(session), therefore the first parameter is self (the object we’re operating on; in reality, lua translates this syntax sugar to someObject._get_crop_list(someObject, session)).

The same goes for _initialize. Something to keep in mind is that __init (not __initialize), aka the constructor of a class, cannot be changed this way (there’s different ways, but if the object was already created, it’s tough luck).

Addendum: You do not need to mess around with _initialize at all just to access self. Fix your function definition and it ought to work.

4 Likes