The Modding Requests

excellent question… ideally, we would be able to start a game and just “pick and choose” which mods we want active during the game… whether that’s technically possible is another story…

it may be that that is only an option when beginning a new game, as opposed to continuing existing games (would certainly be easier to implement programatically)…

1 Like

It… depends on what kind of mod it is as well as how their save file structure is built. I haven’t touched this update so all I can make are some educated guesses.

First, let’s start with serialization of computer data, especially with mods.

A horribly bad metaphor for serialization of modular data

You live in a house. Big house. Many furniture things. And windows, doors, that kind of stuff. Now, for the sake of this metaphor, you will need to move all your things out of the house because of a Poltergeist. However, you are quite a lazy person and some things are very fragile, so you will need to hire companies around the world to do it.

Each company does the following: They get into your house, write a sticky note in their language (for example German, French, Spaghetti) which basically means “IOU item at this place”. They place the note and carry/take the thing apart.

Now your house is empty and you decide to go on vacations. You come back a year later and see that the ghost has died of boredom. After giving it a proper burial in an Indian graveyard, you decide that you want your stuff back. So you call all the companies and say “Hey. You’ve got stuff for my house!”

Each company comes, takes a look at the sticky posts they understand, get the part from their warehouse and replace it.

Ideally, you end up with the same house you got before. In any other case, you don’t.

Now, in this example, each company represents a mod. Mods can read all serialized data (I assume after quickly browsing _saved_variables I think it was called) but they can only really make sense of what they know (i.e. in their “language”).

If a mod is missing, the data will not be replaced - i.e. the entity knows “Well, I had a “Curse of the Pharaoh” entity attached to myself”, but nobody cares.about extra data (most likely).

Now we get into the beautiful world of case-by-case, which - again - is just guessing.

Case 1: resource file overrides

This is sound, models, graphics, pure data (like journal entries, but not entity data). Adding overrides for these or removing them should work fine. There might be some strange side-issues with collision boxes or similar (if they were cached/accessible, which is currently not the case), but assuming you have an almost-identical thing, it should work in any case. The engine is merely referencing to those things, i.e. it saves the path, not the resource itself. The engine’s virtual file system takes care of handling overrides.

Case 2: Scripts that modify behaviour

This would apply to most of my mods - different name generator, different landscape generator, add buffs, remove buffs, that kind of stuff. This will be a case-by-case too, but if coded properly, removing should work - adding might not if the script relied on some kind of “starting state”.

Case 3: Removal, addition and modification of entities, buffs, and other runtime-data-stuff

This is where it gets tricky. Assume that you had an override that changed something about an entity, or had a mixinto, or something like that. All entities that were already created had - I suppose! - copy of this data, not the plain data itself. So when the override/mixinto is gone, the old entities will still use the old data. New entities will use the current data.

If you add a new entity, everything should be OK. If you modify an entity, it may affect older ones in a negative way (as in, dead references, missing buffs, that sort of thing). Removing an entity completely will cause… problems.

How to handle missing or invalid references to things.

So assuming something did change, an entity, buff, data file, callback handler or whatever was removed and the current code wants to access it (because the old data told it to). There are… a few ways that this could be handled.

  • Ignore. This is a sort of Minecraft way, I think. Everything that can’t be assigned will be deleted. This is safe-ish, as it results in a somewhat playable state, but may have cascades where you end up with a lot deleted. Especially painful if it was a simple rename, and not a deletion per se.
  • Fail. Throw a big exception, back to the main menu and error screen. Your save is corrupted, for whatever reason, and cannot be loaded. This is the very safest way because it guarantees, in some way, that code and data are consistent.
  • Try. If there is some sort of meta-data, the code may repair old data. For example, mods have some sort of “update reference” callback, or a “update entity to current version” callback thing. This will greatly increase the necessary work for programmers, as they have to write routines that patch stuff from A to B. This can be annoying, up to impossible (as in, infeasible to do) for complex things that were rewritten.

In essence…

Mods that are simply overwriting models, sounds, or graphics and touch no code are very likely to be interchangeable or can be made interchangeable rather easily.

Mods that touch code, or live in code, are horribly case-by-case. It will depend a lot on what the engine can offer, what mod authors are willing to do and what their mod does. A mod that solely lives on a few events to do things will be almost unnoticed in most cases, where a total conversion mod will break the game hard.

In addition, the more a mod saves, the more… problematic this change can become. Terrain was mentioned and that’s one of those things: If you save the terrain color in the save, then changing the mod won’t change the colors and a new mod won’t have an effect on that.

If you re-choose the color upon loading, you get the proper color for the activated mods. However, this can also increase loading times…

The menu will likely become a third, independent lua state. Mods would not be loaded at this point - but rather become loaded as soon as the game itself is loading. That’s the way GMod is doing it, at least, and it’s pretty reasonable.

Every time you start a game you create a new lua state, load the mods, and run it. As soon as you quit the current game, the lua state is destroyed - and all mods neatly unloaded.


Hmm… as we mentioned sometime we should be able to choose what mods do we want to load for the game. If we leave all the mods in the folder, I think the best would be to choose the mods upon starting a new game, but not being able to turn them off later after saving. That way each savefile (of a particular new game) would only reference a constant collection of mods. (If the player don’t delete the folders in one or other way…)

Other problems I see now is that if a mod changes or updates, it has to be retrocompatible for the sake of those savefiles.

Anyway, it’s so problematic…

1 Like

Educated guess: The data is stored as json, or any kind of serialized lua. So effectively, lua is filling a table and then later reads it.

That means that, unless a mod decides to change this structure, backwards compatibility is easily achieved: You simply don’t change a thing. If you decide to rename something, that could possibly be bad, but maybe not entirely.

I mean, let’s get down to it: If it’s in lua, and especially if it’s a table, we can do metatable hacks. Ooh, the metatable hacks we can do.

Let’s assume for a second that a mod author isn’t satisfied anymore by having the field called “cropNum” and wants to call it “m_iCropNum” instead. Now he can either do, during de-serialization (i.e. loading) self:set_crop_num(data.cropNum or data.m_iCropNum) or he defines, at some points, something like SAVE_REDIRECT(data, 'cropNum', 'm_iCropNum') and then simply accesses self:set_crop_num(data.m_iCropNum).

The advantage is that you can chain those things and the metatable indexer would recursively look it up. You could even separate or merge attributes this way, if necessary. Ah, lots of fun can be had…


Yes, well, I was referring to a mod that adds some new entity to the game and later the author decides to delete that entity for whatever reason, or completely rename it. If the save file contained the entity it wouldn’t find it on the later version of the mod so it would probably throw an error.

Your solution clears my mind, now. Though it’s still better to keep things simple. All depends of what kind of mods are involved.

Well…what can I say? Thanks, I would have to say, for such an in-depth answer! :slight_smile: not too sure I can do much with this knowledge, but cool to know :slight_smile:

There’s ways around that too, although they are not as pretty as the other one.

As mentioned before, the manifest could define a bunch of helper functions. After all, you know which mod the entity belonged to (because of the mod:name format). That means you could have a function à la resolve_missing_entity(uri, data), which could then create (and return) a new entity, or nil if the entity was obsolete… Just a few ideas.

Of course this is more difficult for mod makers too, but we can write nice utilities to keep it at bay. In the very end, it’s all a question of if it’s worth it, not necessarily if it’s possible.