New Item Structure, Overrides and Mixintos

Edit: Before anyone takes this the wrong way I ask the question in my second post not as a rage but as a non profosional coder hoping someone in the biz can explain how more complex with no apparent improved funcionality = improvments.

So this latest push to unstable broke everything in my mod, it all needs to be redone except for the model swaps. Why? because to do something as simple as a name swap I need to do an override of the whole file, so when all the changes came in I now need to redo all these files. I get its alpha and these things are bound to happen and I am changing my modding schedule because of it (Not going to try implementing anything until stable releases happen, which will really slow things down), but as the game becomes more complex I can see this happening a lot more.

Looking at the planed features and where we are, I can see a lot of things that are going to be brought in post release that are going to require a rewrite when they get implemented, that is until a way comes about to simply overwrite one line in a file with another.

3 Likes

So im calling out @sdee and the other programs on this.

Can you please explain to the modding community the changes to the crafted items, how is it better to go from a system where we need 2 .json for an item, both of which are directly called to needing 3 .json files, only one of which is called by the recipe and hiding the previously available details behind a luac? We upped the number of files, hid previously open information, and upped the number of different calls that needs to be made in different files. I know radiant doesn’t have to tell me anything, and this game is already quite mod frendly, I just would like given every single entry in every single mod now needs be redone by all the modders if you could explain how switching to a more complex, higher file count system will help us.

Whilst I’m not involved with any form of modding, I would just say that with everything up in the air in terms of development it seems somewhat unreasonable to expect the modding process as it is now, and as it was say in January, to be the way the process will be by the end of the year.

I obviously have no insight into the changes that were made, but I can only assume that changes are to be expected and breakages will likely be common throughout the alpha and beyond.

I agree with Geoffers. I haven’t looked into what has been changed but if they switched something that majorly, I’m sure there’s a reason. They’re not “hiding” anything behind luac either, it’s merely… a bit difficult to get to. Moving things from data to code sounds a bit weird though, I’m sure there’s more to that.

That being said, I can’t currently check what has been changed, or why - but I’ll make sure to do that. Rest assured that this is not the first change that “broke” stuff. In general, if you’re complaining about too much effort to update your mod within a single update, you did something wrong.

In this case: To change or add things, use mixintos, which allow you to only adjust the values you need. It’s obvious that an override will break way more often. In fact, I wouldn’t use overrides at all, as they’re breaking mod compatibility with other mods, too.

mixintos are the solution you are looking for and they have been around since the introduction of override. There’s a decent blog post about them, too.

2 Likes

Right. So let’s get down to business.

The game differentiates between the object itself (a placed chair) and so called proxies (now called “icons”… iconic things are icons right?).

Tʜᴇ ᴘᴀsᴛ

The previous system therefore based on two entity definitions, let’s use an imaginary example of stonehearth:table. The placed, usable table in the world was stonehearth:table, the “proxy” (which is carried around and stored in stockpiles) was stonehearth:table_proxy. That means to create any furniture (or anything that can be placed), you had to effectively create two entity definitions (two jsons, one per definition) - and they both contained about the same thing, the definition for said entity. They may had different models and small definition changes (i.e. the placed one said “I proxy to stonehearth:table_proxy” and the proxy had “upon placement, I turn into stonehearth:table”) but they were roughly the same.

That’s redundancy and redundancy is bad. It means that whenever I change something in stonehearth:table, I also need to adjust stonehearth:table_proxy, or weird things happen. It’s also clustering up mods that search for entity definitions (for example spawn menu mods had to explicitly filter proxy entities). It doesn’t make a lot of sense to have two different entities that describe the same thing in the end, right?

Note for the advanced: It means that whenever an entity was actually moved, the old one was destroyed, a proxy was created, and upon placing the proxy was removed and the new entity was created. I’m not sure how/if there’s any serialization going on now that we have it or if this behaviour changed, but I somewhat doubt it - so the placed object and the original one are very likely different entities.

Tʜᴇ ɴᴏᴡ

This all just experienced guesswork from taking a look at the files and changes, I haven’t really looked closely - but I would do it this way, which I figure is as good as any guess.

Instead of two definitions, there’s just one now, stonehearth:table. It contains three jsons: The main entity json (which every entity has, table.json), the “iconic” version (table_iconic.json) and the “ghost” version (table_ghost.json). Note that the naming can be different, see below.

The main entity definition (table.json)

The main entity definition mixins the ghost definition ("mixins" : "file(table_ghost.json)"), which means that it takes over all settings from the ghost file. In addition, it defines a few things on its own that the ghost entity does not need, such as the stonehearth:entity_forms component and a collision shape. It defines all components actually, since ghost entities are not allowed components (at least, not over JSON - potentially to avoid mistakes in entity definitions).

The ghost “entity definition” (table_ghsot.json)

I’m not sure how real this entity definition is, seeing as it’s not defined in the manifest, yet it is used as such.

To cut to the chase, this file is about equivalent to your old table.json. A few things can be removed, such as the collision regions (unless you want your ghosts to be solid). All components should be moved to the new table.json however, as ghost and iconic components are not allowed to have these (unless I read this code wrong).

I would hazard a guess that this is the “placeholder” that we see in the world upon placing something down (hence “ghost”?), therefore it makes sense that name, description and all that stuff is already available.

The iconic “entity definition” (table_iconic.json)

This is really just the old proxy. It usually mixins stonehearth:mixins:item_properties which defines most things that are required for placeable items, defines a model and that’s it.

What you described as “hiding stuff in the luac” is actually doing you a favour. Iconic entities will copy over display_name, description, icon and portrait of the “real” entity - therefore you don’t have to define all these things yourself. That means that the iconic entity definition (aka the “proxy definition”) only contains the very bare bones.

Conclusion

This horrible, horrible change is actually making your life easier if you let it be. Instead of updating two files (and keeping them synched) you can now use three independently. Likely, you will touch table_icon.json once or twice, table_ghost.json whenever the description/name changes and table.json a lot - whenever your entity needs a change.

Note that the file names here are assuming the current naming schema, but it’s not fixed: The stonehearth:entity_forms component (defined in the main entity definition) sets these files. You can share these, of course and you could even use four or more files (for example, use a “scaffold” json that both ghost and normal item mixin, instead of the main definition mixin the ghost one). That might not be a too uncommon case for some things - for example, you could attach a sparkly effect to your ghost items, but not the placed one, which isn’t possible otherwise with our current mixin(to) system.

7 Likes

@Wombat85, excellent question, and @RepeatPan, excellent answer. :slight_smile:

All I have to add to RepeatPan’s description is a confirmation that yes, the ghost item is a description of the semi-translucent placeholder that you see in the world when you order an item placed/moved but before a worker actually relocates the actual item. The ghost item used to be created in a kind of messy/fragile way, in lua, and now that it has its own json, there is a clear inheritance from it (all the visible, but non-interactive properties) to the regular item (all the interactive properties) to the miniature version of the item (for use in carrying). The fact that we have three items that are related but redundant used to bother Tony a lot, so this change represents him cleaning house, so to speak.

I’m sorry that changing it means more work for you right now, as a mod author, but if your mod is something you plan to update going forward, it means much less work for you in the future. :slight_smile:

Edit: Updated the post title to reflect the direction of the thread

4 Likes

Thanks for that, as I convert more things I am seeing how its beneficial to be this way. I was just kinda fuming when I saw it becuase I had just added close to 30 new items to the starhearth mod and then was like “omg they changed it and now it all needs to be redone”. A little primer that such a radical change was coming mid release implementation would have been nice, but I do appreciate that changes like this are to be expected at such an early stage. I still have one question though:

How can you adjust existing values in a json with a mixinto, I have never been able to get them to change a value of an existing entry, only add new ones. For example changing the name has required copying the file, editing the name and using an override because trying to mixin the new name does nothing.
Thanks again.

I dont think you can. Overrides do overwrite everything. While mixinto does not overwrite.
It would be really useful tho.

1 Like

You’re… doing it wrong? Mixintos by definition override stuff, except arrays. In arrays they only add, properties they always overwrite. They’ve been doing that for quite some time and usually fine, except that little ordering mismatch we’ve once had (/still have?) with mixins and mixintos (where mixintos are applied after the files have been mixed in, therefore rendered useless or something like that).

##manifest.json

{
“info” : { “name” : “¯\(ツ)/¯” },

“mixintos” : {
“stonehearth:small_oak_tree” : “file(shrug.json)”,
“stonehearth:medium_oak_tree” : “file(shrug.json)”,
“stonehearth:large_oak_tree” : “file(shrug.json)”
}
}

##shrug.json

{
“components” : {
“unit_info” : {
“name” : “Just a tree vOv”
}
}
}

Result

(also, @sdee, we could use some UTF8 support here!)

Right! If mixintos don’t work as described here: http://stonehearth.net/2014/01/14/desktop-tuesday-a-peek-inside-the-modding-api/

Then definitely let us know. :slight_smile: With specifics!

@RepeatPan, it’s on the list! But the list is very long. :wink:

2 Likes

It was mixing in stuff. Thats how I did understand it. overwrite does overwrite.
And mixinto does mixing. If you mix water and orange juice, you get watered down orange juice. Not water. Ah well. Better late than never :blush:

1 Like

mixintos don’t quite work like that. Basically you can imagine it this way, if we want to use bad analogies:

With overrides, you’re not just exchanging the content of the glass, you’re exchanging the whole glass. Mixintos are kinda like filling the glass with molten lead; it will just dispense all the water out of it.

So overrides overwrite complete files, while mixintos only replace and add parts.

2 Likes

For probably this reason, @Tom has been looking for another name for mixintos. Suggestions? Moltenglassintos?

1 Like

alternative naming convention for “mixintos”:

  • supplements
  • addins
  • adjustments
  • gobbledygook
3 Likes

It really depends, seeing as you do mix stuff into arrays, the name fits. That it overwrites properties in doing so… probably not.

I mean, if we want to go nitpicking, then there would be three categories: replacements (the current overrides), mixintos (adds stuff to arrays and, if not existent, properties) and overrides (which sets arrays/objects completely and overwrites properties). But I’m not a huge fan of having three different things because overrides and mixintos could be used in the same breath, so there would need to be something like “override+mixinto” that does both.

2 Likes

Cheers @RepeatPan the different syntax between mixintos and overrides was why I hadn’t got it to work, that example helped immensely.

1 Like

How about:
Mixins: Pull-ins (or Pull-intos?)
Mixintos: Push-ins (or Push-intos?)

1 Like