Jofferson 1.5.3 - Manifest Validator Tool

Jofferson (not to be mistaken with “Geoffers”) is a tool designed to help you with manifests, aliases, overrides and mixintos.


The main interface. Red means that there’s something wrong with this mod, resource or reference.


The resource inspection window. It shows what points to this resource and what this resource (likely) points to.

Features

  • Scans all folders for valid manifests
  • Creates a list of all references inside a manifest. This currently includes aliases, overrides and mixintos.
  • If a reference points to a json file, said json file is (kinda “stupid”) parsed for other references too.
  • Able to list all references to and from a resource.
  • Filter functions to quickly search for stuff.
  • Able to deal with directories and smod files. Even mixed!
  • Context menus to copy (absolute) paths, open files or open folders directly.
  • Error reporting window that lists all errors that were encountered while attempting to parse this mod.

What it’s useful for

  • Get an insight into a mod’s aliases, overwrites and mixintos
  • Validate that all your file()s are properly formatted
  • Find missing resources
  • It’s also a kinda nice wallpaper.

Requirements

The usual .NET Framework 4.5. This time WPF is also a requirement, but that has been part of Windows since Vista - so you most likely already have it.

Download

Repository on GitHub
Latest release download on GitHub


This is mainly a debugging/validation tool. Since I’ve seen that it might not be obvious how file() or mixintos/overrides work, there’s lots of errors that could arise.

What do you think should Jofferson offer as functionality? Should it go beyond the normal entity/jsons and parse the UI/components/functions too (only in the manifest - i.e. not recursive like it does the .jsons)?

11 Likes

excellent! :+1:

this, will be immensely helpful…

hmmm… inferior? sounds about riiight

3 Likes

Right, first release, link’s in the first post. There’s still lots of things that are kinda counter-intuitive and could be improved, including lots of ContextMenus. This is the first thing I’ve done with WPF, soo… yeah.

Currently, the exe has to be inside /mods (or at least, it needs to be executed there - either via batch file or shortcut). Clicking on a mod will show its resources (all files that are, somehow, linked from the manifest.json). Double-clicking a resource or clicking on a link opens a new window that shows some more information about this resource. Double-clicking an (outgoing) reference opens the resource it links to; clicking on an incoming reference opens the resource the link was defined in.

It’s horribly space-consuming with long path names, I’m not too certain how I’m going to fix that in a pretty way. Perhaps some sort of multi-line listbox entry… We’ll see!

Sooo… what if it’s highlighted green in Jofferson, but SH still pops out an error say it can’t find it?

Then SH is obviously broken.

Jofferson won’t (and can’t) parse everything. It won’t catch, for example, if you try to create something in lua that doesn’t exist.

It really depends - what is the error?


11:05:

#Update 1.1

Update time. Not much, but since @Miturion found out what happens when you feed it invalid JSONs, I should kind of push that one out before heading to bed.

Changes, more or less:

  • Ability to copy mods, resources and reference paths by using CTRL-C or the context menu. References will always be copied as absolute paths, i.e. it’s not caring for aliases.
  • Added error reporter. Just click on the little number in the left column to see all errors that were encountered. This logging is far from complete but should deal with the most common cases right now.
  • Better formatting: Mixintos and overrides are now three lines tall - to make it better visible. In the resource window, however, they will be displayed context-based: Outgoing references won’t have the source (because, well, that’s obvious) and incoming references won’t have the destination.
resources | could not find resource for override:ascendancy while processing mixintos for robo.  ignoring.

Which, if I understand it correctly(the error messages can be a bit vague), means that it can’t find what the it either can’t find the resource that override:ascendancy is pointing to, or it can’t find alias override:ascendancy.

Though on an interesting note, that particular override is the only one which doesn’t appear in Jofferson.

Nevermind, I figured it out, I forgot to include the mod name in the override(stupid me). Which I guess is why it didn’t appear in Jofferson. Sorry, I really need to break the habit of doing this sort of thing until 3-5am >.<;

3 Likes

It didn’t appear in Jofferson 1.0, but it definitely got noticed. In 1.1, "stonehearth/..." : "not:existant" throws an error. I probably should still display it, however, so that’s a good call.

Meanwhile, I start to understand how this whole WPF stuff works. The next version will have two buttons for normal explorer files (i.e. non-zipped archives): Open resource and open resource folder. That way you can quickly open stuff and edit errors.


Update 1.2

Update time.

  • Added context menus and CTRL-C functionality to items.
  • Mods just support copying their name.
  • Resources have three options: Copy their absolute path, open the file (with your default program) or open the file location. For obvious reasons, this doesn’t work with smod files, although I could imagine adding 7z support for that.
  • References by default copy origin : target (i.e. something that could, with quotes, be used in mixintos/overrides). For each of the parts (definition, origin and target) its’ possible to copy the location, open them or their folders.
  • Errors cannot be copied yet. Working on that.
  • “Open Folder”/“Open File” functions for the resource window - just like the context menu. Does its job pretty well.
  • The error link is now disabled if there are no errors.

Coming up next: Fixing the aliases and perhaps some sort of refresh feature.

3 Likes

Jofferson 1.3 has been released!

This was a larger update, inspired by @Relyss’ recent report of mixinto arrays not working. I’ve taken the liberty to make a minor update instead of a simple bugfix release.

Changes:

  • Semi-auto refresh: Upon detecting any file change to a .json inside your mods folder, Jefferson will ask whether it should reload all data or not. Upon reloading, it will try to display the same mod/resource/reference again - useful for debugging!
  • Arrays for mixintos are now supported (i.e. "stonehearth:small_oak_tree" : [ "my_mod:something" ])
  • Added what I like to call “mixinto preview”. It can show you what the final json might look like after all mixintos have been processed. Note that, just like in the game, the order is arbitrary and therefore might not be correct/representative when multiple mods mixinto the same area of a file.
    (Yes, this function can be abused to view json files inside .smods)
  • Incoming references are now displayed in the main window too.
  • References are now ordered by validity, then name. That means invalid stuff is at the top and equal-valid stuff is sorted after their display name, ascending.
  • Fixed files in .smod archives sometimes not being found/falsely reported as missing
  • Any error causes a mod to be invalidated now (i.e. appear red), even when all resources and aliases would be valid
  • strict type checking: If a type in the manifest is encountered that was not expected an error is logged
  • Fixed error reporting window having an useless column (and therefore wasting space)
  • ui is now parsed too. This includes html, js and less. Note that none of these files are searched for other references.
  • functions is now parsed too. They are treated and displayed as aliases, which they kind of are, I guess.
  • Components are parsed too. Just like functions.
  • Keys in objects are now considered as links too. That means that component parsing, for example, should work now.
  • Improved heuristic to determine what is considered a “link”. It should now be able to deal with missing aliases for mods that exist
  • Missing aliases will now show up in the mod’s window. Their location will be the same as their name - because, well, we don’t know what they are supposed to point to, right…
  • Probably a lot more that I’ve simply forgot.

The design for certain parts is still terrible (I’m still learning this whole WPF stuff after all…!) but I think this update is a great step towards a (more usable) tool. I like it. After all, I found a few bugs in my own mods now that I probably would not have caught otherwise.

Note that modifications to *.smod files are currently not possible while Jofferson is running (as it read-locks the files). That’s something that might will change in the future, along with a more stable refreshing function (for example, adding a previously missing, non-json is not processed as refresh opportunity).

On my future todo list is a more useful JSON view (that resolves aliases and file() commands to the absolute paths - something I wanted to do with this release but as always forgot last minute) but I’m not in a hurry. Jelly and the Stonehearth Kitchen take priority right now - after playing OpenTTD…

4 Likes

Did you update the download somewhere?

1 Like

Woops, my bad. I didn’t save the updated zip. The zip in the first post is now updated.

A-Are you sure? I don’t think so… ^^U
I’m downloading it several times and executing it but it shows the old 1.2.0.0 version and doesn’t read my mixintos nor anything you mentioned…

Well this is getting ridiculous - Dropbox wasn’t running. Now. For real.

I so need to finish the kitchen.

#1.4.0 is here.

With regard of the recent change of model_variants from an object to an array, I’ve realised that Jofferson had code in place to detect those changes - and break in the debugger.

Without a debugger, however, the code was optimized away. Now it’s back and it’s here to stay.

  • Different mixinto types are detected and reported. If you try to mixinto something wrongly (i.e. attempting to set something to a string instead of a number, or an object instead of an array), your mod will receive an error entry including the “path” to the entry. I’m not sure how useful that path is - time will tell. At least you’ll get some hint about where you’re wrong. A broken mixinto will also be displayed as invalid.
    Exceptions: If either value is null or both are either integer or float, they’re accepted.
  • Errors are now more obvious. The link is red and bold. That should be obvious enough.
  • Missing JSON files can no longer be inspected. They just displayed “null” anyway.
  • Missing manifests are now properly recorded as error.
  • Broken JSON causes resources to invalidate. Also, new fancy error reporting for this, although a bit redundant in some cases.
  • Failed mixintos are reported too. If you try to mixinto a broken JSON into another file, it triggers (another) error for the broken mixinto.
2 Likes

I like this update of Jofferson n_n :+1:
Before, the program would not open if I had some problem with the files, even the slightly missing comma (and I wouldn’t know which file caused the problem).
And know it can tell you exactly what part of the mixinto gives the problem and why.
Thank you for your hard work.

1.4.1

  • Fixed file pointing to invalid files (i.e. not-existing files) not being properly handled. (Thanks @Avairian!)
  • Introduced “fake resources” to deal with non-existing things like stonehearth:customization_variants. Fake resources will be displayed as existing, but non-valid, i.e. things that point to them will be valid, but the mod that contains them is not. Seeing as Stonehearth has a bunch of invalid things already, I don’t think that’s too much of an issue :wink:

1.5.1

  • Added what I call the “resolved” JSON view. In this third column, Jofferson attempts to “resolve” aliases and file paths to the relative paths.
  • The JSON view’s columns are now resizable. Also, dashed lines have been replaced with more proper lines. I’m slowly getting the hang of this WPF stuff. Emphasis on slowly.
  • client_init_script and server_init_script are now checked too.
  • Fixed several cases where an invalid file() would not be picked up as error/wrong reference.
  • Maybe slightly improved loading times. Stonehearth is still very heavy on the CPU - I’m not going to fix this until 2.0 (which will utilise fancy async, multi-threaded buzzword features to deal with this)
  • Fixed mixinto bug. Jofferson is now properly aligned with Alpha 2 Release 53’s/pre-R48’s mixinto behaviour.

I have some plans for Jofferson, mainly to get rid of the (ugly) JSON view that we have right now. Instead, I would like to have a rich-text like thing where references are highlighted as wrong/correct, where one can immediately open them by clicking on them, or checking what aliases they have, what they resolve to and more.

Jofferson, while great for checking smaller things, seems to hit its limits with bigger mods. Of course, one issue will always be it can impossible know what string is referring to a file or alias in all but the manifest’s cases. I’m not sure how to tackle this, except with red flashing warning lights and a clever heuristic - cleverer and better written than the current one in any case.

1 Like

We’re open source. The code is, as mentioned, pretty… awkward to say at least. It was the first time I’ve worked with WPF so a lot in that regard has been hacked together, quite literally.

1 Like

Woop woop. Personally, I’m not quite satisfied… I believe the quotation marks should not be part of the link too. But it’s a start, I guess. In other news, I found out that WPF actually has very decent 3D graphics support, including cameras, animations and all that stuff. It will probably more than enough to have a solid rendering system.

I think sticking with Unity3D would have the advantage of potential cross-platform support. However, as I don’t own a Mac and very rarely use Linux, I don’t think I could do much support on any of those platforms anyway. That plus Stonehearth currently is Windows only anyway, which plays nicely into my laziness…

My future is more or less sorted out, I believe, so I think I’ll wrap some things up. On that list would be a proper rewrite of Jofferson. That means proper variable naming (I’m not quite sure how to call private stuff in .NET yet, but I’ll go with _foo and void blorp() - or rather _blorp()?), proper usage of data binding (without any middle class fingers crossed), usage of styles (with conditions? something like that?) and all in all separate code and design more. There should also be at least a more interactive JSON viewer (as seen above: Links to entities and other files could be clicked directly, rightclick on such detected or undetected links could open some sort of whitelist/blacklist). Future plans could include to have a JSON editor too (i.e. double-click on a value, type in new value, Jofferson edits the JSON properly, even in .smods). The Kitchen’s model viewer would certainly be included in that too, after a rewrite of Panicle to move away from Qubicle’s .qb as main format (and rather offer it as a module than the base)

Speaking of the Kitchen, I’m unsure where to take it. It certainly sounds fancy and might even be useful, but the community at the moment seems to be too… err. small to do something on that scale. Instead, I thought about going a step into that direction, but with smaller steps. This means that instead of one big thing, there would be rather a few smaller apps.

One of those could be some sort of pastebin-like storing tool for models and animations. Modders could upload their models to the server, which would then give them a link to the model previewer. They could then create collections that contain exactly those models and animations that they want to. If you’re familiar with imgur, we’re speaking of images and albums so to say.

I would probably do lots of fancy magic that would allow modders to, say, exchange animations. For example, a modder could upload a new fox model and use another modder’s critter animation to show it off - that kind of thing. It sounds fairly reasonable, I think, and pretty much doable, I hope. Especially if I can get ASP.NET working - but in the end, some stupid C# server and some fancy JS could do the same trick.

However, current priority would be the remake of Jofferson, perhaps re-writing a few parts of RP that really need to be updated (for example, the start menu part which was made obsolete and therefore broke rp_spawnstuff). As always, I’m very demand driven: If something is demanded (or rather, somebody is interested in something), I’m more likely to give that more attention than the other stuff.

@RepeatPan Currently I get loads of red highlighting when .lua files are referred to. This happens with the Stonehearth.smod aliases that refer to Lua. Is it possible that Jofferson does not understand that when .lua files are referred to, the game is (usually) actually talking about .luac?