The Happy HTTP Hack
Ladies and gentlemen. I understand that you have come tonight to bear witness to my journey on an insane path of monkey-patching Lua. I regret to announce that this is not the case, as instead I come tonight to bring you the codified recreation of the end of the world.
HHH is a patch for Lua that allows Stonehearth mods to do HTTP calls to remote endpoints. This allows the game to interact with web services, because we all know HTTP and the cloud are the future.cough
Using HHH is pretty straight forward with the included httplib.lua
:
local http = require 'httplib'
if not http then
return -- HTTP is not available
end
-- Fetch the resource. Make sure the endpoint is whitelisted!
http.get('https://my-awesome-web-service/feed.json')
:on_success(function(ctx)
-- Feel free to do whatever you want in here. As example, let's get the response content:
local response = ctx:get_response()
end)
:on_failure(function(ctx)
-- Uh oh, something went wrong.
logger:error('Could not fetch %s: HTTP status %d, error %s', ctx:get_url(), ctx:get_status_code() ctx:get_error())
end)
Download v1.0.0
Included is everything you need to get started: The two patched lua-5.1.5.dlls and an example mod that loads the latest topics from this Discourse, sends it to the GUI, and displays them there along with the avatars of the last posters.
Simply copy the content of the zip over your base game directory - not your mods folder! If done properly, Windows should ask you to overwrite lua-5.1.5.dll twice. Keeping a backup of the original files is recommended if you wish to go back to the official Lua.
What’s different in this Lua dll of yours?
It’s built completely from scratch, as I had no access to Stonehearth’s original Lua sources. Thankfully @max99x helped me here and there with a few missing functions, so that overall the dll should do mostly the same. In fact, there is a one-in-a-million chance that it actually runs somewhat (probably in the unobservable realm) smoother, as I’ve fixed one (1!) undefined behaviour that happened in one of the patches, but that’s mostly just semantics.
My patched DLL comes with a HTTP client that can open HTTP and HTTPS connections (technically, FTP, SFTP and whatever else curl offers would be possible, too, but for safety reasons, only HTTP is enabled for the time being), as well as lua bindings for it. To top it off, there’s a proprietary (read: not the one SH uses) JSON parser included to parse the whitelist.
To be able to actually use it, I’m hitchhiking on some other calls that Stonehearth necessarily needs to make in order for Lua to work.
Additionally, I’ve removed several bits that could be used to do bad things, such as the io and debug library. Turns out, those have already been removed as part of the multiplayer update, so they’re gone… twice now, I suppose?
On the topic of Security
You might ask now, isn’t this kind of insecure? Well, yes, it is! There’s a good reason that Stonehearth doesn’t allow HTTP connections out of the box. Lua itself is a sandbox (let’s ignore the fact that Lua has native support for binary modules), especially in games. By allowing outgoing requests, we’re breaking the sandbox.
In addition, you’ll need to trust the .dll that some stranger made. This is inherently unsafer than just any mod, because if I have messed up somewhere, bad people could do bad things. The chance for that is slim, but it’s not exactly non-existent.
If anything, this is a proof of concept. Depending on the interest generated and the use cases, there might be additional development, and the whole source code might go Open Source as well.
Any mod that is requires this library will also require its users to download HHH, as it cannot be distributed over the Steam Workshop.
To get some level of security going, the user needs to maintain a whitelist himself. This whitelist cannot be overridden, mixinto’d or anything else, as it exists completely outside of Stonehearth’s realm of influence. Details on the whitelist are on the GitHub README.
More examples!
Posting some JSON
If you wish to control what your request body will be like, simply pass a string (containing the data) and optionally a Content-Type
. In this example, I’m posting some simple JSON and specify the Content-Type as application/json
.
http.post('https://my-awesome-webservice/api/search', '{ "query": "sheep" }', 'application/json')
:on_success(function(ctx)
local response = ctx:get_response()
-- Feel free to do whatever you want here. Parse it, print it, send it to someone!
end)
:on_failure(function(ctx)
-- Uh oh, something went wrong.
logger:error('Could not fetch %s: HTTP status %d, error %s', ctx:get_url(), ctx:get_status_code() ctx:get_error())
end)
Posting a form
Forms are posted by passing a lua table. Both keys and values of the table must be strings, or stringify-able.
http.post('https://my-awesome-webservice/api/search', { query = 'sheep' })
:on_success(function(ctx)
local response = ctx:get_response()
-- Feel free to do whatever you want here. Parse it, print it, send it to someone!
end)
:on_failure(function(ctx)
-- Uh oh, something went wrong.
logger:error('Could not fetch %s: HTTP status %d, error %s', ctx:get_url(), ctx:get_status_code() ctx:get_error())
end)
Reading the response
There are multiple ways to dealing with a HttpRequest
as is passed by the on_success
and on_failure
callbacks. The most interesting properties are:
-
HttpRequest:get_status_code()
: Returns the HTTP status. Defaults to 0 in case the request wasn’t started yet, or there was a transport error. -
HttpRequest:get_response()
: Returns the response content as string. -
HttpRequest:get_response_base64()
: Returns the response as base64 encoded string. -
HttpRequest:get_response_header(name)
: Returns the value of the response header with the namename
, or nil if the header was not present. -
HttpRequest:get_error()
: Returns a string containing the error (if any), or nil. -
HttpRequest:get_error_code()
: Returns the libcurl error code associated with the error. Defaults to 0, which is OK.
More advanced usage
It’s also possible to go barebone on HttpRequest. In this case, usage of httplib and especially http.prepare
is still recommended, because it’s still making things somewhat easy.
In this example, I’ll perform a PUT request on a resource, using a custom request header (Authentication) and as payload some plain text.
local req, promise = http.prepare('https://my-awesome-webservice/api/content/12')
req
:set_method('PUT')
:set_request_header('Authentication', 'Bearer XYZ')
:execute('Baaaawk!', 'text/plain')
promise:on_success(function(ctx)
-- Same as the other examples.
end)
:on_failure(function(ctx)
-- Same as the other examples.
end)
And more!
The full documentation of the interface can be found on GitHub. The README contains more bits too, so I’ll keep this one short.