How to build a VR retro arcade: do it yourself!

In part 1, you got an intro and team retrospective of the “retro arcade” hack week project, where the whole team banded up and built a fun project together that really put Alloverse to the test. In part 2, you got an architecture overview, explaining the components involved in bringing a SNES emulator into a collaborative VR environment.

In this part 3, you’ll get a step-by-step tutorial in building it yourself! This is going to be intermediate-to-advanced, and a lot of code. The point is to demo the capabilites of the Alloverse platform, and show you some of the really advanced stuff that is possible with LuaJIT and AlloUI. If that sounds appealing to you, strap in and let’s go!

Let’s get to coding!

We start out by creating an AlloUI project somewhere on our computer. (This is the same as the first step in our Getting Started guide for the Lua language. If you want an easier starter project, I recommend following that guide!)

If you open up the Alloverse app and connect to sandbox.places.alloverse.com, you should now be seeing your app in there as a flat surface with a button on it.

Stubs and dummies

This flat slab is an eyesore. Let’s spice it up and add some basic UI, make it feel like a real object that the user will want to walk up to and interact with. Replace lua/main.lua with the code below. It instantiates stubs for Emulator and RetroMote; loads a fancier model file; creates the "main UI" view that everything will attach to; places game controllers at some good default location; and kicks off the runloop for the app.

You’ll need three more files before this runs.

1st file: Download and place the arcade cabinet model file into . This’ll be the fancy arcade cabinet model that the user will want to walk up to.

2nd file: Create lua/Emulator.lua, where we just stub out an Emulator class. We will later use this to wrap all of libretro in a self-contained class.

3rd file: Create lua/RetroMote.lua, which will be our game controller object that we can pick up in VR to control the retro game being played. For now, this is also just an empty stub subclassing .

Run it…

and you should have a blank cabinet in the Sandbox place, yay!

Using FFI to hook up libretro to Emulator.lua

Now to make the arcade machine actually DO something. This is going to be a bit of a journey: we’ll be using LuaJIT’s FFI (“foreign function interface”) to talk directly to libretro’s C API. This way, we’ll be able to tell libretro to emulate games, send controller input to it, and receive back audio and video from the emulated system.

The libretro API is thankfully all in a single header, but LuaJIT can’t read it as-is, since the LuaJIT FFI lacks a preprocessor (and also a few other C features). I’ve gone ahead and preprocessed the header for you using the advanced tool “my brain and hands”, so that LuaJIT can read it. It’s too long to paste here; instead, download it from here, and put it into lua/cdef.lua.

Head over to lua/Emulator.lua. Splat this at the top:

(we’ll be needing all those other requires later, so might as well get them in there now).

That last ffi.cdef line is all that's needed for you to be able to call all of libretro as if it was a lua library.

We just need to dynamically load it. Which means we need libretro somewhere on your system…

Installing libretro on your machine

Right, okay. If you’re on Linux:

On a Mac:

1. Install RetroArch from https://www.retroarch.com/
2. Launch RetroArch, and use the menus to install the following cores: Snes9X, Genesis Plus GX, Nestopia.

On Windows:

It should be totally doable to get retroarch installed in a way that’s compatible with this project on Windows, at least if you’re using mingw or something; but it’s not a setup I’m familar with so I can’t provide a detailed guide here.

Initializing libretro

Let’s dynamically load our newly installed libretro. Below are some helpers that…

1. locate the libretro dynamic library on your platform

load it using ffi.load() so you can call functions from it!

1. set all the callbacks to closures that call our own functions, so we can start reacting to events happening in libretro. (just setting self.handle.retro_set_input_state = self._input_state wouldn't work, because the callback wouldn't be called with the correctself instance. So we need to capture self with a closure and call the own method with it.)
2. and finally, set up controllers and init the library!

Go ahead and add this code to lua/Emulator.lua:

So as you can see, we’re able to call C methods like retro_set_controller_port_device(unsigned port, unsigned device) directly from Lua now that we've cdef'd its interface. The only bummer is that we can't use the handy macros like RETRO_DEVICE_JOYPAD, so we have to send in the raw numbers that they map to. But at least we don't have to manually allocate ffi memory for the arguments! It's just all handled automatically.

(It’s also blows my mind every time an FFI interface is able to assign a closure to a regular old C function pointer.)

Loading games

While we’re at it, let’s load the game too.

We use ffi.new to allocate named C structures (which have previously been handed to luajit with the ffi.cdef method). With retro_get_system_info we're then passing the allocated structure to get it filled in so we can use it later; and with retro_load_game we're passing in a structure that we have filled with data so that libretro can work with it. The data in question is the full game just read from disk and mashed into RAM. (there are also APIs for streaming game data so that one could play bigger games, but that's overkill for what we're doing here).

Constructor

Later on we’re going to want to render the game’s graphics, so we need to know what the size of the screen is going to be (“geometry”), so we set out to fetch that. Let’s also get the class’s constructor in place and finish up all the setup we needed:

Run-fix-repeat: The “environment”

At this point, it’s kind of easier to just try to run it and fix all the runtime errors, than to read documentation and figuring out exactly what needs to be configured to make things work. We’ll be doing run-fix-repeat cycles now until we have a working emulator.

Let’s make the code load up a game. Please legally acquire a NES or SNES rom file for a game you enjoy, and put it in roms. Then, change the bottom of main.lua to read:

So, first try: where do we crash?

Okay. So immediately upon loading a game, libretro is asking us for “environment”, which means it’s asking us for runtime settings. Let’s implement it and check which setting it’s asking for:

34. What does 34 mean? If we go back to libretro’s API and just search for 34, we’ll find it means RETRO_ENVIRONMENT_SET_SUBSYSTEM_INFO. We don’t care about subsystems, so we can just return false to say “we have no environment value for setting 34, sorry”.

If you’re feeling extra ambitious, you can now go ahead and run-fix-repeat for all the requested environment settings until it works… or you can copy-paste my implementation here 😅

You’ll note that we just return false for all the things we don’t care about. Some notes:

• libretro MUST have a logging callback. And, it must be a vararg function. luajit’s FFI doesn’t support that, so we’re going to have to implement that in C. BUMMER. We’ll get to that later.
• You’ll note that data is a void pointer, so we have to cast it to whatever is appropriate for the given setting, and then set it with pointer dereferencing. Since pointers and single-value-arrays are the same thing in C, we can dereference a pointer by referring to its first array value.
• The rest of the settings should be fairly self-explainatory; if not, ping @nevyn on our Discord and ask him to explain it to you 😅

Run-fix-repeat: The logging helper

Running the code above will crash on field 'helper' is a nil value. So let's implement the helper. Download libretro.h into c/libretro.h. Implement :

Write a Makefile to build it (in the project root):

We’ll also have to load this new libhelper.so at runtime. In Emulator.lua, right after self.handle = _loadCore(coreName), add:

Try it out:

Woah. It works?! It’s even using our logging callback to print the game name, which means we’re officially emulating a game inside a VR (even though we can’t see it yet).

Hey, you. Well done getting here! You deserve a fika break. Get a coffee and a cinnamon bun. I’m going to do that too, because the above was quite a handfull. Hopefully by the time you’re done, I’ve published part 4, and we can dig into displaying video, playing audio, and receiving controller input.

If you have any questions or feedback, please head over to our Discord and let us know!

Originally published at https://alloverse.com.