Scientifically (in)accurate crane simulator

My company ( is working a lot with the manufacturing and power industry. I wanted to see how I can take a physical controller and bring it into the VR environment. For simplicity, I went with a Throttle Qudrant joystick , often used for flight simulators. I modeled the joystick and mapped it 1:1 with the real device.


And then VR’ed the whole thing. It’s surprisingly fun. I’m going to demo this at Manufacturing Matters next month.

VR for Industry

Geiger Counter

Training and testing on the usage of a hand-held radiation detector.

This is a pilot project for the nuclear power plant in Pickering, Ontario. An operator places a virtual radiation source on the table and the trainee has to find it. Only the operator can see the source on screen. In VR, this test can be conducted in an office or even at home, without expensive equipment and without exposing trainees to a radiation source.

Crane Demo

This is a demo of combining a virtual device (the crane) with a real-life controller (the levers). The user can see the levers in VR and operate the controller with their hands. The virtual and physical controllers are mapped 1:1 (position and scale). It is also possible to show the user’s hands in VR, for easier orientation.

Audio Testing Room

This was is a test developed for the audiology department at the University of Western Ontario. A user is presented with a VR room that includes several audio sources. The sources can be moved in 3D space and their volume / reach adjusted. This VR setup could potentially replace audio testing rooms, which have dozens of custom speakers and cost upwards of $100k.

Final Thoughts about the MS/Acer VR headset (I refuse to call it Mixed Reality)


About a week ago I got an Microsoft / Acer VR headset, which MS insists on called “Mixed Reality”. I reviewed it, I took it apart. I made some tests in Unity3D. I emailed MS support. Here are my final thoughts:


It’s cheap. Currently priced at $300. While a few weeks ago I would have considered this incredibly cheap, in light of the Rift’s recent price cut, the cheapness factor has diminished somewhat.


The unit is light (which is good) but feels really cheaply constructed. The strap is minimalistic and not very comfortable, consisting almost entirely of hard injection molded plastic. The forehead rest has some cushioning, thank goodness.


The resolution is great, sharp image with lots of detail that just pops right at you. Screen door effect is still there though. It’s hard to compare it to the Rift or Vive. I feel that it’s less pronounced, but that’s entirely subjective. The optics and field-of-view are noticeably inferior to Vive/Rift, with the Acer having a smaller “sweet spot”, lots of blurriness in the borders and a distinct “tunnel vision” effect.


There is almost none, which is fantastic. The headset plugs into the computer and that’s it. There are no cameras or lighthouses to install. It doesn’t even require external power. The minimal spec is very low, but I haven’t tested how well it functions on a low-end system. I suspect the experience will be greatly diminished.


Tracking is good, not as good as the Vive (it’s a bit jittery), but better than any mobile VR headset. Positional tracking is solid. Interestingly, the headset doesn’t function in the dark. Turn off the lights and it will lose tracking.


There is none. I found a demo for the Hololens, and that’s about it. You can “inject” regular 2D UWP apps into your 3D virtual space, as floating screens. While this is really cool to have on a Hololens, I don’t see the point of doing it in VR. Some people might find a use for this feature, to me it’s not very alluring. The display resolution is good, but not good enough to read small text and certainly not as good as my 4K monitor. Also, normal “desktop” software can’t be injected, which rules out 99% of the apps I normally use.

Mixed Reality

Unlike the Hololens, these new “MR” headsets do not have 3D cameras and do not map the environment around them. Furthermore, the two cameras on the front are used for tracking and nothing else. They are not currently accessible via the SDK, and there is no way to get a video feed from them. This is confirmed by a MS rep over email.
Microsoft is pushing new terminology of a “mixed reality spectrum”, it’s just that these headsets are waaaay over on the VR side of things. They “mix” 0% real and 100% virtual. Honestly, I’m disappointed. I was hoping for some kind of poor-man’s Hololens, but it’s really not. If the Acer is a MR headset, then by the same logic so is this:


The Acer headsets requires Unity 2017.2 Beta to build and run custom apps. No other version will work. Being a beta, 2017.2 is very unstable and can be frustrating to work with. I spent a few hours making a simple “hello world” app, consisting of a few floating textured cubes, and I had to restart my system twice. The headset stopped responding completely after the first build attempt. The second time I got a very weird “double vision” effect, where my app was visible twice inside the VR world, one in front and another in the back. I can’t explain that one, because it didn’t happen again after a restart.
Overall it works, but it’s finicky and unstable at the moment.


Being Microsoft, the headset will only run UWP apps. For anyone hoping for an OpenVR bridge or driver, this is a problem. UWP is all about layering, security and sandboxing. It might be very tricky to convince this headset to run OpenVR or SteamVR applications.
MS announced that these new headsets will support SteamVR, however, this will not happen at launch (Fall 2017). Work on this has just begun, and there is no release date yet.


I love the simplicity of this device, the ease of setup and the display resolution. I don’t like the optics and being stuck with UWP (for now?). I intend on using mine as a mobile demo tool, at least for the time being.
Is it worth buying? I don’t think so. Not yet anyway. For an extra $100 you could get an Oculus Rift, with two controllers and a lot of supported content, apps and games.




Teardown of the Acer / Microsoft “Mixed Reality” VR headset

I wanted to see whats inside the new Acer VR headset that my company (Packet39) received a few days ago. So I took it apart and documented the process.


Velcro comes off, then 6 small screws. Then the plastic “face cover” detaches, and the headband with it.




The two lens holders (or maybe protectors, as they don’t actually hold anything) pop off. Four more screws and then another plastic cover comes off (not easily though).




Now the IR proximity sensor and audio jack are exposed.




One more screw and the audio jack comes out.




Four additional screws and the LCD+lenses module comes off. I didn’t open that one, because they are usually hermetically sealed and I don’t have a clean room to put it back together. The LCD module is connected to the main board by two ribbon cables, and the IR sensor is the third.




Back side of the main board, and the QR code / serial number.





The two cameras, each have their own ribbon cable and a fancy cable protector. I peel off the shielding but there is no label on the camera itself. Sadly, I still have no idea what kind of cameras these are. Each camera has a flat and flexible heatsink glued to the back, that extends to the sides of the headset.




There is nothing else in the headset, no additional sensors or emitters.




Here is the front side of the main board, with some closeups on the chip numbers. I haven’t looked these up yet, if you know what they do please comment on this blog post.





Everything was put back together in reverse order without much difficulty. Headset still works, so great success. I hope you find this useful.

3D modeling curved objects with a flatbed scanner

Here is a neat trick that can save a lot of time when modeling objects.

Say you have an object that you want to model with high fidelity. And also say you are planning to attach a Vive tracker to this object and swing it around in VR (coming soon). Like so:


How would you go about measuring the curves?
You can use one of these, maybe?

Or these?

Surly there is an easier way? Yup. There is. The humble flatbed scanner. Available for pennies at your local second-hand store or ebay. Place your object on the scanner, like so:

Object is too big? Scan it in part, assemble automatically using a panorama stitching tool, or manually in any photo editor. It’s fairly easy. You’ll end up with a scan that looks like this:


It’s going to look bad. Fuzzy with a lot of color noise and crazy pixels. That’s fine. We don’t care about picture quality, we just need the curves. I also scanned the cross section of the handle, because I couldn’t find my caliper and for the sake of completeness. Now I can say that everything about this model was flatbed scanned.

The images should come out of the scanner exactly to-scale, meaning you can import them directly into your 3D modeling software of choice, trace the lines and have a to-scale replica of your object. First I traced the curves:


Then some sweeps, extrudes and lofts, and voila, one world-scale racquetball racquet is reading for texturing:

Coming up soon, attaching the Vive tracker and bring the racquet into Unity.