Hands-on with Swift Playgrounds for educational robots

What if you could use a single programming environment to learn to code at an early age, then use that same environment to explore programming for robots, IoT, Augmented Reality, and then take those projects into a full-blown programming environment? Apple’s Swift Playgrounds is a free app that aspires to do all of these things. I spent some time working with their new features for controlling educational robots, and here is what I found.

The Swift programming language was introduced in 2014, along with a feature called Playgrounds in Xcode, their main developer tool. The Swift Playgrounds app for iPad was introduced in 2016 as a “learn to code” environment aimed at children, but it is very much real programming. This spring (2017) it received substantial updates, including the ability to program several educational robots.

I spent some time using Swift Playgrounds with these robots:

Other robots are supported, including Parrot drones and Meebot, but I didn’t have a chance to test them.

What I found is a VERY promising beginning. The core Swift Playgrounds tutorials (Learn to Code) are really solid, and provide for a seamless transition into robotics. The robotics support is good, even though the level of finish and polish is variable.  The ability to bring projects into XCode (Apple’s IDE) is an important bonus, and opens the path to more sophisticated projects.

  • Lego Mindstorms is very polished, with a useful Live View, and very complete support for the API. Documentation is extremely thorough. You can do everything you can do with the Lego software. It feels like an advanced step for students used to the Lego apps.
  • Wonder Workshops Dash is also well designed, but it seems like the API support is limited. It feels like you can only the use the robot to a fraction of its capabilities. The tutorials are very easy, but without much of a growth path. It feels like an extension to basic Swift tutorials, instead of using this tool as a way to explore more advanced concepts.
  • The Sphero playgrounds are very solid, with complete API support, Live Views, and thorough documentation. Well structured Star Wars themed tutorials for the R2-D2 robot, and more general tutorials for the entire Sphero product line. This also feels like an advanced step past the native apps.

Keep reading for more detail on my experience.

Continue reading Hands-on with Swift Playgrounds for educational robots

10 ways to get more out of Lego Mindstorms

I’ve been spending some time making stuff with Lego Mindstorms EV3, and have learned some things that might be useful. Out of the box, the kit is extremely complete and most of the materials on-line and in print start with the assumption that ALL you have is the basic kit. However, with a bit of judicious spending, you significantly increase the range of things you can do. (Bear in mind that some of these extensions are not allowed in some competitive robotics leagues, like FIRST.)

Read on for a list of 10 ways to get the most out of your Mindstorms kit!

Continue reading 10 ways to get more out of Lego Mindstorms

The next 5 years in 3D printing

I’ve spent the past 7 years working extensively with 3D printing, everything from low-cost hobby level printers to very high end plastic and metal 3D printers. I’ve been exploring the elusive space in between, using low-cost printing to serve enterprise and volume production applications. I’ve experienced first-hand how the low cost and high quality of 3D printed parts can change the world.

Here’s my take on the next 5 years in 3D printing.

  • Innovation by newcomers to the industry will dramatically drive down costs, enabling the move to additive manufacturing
  • Cost reduction in machines and processes will enable systems capable of volume production
  • Dramatic innovation in process and materials will also broaden the application space
  • Existing applications will become cheaper and more accessible.
  • Entirely new applications will emerge to take advantage of these changes
  • 3D printing will start to encroach on other technologies for some use cases.

By far the most important change over the course of the next 5 years will be the decline in cost of 3D printing. This will be driven by newcomers to the industry, many of them coming from the low-cost segment of the industry. This dramatic reduction in cost will shift 3D printing from high value, high cost, low volume applications to low cost, high volume applications. Cost reductions will affect machines, labor cost and materials.

Let’s look at a specific example of cost reduction, comparing high-end “pro” machines using the FDM technology with some that come from the “desktop” space. In order to normalize for size, I’ve compared the price per cubic inch of build volume for various printers:

On the left, two generations of Stratasys 3D printers, the Fortus and F123, all with heated chambers and soluble support. In the middle, the Replicator2X and Z18 are 4th and 5th generation Makerbots. On the right, competitive low-cost printers built for robust applications. It is worth pointing out that that the Raise3D N2+ has a build volume comparable to the Fortus 450mc, dual extruders and network control – for 1/40th of the price.

Continue reading The next 5 years in 3D printing

An Alchemists Wand for the 21st Century

I have been working on a project with Lynn Hershman, called Alchemists Wand for the 21st Century. This art piece incorporates a number of brooms that have been modified to detect toxic substances, registering this on LEDs embedded in the broom handle, and transmitting their readings to a screen that presents information on what to do. Each broom detects a different substance, currently: alcohol, ammonia, carbon dioxide, carbon monoxide, natural gas and various solvents. The piece showed at Bitforms gallery in New York, as part of a show called “Touch: A Space of Relations”.

Alchemists Wand for the 21st Century at Bitforms gallery

Alchemists Wand for the 21st Century at Bitforms gallery

The brief is simple: build a set of brooms that can detect toxic substances in the environment, and display their findings via lights on the broom itself, and on a connected screen. The initial concept design used a square janitor’s broom:

First concept diagram

First concept diagram

Read on to see how this concept became reality…

Continue reading An Alchemists Wand for the 21st Century

RAW/WAR at Sundance Film Festival

Since October of last year, I’ve been working on an exciting project with filmaker and artist Lynn Hershman Leeson, an interactive installation integrated with a web-based media database called RAW/WAR. The interactive installation is constantly updated with multimedia from a companion website at

I’m going to describe some of the aspects that I worked on, especially in the area of physical interface design. We worked with Paradiso Projects extensively for the design and coding of the installation and website software.

The genesis of the project comes from the film “!Women Art Revolution“, a documentary by Heshman about women in contemporary art, and how they created an entirely new, and parallel, body of work from the 1960’s onward. The film is based on over 40 years of interviews with contemporary women artists. However, Hershman realized that it was not possible to present all the stories she had gathered in a single documentary, let alone those of artists she had not been able to document. Therefore, she made available all her original material via an online archive at Stanford University, and created a website, RAW/WAR, that allowed anyone to upload documentation of their story.

For the Sundance Film Festival, Hershman was extremely interested in creating an interactive installation that would allow casual exploration of the RAW/WAR archive in an engaging manner. Starting from an observation about the film, that it was like exploring a dusty attic with a flashlight, we set about bringing this to life.

Continue reading RAW/WAR at Sundance Film Festival

Making a Makerbot (part 6) – printed replacement XY stage

One of the most interesting things about the Makerbot 3D printer is that it can be used to repair, or even upgrade, itself. After a substantial run of large prints, the bushings that hold the X & Y platform had started to wear out, and the print stage was becoming wobbly. This translated into unsightly marks and loss of precision on printed parts.

Fortunately, someone on Thingiverse (a repository for uploaded fabrication files) had already developed a replacement XY stage for the Makerbot, based on the RepRap Mendel, that uses bearings instead of bushings. I ordered the parts (nuts, bolts, bearings) and printed the 18 pieces that make up the replacement stage. The design for the new XY stage has gone through various iterations, with lots of useful changes proposed by the user community. Not only that, but there are even upgrades for the upgrade, as I will discuss later.

Old vs. new XY stages

Old vs. new XY stages

The new XY stage, mostly printed on the Makerbot, uses bearings instead of bushings, and should experience a lot less friction while moving. The design is also much lower, and allows for more than 1cm increase in vertical travel.

As I was in the process of putting it together, I discovered yet another upgrade for the upgrade: a quick release clamp system for the new XY stage. The print platform is normally held to the XY assembly by magnets, to allow for easy removal. While this makes sense in some circumstances, it can also lead to quite a bit of wobbling. I went ahead and printed two parts as well, and incorporated it to the build.

Print in progress on the new XY stage

Print in progress on the new XY stage

Continue reading Making a Makerbot (part 6) – printed replacement XY stage

Hacking Spykee the robot for increased range and power

I have had this Spykee robot for a while now, had encountered some issues, and decided to fix it.

Spykee is a “toy” robot that is suprisingly sophisticated. It has a video camera and microphone, and can be controlled remotely from a computer via local WiFi, or over the Internet. However, it has a design flaw in the battery charging mechanism that overcharges and soon destroys the included battery, getting to the point where you can only operate the robot for 10 minutes. The built-in WiFi antenna is cheap – a piece of PCB with etched traces – so the range is very poor.

Spykee with new WiFi antenna and external battery

Spykee with new WiFi antenna and external battery

I fixed both of these issues very quickly: for the WiFi range issue, I ordered an RP-SMA to UFL pigtail connector. This is a fancy wire that connects to the WiFi connection on the robot’s motherboard on one end, and to an external WiFi antenna on the other. I ordered this one. Disassembled the robot, unplugged the internal WiFi antenna, and plugged in the pigtail. Drilled a hole in the case, on the left side near the top, and threaded the RP-SMA connector through. The antenna, from a defunct D-Link router, just screwed on. You can probably use higher power antennas as well, as long as it has an RP-SMA connector.

For the power issue, I did a little research. It turns out that Spykee runs fine on a 12V power supply, not just the 9.6V battery that it ships with.  I had good experience with Lithium-Polymer batteries I found on eBay, that go up to a whopping 6800MaH (vs. 1600MaH at best for the factory battery). I use the same battery for powering the LEDs on my baby’s stroller, and it has been rock solid. So I ordered this one from ebay, and set about adapting it to the robot.

Spykee's new super battery

Spykee's new super battery

I wanted the mod to be as reversible and low-impact as possible, so I looked at the battery connector inside the robot (Tamiya style), cut the equivalent battery-side connector off a dead battery, and soldered it via some wires to a standard 2.1mm power jack from Radio Shack. I drilled a hole in the robot case to accomodate the jack, and plugged the other end into the robot’s existing battery connector. The battery attaches to the robot using Velcro adhesive strips. I can revert to the “factory” configuration by just unplugging the new cable, and plugging in the old battery.

One disadvantage of the LiPo pack is that you cannot use the existing charging dock, you have to connect the LiPo battery’s own charger. The robot has some nifty functionality for auto-docking, so this is a bit of a loss. Since the battery is connecting via the existing circuit path in the robot, it should be possible to adapt the LiPo charger to connect through the robot’s charging dock.  Haven’t gotten around to doing this yet. (Warning: LiPo batteries require a lot of care when charging. Use only the charger that comes with it. Don’t do this if you don’t know what you are doing.)

Javier and the robot

Javier and the robot

Results of all this? The robot can now be run very actively for over two hours at a time, at high speed. WiFi range is roughly 25% better, the robot can now roam freely all over the apartment without losing the connection. It is now a far more practical proposition to use the robot to play with my son!

Timelapse toolkit for the Exploratorium

I’ve been doing some development work at the Exploratorium, and my latest project is now ready to go live. It is a general purpose, highly configurable timelapse recording program, that will soon be incorporated into an interesting experiment in social dynamics (I’ll tell you what it is when it goes live).

Exploratorium timelapse toolkit

The program is developed in Max/MSP, and contains a lot of useful features:

  • Live preview of camera input
  • Timestamping of recorded frames can be turned on or off
  • Can be controlled with a rotary encoder (big free spinning knob) allowing users to “travel backwards in time”
  • Timed recording, you can have it start and stop recording a timelapse movie at specific times
  • User configurable recording interval and/or difference threshold (ie. record a frame when something changes)
  • Disk based buffering of the current recording, so you can scroll back and forth while it is recording
  • Program saves all settings automatically, making it possible to run unattended and on a schedule

The system is designed to be sufficiently flexible and configurable that it can be deployed in multiple situations where timelapse recording is interesting.

Baby stroller hacking – multicolor, remote-controlled LEDs

Nik and I had an awesome baby! His name is Javier Jun-Hong, and his special superpower is breastfeeding. Here he is:

Javier is preoccupied by the situation

Javier is preoccupied by the situation

Now, as everyone knows, having a baby is a great opportunity to launch a variety of different toy-making and hacking projects. The first one is to enhance his stroller with remote-controlled multicolor LEDs. I picked up some parts from Elemental LED, and put it together over the course of an evening. I used zip ties to hold the LED strips to the bottom of the stroller basket. The power pack and controller fit nicely under the footrest, and are held in place with velcro. Stroller folding is completely unaffected. Looks pretty good! And so does Javier’s mom!

Variable rate time-lapse video using motion detection

As part of a larger project involving timelapse video, I developed a technique that uses motion detection to identify frames for capture, and skips over frames where nothing is happening.

The video on the left implements this technique. You can see how it works, by skipping the stops at the stations in the video. The video on the right is original source video, sped up to match the duration of the video on the left. You can see that it does not skip the stops at the station.

I implemented this in Max/MSP/Jitter. The basic algorithm keeps track of the last frame written, and constantly compares the incoming video feed with that frame, calculating a constant difference score that tracks not only the number of differing pixels, but also the amount of difference. When this difference exceeds a defined threshold, then it captures that frame.

Comparing against the last frame captured, instead of simply the previous frame, ensures that even gradual changes will be recorded.