Sonic Screwdriver

x

Alpha Prototype

First prototype with the functions - The development blog is documented on Hackaday.

Display and Menu

I used a webpage that allowed pixel level drawings to be turned into bitmap arrays that can easily be stored.

CAD and Enclosure

While I waited for the electronics to arrived, I decided to start working on the enclosure. The internal volume is similar to the final version of the SS.

Testing Modules

The first round of ordered modules arrived from Digikey, and I spent the better part of the weekend testing their functionality. Most of the tests I ran on the modules were simple call-and-response, or sensor reading. I discovered a website (Tinycircuits.com) which sold, unsurprisingly, tiny circuits. These are perfect for a space limited project.

Putting Everything Together

Assembled modules and wired on protoshield.

Testing function menu

Tested the "tier-1" functions of the SS, which largely comprises of scientific modules. Here's a list of modules tested, and their basic functions added:

• IMU (accelerometer, gyroscope, magnetometer)
• Hygrometer
• Barometer (and altimeter)
• Thermometer (IR and ambient)
• RGB Led
• IR Led (and receiver)
• Radio Module
• Wifi Module

Shuffling Functions Around

I decided to shuffle the functions slightly. There are a lot of basic / measurement functions on the SS, that are useful, though not all the time. Some of the functions will need additional hardware not yet purchased (e.g. C12666MA Spectrometer, or a GPS module), but I've included them for completeness sake. Here's the current break down of functions I've added:

1. Pen. Still not sure what the best approach is in this case. ML models are too hungry for the ATSAMD21. Other ideas like Hopfield Networks could work be implemented, though they are significantly limited in terms of memory. I could also offload the identification / classification to a server, and just have the SS record the pen strokes, but this is less fun.
2. Analog Broadcast. The ESP8266 can be severly overclocked to transmit analog TV signal. I swapped to a TI wifi module instead, so would likely have to scrap this function.
3. Smart Home. Connecting the SS to other projects at home. I can use the wifi router and compass to localize the sonic - I can trigger different projects by pointing at them.

x

IMU Pen

This was a submodule is part of the sonic screwdriver project. The sonic screwdriver will have a pen function, that can be used on any surface, recording IMU data, that will immediately be turned into alphabets. The prototype below was made to collect data.

The core electronics of the prototype are the same as for the sonic screwdriver. For collecting data, I used miniature modules from TinyCircuits.

Data

The pen collects motion data from the 9-axis module (accelerometer, gyroscope, magnetometer), and uploads them. The data can then be downloaded and used to train a machine learning model to determine the letter being written from the motion data. The motion data for the letter "a" is shown:

Rather than collect 1000s of examples, I opted to first use a combination of unsupervised learning methods, and a priori knowledge on letters for classify motion into alphabets. For aprior knowledge, I first analyzed the use of dashes and dots, and investigated the length (of time) used in writting different letters. I investiagted how the rate of writing affected the quality of motion data. There was ample repeatability when written at different rates. This means there is less of a concern about how quickly letters are written, since the motion would still remain representative of the letter once normalized.

Unsupervised Learning

t-stochastic neigheborhood embeddings (tSNE) was used to learn latent structures in the data space of all recorded motion. Three letters that are similar (a, d, and g) are shown below. In the plots, show that they are similar (grouped nearby), but distinct enough that they are clustered seperately.

The remaining letters (excluding letters involving dots and dashes) are shown below. The distinctive clustering shows that there are latent structures in our data. Thus, a trained DNN model will be able to extract the correct letter from the motion data.

x

The final PCB board is generated with KiCAD again, and fabricated at OshPark.

PCB Schematic rev. 3, KiCad. Originally planned to use the ARM Cortex M0+ microcontroller (similar mcu to Arduino Zero's).

Repurposed "reflow oven". Marked improvement from the frying pan model.

Populated PCB solder in reflow oven; remaining will be hand-soldered. I replaced the ESP8266 with the more robust ESP-12S in this revision. This is especially useful for adding new functions over wifi and planned communication with the Raspberry Pi server.