Bench Talk

Thanks to modern technology and general network availability, it's possible to develop and create prototypes rapidly. For instance, getting inspired by a very simple concept like the J.A.R.V.I.S. artificial assistant from the Marvel movie "Iron Man,” we decided to develop a voice-activated “butler” of sorts that could correctly interpret questions and provide answers. The project have been extended further to have Alexa respond to voice commands by completing custom tasks such as turning lights on and off. We’ve all seen Alexa in action, but creating a voice-activated butler also made a useful and fun engineering project for students.

Raspberry Pi 3 and Alexa

For this project, we used two technoloiges:

The single-board Raspberry Pi 3 was used in conjunction with the REST APIs offered by the cloud service from Amazon’s Alexa. Raspberry Pi 3 is open-source hardware and a great platform for teaching and rapid prototyping. Everything is freely available, starting from the basic building blocks in order to get a fully working system.

Alexa is a free commercial service provided by Amazon for use in development and prototypes, but require a subscription for industrial products. Alexa is intrinsically open in spirit, meaning users can interact with it using standard state-of-the-art methodologies and different languages.

Hardware, Software, and Application Software

Raspberry Pi and Alexa worked together by way of these additional pieces to complete the workflow and project:

• Hardware for emulating an assistant: A processor, microphone for sampling voice, and speakers/headphone for emitting voice).
• Software for interacting with the hardware: In our case, we had a customized Linux distribution powered by Yocto. The Raspberry Pi 3 board alone is not capable of doing the whole job of virtual assistant. It needs to have proper software on top of acceptable processing cores to enable functionality of the hardware and programs (ie., the so called “userland,” in this case the Alexa Services clients).
• Application software: In this case, we needed application software running on top of the Linux distribution with the capability to interact with both the hardware devices via abstraction and Amazon’s Alexa cloud services. Different solutions were explored—the final winner found in a Python software stack under permissive license.

Interface to Alexa Cloud Services

With the hardware, software, and application software in place, the next step was to learn the basics of interacting with Alexa’s cloud services. The whole point here is that Alexa is a service operating on the public Internet, and it has a set of rules that need to be understood. At Amazon, they do not disclose all the service functionalities, but rather, they provide a sort of communication contract—an interface—that defines the rules for communicating with Alexa’s cloud services.

Alexa has two different services, each independent but capable of complex co-operations:

• Alexa Voice Service is the service capable of offering voice and speech recognition. It can record from the microphone, send the track to the service, and yield back the Alexa voice output. As an example, recording a voice input like “What time is it in Rome?” will produce the time in Rome as output computed by Alexa Services.
• Alexa Custom Skill helps define actions in a description language and provides trigger words that enable Alexa Voice Service to recognize what’s being asked. An examplewas to define “Raspberry Pi” as the phrase that triggered the custom task of turning designated LEDs on and off. Whenever Alexa Voice Service recognized the aforementioned action, it commanded a web service via standard REST API interfaces.

Developing Interfaces

The last step of this prototype development was to write down the so-called “glue logic,” which is a set of programs capable of interfacing with the hardware and Alexa.

For this project, the Linux distribution alone was not enough for interacting with Alexa. Even if we had an operating system capable of talking with all the Raspberry Pi 3 hardware blocks, we still needed a piece of software to manage the input and the output (microphone and speakers, respectively). The flow—and hence, interfaces needed—looked like this:

1. The software received input from the microphone.
2. Data was processed and sent to the Alexa server.
3. The Alexa server computed the raw input data received.
4. The Alexa server produced an output that was sent back to the software on the Raspberry Pi.
5. The software that initiated the communication produced an output.

To accommodate these requirements, Python was chosen as the platform to develop the glue logic. We used a project that already existed on GitHub and forked it to meet our needs. Standard python libraries already existed for the Custom Skill Kit, and in this case, we integrated it in the existing project as a separate application.

Results

After some wash-rinse-repeat iterations, the project was fixed and finished. The final system had everything required working out-of-the-box, except for the authentication tokens. Using Alexa Voice Service and Raspberry Pi 3, the final process looked like this:

• Alexa Assistant trigger: Call “Alexa!” into a microphone.
• Question to Alexa Assistant: “What time is it in Rome?”
• Alexa's response: The correct time!

Using Alexa Custom Skill and Raspberry Pi 3, Alexa could respond to voice commands with actions such as turning LED lights on and off. Figure 1 shows the extending prototype circuit with buttons and the LEDs interacting with the Alexa Cloud Service. Here, the button was used to trigger the Alexa input (microphone recording); the green LED is turned on/off via voice commands (the other two are status LEDs).

Figure 1: Prototype breadboard in an early stage of development (Source: Politecnico di Torino)

Overall, this was a great project! All of the base material for this project is publicly accessible on GitHub at https://github.com/fgpolito. The same is true for the tools used for building and integrating the custom Linux distribution. Everything can be easily done from scratch, and this is the real strength of using open components that can be modified.