mallard2 The Mallard device was a UI-development device created for Nokia around 2007-2008. I was in charge of the development team at Tampere University of Technology, with a permission for open-ended exploration. The specification of the device was pushed far beyond what was initially asked for, and was a conscious risk. My team consisted of two engineers and myself, and when the electronics was finalized, Nokia hired an industrial designer to develop a suitable casing and the final shape for the Mallard device. I defined the specification for the electronics, based on user tests and an educated hunch on what might be useful for a longer time.

The team consisted of me, a researcher and a research assistant, where the initial development was mostly done by me, with a research assistant doing tasks I assigned. The development of the final circuitboard, as well as the programming, was done by both the research assistant as well as the researcher, under my control. Overall, the project had other teams involved as well, such as IHTE, also from Tampere university of Technology, and Tauchi from the university of Tampere.

The project started with a request from Nokia, to create a handheld device with a single controllable vibration feedback built in. This was discussed in the preliminary meeting, and I directly said that there is no point in us to do that, since there is nothing that could be considered novel. Thus I was given free hands to make a suggestion.

We did a extensive development in a relative short amount of time, starting with the way things are held, and how touch can be detected. Figure 1. shows samples from the ideation phase, and Figure 2. shows some of the holding tests, with the electronic grasp-visualisation device we created.

screen-shot-2016-09-12-at-16-45-46

Figure 1. Initial Sketches

From the sketches, a selection of models that were simply appealing to the team were further selected for development, which resulted in the creation of styrofoam mockups. Once satisfied with them, a simple test-kit was created, and the mock-ups modified to include vibration motors and weights, shown in Figure 3. The size of the mockup was explored with the aim of getting a good feeling when held. Being satisfied with the physical shape, the development of tactile feedback was started. Using the test-kit, rapid development of different tactile effects was possible.

screen-shot-2016-09-12-at-16-45-55

Figure 2. Grasp-tests

screen-shot-2016-09-12-at-16-46-03

Figure 3. Testing kit for vibration feedback, and the unmodified mock-up for scale.

In addition to the mockups and vibration, we wanted to push touch-detection as well. We built a simple circuitboard as a kit, and some test-touchpads, shown in Figure 4. The kit consisted a processor, six LEDs for direct visual feedback, a connector for the PCB:s with the test patterns, as well as for data-communication for transmitting the exact touch-based values. The kit was designed to be a test-bed for evaluating different kinds of capacitive touch-patterns: patterns that would work were then suggested to be tested with the final prototype.

screen-shot-2016-09-12-at-16-46-12
Figure 4. The touch tester-kit, and two different capacitive touch-pattern PCB:s.

After this point, the concept was looked at again, using available electronic components for setting the baseline for concept creation. I had an idea of a button which could give vibration feedback, and that became one of the core ideas around which to build the final concept. The concept drawings, as well as the wooden mockup used to present our initial suggestion for the capabilities of the final prototype are shown in Figure 5.

screen-shot-2016-09-12-at-16-46-26

Figure 5. Sketches and initial suggestion for the capabilities of the final prototype.

As our work was not on the industrial design nor form giving, our suggestion was intended for giving a better understanding for the capabilities and requirements for the electronics. Thus, we suggested to design a small circuitboard, which would allow touch detection over curved surfaces, force-sensitive buttons with haptic feedback, as well as other requirements typical of a prototype intended for user testing. Since the work was done roughly in a span of a month, Nokia decided to refocus the project and hired an industrial designer to design a casing, which would be able to use all of the capabilities of our suggested circuitboard. With this, we created a circuitboard, layout of which is shown in Figure 6. It was then used as the core for the final Mallard-device, shown in Figure 7. Roughly at the same time the circuit-board was finished, I started to work at Helsinki University of Arts and Design.

xxx_mallardcore
Figure 6. Dimensions of the circuitboard.

proto_kadessaFigure 7. Final “mallard”-prototype

The device has contributed quite broadly to the research. There are at least two PhD theses [1] [2] (excluding mine) with direct contribution, and it was used in several publications [3] [4] [5] [6] [7] [8]. At the time of writing, I was already working at University of Arts and Design in Helsinki, and was not part of the writing process. The development process is described in more detail in my own doctoral thesis.

Due to the impact of the work, Nokia wanted to continue working with me, resulting in several Design Sprints, two sponsorships for the Interactive Prototyping-course, as well as a project on organic user interfaces, resulting in the MA-thesis of Yi-Ta Hsieh.

References.

[1] Salminen K. (2015). Emotional Responses to Friction-based, Vibrotactile, and Thermal Stimuli. PhD thesis, Acta Electronica Universitatis Tamperensis 1543 ISBN 978-951-44-9795-7 (pdf), ISSN 1456-954X.

[2] Rantala, J. (2015). Spatial Touch in Presenting Information with Mobile Devices. PhD Thesis, Acta Electronica Universitatis Tamperensis; 1491 ISBN 978-951-44-9652-3 (pdf), ISSN 1456-954X

[3] Heikkinen, J., Rantala, J., Olsson, T, Raisamo, R., Lylykangas, J., Raisamo, J., Surakka, V., & Ahmaniemi, T. (2009). Enhancing Personal Communication with Spatial Haptics: Two Scenario-based Experiments on Gestural Interaction. Journal of Visual Languages & Computing, 20(5): 287-304, ISSN 1045-926X, http://dx.doi.org/10.1016/j.jvlc.2009.07.007

[4] Heikkinen, J., Rantala, J., Olsson, T., Raisamo, R., & Surakka, V. (2011). Exploring the Effects of Cumulative Contextual Cues on Interpreting Vibrotactile Messages. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI ’11). ACM, New York, NY, USA, 1-10. DOI=10.1145/2037373.2037375 http://doi.acm.org/10.1145/2037373.2037375

[5] Hoggan, E., Trendafilov, D., Ahmaniemi, T., & Raisamo, R. (2011). Squeeze vs. Tilt: A Comparative Study using Continuous Tactile Feedback. In CHI ’11 Extended Abstracts on Human Factors in Computing Systems (CHI EA ’11). ACM, New York, NY, USA, 1309-1314. DOI=10.1145/1979742.1979766 http://doi.acm.org/10.1145/1979742.1979766

[6] Rantala, J., Raisamo, R., Lylykangas, J., Ahmaniemi, T., Raisamo, J., Rantala, J., Mäkelä, K., Salminen, K., & Surakka, V. (2011). The Role of Gesture Types and Spatial Feedback in Haptic Communication. IEEE Transactions on Haptics, 4(4): 295–306

[7] Rantala, J., Salminen, K., Raisamo, R., & Surakka, V. (2013). Touch Gestures in Communicating Emotional Intention via Vibrotactile Stimulation. International Journal of Human-Computer Studies, 71(6): 679–690

[8] Rantala, J., & Raisamo, R. (2014). Preferences for Touch Gestures in Audio-tactile Communication. In Proceedings of IEEE Haptics Symposium 2014 (HAPTICS ’14), 247–250. Washington, DC, USA: IEEE

The Mallard Device