Artificial hand learns to play the piano by listening
A robotic hand can immediately play a piano piece once it has heard it. This ability could enable robots to help people with restricted mobility in the future.
If you ever wanted to learn to play an instrument, you might be jealous: Researchers at the USC Viterbi School of Engineering in California have developed a robotic hand that can play a melody on a piano immediately after hearing it once - without sheet music or pre-programmed scores.
The hand consists of four fingers whose tendons are controlled by small electric motors. Neural networks analyse the sound of a melody and convert it into commands that control the motors. However, before the hand is able to play musical pieces, it has to be trained once. To do this, the fingers press the piano keys with alternating force for two minutes. During this phase, which the researchers call »motorised babbling«, the neural networks learn which motor signal is linked to which sound. The robot can then immediately play the melodies it hears.
For test purposes, the experts pitted the robotic hand against nine humans, four of whom were trained pianists and five of whom were piano beginners. They were asked to play three melodies of up to 24 strokes on four piano keys. Two professional composers blindly assessed the melodies they played. As it turned out, the artificial hand not only reproduced the melody better than the piano beginners, the level of the robotic playing was even comparable to that of the trained pianists.
However, the researchers do not see the actual purpose of the developed technology primarily in playing the piano. Instead, they want to develop robotics for various applications that recognise their surroundings and, after a short learning phase and without extensive training, help people with complex movement sequences. As an example, they cite Parkinson's sufferers who, shortly after their initial diagnosis, train an exoskeleton to perform their typical movements. At an advanced stage of the disease, they could put the exoskeleton back on and use it to move as before.
Spectrum of science
We are partners of Spektrum der Wissenschaft and want to make well-founded information more accessible to you. Follow Spektrum der Wissenschaft if you like the articles.
Experts from science and research report on the latest findings in their fields – competent, authentic and comprehensible.
From the latest iPhone to the return of 80s fashion. The editorial team will help you make sense of it all.
Show all
