1 Haptic Feedback
When bakers bake a cake, they know the result only after eating it. When pianists “bake” a tone, they know the result only after hearing it. Only after finishing the baking process, the hammer will hit a string, and/or the key will hit the key bottom, and the tone starts to be generated, just mil- liseconds later. The only real time control there is left is one that de- fines the length of the tone, not the quality. It is the damper which may act as an extension of your body, directly touching the string. In order to make good cakes a lot of experience is needed.
But what about touch? How much real time control do we have over the movement we use for each keystroke? Before the ham- mer reaches the string our work is by definition already done. But while playing the key, we touch it. Don’t we at least have real time control on the motoric side of the business? No, not on most keystrokes, because they are sim- ply over before we know it. Liter- ally, they are over before our brain even receives the sensory informa- tion from the touch itself. We have recorded actual (forte) keystrokes that last about 10 milliseconds. For our touch and pressure recep- tors it would take at least three times longer just to reach our brain, let alone interpret the in- formation and adjust our muscu- lar activity. So, touch is not re- ally real time enough. But there is vestibular data, our awareness of our angle towards gravity and proprioception, our spatial awareness measured by muscle tension receptors. You know, the sensors that allow you to touch your nose when your eyes are closed, or the ones you need to train so hard for La Campanella. How fast is it? The vestibular organs are directly attached to your head and therefore this type of sensory information reaches our brain very quickly. The proprioceptive information can travel with velocities of up to 120m/s, but has to travel long distances for example from your finger hand or arm. Let’s say that your finger is one meter from your brain, the information would reach your favorite gray mass at 80% completion of your fastest keystroke, in about 8 milliseconds. That is blazingly fast indeed. But the distance is still too long to be usable for real-time precise control, as the signal only reaches your brain once 80% of your keystroke has already been executed. Conclusion: while playing fortissimo, you do not have any type of real time control, because our nerve conduction velocity is simply too low.
But what about when you play pianissimo? Those long and slow keystrokes? Do we have any real time control over those, at least in regard to touch? It seems that we have some limited closed loop control, as long as we do not need to accelerate too fast at any specific part of the keystroke. But would the amount of data that reaches our brain before the keystroke is over, be rich enough for actual closed loop con- trol? For instance the sense of touch is much slower than our proprioception (our spatial awareness based on our muscular tension). So you can rely on your proprioception before you can rely on your touch. In other words, combining the information of different sensors, the so-called sensor fusion, costs time and you cannot directly rely on rich enough material. I would say that relying on real time human sensory information is close to impossible while playing the piano.
Great news for all piano builders! You do not need to regulate the action anymore, because you don’t feel it anyway, right? Well. . . That doesn’t seem to be the case, either. You can feel how well a piano is regulated after the keystroke. But, and here comes the important part, that is before the next keystroke. The central nervous system is combining vestibular and proprioceptive information, touch and pressure, in order to help you to learn how to program your next keystroke. This is what musicians also call “practicing”. It is undeniable that practicing can provide you with more interesting results on a more capable instrument.
The sad conclusion is that in playing the piano you do not actually have any real time closed loop control; not from auditory feedback and not from haptic feedback. However, we are train- ing our muscle memory and we are training circuits that help us to adapt to different touch as if we have a volume knob that we can use when we change instrument to scale our fine motor memory. Your muscle memory system has a shorter, cheaper and thus faster route, but is still reporting after the fact to your brain. Your brain is able to scale your performance in order to adjust it to a different instrument. Needless to say, a trained professional can adapt much more easily than an untrained beginner, in the same way as learning to play is different from learning to play a piece while you can already play. Also, a pro- fessional who has trained countless hours on a not so capable instru- ment, might still be rhythmically precise for example, might have some reasonable control of dynam- ics, but will never be able to really control the tone if this was never practiced.
2.1 The action
So what is a good action? What properties does it have? What properties help you to depress the next key in a way that is closer to achieving the musical result that you aim for compared to your last bet with the previously depressed key? And what else should it do? In the end, when the piano was born out of father hammered dulcimer and mother church or- gan (yes, it got its teeth from the mother), the piano action was not primarily developed as a nice user feedback device, but as a power transmission line to convert that stored chemical energy out of your sandwich with ham and cheese into acoustic energy in the room. It is a leverage system that makes it more efficient for a human to drive all the strings on the instrument and it needs to be pretty fast as well.
Still the piano is mainly an acceleration sensitive instrument. The complete acceleration curve has influence over your tone. It determines not only if you hear louder or softer, earlier or later keybed sounds complementary to the hammer-string induced tone, but it also determines the quality of the transients in general, that is the quality of the beginning of each tone, the first few milliseconds.
For context sake, it might be interesting to notice, that the neural response to transient portions of a stimulus is typically much larger than to steady-state portions, as for evolutionary reasons we use sound to decipher the physical world around us. The physical information of an event is stored in the very beginning of each sound. Humans need to know if they heard a stone fall into the mud or into the grass. The dif- ference in the auditory informa- tion is just stored only in the tran- sient. Humans are very keen on subconsciously deciphering transients. Even the information in speech is stored mostly in the transients. When you delete 98% of the steady-state sound of both consonants and vowels in a spoken text recording, the remaining 2% of the sound, namely the tran- sients, are still equally intelligible as the whole word. Leaving a different portion of 2% of the sound, not including the transients, creates nothing understandable at all. Even most people happen to be unable to discern the sound of a piano from a violin if one cuts off the first 40 milliseconds of one tone.
So your main concern while “operating” the piano, is controlling the acceleration curve of the hammer. How do you feel acceleration? Simple, by its inertia; its unwillingness to change of state or here change of movement. The faster you accelerate, the harder it gets, or as good old Isaac likes to say it: F = m a. So the most important part of the haptic feedback is to feel the acceleration of the hammer as clearly as possible, with as little as possible clouding by other moving masses. Also, you even feel when the ham- mer is away and when it recon- nects.
The second part of haptic feedback that comes in handy, would be what I call the resistance grid. When you push a key down, you feel all kinds of blobs of resistance, most notably, when the key hits the keybed and also very noticeably about 1 mm before the end, when the jack flips away underneath the action. But before those highly notable events occur you have a rich experience of all kinds of little blobs of resistance that are related to mechanical parts scraping on each other while playing the key. This is related to position. However in reality, unfortunately, the parts in a real piano tend to bend. So the more you accelerate, the less precise this grid becomes. But let’s neglect that part for now. The point is, this resistance grid still gives you information about the depth of the key.
Of course your vestibular and proprioceptive system are working hard as well to do some nice sensor fusion on your central nervous system in order to enrich the data of the touch. But the really important aspect about the touch is the inertia. And the resis- tance grid just really helps, espe- cially for beginners. All other arti- facts that you feel are only cloud- ing your precious haptic informa- tion that you need for some suc- cessful hammer launches.
With an acoustic instrument however, you do not only control the instrument, but you also power the instrument. The acoustic energy in the room, thus the sound that you created, was all converted out of the stored chemi- cal energy in that nice little sand- wich that you ate three hours be- fore playing. So next time that you hear a piano, you know that you sort of listen to your sandwich with ham and cheese. All the wonder- ful aspects of user feedback that I described above, emerged only as (useful) artifacts of a power trans- mission line that was invented in order to enable you to develop me- chanical power and to transmit it into the instrument. The basic architecture of this power transmission line is a lever with roughly a 1 : 5 ratio. You press the key 1 cm down, and you move the hammer 5 cm up. The movement of the lever is restrained to that range. So when you perform a full keystroke the hammer can decou- ple after roughly 5 cm of connected flight and have his miraculous 1 mm long and lonely free flight un- til it hits the string. We can con- clude that the action builders had the idea that it was easier for peo- ple to press a key down, than to throw a hammer up, and that it was easier to press a key 1 centime- ter down with 5 times the force, than to press it 5 centimeters down with 5 times less force than be- came common.
Piano builders had to figure out what mix would work best for their idea of operating their own ballistic instrument. The im- portant parameters involved were power transmission efficiency, rep- etition time and tonal control. Ev- ery action had to be appropriate for its purpose. An upright piano and a grand piano are members of the same family of ballistic piano instruments, their actions operate on the same ballistic principles, but they are very different. An upright piano action is not made as a surrogate grand piano action or something as ridiculous as that, but it is made as the best ac- tion to operate that specific ballis- tic instrument in the way that the maker believes is the best mix of parameters. This makes both in- struments very different from each other, but also fully compatible and interchangeable. You can use the vertical and horizontal piano for the exact same repertoire with- out any artistic disadvantage. Of course there are better and worse, more interesting, less interesting instruments of both kinds. But they remain interchangeable with- out a problem. You can influence the dynamics in the same way, you can influence the timing of the percussive keybed sounds, you can influence the color of the tone in the same way. Etc. Only the amount of effort is different. For instance a horizontal piano is much less en- ergy efficient than a vertical piano, thus as a result, it plays heavier. However it can repeat a bit faster, and has a more outspoken keybed sound response. On top of that it provides you with a slightly longer time frame that arguably makes controlling the tone a bit less diffi- cult. Still they remain fully inter- changeable. Notice that the digi- tal “piano” is not interchangeable with the other two, as the only thing it can do is play a note louder or softer. The way to make it louder or softer has nothing to do with actual ballistics and accelera- tion, but with the average velocity of a full keystroke. So it can only mimic full keystrokes played in one predetermined way, a bit louder, or a bit softer.
There is no such thing as the grand piano action, or the upright piano action, or the digital piano action. All those instruments have a multitude of action configurations, even when they are based on a certain fixed principle. When thinking of grand pianos, who remembers the Viennese actions? B¨osendorfer even produced them into the twenti- eth century. Light, slow and by today’s standards weird. Most of the actions are replaced by the 1821 Erard design, bulky, heavy, but faster in repetition. There were some notable exceptions such as the smooth and light (however slower–repeating– than–Erard ) Blu¨thner action, or the Hickman action that tried to solve a multitude of problems at once, that can now be solved even in different ways. Even the Er- ard type of action has many fam- ily members, choosing different op- timizations. After Erard, Herz and Schwander were the big play- ers. Other action companies fol- lowed with designs based on theirs. And different piano manufactures were interested in different optimizations. Steinway was for example at one side of the spectrum mainly concerned with maximizing power transfer at the cost of tonal control, while on the other side of the spectrum Bechstein was mainly concerned with optimizing control during the whole keystroke at the cost of having to rely on sharper contact angles and thus less power transfer. Steinway is optimized for percussive quality, Bechstein is optimized for singing quality and control of color. These are just two very opposing examples, but there were many different choices that instrument makers made. The end of wwi was also the end of the high quality pi- ano industry as Germany, where the main developments of the era took place, bankrupted, and only the cheapest action builder could survive. Expertise was lost and we were left with the bulky mod- ern action that is not optimized for anything, not for control, not even for power transfer, but only for lowering production costs. Mod- ern actions look very much like good actions. At first sight, noth- ing changed over the last century. But when you play them, you know it is just the looks that did not change.
The first question to answer before we can answer what your action should feel like is thus, “what is the purpose of your action”. This is complicated to answer within the framework of a good mechanical piano. Within the framework of a digital however it is much simpler. You don’t need to find the sweet spot between op- timizing for control or for power. You only need to optimize for con- trol since you don’t power the in- strument yourself; electricity does.
A “superfancy 3-sensor super optical sensing” keyboard only measures your average velocity during a full keystroke towards the bottom of the keybed. Obvi- ously inertia, thus a flying mass in the action such as hammers, would be the most devastating mislead- ing property to implement in the action as inertia suggests a sensi- tivity to acceleration while there is none. Additionally, as in the cur- rent digital “pianos” it lacks any mechanical purpose, inertia should just really not be there. It actively misleads the user as it will signal our brain to deal with the instrument as an acceleration sensitive instrument, while it is average velocity sensitive instead. How can the instrument give you velocity feedback? Via a resistance grid. Your brain keeps track of position and plots it over time. And why do those digital “pianos” still use fly- ing masses? It seems to have to do with a complete lack of taste and understanding, a good dose of mis- leading information and the sim- ple fact that those instruments are never built to be musical instru- ments, but surrogate instruments to use when there is no real one. Pianists often want them heavy so that they keep training their mus- cles even when no musical instru- ment is available.
In contrast to the digital home trainers that are widely available at all price ranges, Respons, is actually part of the piano-family. You play Respons as you play piano. It is sensitive to the acceleration curve and it is a ballistic musical instrument, as is the piano. That means that you, in this case, do need to have the exact same kind of feedback as in every member of the piano–family: inertia and re- sistance grid. Since it is digital, we still do not need to worry about the actual power transfer, so the optimization is purely for control and we rely on the same mental map or conceptual model that the user has from playing other pianofamily instruments. Now we need to determine only how much feedback we need. For that we could look into how a child learns to grab a pencil.
2.2 The human
Back to training and your level of expertise; how should your piano action feel? How does a child grab a pencil? With his whole body. There is no fine motoric control. Instead the whole body, with the whole body mass, is working to grab it. When the child grows and learns, smaller areas of the body are being used for the same task. This gives a more elegant result, also more efficient and more re- fined. Precision is a fine motoric task that can only be learned by training. So what about the action? Maybe contrary to intuition, children and other beginners need relatively heavy actions. Note the word relatively, because obviously it should be a bit heavy for them, not for an elephant. When the action is heavy, the feedback is more clear, and a keystroke lasts longer. You feel the inertia bet- ter, and you feel the position bet- ter, and above all the information is provided to you in a more relaxed timeframe so that it is easier to learn to recognize what you feel. Also its resistance helps you not to overshoot as easily when you try to actuate the keys in a somewhat controlled manner. When you get better and better, the resistance becomes more and more annoy- ing and even problematic. You need it to become non-obtrusive, it should not hinder you in biomechanical terms more than can be justified by mechanical necessity to drive the instrument. Also, it should always give you a clear and rich mix of haptic feedback, with as little as clouding as possible. The better you become, the less obtrusive feedback you need. And everything has to do with the Just Noticeable Difference (jnd). The jnd is one of the most discussed terms within the psychoacoustics and actually in the psychophysics in general. It is the branch of experimental psychology that studies sensation and perception. The jnd is the amount that something must be changed for the difference to be noticeable, defined to mean that the change is detectable half the time. This is why so many people who have their weekly lesson and learn some of Chopin’s nocturnes 90% of the time prefer a much much heavier action than what Vladimir Horowitz would have ever consid- ered even touching.
2.3 Our Respons
When designing actions we optimize it for certain mechanical specifications, but also for psychophysical specifications. In terms of the action of Respons, we optimize it for users of all cate- gories as you can adjust it to your needs. Our action is active, that means it is motor controlled. We create a feeling as clean as possi- ble with inertia and a resistance grid fully connected to how the tone is being generated, in order to make it operable as any other real musical instrument in the piano- family. As you develop as a pi- anist, your jnd of many things is shifting. One of those things is your sensibility for the inertia and for the resistance grid. In our action you can adjust that as a user. The software in fact makes ultraprecise alterations in the physical mechanics.
Everything we do within our company has to do with the search for this fine line of the jnd. The action of Respons is no exception. It is very important to have a deep understanding of the qualities of the historic implementations of actions in order to make something that is very good today. We need to understand both the mechanical purpose and the hu- man centered purpose of the ac- tion and we need to be aware of the nuances in all kinds of designs. Simultaneously, just as is done with the vertical piano compared to the horizontal one, we must de- velop the best and the cleanest ac- tion possible to operate our instrument. We don’t break with the piano family, but we do it differently as we do not create any surrogate grand piano, but a real self wor- thy musical instrument. You could use Respons as a heavy home trainer (if you turn the settings to heavy), but you would most likely train in order to later perform on a worse instrument. And remember, Respons is portable. We would therefore encourage anyone to set it as sensitive as one can manage, and to enjoy the extreme responsiveness.
With Respons, it is possible to have more precision than with the best instruments of the piano heyday. The only bottle neck could be your sound module. But we have one in the making as well: Tendens. More about Tendens in the next newsletter.
Thomas Kaduk
Commenti