The title, and even their own abstract, does not do this project justice. They did not simply map piano keys to a controller's buttons. About a quarter through the post:
A simple solution would be to link every piano key to a controler button. This could be fairly easy to build, but seemed that it would lean way too much towards non-music, providing too much of a simple translation, instead of a metaphor. We wanted a system that would allow us to say: “If you’re a good pianist, then you’ll be a good gamer”. ... Simply put, we had to find a way to translate what a good pianist would play into what a good gamer would play.
This talk then continues much later:
To tackle this situation, we decided to adopt a clear and strict separation of concerns, in which our firmware would be conceptual center of command:
* the single role of the piano is to play notes (key hits) * the single role of the console is to play controls (button hits) * the role of the pianette, our firmware, is to receive notes from the piano, somehow turn them into controls, and send these controls to the console
That became the basis for a refactor of the whole firmware.
A one-to-one mapping from piano keys to controller buttons would be interesting, but not... satisfying. They really thought about this deeply, and mapped music to fighting.
Thank you scott_s for quoting _these_ words from the post. They are our most sincere attempt to express why we even put so much effort developing the idea. It is great and humbling that they come across as such.
They tried, but I'm not convinced that they succeeded. Their video resembles Street Fighter in the way that https://youtu.be/-2gJamguN04 resembles Football. It's possible that their mapping works, in that there's some non-unpleasant piano sequence that maps to at least superficially convincing gameplay, but their video didn't show it.
The connection of analog (piano) to digital is impressive. The additional claim of having mapped music to fighting is not yet substantiated.
> The [...] claim of having mapped music to fighting is not yet substantiated.
(project and article co-author here)
Hopefully our writing does not come across as too much of a claim that we reached the single one and only best way to map music to fighting!
Our goal was to bring the most fun from both these worlds combined. Not an easy proposal to make, and obviously a personal one.
The overwhelmingly positive reactions from the audience during the public performance really made our day and validated this proposal to our eyes, but we'd be even happier if the project lived on with third parties pitching in other music/game metaphors. This is why we made it 100% open source/open hardware.
Speaking of the audience, we noticed that people were intimidated or thrilled by the possibility to play piano, others by the possibility to play a Street Fighter game, usually on a mutually exclusive basis. But if they dared to come and play, a balance could be quickly reached, and we considered this a success.
Obviously, it was't always the case. Some moments were just plain unlistenable.
But some other moments during were pure magic. We would like to give immense credit to pianists Alvise Sinivia and Léo Jassef (who appear in the video) from the Conservatoire National de Paris. They are incredibly talented musicians, and they played with our installation long enough that they knew the combos inside and out. It was an absolute delight to see and hear them play/fight during the public event.
It's neat that they can play a duet that results in actual gameplay; but generally I think you are correct, a player trying only to win would inevitably play something pretty awful sounding. (It would be hard to establish a musical parallel to game play on one piano - let alone two!)
Take the demo of a fight between skilled players and derive what keys they would have had to press on the piano to make it happen. I know that some (most modern fighters?) fighting games have options to show inputs on screen.
Sortof. For one if it could be done it would only work for that particular fight. Even that is highly unlikely though.
For one consider that music at least in terms how we (culturally, or biologically) are predisposed to listening to it: must keep time, you can change from 4/4 to 5/4 in the next section, but not arbitrarily (if you want it to sound like what people are inclined to call music at least).
(i.e. the drum-pad as a controller video someone linked in one of the responses, which is totally non-musical - is a good example).
Of course in a live-fighting game you don't care about structured timing, only about relative timing as to what your opponent is doing. Perhaps you could run a game through an emulator that enforced actions into particular time-blocks. Anyways I think it's very difficult, even to get something satisfactory rhythmically, not to mention tonality and a second piano.
I totally get what you are saying and I'm glad for the effort they put in, but...
I'd almost rather see a 1-1 mapping of controllers to piano keys and see people hacking on the piano, to see how good Street Fighter play corresponds to sound.
Even if it sounded terrible, I think that would be more interesting to me than this produced video, which I don't feel like I pulled a lot of factual information out of.
It would have been even better if they had been able to rig it so that the commands to operate the game roughly followed the rules of classical harmony, that way, the gameplay would be more obviously musical. As it is, it sounds a bit atonal.
"The concept was pitched for the reopening of the Maison de la Radio, French’s principal and historical radio building situated in the heart of Paris."
\o/ - everybody loves Maison de la Radio, home of the excellent French radio public broadcasting (France Inter, France Musique and especially France Culture - which is some serious public broadcasting with no compromise on quality.)
A simple solution would be to link every piano key to a controler button. This could be fairly easy to build, but seemed that it would lean way too much towards non-music, providing too much of a simple translation, instead of a metaphor. We wanted a system that would allow us to say: “If you’re a good pianist, then you’ll be a good gamer”. ... Simply put, we had to find a way to translate what a good pianist would play into what a good gamer would play.
This talk then continues much later:
To tackle this situation, we decided to adopt a clear and strict separation of concerns, in which our firmware would be conceptual center of command:
* the single role of the piano is to play notes (key hits) * the single role of the console is to play controls (button hits) * the role of the pianette, our firmware, is to receive notes from the piano, somehow turn them into controls, and send these controls to the console
That became the basis for a refactor of the whole firmware.
A one-to-one mapping from piano keys to controller buttons would be interesting, but not... satisfying. They really thought about this deeply, and mapped music to fighting.