Focusing parts by natural movement

The Sound Scope Phone introduces a new way of listening to music that enables you to focus on the part you want to listen to simply by facing the corresponding section and putting your hand on your ear.

^{_{Special thanks to: MIRU DESIGN, Shiseido, SOMARTA}}

RIKEN’s Sound Scope Phone introduces a new way of actively listening to music by enabling you to focus on the part you want to listen to. All you need is an iPhone and earphones or headphones for this active music listening experience using spatial sound and AI.

– Spatial acoustic experience

Put on your earphones or headphones and press the play button on Sound Scope Phone. You will hear performers playing instruments around you. If you hold your iPhone in front of you, you’ll see the application tracking your face on the display. By turning your head left and right, you can follow the sound of the performers on their respective sides. Likewise, if you turn around with your iPhone in hand, you will be able to face the performers behind you.

– Active music listening experience

This app provides an active music listening experience by enabling you to focus on the certain performers you want to hear. If you turn to face the performer of your choice, it will sound as if they are directly in front of you. By moving your left or right palm closer or further from your ears, you can adjust the range of the performance that you want to focus on. You can highlight one part or multiple parts such as the horn section and the rhythm section. You can check the direction of the performer and the direction you are facing on the position display.

– Mixing console and boost mode

Press the settings button at the bottom right to open the mixing console. The mixing console enables you to operate the faders to fine-tune the volume of each part. When the boost switch is turned on, the targeted sound will be more amplified. You can lower the boost limit fader to adjust the level.

– Development history

The Music Information Intelligence Team, part of the RIKEN Center for Advanced Intelligence Project, is exploring potential new ways of creating and listening to music using AI. When listening to classical music, for example, you may want to hear the first violin a little more clearly, or suppress the sound of the trombone sound, or only hear the flute. Sound Scope Phone is an app that enables you to specify the sections you want to listen to.

The app is available to the public to gauge the overall interest in such a way of listening to music. We ask that users the app who are prompted to fill out a questionnaire to please submit their responses. We plan to add songs in the future. If you are interested in the app and technology, please contact us from below.

New Contents!

We have added new contents on the app.
– “Wild Birds”: sounds of wild birds in and around the evacuation area of Fukushima Daiichi nuclear power plant accident, recorded by the National Institute for Environmental studies Japan, Biodiversity Division.
– “Metal Spirits 2022”: composed by Daisuke Hashimoto (guitarist, based in Shiga)
You can download the new contents by updating the app.

Reference

[LINK] [SoundScopePhone] Masatoshi Hamanaka: “Sound Scope Phone”, ACM Siggraph2022 Appy Hours AH-3, August 2022.
[LINK] [SoundScopePhone] Masatoshi Hamanaka: “Sound Scope Phone: Focus on a Specific Part Using Face and Hand Tracking”, the 22nd International Society for Music Information Retrieval Conference (ISMIR2021), Late Breaking, November 2021.

SoundScopePhone