I often sit on my sofa and hum tunes that I made up in my head. While doing that, I also imagine that my automatically gets converted to a beautiful musical piece. Google‘s new machine learning algorithm experiment, Tone Transfer, makes that fantasy come true — at least to an extent.
The idea is simple. Go to the Tone Transfer site from your Android phone or desktop (Windows or Mac), select “Add your own,” and record your 15-second hum or tune. You can use your voice, tap on your table, or play an actual instrument. Google’s machine learning algorithm will convert that tune into a digital signal, and then you can convert it into a tune with Flute, Saxophone, Violin, or Trumpet.
Now, the quality of output will depend on your mic and background noise while you’re recording. So, it might take a few tries for you to get a good recording. But, it’s a super fun tool and you won’t mind trying out new recordings.
My colleague Abhimanyu tried to record a guitar solo, but the algorithm turned the noise of the pick hitting the strings into high pitched notes.
Google’s Magenta AI team, which builds open source technologies to explore the use of machine learning in art, built called ‘Differentiable Digital Signal Processing (DDSP)’ library to make this tool possible. As explained in this blog post, if you control certain parameters of Digital Signal Processors (DSP), you will be able to produce natural sounds of instruments. With DDSP, the tool uses a neural network to convert user audio input into a DSP; that in turn opens it up for conversion to many instruments.
With the DDSP library, Google’s team can train audio synthesis models with fewer parameters and less data. These models help tools such as Tone Transfer create high-quality audio from user input.
Even if you’re not fascinated by the technology, you might be inspired to create a song just by your voice using this tool.
You can check out the paper related to DDSP here and try out the tool here.
So you’re interested in AI? Then join our online event, TNW2020, where you’ll hear how artificial intelligence is transforming industries and businesses.
Published October 1, 2020 — 08:47 UTC