Click the pause button to pause and the play button to resume. Make sure to not press the pause button too quickly after you un-pause the game, otherwise the music will get all janky and stuff.
Credit to Yuzo Koshiro for all actraiser music.