javaaudiosynchronizationbackground-music

Play a soundfile (.wav) in the background and have a block vibrating to it


I would like to play a soundfile (.wav) in the background and have a block vibrating to it. So i have already programmed a JFrame with a Panel and a Canvas in it, in which i have sort of a Block that is controllable by the Arrowkeys of the Keyboard. My main method has a loop that works roughly said like so:

while (play) {
        Date delay_time = new Date();
        delay_time.setTime(delay_time.getTime() + (int) (1000*(1.0/FPS)));
        Graphics g = can.getGraphics();
        can.update(g);

        //Here is my other stuff, like the Motion of the Block etc...


        while(new Date().before(delay_time)) {

        }

    }

My FPS variable is a static final int that is currently set to 30.

Now I would like to implement a Background Music that is playing constantly. It will be some eletrical/dubstep stuff.

Now my Question is: How do i implement that Music and (most important!) how do I make the Block vibrate according to the Music. (If you dont understand what I mean by vibrating to the music, maybe you could help me by explaining how I can get the itensity, resonance, or something like that, wich should be changing constantly during the Playtime of the Music.)

Thanks


Solution

  • Frame

    A frame is a cross section of samples across all channels in the audio file. So, a 16-bit stereo (two channel) audio file will have 32-bit frames (16 bits per sample * 2 channels per frame = 32 bits per frame).

    Load the Raw Data

    Java reads raw audio data in 8-bit bytes, but most audio has a higher sample size. So, in order to represent the audio, you'll have to combine multiple bytes to create samples in the audio format. But first, you'll need to load all of the audio into a buffer before you combine the bytes into samples.

    Start by getting an audio stream from a file:

    File file = new File(filename);
        AudioInputStream audioInputStream = AudioSystem.getAudioInputStream(file);
    

    Now that you have the AudioInputStream, you can read in the audio data. AudioInputStream has a read() method that takes an unpopulated byte[] and reads in data the length of the byte[]. To read in the entire audio file in one shot, create a byte[] the length of the entire audio file. The complete length of the file in bytes is:

    total number of bytes = bytes per frame * total number of frames

    You can get the number of frames for the whole file (frameLength) and the size of the frame (frameSize) from the AudioInputStream:

    int frameLength = (int) audioInputStream.getFrameLength(); int frameSize = (int) audioInputStream.getFormat().getFrameSize();

    You can create the byte[] with the length set to frameLength*frameSize:

    byte[] bytes = new byte[frameLength * frameSize];
    

    Finally, you can read in the audio, passing the AudioInputStream the empty byte[] and catching the appropriate exceptions:

    int result = 0;
        try {
            result = audioInputStream.read(bytes);
        } catch (Exception e) {
            e.printStackTrace();
        }
    

    Convert to Samples and Channels

    The raw audio data isn't very useful. It needs to be broken up into channels and samples. From there, it's easy to paint the samples.

    The bytes will be converted to samples and represented as ints. You'll need a container to store the samples across all channels. So, create a two dimensional int[][] referencing the channel and samples per channel. You've already seen how to get the frame length from the AuduioInputStream, and you can get the number of channels the same way. Here is the code to initialize the int[][]:

    int numChannels = audioInputStream.getFormat().getChannels();
    int frameLength = (int) audioInputStream.getFrameLength();
    int[][] toReturn = new int[numChannels][frameLength];
    

    Now, you need to iterate through the byte[], convert the bytes to samples, and place the sample in the appropriate channel in the int[][]. The byte[] is organized by frames, meaning that you'll read in a sample for every channel rather than all of the samples for a specific channel in a row. So, the flow is to loop through the channels and add samples until the byte[] has been iterated completely:

    int sampleIndex = 0;
    
    for (int t = 0; t < eightBitByteArray.length;) {
        for (int channel = 0; channel < numChannels; channel++) {
            int low = (int) eightBitByteArray[t];
            t++;
            int high = (int) eightBitByteArray[t];
            t++;
            int sample = getSixteenBitSample(high, low);
            toReturn[channel][sampleIndex] = sample;
        }
        sampleIndex++;
    }