swiftuiaudio-playeravaudioenginegcdasyncsocketavaudiopcmbuffer

Audio data (getting in AvAudioEngine) plays sizzling audio in SwiftUi


I am trying to send audio (getting over iPhone microphone) to another listening device via GCDAsyncUdpSocket, I get audio with AudioEngine and I converting AVAudioPCMBuffer to Data and sending via socket but When I listen to coming audio sounds, it plays sizzling sounds,I shared below sended Audio and converting method AVAudioPCMBuffer to Data, and listening Audio Method, Why I getting sizzling audio from sended Audio?,thanks..

func setupAudio () {
    let sampleRate = 16000
    self.audioEngine = AVAudioEngine()
    self.mixer = AVAudioMixerNode()
    //audioFormat = AVAudioFormat(standardFormatWithSampleRate: Double(sampleRate), channels: 1)
    self.mixer = AVAudioMixerNode()
    self.mixer.volume = 0
    self.audioEngine.attach(mixer)
    
    self.socket = GCDAsyncUdpSocket(delegate: self, delegateQueue: DispatchQueue.main)
    audioFormat = AVAudioFormat(standardFormatWithSampleRate: Double(sampleRate), channels: 1)
}

//Sending Audio
    func sendAudio {
    try! AVAudioSession.sharedInstance().setCategory(AVAudioSession.Category.playAndRecord)
    try! AVAudioSession.sharedInstance().setActive(true)
    
    let format = AVAudioFormat(commonFormat: AVAudioCommonFormat.pcmFormatInt16,
        sampleRate: 44100.0,
        channels: 1,
        interleaved: true)
    
    self.audioEngine.connect(self.audioEngine.inputNode, to: self.mixer, format: format)
    self.audioEngine.connect(self.mixer, to: self.audioEngine.mainMixerNode, format: format)
    
    
    DispatchQueue.global(qos: .background).async { [weak self] in
        guard let self = self else { return }
        do {
            self.socket.setIPv4Enabled(true)
            self.socket.setIPv6Enabled(false)
            try self.socket.connect(toHost:"235.10.10.100" ?? "", onPort: 4646 ?? 0)
            try self.socket.beginReceiving()
            print("Socket started")
        } catch {
            print("Socket Started Error: \(error)")
        }
    }
    
    self.mixer.installTap(onBus: 0, bufferSize: 2048, format: format, block: { (buffer: AVAudioPCMBuffer!, time: AVAudioTime!) -> Void in
        
        let data = Data(bytes: (buffer.int16ChannelData![0]), count: Int(buffer.frameLength))
        print(buffer)
        
        DispatchQueue.global(qos: .background).async { [weak self] in
            guard let self = self else { return }
            do {
                self.socket.send(data, withTimeout: 0, tag: 0)

            } catch {
                print("Socket send Error: \(error)")
            }
        }
    })
    
    audioEngine.prepare()
    do {
        try audioEngine.start()
    } catch {
        print("Can't start the engine: \(error)")
    }
    }

// Listening Audio

func udpSocket(_ sock: GCDAsyncUdpSocket, didReceive data: Data, fromAddress address: Data, withFilterContext filterContext: Any?) {
    audioPlayer.scheduleBuffer(getComingAudio(with: data), completionHandler: nil)
}

func getComingAudio(with data: Data) -> AVAudioPCMBuffer {
    let audioBuffer = AVAudioPCMBuffer(pcmFormat: audioFormat, frameCapacity: UInt32(data.count) / 2)!
    data.withUnsafeBytes { (bufferPointer: UnsafeRawBufferPointer) in
        let int16Array = Array(bufferPointer.bindMemory(to: Int16.self))
        let floatArray = int16Array.map { Float($0) / Float(Int16.max) }
        floatArray.withUnsafeBufferPointer { audioBuffer.floatChannelData!.pointee.assign(from: $0.baseAddress!, count: floatArray.count) }
    }
    audioBuffer.frameLength = audioBuffer.frameCapacity
    return audioBuffer
}

Solution

  • On your sending side, you fetch float data:

    let channels = UnsafeBufferPointer(start: PCMBuffer.floatChannelData, count: channelCount)
    

    But on your receiving side, you interpret it as Int16 data:

        let int16Array = Array(bufferPointer.bindMemory(to: Int16.self))
    

    That's at least a part of your problem. You also need to more generally ensure that your two formats are the same. It's not unusual to have different sample rates in particular. But the "static" sound is most likely due to your float/int16 conversion.

    I would start your project a little more simply, and write your data to memory, then play it back on the same device. This will ensure that your basic logic works.