Google has a page describing how to use one of their Gemini-1.5 models to transcribe audio. They include a sample script (see below). The script grabs the audio file from Google Storage via the Part.from_uri() command.
I would, instead, like to use a local file. Setting the URI to "file:///..." does not work. How can I do that?
Google's (working, cloud-based) code is here:
import vertexai
from vertexai.generative_models import GenerativeModel, GenerationConfig, Part
# TODO(developer): Update and un-comment below line
# PROJECT_ID = "your-project-id"
vertexai.init(project=PROJECT_ID, location="us-central1")
model = GenerativeModel("gemini-1.5-flash-002")
prompt = """
Can you transcribe this interview, in the format of timecode, speaker, caption.
Use speaker A, speaker B, etc. to identify speakers.
"""
audio_file_uri = "gs://cloud-samples-data/generative-ai/audio/pixel.mp3"
audio_file = Part.from_uri(audio_file_uri, mime_type="audio/mpeg")
contents = [audio_file, prompt]
response = model.generate_content(contents, generation_config=GenerationConfig(audio_timestamp=True))
print(response.text)
# Example response:
# [00:00:00] Speaker A: Your devices are getting better over time...
# [00:00:16] Speaker B: Welcome to the Made by Google podcast, ...
# [00:01:00] Speaker A: So many features. I am a singer. ...
# [00:01:33] Speaker B: Amazing. DeCarlos, same question to you, ...
Although one might expect the following base-64 encoding and decoding to cancel, the following code appears to work. (The code is a slightly modified version from this page.)
...
encoded_audio = base64.b64encode(open(audio_path, "rb").read()).decode("utf-8")
mime_type = "audio/mpeg"
audio_content = Part.from_data(
data=base64.b64decode(encoded_audio), mime_type=mime_type
)
contents = [audio_content, prompt]
...