Comments (11)
Yeah, I don't think we need it in close_websocket.
from whisperlive.
Reproduced on my end. Let me know if #43 doesnt fix the issue
from whisperlive.
Unfortunately that didn't seem to fix the problem. See attached screen recording.
Getting a few more tokens back, but not the full transcript
And so, my fellow America, ask not what your country can do
for you, as...
whisper_live_720p.mov
from whisperlive.
I dont see this on my end when using a GPU
, over the CPU
yes.
I think your server is not able to compute fast enough to be real-time. Its something that the client needs to take care of i.e. wait sometime before closing the websocket connection so that it receives everything. If I run the server on the cpu
I see the same issue.
After your audio ends, I have added a simple wait for 15 seconds before closing the websocket to consume the server responses. Feel free to change 15 to 20 if its still not showing the complete response.
from whisperlive.
@makaveli10 This makes total sense. Trying this out now. I am running on CPU
for development as I want to experiment with what I can accomplish with the small and tiny models as they do seem to work well for most situations from what I'm seeing. If this is the case, I can see some very interesting potential applications I'd like to build.
from whisperlive.
Hmm, seems that there are still some cleanup and timeout issues when running on cpu. Quite often I'll get the following error output from the server
ERROR:root:[ERROR]: 'WhisperModel' object has no attribute 'model'
^CTraceback (most recent call last):
File "/Users/justinwinter/projects/whisperlive/./run_server.py", line 5, in
server.run("0.0.0.0")
File "/Users/justinwinter/projects/whisperlive/whisper_live/server.py", line 136, in run
server.serve_forever()
File "/opt/homebrew/Caskroom/miniconda/base/envs/whisper_live/lib/python3.9/site-packages/websockets/sync/server.py", line 226, in serve_forever
poller.select()
File "/opt/homebrew/Caskroom/miniconda/base/envs/whisper_live/lib/python3.9/selectors.py", line 562, in select
kev_list = self._selector.control(None, max_ev, timeout)
KeyboardInterrupt
from whisperlive.
If I add the following to the client play_file
: method before the stream is closed, I can get the transcription to work every time now
elapsed = time.time() - self.LAST_RESPONSE_RECIEVED
while elapsed < self.DISCONNECT_IF_NO_RESPONSE_FOR:
continue
from whisperlive.
@justinlevi I did that in this commit. So, we can merge this now?
from whisperlive.
@makaveli10 I think your commit only has that addition in the close_websocket method
def close_websocket(self):
while time.time() - self.LAST_RESPONSE_RECIEVED < self.DISCONNECT_IF_NO_RESPONSE_FOR:
continue
I think it also need to to be in play_file
def play_file(self, filename):
# read audio and create pyaudio stream
self.wf = wave.open(filename, 'rb')
self.stream = self.p.open(format=self.p.get_format_from_width(self.wf.getsampwidth()),
channels=self.wf.getnchannels(),
rate=self.wf.getframerate(),
input=True,
output=True,
frames_per_buffer=self.CHUNK)
try:
while Client.RECORDING:
data = self.wf.readframes(self.CHUNK)
if data==b'': break
audio_array = Client.bytes_to_float_array(data)
self.send_packet_to_server(audio_array.tobytes())
self.stream.write(data)
self.wf.close()
elapsed = time.time() - self.LAST_RESPONSE_RECIEVED
while elapsed < self.DISCONNECT_IF_NO_RESPONSE_FOR:
continue
self.stream.close()
```
from whisperlive.
Otherwise the stream is closed right when the file is done playing which we don't want if we're waiting for the server to send back transcription text.
from whisperlive.
@justinlevi In that case, could you test if checking the elapsed_time
once in play_file
solves the issue. Wondering if we really need to check again in close_websocket
?
from whisperlive.
Related Issues (20)
- Only one line in the output srt file! HOT 2
- how to use a finetuned nonlocal model? HOT 1
- Failed to transcribe audio chunk HOT 2
- Infrastructure for multiple sessions
- Deadlock with pipe_stderr=True in FFmpeg subprocess HOT 1
- Maybe a way to avoid hallucinations HOT 2
- Not detecting correct mic input
- Unexpected error: Required inputs (['state']) are missing from input feed (['input', 'h', 'c', 'sr']). HOT 9
- Question about parallelism using whisper-live vs. faster-whisper on a single GPU HOT 1
- Problem about Windows running client HOT 1
- Problem about long stop when processing a given audio file HOT 1
- vad.py crashes after recent change to silero-vad HOT 1
- Required inputs (['state']) are missing from input feed HOT 7
- Request: Please expose ClientManager arguments (max_clients, max_connection_time)
- Can I use WhisperLive to transcribe zoom call in real time? HOT 2
- Unsupported Model Binary Version
- TensorRT backend gives no output
- ERROR:root:Error during new connection initialization: [WinError 2] 系统找不到指定的文件。
- Error: compute type float16
- error during new connection initialization: 'model' HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from whisperlive.