sean1832 / sumgpt

Bulk summarization of documents using ChatGPT API

SumGPT

Achieve detailed summarization of extensive documents through 🚀ultra-fast parallelized predictions, utilizing GPT-3.5 and GPT-4 APIs provided by OpenAI.

🌐 Web App: https://sumgpt.streamlit.app

---

⭐️ Like this repo? please consider a star!

💡As I am not a professional programmer and am fairly new to Python, this project may contain bugs. If you encounter any issues, please suggest them in the Issues section.

---

🌟 Features

• 📄 Summarize document (.pdf, .docx, .txt, .md).
• 🎥 Summarize YouTube video with subtitles.
• 🤖 Customizable parameters and bot persona for refined response generation.
• 🚀 Facilitates parallel processing of chunks, enabling ultra-fast generation speeds.
• 💼 Export & import configs for easy sharing and reuse.
• 🧠 Supports GPT-3.5 and GPT-4.

💡 What you need

• 🔑 OpenAI API keys

Note: To access GPT-4, please join the waitlist if you haven't already received an invitation from OpenAI.

💻 Running Locally

• Make sure you have python 3.11 | python installation tutorial (YouTube)

1. Clone the repository

git clone https://github.com/sean1832/SumGPT

2. Execute RUN.bat

sumgpt's Issues

[FEATURE] Support Chinese caption video

I tried to summarize a youtube video with Chinese (traditional) subtitles and it tells me the following message：

❌ Caption language currently not supported.

{'zh-TW': <Caption lang="Chinese (Taiwan)" code="zh-TW">}

Describe the solution you'd like

Would like to request a feature to support this kind of video.

RateLimitError

每次執行的結果都是出現

openai.error.RateLimitError: This app has encountered an error. The original error message is redacted to prevent data leaks. Full error details have been recorded in the logs (if you're on Streamlit Cloud, click on 'Manage app' in the lower right of your app).

[BUG]中文文字稿总结成英文

不知道为啥一直总结成英文

[BUG]Can't Launch the repo:unsupported operand type(s) for |: '_GenericAlias' and 'type'

Describe the bug
启动RUN.bat脚本后

To Reproduce

发生异常: TypeError
unsupported operand type(s) for |: '_GenericAlias' and 'type'
  File "E:\github\SumGPT\src\GPT\misc.py", line 48, in <module>
    def predict_token_single(chunk: Dict[str, Union[str, float]] | str, max_tokens: int = None) -> int:

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots

Envrionment (please complete the following information):

• Web or Local: local
• Version: [e.g. 0.6.1]
• OS: WIndows11
• Browser chrome

[BUG]尝试用ChatGPT把.bat转化成.sh，但出现报错

不会代码，使用ChatGPT转化的

转化的sh文件

#!/bin/bash

设置虚拟环境变量

VENV_NAME=venv

如果虚拟环境不存在，则创建一个虚拟环境

if [ ! -d "$VENV_NAME" ]; then
echo "Creating virtual environment $VENV_NAME..."
python3 -m venv $VENV_NAME
fi

显示虚拟环境已准备就绪

echo "Virtual environment $VENV_NAME is ready."

激活虚拟环境

echo "Activating Virtual environment!"
source ./venv/bin/activate

检查库更新

REQUIREMENTS="requirements.txt"
LAST_MODIFIED="requirements.temp"

检查 requirements.txt 文件是否存在

if [ ! -f "$REQUIREMENTS" ]; then
echo "Error: requirements.txt not found"
exit 1
fi

检查 last_modified.txt 文件是否存在

if [ ! -f "$LAST_MODIFIED" ]; then
echo "0" > "$LAST_MODIFIED"
fi

检查 requirements.txt 文件是否被修改

mod_date=$(stat -f "%Sm" -t "%Y-%m-%d %H:%M:%S" "$REQUIREMENTS")
last_mod_date=$(cat "$LAST_MODIFIED")

if [ "$mod_date" != "$last_mod_date" ]; then
pip3 install -r $REQUIREMENTS
echo "$mod_date" > "$LAST_MODIFIED"
clear
echo "Requirements file has been modified. Updated complete!"
else
echo "Requirements file has not been modified. Skipping update."
fi

运行 SumGPT.py 程序

streamlit run src/SumGPT.py

---

报错内容

2023-04-13 22:47:24.330 Uncaught app exception
Traceback (most recent call last):
File "/Users/sing/Downloads/333333/SumGPT-1.0.5/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script
exec(code, module.dict)
File "/Users/sing/Downloads/333333/SumGPT-1.0.5/src/SumGPT.py", line 2, in
import Components
File "/Users/sing/Downloads/333333/SumGPT-1.0.5/src/Components/init.py", line 1, in
from Components import sidebar
File "/Users/sing/Downloads/333333/SumGPT-1.0.5/src/Components/sidebar.py", line 2, in
import GPT
File "/Users/sing/Downloads/333333/SumGPT-1.0.5/src/GPT/init.py", line 1, in
from GPT import misc
File "/Users/sing/Downloads/333333/SumGPT-1.0.5/src/GPT/misc.py", line 48, in
def predict_token_single(chunk: Dict[str, Union[str, float]] | str, max_tokens: int = None) -> int:
TypeError: unsupported operand type(s) for |: '_GenericAlias' and 'type'
2023-04-13 22:47:28.079 Uncaught app exception
Traceback (most recent call last):
File "/Users/sing/Downloads/333333/SumGPT-1.0.5/venv/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script
exec(code, module.dict)
File "/Users/sing/Downloads/333333/SumGPT-1.0.5/src/SumGPT.py", line 2, in
import Components
File "/Users/sing/Downloads/333333/SumGPT-1.0.5/src/Components/init.py", line 1, in
from Components import sidebar
File "/Users/sing/Downloads/333333/SumGPT-1.0.5/src/Components/sidebar.py", line 2, in
import GPT
File "/Users/sing/Downloads/333333/SumGPT-1.0.5/src/GPT/init.py", line 1, in
from GPT import misc
File "/Users/sing/Downloads/333333/SumGPT-1.0.5/src/GPT/misc.py", line 48, in
def predict_token_single(chunk: Dict[str, Union[str, float]] | str, max_tokens: int = None) -> int:
TypeError: unsupported operand type(s) for |: '_GenericAlias' and 'type'

[BUG] Failed to fetch data from YouTube after 5 attempts

Cannot extract transcript from Youtube link

Error message：

Failed to fetch data from YouTube after 5 attempts. Please "🔃 Refresh" button to try again.

input link: https://www.youtube.com/watch?v=hVWLNj0td3k&t=137s&ab_channel=PhocusWire

[BUG] 运行完bat 如何浏览网页呀

Checking library updates...
"Requirements file has not been modified. Skipping update."

Welcome to Streamlit!

If you’d like to receive helpful onboarding emails, news, offers, promotions,
and the occasional swag, please enter your email address below. Otherwise,
leave this field blank.

Email:
运行完，没显示端口号，我用8501 不对