tencentblueking / bkpaas-python-sdk Goto Github PK
View Code? Open in Web Editor NEW蓝鲸 PaaS 平台 Python 工具集
License: MIT License
蓝鲸 PaaS 平台 Python 工具集
License: MIT License
背景信息:
django 3.2.13 版本, django 默认的Storage 新增了 validate_file_name 方法,该 方法默认接收一个相对路径:
在最新的 FileSystemStorage 实现中, _save 方法接收一个绝对路径,但是会将路径转换成相对路径,保证下一步的 validate_file_name 不出错:
paht = "/app/xxxx"
经过_save() 处理之后 会变为 app/xxxx
bkstorages 使用制品库时使用 BKRepoStorage
实例。该实例的 _save() 方法并未将绝对路径转化为相对路径的操作。
于是就会引发异常:
首先不知道是不是我对 file_overwrite
的释义有误解,在我的理解上,file_overwrite
表示同名文件会被覆盖
❌ 目前出现的问题: file_overwrite=True
时,同名文件上传后并未覆盖,而是原文件名拼接随机字符串后缀并另存
1.tar.gz
,最后是多出一个 1_6OFSWbn.tar.gz
的文件,期望是覆盖原文件保存Django 提供的 Storage 在 save
时,并不直接使用传入的 name
,而是通过 get_available_name
方法,检测重名及文件路径长度,如果重名,该方法会生成随机串拼接到原文件名直到没有文件重复:
def get_available_name(self, name, max_length=None):
"""
Return a filename that's free on the target storage system and
available for new content to be written to.
"""
dir_name, file_name = os.path.split(name)
file_root, file_ext = os.path.splitext(file_name)
# If the filename already exists, add an underscore and a random 7
# character alphanumeric string (before the file extension, if one
# exists) to the filename until the generated filename doesn't exist.
# Truncate original name if required, so the new filename does not
# exceed the max_length.
while self.exists(name) or (max_length and len(name) > max_length):
# file_ext includes the dot.
name = os.path.join(dir_name, "%s_%s%s" % (file_root, get_random_string(7), file_ext))
if max_length is None:
continue
# Truncate file_root if max_length exceeded.
truncation = len(name) - max_length
if truncation > 0:
file_root = file_root[:-truncation]
# Entire file_root was truncated in attempt to find an available filename.
if not file_root:
raise SuspiciousFileOperation(
'Storage can not find an available filename for "%s". '
'Please make sure that the corresponding file field '
'allows sufficient "max_length".' % name
)
name = os.path.join(dir_name, "%s_%s%s" % (file_root, get_random_string(7), file_ext))
return name
如果配置中提供文件覆盖的选项,那我觉得是需要重写该方法,在self.fileoverwrite=True
时,对该方法的行为进行改写
因为在 节点管理 -> https://github.com/TencentBlueKing/bk-nodeman 我们对自定义的 Storage
都有覆盖写的需求,所以我的处理方式是直接重写了 get_available_name
方法,加上了 self.fileoverwrite
的判断,定义了 StorageFileOverwriteMixin
:
class StorageFileOverwriteMixin:
file_overwrite = settings.FILE_OVERWRITE
def get_available_name(self, name, max_length=None):
"""重写获取文件有效名称函数,支持在 file_overwrite=True 时不随机生成文件名"""
dir_name, file_name = os.path.split(name)
file_root, file_ext = os.path.splitext(file_name)
def _gen_random_name(_file_root) -> str:
# 在文件名的起始位置添加随机串,源码规则为 "%s_%s%s" % (_file_root, get_random_string(7), file_ext)
# 上述规则对 .tar.gz 不友好,会在类型后缀中间加随机串,所以改为随机串作为前缀
return os.path.join(dir_name, "%s_%s%s" % (get_random_string(7), _file_root, file_ext))
# not self.file_overwrite and self.exists(name) 利用 and 短路特点,如果 file_overwrite=True 就无需校验文件是否存在
while (not self.file_overwrite and self.exists(name)) or (max_length and len(name) > max_length):
# file_ext includes the dot.
name = name if self.file_overwrite else _gen_random_name(file_root)
if max_length is None:
continue
# Truncate file_root if max_length exceeded.
truncation = len(name) - max_length
if truncation > 0:
file_root = file_root[:-truncation]
# Entire file_root was truncated in attempt to find an available filename.
if not file_root:
raise SuspiciousFileOperation(
'Storage can not find an available filename for "%s". '
"Please make sure that the corresponding file field "
'allows sufficient "max_length".' % name
)
name = name if self.file_overwrite else _gen_random_name(file_root)
return name
期待讨论该问题是否符合预期,以及是否有更优雅的处理,也很乐意提pr去修复该可能存在的问题~
请求 login 等接口失败时,打印的错误消息中,包含了请求参数,其中有 bk_token 信息
例如 http.py 中的:
logger.exception("http request error! method: %s, url: %s, kwargs: %s", method, url, kwargs)
目前蓝鲸环境中存在多个插件网关,当新系统接入插件服务时,需要拥有一个能够为新系统进行批量插件网关授权的管理应用账号
https://pypi.org/project/blue-krill/
https://pypi.org/project/bkstorages/
两个项目页面过于简陋,开发者很难找到细节信息。
参考 django setup.cfg 修正当前 setup.py
的一些问题:
https://www.djangoproject.com/weblog/2021/jun/02/security-releases/
限定 Django > 2.2.24,>3.2.4
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.